Make max_time dynamic

[wwwutz]

It would be great win for resource usage if we allow max_time to be more, maybe dynamic.

Currently we have a system with a 'low' max_time of 15 minutes to provide the system for interactive usage within 15 minutes. In the past this 'freeing up' a system only happens 3 or 4 times a month for 1 or two days. The system more or less idles most of the time.

What if we would set up a 1hour step / 24h max_time - table like "from 8:00 to 17:00 only accept 15 minute jobs, after 17:00 longer jobs which have to be finished at 8:00 next day"

07:00: 15 min
06:00  60 min
05:00 120 min
04:00 180 min
03:00 240 min
02:00 300 min
01:00 360 min
00:00 420 min
23:00 480 min
22:00 540 min
21:00 600 min
20:00 660 min
19:00 720 min
18:00 780 min
17:00 840 min

so for example a job with maxtime of 1hr submitted at 06:59 will run until 08:00.

but as a win, after 17:00 jobs with 14 hr max_time will be accepted for at least 1 hour.

Or put if in a formula, but a table with 24 entries might be more comfortable/flexible to edit.

[donald]

Hmm, okay. Another thought behind the time limit was, that we want to keep some capacity available for short jobs and not allow long jobs to occupy the whole cluster. The wait time (until the job starts running) should be in the same magnitude as th run time. If your jobs runs several days, it is okay two wait a few hours. But if your jobs runs 15 minutes, you don't want to wait hours before it gets started.

[wwwutz]

Yeah! That's exactly what I don't want to spoil. This is why I introduced the time table: In our use case we only want the "short time availability" in between working hours. Noone needs a machine at night so fast. We still don't lose this function. We are just able to strech it for "nightly" usage.

It could also be done by an external program, stopping and restarting mxqd with different options at different times. Maybe that would also be an option, but I wanted to discuss this first.