Add --blacklist/--whitelist to mxqsub, add mxset #82

Avoid unneeded warning when scanning JOB_TMPDIR_MNTDIR for possible leftover job mounts. The parent directory of JOB_TMPDIR_MNTDIR is created when the first job with the --tmpdir feature is started. So it won't exist in a new server. This cleanup path is only used in the exceptional case that the usual cleanup path (via job_has_finished) didn't succeed, e.g. when mxqd was killed. The cleanup is not essential for the currently running server.

Add function attribute ((unusged)) to avoid a compiler warning when the static inline function defined in the header file is not used by the compilation unit.

Add header file for the quasi-standard [1] xmalloc call. The current approach implemented in mx_util is to wait and retry on ENOMEM malloc failure. However, this overhead doesn't seem to be justified, because it is difficult to imagine a case, where a malloc would fail with ENOMEM at one time and a retry would succeed. [1] https://www.gnu.org/software/libc/manual/html_node/Malloc-Examples.html

Add a utility module which can store a set of keywords. The set can be created and updated from a string and can be serialized into a string. The canonical string representation produced by this utilits is the space separated list of the sorted keywords in lexical order. Input strings used to create or update a keyword set contain keywords separated by whitespace. The keywords may be prefxied with "+" and "-". For updates, first all unprefixed keywords are processes to create the initial set, than updates are applied from the prefixed keywords. An empty string used for updates is considered "no change", not "set to empty set". Usage example: struct keywordset *kws = keywordset_new("xx yy") keywordset_update(kws, "-yy +zz") // remove yy, add zz keywordset_update(kws, "aa bb") // set to aa, bb if (keywordset_ismember(kws,"bb")) ... // true char *s = keywordset_get(kws): // s now "aa bb" free(s); // caller must free() keywordset_purge(kws) // set nowto empty. keywordset_free(kws);

Add list of blacklisted and whitelisted servers to group. The string is supposed to be canonical (lexical sorted, space separated) as produced by keywordset_get(), so that groups with equal sets can be found by sql string compare.

Add a new options so that specific mxqd servers can be excluded from starting jobs for this group. mxqsub --whitelist "acedia avaritia" sleep 10 // start only on these mxqsub --blacklist dontpanic sleep 10 // start on any node but this one Blacklist has priority, so mxqsub --white "kronos, uselessbox" --black kronos would not start on kronos. The lists of blacklisted and whitelisted servers can be modified with mxqset.

Add a new command which can be used to modify an existing group. Examples: mxqset group 123 --closed mxqset group 123 --open mxqset group 123 --blacklist "dontpanic" # replace mxqset group 123 --whitelist "+uselessbox +gula" # add mxqset group 123 --blacklist -dontpanic # remove mxqset group 123 --whitelist "" # clear The flags open and closed can be set from mxqadmin as well (`mxqadmin --close=123`) , but the synopsis, in which the options take the groupid as a value is difficult to expand to new options, which take a value.

Check, whether this server is qualified to start jobs from a group. Lazy-evaluate qualification criteria defined in the group and cache the result. The qualification criteria need to be reevaluated when the active groups were reloaded, because they may be changed by the user in existing groups. For now, the only qualification criteria are the groups blacklist and whitelist. If a group has a whitelist, the short or long hostname of the mxqd server needs to be on that list, otherwise the server is not qualified for the group. If the servers name is on the blacklist of the group, the server is not qualified for the group. Don't start jobs we are not qualified for. This can later be expanded to additional criteria (e.g. hostconfig or processor flags).

Add --blacklist/--whitelist to mxqsub, add mxset #82

Add --blacklist/--whitelist to mxqsub, add mxset #82

Commits on Apr 15, 2020