Add exclusive mode and prerequisites #86
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The only two functions from the mx_strvec familiy which are used by mxqd and mxqsub have no tests currently. Add test.
Remove test for a "cache bug", as we are going to remove the mx_strvec cache feature in the next commit.
The idea of the cache was to store the address and the number of the elements of a single vector in variables each time the length was determined. However, this aliases the pointer to the string vector owned by the user. If the users free()s his pointer to the vector, the cached pointer becomes invalid. An approach would be to require the user to always use a library routine to free the vector. This library routine would need to invalidate the static alias pointer, too. However, the assumed performance win doesn't seem worth the complexity. Remove the cache code.
Closed
Only unmount and remove job temporary directories for jobs, which actually requested one to avoid meaningless warnings in the logfile.
We'd like to have a daemon flag, namely "exclusive". This flag should be published by the daemon into the database, so that is visible to the web interface. Although we currently need only a single boolean flag, make it a integer, so that we can add more flags at a later time without changing the database scheme.
When a daemon is started, with --exclusive it will set daemon_flags to 1 to indicate, that it is running in exclusive mode. In exclusive mode, the daemon should only start jobs which have an implicit whitelist (`mxqsub --whitelist`) which includes the daemons name. This is implemented in the next commit.
If the daemon was stared with `--exclusive`, only start jobs which have the daemon in their whitelist.
donald
force-pushed
the
add-prerequisites
branch
from
4f35ed7
to
65e928c
Compare
Do not match "abcdef" with "abc" in find_name().
Update of "abcdef" with "+ab" wrongly resulted in "ab", because the substring was matched for the keywords. Add a test for the bug fixed in the last commit.
Add function keywordset_add to update a keywordset with names from a string, not requiring embedded "+" markers.
Add function keywordset_free_byref to be used with `__attribute__ ((cleanup))`.
donald
force-pushed
the
add-prerequisites
branch
2 times, most recently
from
eace747
to
2d9ba1f
Compare
The inlined keywordset_free_byref from keywordset.h uses keywordset_free, so link keywordset.o in, too.
Add a simple bison based expression parser and evaluator for
boolean expressions with identifiers from keywordsets.
Identifiers existing in the supplied keywordset evaluate to true,
others to false. This can be combined with boolean arithmetic w
using '!', '&', '|', '(' and ')'. White space is ignored.
Example valid expressions:
cuda
cuda & ssse3
amd & !desktop
theinternet | deadpool
Add a test for the expression parser.
Add field "prerequisites" to mxq_daemon which hold the prerequisites the server offers (e.g. cpu feature flags and hostconfig keywords). This field is used to publish the discovered prerequisites.
Add add column to store a prerequisites expression which the group requires from the server.
Initialize set of prerequisites available on the server from hostname, short hostname, hostconfig and cpu features. Also include a keyword "true". Store the preqrequisites as keywordset in the server structure to use for cehcking group requiresments and as as string in the daemon structure to be published to the database for documentation.
Reformulate server_is_qualified to make it (IMO) easier to understand and easier to expand.
If the users specifies a prerequisites expression with --prerequisites, parse it to verify its syntax and store it as a group attribute.
donald
force-pushed
the
add-prerequisites
branch
from
2d9ba1f
to
63d6f12
Compare
Sign in
to join this conversation on GitHub.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
mxqsub --whitelist)mxqsub --group-idSupersedes #85