diff --git a/mxq-rolling-reboot/README.md b/mxq-rolling-reboot/README.md new file mode 100644 index 0000000..a656d7b --- /dev/null +++ b/mxq-rolling-reboot/README.md @@ -0,0 +1,112 @@ +# mxrolling-reboot + +Rolling kernel reboot of the `mxqd` cluster nodes: reboots every node into a +new kernel **without killing running jobs** and **without draining the whole +cluster at once**. + +## What it does + +For each node still running the old kernel it launches, detached via +`systemd-run`, the sequence: + +1. `mxqdctl-hostconfig stop` — mxqd stops accepting new jobs and exits once its + running jobs finished. +2. wait for the `mxqd` process(es) to disappear — i.e. wait (indefinitely) for + the node to be fully drained. `mxqdctl-hostconfig stop` only *signals* mxqd + and returns immediately, so this explicit wait is what makes the reboot + safe. +3. install a one-time systemd unit that runs `mxqdctl-hostconfig start` on the + next boot and then removes itself again (see below). +4. `mxgrub