[RFC] Add mx-networkd for reliable network device detection with systemd-networkd #566
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…tion
mxnetctl, mxvlanctl and the two oneshot units around them do three things
that systemd-networkd and udev can do themselves: rename interfaces by MAC
address, put the primary address and the default route on one of them, and
create the VLANs of the local host.
mx-networkd is a systemd generator that translates the existing
configuration files into .link, .netdev and .network files. Generators run
before any unit starts, including systemd-udevd, so the .link files are in
place before the first device is coldplugged. Everything is written to
/run/systemd/network/, which keeps /etc/systemd/network/ empty as required
and leaves /etc/local/mxnet, /etc/local/mxhost.conf and /etc/mxvlans as the
single source of truth, unchanged in format.
Three subcommands: `generate` (what the generator runs), `check` (parse and
report, for the master before pushing /etc/mxvlans) and `learn` (assign
netXX names to unknown MACs on a first boot, the one thing .link files
cannot do).
Notable details:
- systemd-networkd applies only the first matching .network file to an
interface, so all settings of one interface go into one file even when
they come from different input files.
- The output is deterministic, so re-running the generator on every
daemon-reload rewrites nothing.
- A broken line is skipped and logged rather than fatal: at boot a
partially configured network beats no network. `check` turns the same
problems into a non-zero exit status where a human can still fix them.
- The prefix length, gateway and broadcast address that were hard coded
in network.service are defaults now and can be overridden per host with
MX_PREFIXLEN, MX_GATEWAY and MX_BROADCAST.
No dependencies beyond the standard library. Nothing is enabled by this
commit; see mx-networkd/README.md for the per machine migration.
`cargo test` passes (21 tests). Tested with follow-on patches on
*dose*, *sigusr2*, *fluffybutt*.
Assisted-by: Claude Fable 5 <noreply@anthropic.com> (thanks to @boris)
Resolves: #556
Link: pmenzel/oc-exchange#3
Link: pmenzel#1
`make` or `make mx-networkd/target/release/mx-networkd` builds the binary with cargo, `make install` puts it into `/usr/sbin/mx-networkd` and installs the same binary as the generator `/usr/lib/systemd/system-generators/mx-networkd-generator` (it recognises being called under that name). The install is skipped when the binary was not built, so `install.sh` keeps working standalone on a machine without a Rust toolchain. Installing the generator does not change any behaviour by itself: it only writes to `/run/systemd/network/`, which nothing reads unless `systemd-networkd` is enabled on that machine. Assisted-by: Claude Fable 5 <noreply@anthropic.com> (thanks to @boris)
network.service removes the primary address in ExecStop, so 'systemctl disable --now network.service' over SSH drops the connection and locks the admin out of the machine. Flip the units without --now and let the reboot do the swap: the old units run until shutdown, the generator and the .link files take over at the next coldplug, and the two mechanisms never overlap. The rollback gets the same treatment. Assisted-by: Claude Fable 5 <noreply@anthropic.com> (thanks to @boris)
…nerator Make the hostconfig tag mx-network-generator the single per-machine switch between the old and the new network setup (hyphens are fine in tag names: hostconfig matches tag terms with [a-z][a-z0-9$_-]*): * network.service and mxvlan.service get ConditionPathExists=!/node/tags/mx-network-generator, so they keep running everywhere until a machine carries the tag. * systemd-networkd.service gets a drop-in with ConditionPathExists=/node/tags/mx-network-generator, so it never fights the old units on a machine that has not been switched over. * mx-networkd generate writes nothing while the tag is absent and prunes files it wrote earlier, so a rollback needs nothing beyond removing the tag. /node/tags/ persists across boots and generators run before startup-tags.service, so the previous boot's populate-node result is what gates the generator. --dry-run ignores the tag so the preview always works; check now reports the tag state. Assisted-by: Claude Fable 5 <noreply@anthropic.com> (thanks to @boris)
Add mx-networkd/migrate-to-mx-networkd.sh, installed as /usr/sbin/migrate-to-mx-networkd. It prepares the reboot-based switchover without touching the running network, so it is safe over SSH and idempotent: * checks that binary, generator, drop-in and conditioned units are installed and that /etc/local/mxhost.conf and /etc/local/mxnet exist * validates the configuration with mx-networkd check and shows the generate --dry-run preview for comparison with the running state * verifies the mx-network-generator tag is set in /etc/hostconfig (and prints the exact line to add on the distmaster if not - /node/tags/ is rebuilt from /etc/hostconfig at every boot, so a locally touched tag file would not survive) * runs hostconfig --populate-node and enables systemd-networkd.service for the next boot only - nothing is started, stopped or reloaded * prints a summary and reboot-based rollback instructions Rewrite the README migration section around the tag and the script. Assisted-by: Claude Fable 5 <noreply@anthropic.com> (thanks to @boris)
hostconfig does not support hyphens/minus in tag names after all:
== checking hostconfig tag 'mx-network-generator'
Invalid [] range "x-n" in regex; marked by <-- HERE in m/^[a-z][a-z0-9mx-n <-- HERE etwork-generator-]*$/ at /usr/sbin/hostconfig line 43.
Leave the the hyphens in the drop-in name, as it’s easier to read. The
tag is going to be removed after the migration finished.
Sign in
to join this conversation on GitHub.
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Request for comment.
This is to solve issue #556.
Tested on dose, fluffybutt and sigusr2. Still needs testing on devices with lots of VLANs.
systemd-networkd uses around 5 MB of RAM, and is otherwise not consuming any resources.
CONFIG_DUMMY=yResolves: #556