Skip to content

misc_systemd_units: Add udev-settled.target #31

Merged
merged 1 commit into from
Mar 14, 2018

Conversation

donald
Copy link
Collaborator

@donald donald commented Mar 14, 2018

After the upgrade of systemd from version 27 to version 238, we had the
problem, that mxnetctl start from network.service ("Network
Connectivity") was run before all network interfaces were available and
thus didn't rename them.

network.service is configured with DefaultDependencies=no to prevent
problems with nfs umounts during shutdown. But even when
network.services was returned to DefaultDependencies=yes and waited
for basic.target ("Basic System") the problem persisted:

root@nsa:/etc/systemd/system (master)#  journalctl --system --boot=-0|egrep 'mxnetctl|Gigabit|renamed|Basic System|Network Connectivity'
Mar 14 08:36:19 nsa.molgen.mpg.de kernel: bnx2x: QLogic 5771x/578xx 10/20-Gigabit Ethernet Driver bnx2x 1.712.30-0 (2014/02/10)
Mar 14 08:36:19 nsa.molgen.mpg.de kernel: igb: Intel(R) Gigabit Ethernet Network Driver - version 5.4.0-k
Mar 14 08:36:19 nsa.molgen.mpg.de kernel: igb 0000:05:00.0: Intel(R) Gigabit Ethernet Network Connection
Mar 14 08:36:19 nsa.molgen.mpg.de kernel: igb 0000:06:00.0: Intel(R) Gigabit Ethernet Network Connection
Mar 14 08:36:19 nsa.molgen.mpg.de kernel: igbvf: Intel(R) Gigabit Virtual Function Network Driver - version 2.4.0-k
Mar 14 08:36:19 nsa.molgen.mpg.de kernel: ixgbe: Intel(R) 10 Gigabit PCI Express Network Driver - version 5.1.0-k
Mar 14 08:36:19 nsa.molgen.mpg.de kernel: ixgbe 0000:02:00.0: Intel(R) 10 Gigabit Network Connection
Mar 14 08:36:19 nsa.molgen.mpg.de systemd[1]: Reached target Basic System.
Mar 14 08:36:19 nsa.molgen.mpg.de systemd[1]: Starting Network Connectivity...
Mar 14 08:36:20 nsa.molgen.mpg.de kernel: ixgbe 0000:02:00.1: Intel(R) 10 Gigabit Network Connection
Mar 14 08:36:20 nsa.molgen.mpg.de mxnetctl[412]: rename eth0 to net00
Mar 14 08:36:20 nsa.molgen.mpg.de kernel: igb 0000:05:00.0 net00: renamed from eth0
Mar 14 08:36:20 nsa.molgen.mpg.de mxnetctl[412]: rename eth1 to net01
Mar 14 08:36:20 nsa.molgen.mpg.de kernel: igb 0000:06:00.0 net01: renamed from eth1
Mar 14 08:36:20 nsa.molgen.mpg.de mxnetctl[412]: rename eth2 to net02
Mar 14 08:36:20 nsa.molgen.mpg.de kernel: ixgbe 0000:02:00.0 net02: renamed from eth2
Mar 14 08:36:20 nsa.molgen.mpg.de mxnetctl[412]: rename eth3 to net03
Mar 14 08:36:20 nsa.molgen.mpg.de kernel: ixgbe 0000:02:00.1 net03: renamed from eth3
Mar 14 08:36:20 nsa.molgen.mpg.de systemd[1]: Started Network Connectivity.
Mar 14 08:36:20 nsa.molgen.mpg.de kernel: ixgbe 0000:03:00.0: Intel(R) 10 Gigabit Network Connection
Mar 14 08:36:20 nsa.molgen.mpg.de kernel: ixgbe 0000:03:00.1: Intel(R) 10 Gigabit Network Connection
Mar 14 08:36:20 nsa.molgen.mpg.de kernel: ixgbe 0000:81:00.0: Intel(R) 10 Gigabit Network Connection
Mar 14 08:36:20 nsa.molgen.mpg.de kernel: ixgbe 0000:81:00.1: Intel(R) 10 Gigabit Network Connection
Mar 14 08:36:20 nsa.molgen.mpg.de kernel: ixgbe 0000:84:00.0: Intel(R) 10 Gigabit Network Connection
Mar 14 08:36:21 nsa.molgen.mpg.de kernel: ixgbe 0000:84:00.1: Intel(R) 10 Gigabit Network Connection

Similar problems are possible with other startup services (e.g.
mxmount) as they are expecting the local devices to be available.

So we want to start and wait for systemd-udev-settle.service ("udev Wait for
Complete Device Initialization") before our services. We do this by
adding our own target with Before=basic.target and
After=systemd-udev-settle.service. The normal services will wait for
basic.target per default. Because network.service has
DefaultDependencies=no, we add the order dependency explicitly. Because
the unit file of network.service is system specific and not distributed
by the distmaster, we add this order dependency in the unit file of the
new target.

Add new target unit udev-settled.target.

@pmenzel
Copy link
Contributor

pmenzel commented Mar 14, 2018

In the commit and merge/pull request message the version should be changed from 229 to 238.

The rest looks fine.

Somebody also added systemd-network-after-root-writable-229-0.bee in the past by the way.

After the upgrade of systemd from version 27 to version 238, we had the
problem, that `mxnetctl start` from network.service ("Network
Connectivity") was run before all network interfaces were available and
thus didn't rename them.

network.service is configured with `DefaultDependencies=no` to prevent
problems with nfs umounts during shutdown. But even when
network.services was returned to `DefaultDependencies=yes` and waited
for basic.target ("Basic System") the problem persisted:

    root@nsa:/etc/systemd/system (master)#  journalctl --system --boot=-0|egrep 'mxnetctl|Gigabit|renamed|Basic System|Network Connectivity'
    Mar 14 08:36:19 nsa.molgen.mpg.de kernel: bnx2x: QLogic 5771x/578xx 10/20-Gigabit Ethernet Driver bnx2x 1.712.30-0 (2014/02/10)
    Mar 14 08:36:19 nsa.molgen.mpg.de kernel: igb: Intel(R) Gigabit Ethernet Network Driver - version 5.4.0-k
    Mar 14 08:36:19 nsa.molgen.mpg.de kernel: igb 0000:05:00.0: Intel(R) Gigabit Ethernet Network Connection
    Mar 14 08:36:19 nsa.molgen.mpg.de kernel: igb 0000:06:00.0: Intel(R) Gigabit Ethernet Network Connection
    Mar 14 08:36:19 nsa.molgen.mpg.de kernel: igbvf: Intel(R) Gigabit Virtual Function Network Driver - version 2.4.0-k
    Mar 14 08:36:19 nsa.molgen.mpg.de kernel: ixgbe: Intel(R) 10 Gigabit PCI Express Network Driver - version 5.1.0-k
    Mar 14 08:36:19 nsa.molgen.mpg.de kernel: ixgbe 0000:02:00.0: Intel(R) 10 Gigabit Network Connection
    Mar 14 08:36:19 nsa.molgen.mpg.de systemd[1]: Reached target Basic System.
    Mar 14 08:36:19 nsa.molgen.mpg.de systemd[1]: Starting Network Connectivity...
    Mar 14 08:36:20 nsa.molgen.mpg.de kernel: ixgbe 0000:02:00.1: Intel(R) 10 Gigabit Network Connection
    Mar 14 08:36:20 nsa.molgen.mpg.de mxnetctl[412]: rename eth0 to net00
    Mar 14 08:36:20 nsa.molgen.mpg.de kernel: igb 0000:05:00.0 net00: renamed from eth0
    Mar 14 08:36:20 nsa.molgen.mpg.de mxnetctl[412]: rename eth1 to net01
    Mar 14 08:36:20 nsa.molgen.mpg.de kernel: igb 0000:06:00.0 net01: renamed from eth1
    Mar 14 08:36:20 nsa.molgen.mpg.de mxnetctl[412]: rename eth2 to net02
    Mar 14 08:36:20 nsa.molgen.mpg.de kernel: ixgbe 0000:02:00.0 net02: renamed from eth2
    Mar 14 08:36:20 nsa.molgen.mpg.de mxnetctl[412]: rename eth3 to net03
    Mar 14 08:36:20 nsa.molgen.mpg.de kernel: ixgbe 0000:02:00.1 net03: renamed from eth3
    Mar 14 08:36:20 nsa.molgen.mpg.de systemd[1]: Started Network Connectivity.
    Mar 14 08:36:20 nsa.molgen.mpg.de kernel: ixgbe 0000:03:00.0: Intel(R) 10 Gigabit Network Connection
    Mar 14 08:36:20 nsa.molgen.mpg.de kernel: ixgbe 0000:03:00.1: Intel(R) 10 Gigabit Network Connection
    Mar 14 08:36:20 nsa.molgen.mpg.de kernel: ixgbe 0000:81:00.0: Intel(R) 10 Gigabit Network Connection
    Mar 14 08:36:20 nsa.molgen.mpg.de kernel: ixgbe 0000:81:00.1: Intel(R) 10 Gigabit Network Connection
    Mar 14 08:36:20 nsa.molgen.mpg.de kernel: ixgbe 0000:84:00.0: Intel(R) 10 Gigabit Network Connection
    Mar 14 08:36:21 nsa.molgen.mpg.de kernel: ixgbe 0000:84:00.1: Intel(R) 10 Gigabit Network Connection

Similar problems are possible with other startup services (e.g.
mxmount) as they are expecting the local devices to be available.

So we want to start and wait for systemd-udev-settle.service ("udev Wait for
Complete Device Initialization") before our services. We do this by
adding our own target with `Before=basic.target` and
`After=systemd-udev-settle.service`. The normal services will wait for
basic.target per default. Because network.service has
`DefaultDependencies=no`, we add the order dependency explicitly. Because
the unit file of network.service is system specific and not distributed
by the distmaster, we add this order dependency in the unit file of the
new target.

Add new target unit udev-settled.target.
@donald
Copy link
Collaborator Author

donald commented Mar 14, 2018

I fixed the message.

Good you found systemd-network-after-root-writable-229-0.bee It did arrange network.service after udev-settle.service on systems with the old systemd, but it failed to do so on systems with the new systemd, because on systems with the new systemd, udev-settle.service is never started.

Only on the old systems, basic.target had an own 'Wants' dependency for udev-settle.service. This was provided by udev-164-2.x86_64 as a symlink /lib/systemd/system/basic.target.wants/udev-settle.service. So at least now we know where the difference originated.

I'll check whether remount-rootfs.service is still needed. If so, I might merge that into this unit, too.

@donald
Copy link
Collaborator Author

donald commented Mar 14, 2018

Hmm. remount-rootfs.service now is systemd-remount-fs.service. It seems to run long before basic.target is reached, although I can't figure out, what dependency path enforces this. Ignore for now until problem manifests.

@donald donald merged commit 90e2957 into master Mar 14, 2018
@donald donald deleted the add-udev-settled.target branch March 14, 2018 15:36
Sign in to join this conversation on GitHub.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants