Skip to content

Update systemd from 27 to 238 #621

Merged
merged 76 commits into from Mar 12, 2018
Merged

Update systemd from 27 to 238 #621

merged 76 commits into from Mar 12, 2018

Conversation

pmenzel
Copy link
Collaborator

@pmenzel pmenzel commented Feb 17, 2018

TODO

  • Find solution for new binary location /lib/systemd/systemd (Rebuild systemd v27 with correct symbolic links for smooth upgrade?)
  • Decide on how to handle user session after logout (Screen, …)
  • Decide on updating the package fhs
  • Do not mount /tmp as tmpfs
  • Decide on updating GDM because of /var/run and systemd-logind (better follow-up)

Finally update systemd from the almost seven year old version 27 to the current version 238. Several packages need to be updated, or rebuilt.

Use the commands below to perform the update. Note, /var/run has to be made a symbolic link to /run. Update /etc/pam.d/system-auth, and remove the ConsoleKit line, and make pam_systemd.so optional and remove the parameters.

$ sudo scripts/update-systemd-from-27-to-238.sh
  1. UDisks is only updated to version 2.1.8, as the current version 2.7.6 has storaged dependencies.
  2. Qt 5.10.1 is built without QtWebengine, as it doesn’t built.

Tested on rabammel.

@donald
Copy link
Collaborator

donald commented Feb 17, 2018

Oh my gosh, you've made it. Great! Looking forward to test and merge this!

@pmenzel
Copy link
Collaborator Author

pmenzel commented Feb 17, 2018

UDisks is now updated (rebuilt) from 1.0.2 to 2.1.8. Updating the package shows the messages below.

$ sudo bee update udisks
installing /src/mariux/beeroot/packages/udisks-2.1.8-0.x86_64.bee.tar.bz2 ..
removing udisks-1.0.2-0.x86_64 ..
rmdir: failed to remove ‘/var/lib/udisks’: Directory not empty
rmdir: failed to remove ‘/usr/lib/polkit-1’: Directory not empty

@pmenzel
Copy link
Collaborator Author

pmenzel commented Feb 17, 2018

Reported VLC 3.0.0 build error upstream.

@donald
Copy link
Collaborator

donald commented Feb 17, 2018

Best use full version for bee update and bee install instruction. Latest may not be yours.

@donald
Copy link
Collaborator

donald commented Feb 17, 2018

Why do we need "/var/run" to be a symlink? Systemds var-run.mount unit bind-mounts /run to /var/run, so /run and /var/run are already the same.

@pmenzel
Copy link
Collaborator Author

pmenzel commented Feb 17, 2018 via email

@pmenzel
Copy link
Collaborator Author

pmenzel commented Feb 18, 2018

I couldn’t clarify until now due to the Vodafone Internet outage.

There are several packages, like D-Bus, creating directories under /var/run/ during build/install time. I believe, that newer systemd notices this, and doesn’t bind mount the directory, if files are in there. That’s why it should be a symbolic link, cf. systemd issue #8197.

@pmenzel
Copy link
Collaborator Author

pmenzel commented Feb 18, 2018

$ sudo bee query /var/run
consolekit-0.4.4-0.x86_64
  /var/run/ConsoleKit
dbus-1.4.0-2.x86_64
  /var/run/dbus
fhs-1-1.x86_64
  /var/run
gdm-3.4.1-14.x86_64
  /var/run/gdm
  /var/run/gdm/greeter
qemu-2.6.2-0.x86_64
  /usr/var/run
sudo-1.8.20_p2-0.x86_64
  /var/run/sudo
tsm-molgen-1.0-0.x86_64
  /etc/TIVGUID///var/run/tsm/TIVGUID
  /var/run/tsm
udisks-1.0.2-0.x86_64
  /var/run/udisks

@pmenzel pmenzel force-pushed the update-systemd-from-27-to-237 branch 2 times, most recently from 09bd419 to 4e16688 Compare February 19, 2018 08:15
@donald
Copy link
Collaborator

donald commented Feb 19, 2018

Tried on sigusr2 and bricked it :-)

@pmenzel
Copy link
Collaborator Author

pmenzel commented Feb 19, 2018 via email

@donald
Copy link
Collaborator

donald commented Feb 19, 2018

The rename of fhs-1-1.bee to fhs.be0 is in 75d3acc "fhs: Create directories and symlinks according to latest FHS" not in the previous commit ee07e0d "fhs: Convert to versionless bee file" where it belongs.

@donald
Copy link
Collaborator

donald commented Feb 19, 2018

if /var/run is a mounted tmpfs, bee update fsh will not make it into a symlink. I still don't understand this whole issue. On the current systems /var/run mounts the same tmpfs ans /run. How can the content of /run and /var/run ever become different?

@pmenzel pmenzel force-pushed the update-systemd-from-27-to-237 branch from 4e16688 to 3ad7c39 Compare February 19, 2018 09:55
@pmenzel
Copy link
Collaborator Author

pmenzel commented Feb 19, 2018 via email

@donald
Copy link
Collaborator

donald commented Feb 19, 2018

sigusr2 could boot with added init=/lib/systemd/systemd

@pmenzel
Copy link
Collaborator Author

pmenzel commented Feb 19, 2018 via email

@pmenzel
Copy link
Collaborator Author

pmenzel commented Feb 19, 2018 via email

@donald
Copy link
Collaborator

donald commented Feb 19, 2018

How does the boot work now? initramfs/init defaults to /sbin/init and there is no override in grub.cfg. But /sbin/init is from sysvinit-2.88dsf-2.x86_64 ?

@donald
Copy link
Collaborator

donald commented Feb 19, 2018

Newer systemd doesn’t seem to bind mount it, when there are already files in /var/run, which happens for example, if you install the system with the MarIuX installer.

Looks to me, that it never mounts it (there no longer is a var-run.mount unit) but assumes the symlink.

@pmenzel
Copy link
Collaborator Author

pmenzel commented Feb 19, 2018 via email

@donald
Copy link
Collaborator

donald commented Feb 19, 2018

Stupid me. Its from the kernel config CONFIG_CMDLINE="init=/bin/systemd". Never liked that to be configured into the kernel.

@pmenzel
Copy link
Collaborator Author

pmenzel commented Feb 19, 2018 via email

@pmenzel pmenzel force-pushed the update-systemd-from-27-to-237 branch from 3ad7c39 to e096c53 Compare February 19, 2018 10:58
@donald
Copy link
Collaborator

donald commented Feb 19, 2018

CONFIG_CMDLINE="init=/bin/systemd" should be removed. ( Quote menuconfig: "systems with fully functional boot loaders (i.e. non-embedded) should leave this option set to 'N' "). But currently we have a lot of kernels running with that config. So we (1) either need to provide /bin/systemd or (2) create some logic in initramfs/init if INIT=/bin/systemd and it doesn't exist. I'd vote for option 2, because when the config is removed from the kernel, we need to provide the default in initramfs anyway. We could treat init=/bin/systemd the same as no value given, knowing that this is an obsoleted usage only.

Then there are our few leftover lilo systems which have append="init=/bin/systemd" ini /etc/lilo.conf. But these can be changed easily.

[EDIT: several times...]

@donald
Copy link
Collaborator

donald commented Feb 19, 2018

Put the update script into the repository (perhaps remove it before merge)? So I can reset a test system, pull the branch, run the script.... And if this changed (version, added or removed packages) the script would change, too.

@donald
Copy link
Collaborator

donald commented Feb 19, 2018

We could

  1. re-add the var-run.mount unit
  2. change the (hidden) directory /var/run into a (hidden) symbolic link to /run on all systems
  3. change /var/run into a symbolic link to /run from initramfs/init.

Need to exclude /var/run from dist until all systems have changed.

@pmenzel
Copy link
Collaborator Author

pmenzel commented Feb 19, 2018 via email

From the manual page systemd-resolved(8):

> /ETC/RESOLV.CONF
>        Four modes of handling /etc/resolv.conf (see resolv.conf(5))
>        are supported:
>
>        ·   systemd-resolved maintains the
>            /run/systemd/resolve/stub-resolv.conf file for
>            compatibility with traditional Linux programs. This file
>            may be symlinked from /etc/resolv.conf. This file lists the
>            127.0.0.53 DNS stub (see above) as the only DNS server. It
>            also contains a list of search domains that are in use by
>            systemd-resolved. The list of search domains is always kept
>            up-to-date. […]
Several packages still depend on these shared library versions, so add a
compatibility package to avoid rebuilding them. A lot of them are KDE
packages.
udev rules work as expected, so use them.
Currently, systemd-networkd assigns IPv6 addresses, which is unwanted.
This reverts commit 6aa5581.

Some GnuTLS files are linked against some libunbound.so.2`, so keep the
unbound package around, despite using Unbound anymore.

```
MISS libunbound.so.2                /usr/bin/danetool
MISS libunbound.so.2                /usr/bin/gnutls-cli
MISS libunbound.so.2                /usr/lib/libgnutls-dane.so.0.4.1
```
From the manual page logind.conf(5):

> KillUserProcesses=
>     Takes a boolean argument. Configures whether the processes of a
>     user should be killed when the user logs out. If true, the scope
>     unit corresponding to the session and all processes inside that
>     scope will be terminated. If false, the scope is "abandoned", see
>     systemd.scope(5), and processes are not killed. Defaults to "yes",
>     but see the options KillOnlyUsers= and KillExcludeUsers= below.
>
>     In addition to session processes, user process may run under the
>     user manager unit user@.service. Depending on the linger settings,
>     this may allow users to run processes independent of their login
>     sessions. See the description of enable-linger in loginctl(1).
>
>     Note that setting KillUserProcesses=yes will break tools like
>     screen(1) and tmux(1), unless they are moved out of the session
>     scope. See example in systemd-run(1).
[Announcement][1]:

> CHANGES WITH 238:
>
> * The MemoryAccounting= unit property now defaults to on. After
> discussions with the upstream control group maintainers we learnt
> that the negative impact of cgroup memory accounting on current
> kernels is finally relatively minimal, so that it should be safe to
> enable this by default without affecting system performance. Besides
> memory accounting only task accounting is turned on by default, all
> other forms of resource accounting (CPU, IO, IP) remain off for now,
> because it's not clear yet that their impact is small enough to move
> from opt-in to opt-out. We recommend downstreams to leave memory
> accounting on by default if kernel 4.14 or higher is are primarily
> used. On very resource constrained systems or when support for old
> kernels is a necessity, -Dmemory-accounting-default=false can be
> used to revert this change.
>
> * rpm scriptlets to update the udev hwdb and rules
> (%udev_hwdb_update, %udev_rules_update) and the journal catalog
> (%journal_catalog_update) from the upgrade scriptlets of individual
> packages now do nothing. Transfiletriggers have been added which will
> perform those updates once at the end of the transaction.
>
> Similar transfiletriggers have been added to execute any sysctl.d and
> binfmt.d rules. Thus, it should be unnecessary to provide any
> scriptlets to execute this configuration from package installation
> scripts.
>
> * systemd-sysusers gained a mode where the configuration to execute
> is specified on the command line, but this configuration is not
> executed directly, but instead it is merged with the configuration on
> disk, and the result is executed. This is useful for package
> installation scripts which want to create the user before installing
> any files on disk (in case some of those files are owned by that
> user), while still allowing local admin overrides.
>
> This functionality is exposed to rpm scriplets through a new
> %sysusers_create_package macro. Old %sysusers_create and
> %sysusers_create_inline macros are deprecated.
>
> A transfiletrigger for sysusers.d configuration is now installed,
> which means that it should be uncessary to call systemd-sysusers
> from package installation scripts, unless the package installs any
> files owned by those newly-created users, in which case
> %sysusers_create_package should be used.
>
> * Analogous change has been done for systemd-tmpfiles: it gained a
> mode where the command-line configuration is merged with the
> configuration on disk. This is exposed as the new
> %tmpfiles_create_package macro, and %tmpfiles_create is deprecated. A
> transfiletrigger is installed for tmpfiles.d, hence it should be
> unnecessary to call systemd-tmpfiles from package installation
> scripts.
>
> * sysusers.d configuration for a user may now also specify the group
> number, in addition to the user number ("u username 123:456"), or
> without the user number ("u username -:456").
>
> * Configution items for systemd-sysusers can now be specified as
> positional arguments when the new --inline switch is used.
>
> * The login shell of users created through sysusers.d may now be
> specified (previously, it was always /bin/sh for root and
> /sbin/nologin for other users).
>
> * systemd-analyze gained a new --global switch to look at global
> user configuration. It also gained a unit-paths verb to list the unit
> load paths that are compiled into systemd (which can be used with
> --systemd, --user, or --global).
>
> * udevadm trigger gained a new --settle/-w option to wait for any
> triggered events to finish (but just those, and not any other events
> which are triggered meanwhile).
>
> * The action that systemd-logind takes when the lid is closed and
> the machine is connected to external power can now be configured
> using HandleLidSwitchExternalPower= in logind.conf. Previously, this
> action was determined by HandleLidSwitch=, and, for backwards
> compatibility, is still is, if HandleLidSwitchExternalPower= is not
> explicitly set.
>
> * journalctl will periodically call sd_journal_process() to make it
> resilient against inotify queue overruns when journal files are
> rotated very quickly.
>
> * Two new functions in libsystemd — sd_bus_get_n_queued_read and
> sd_bus_get_n_queued_write — may be used to check the number of
> pending bus messages.
>
> * systemd gained a new
> org.freedesktop.systemd1.Manager.AttachProcessesToUnit dbus call
> which can be used to migrate foreign processes to scope and service
> units. The primary user for this new API is systemd itself: the
> systemd --user instance uses this call of the systemd --system
> instance to migrate processes if it itself gets the request to
> migrate processes and the kernel refuses this due to access
> restrictions.  Thanks to this "systemd-run --scope --user …" works
> again in pure cgroups v2 environments when invoked from the user
> session scope.
>
> * A new TemporaryFileSystem= setting can be used to mask out part of
> the real file system tree with tmpfs mounts. This may be combined
> with BindPaths= and BindReadOnlyPaths= to hide files or directories
> not relevant to the unit, while still allowing some paths lower in
> the tree to be accessed.
>
> ProtectHome=tmpfs may now be used to hide user home and runtime
> directories from units, in a way that is mostly equivalent to
> "TemporaryFileSystem=/home /run/user /root".
>
> * Non-service units are now started with KeyringMode=shared by
> default. This means that mount and swapon and other mount tools have
> access to keys in the main keyring.
>
> * /sys/fs/bpf is now mounted automatically.
>
> * QNX virtualization is now detected by systemd-detect-virt and may
> be used in ConditionVirtualization=.
>
> * IPAccounting= may now be enabled also for slice units.
>
> * A new -Dsplit-bin= build configuration switch may be used to
> specify whether bin and sbin directories are merged, or if they
> should be included separately in $PATH and various listings of
> executable directories. The build configuration scripts will try to
> autodetect the proper values of -Dsplit-usr= and -Dsplit-bin= based
> on build system, but distributions are encouraged to configure this
> explicitly.
>
> * A new -Dok-color= build configuration switch may be used to change
> the colour of "OK" status messages.
>
> * UPGRADE ISSUE: serialization of units using JoinsNamespaceOf= with
> PrivateNetwork=yes was buggy in previous versions of systemd. This
> means that after the upgrade and daemon-reexec, any such units must
> be restarted.
>
> * INCOMPATIBILITY: as announced in the NEWS for 237,
> systemd-tmpfiles will not exclude read-only files owned by root from
> cleanup.
>
> Contributions from: Alan Jenkins, Alexander F Rødseth, Alexis
> Jeandet, Andika Triwidada, Andrei Gherzan, Ansgar Burchardt,
> antizealot1337, Batuhan Osman Taşkaya, Beniamino Galvani, Bill
> Yodlowsky, Caio Marcelo de Oliveira Filho, CuBiC, Daniele Medri,
> Daniel Mouritzen, Daniel Rusek, Davide Cavalca, Dimitri John Ledkov,
> Douglas Christman, Evgeny Vereshchagin, Faalagorn, Filipe
> Brandenburger, Franck Bui, futpib, Giacomo Longo, Gunnar Hjalmarsson,
> Hans de Goede, Hermann Gausterer, Iago López Galeiras, Jakub Filak,
> Jan Synacek, Jason A. Donenfeld, Javier Martinez Canillas, Jérémy
> Rosen, Lennart Poettering, Lucas Werkmeister, Mao Huang, Marco
> Gulino, Michael Biebl, Michael Vogt, MilhouseVH, Neal Gompa (ニール・ゴン
> パ), Oleander Reis, Olof Mogren, Patrick Uiterwijk, Peter Hutterer,
> Peter Portante, Piotr Drąg, Robert Antoni Buj Gelonch, Sergey
> Ptashnick, Shawn Landden, Shuang Liu, Simon Fowler, SjonHortensius,
> snorreflorre, Susant Sahani, Sylvain Plantefève, Thomas Blume, Thomas
> Haller, Vito Caputo, Yu Watanabe, Zbigniew Jędrzejewski-Szmek, Марко
> М. Костић (Marko M. Kostić)

[1]: https://lists.freedesktop.org/archives/systemd-devel/2018-March/040469.html
It looks like, systemd does it for us now with the script
`tools/meson-make-symlink.sh`. No idea, why it wasn’t the case in
systemd 237.

```
[BEE] => entering mee_install_post() ..
mkdir: cannot create directory ‘/dev/shm/bee-root/systemd/systemd-238-0/image/sbin’: File exists
```
@pmenzel pmenzel force-pushed the update-systemd-from-27-to-237 branch from 73bf402 to 2a47330 Compare March 12, 2018 15:10
@pmenzel pmenzel changed the title Update systemd from 27 to 237 Update systemd from 27 to 238 Mar 12, 2018
@donald donald merged commit 4c53fe2 into master Mar 12, 2018
@donald
Copy link
Collaborator

donald commented Mar 14, 2018

Problem with mxnetctl interface renaming and possible fix: mariux64/mxtools#31

@donald
Copy link
Collaborator

donald commented Mar 14, 2018

The systemd builtin DSN resolver suddenly failed for me on my workstation. Additionally it doesn't support the CD (checking disabled) flag on DNS request. Paul reported both ( https://github.com/systemd/systemd/issues/8451 , https://github.com/systemd/systemd/issues/8450 ) but we can not continue to roll out systemd with an unstable resolver, so IMO we need to return backup to unbound.

@donald donald deleted the update-systemd-from-27-to-237 branch March 14, 2018 12:20
@pmenzel
Copy link
Collaborator Author

pmenzel commented Mar 15, 2018 via email

@donald
Copy link
Collaborator

donald commented Mar 15, 2018

I've reenabled unbound on the mx64 distmaster&nodes (enable unbound,start unbound, restore resolv.conf) and commented out the relevant lines in /root/bin/update-systemd-from-27-to-238.sh. If your change is ready we can replace the script from the one in the repository.

@donald
Copy link
Collaborator

donald commented Mar 15, 2018

Next problem: SysRq 'b' : This sysrq operation is disabled. wtf?

@donald
Copy link
Collaborator

donald commented Mar 15, 2018

Next problem: Time is jumping backwards:

Mar 05 17:07:38 sigusr2.molgen.mpg.de systemd-udevd[207]: Process '/bin/mount -o relatime /dev/sda2 /mnt/usbhd-sda2' failed with exit code 32.
Mar 05 16:07:44 sigusr2.molgen.mpg.de systemd-timesyncd[245]: Synchronized to time server 160.45.10.8:123 (zeit.fu-berlin.de).
[...]
Mar 05 16:08:00 sigusr2.molgen.mpg.de mxloadmonitor-collectd[258]: ERROR: /var/mxloadmonitor/log/load.rrd: illegal attempt to update using time 1520262480 when last update time is 1520264199 (minimum one second step)

donald added a commit that referenced this pull request Apr 11, 2018
…to-237"

This reverts commit 4c53fe2, reversing
changes made to 7ebbbac.
Sign in to join this conversation on GitHub.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants