pdist: run the post-update scripts in the client and run make-automaps
#575
+47
−14
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Paul Menzel (pmenzel)
force-pushed
the
pdist-run-post-update-scripts
branch
from
579c705 to
946b43e
Compare
Collaborator
|
Nice! |
Paul Menzel (pmenzel)
force-pushed
the
pdist-run-post-update-scripts
branch
from
946b43e to
ff280e8
Compare
Donald Buczek (donald)
approved these changes
Sep 1, 2026
Scripts to be run after an update live in a directory on the client and
are called by a wrapper, which the master has to name on every push:
ExecStart=… pdist push $host … --post-command /usr/libexec/post-pdist/post-pdist
$ cat /usr/libexec/post-pdist/post-pdist
set -e
for f in /usr/libexec/post-pdist/[0-9]*; do
"$f"
done
The client knows on its own when an update has finished, so let it walk
the directory itself. That removes the wrapper, the argument which has
to be threaded from the unit file through ssh to the client, and the
[0-9] glob, which only exists to keep the wrapper from calling itself.
A push without `--post-command` now runs the scripts as well, which is
what one wants for an ad-hoc `pdist push somehost`.
The scripts run before `--post-command`, which is kept for anything a
single push wants to add.
The first failing script stops the run, as `set -e` did in the wrapper,
so a script may rely on its predecessors having run. Unlike the wrapper,
the failure now also fails the update: the client dies, ssh exits
non-zero and `push_single()` picks that up. A post-update script which
fixes up the installed system is not something to fail silently.
Names are restricted to digits, letters, '-', '.' and '_' so that an
editor backup left in the directory is not executed – the old glob would
have run `0001-foo~`. A script which is not executable stays an error
and is not silently skipped.
Tested with a copy of the script pointed at a scratch directory:
case 1: directory missing → no output, exit 0
case 2: 0001, 0002, 0010 → run in that order, exit 0
README, 0001-first.orig~, subdirectory 0005-a-directory
→ ignored
case 3: 0002 exits 3
.../0002-second: exit status 3 → 0010 not run, exit 3
case 4: 0002 kills itself with SIGTERM
.../0002-second: killed by signal 15 → 0010 not run, exit 255
case 5: 0002 not executable
.../0002-second: Permission denied → 0010 not run, exit 13
case 6: --quiet → per-script line gone, failure still reported
The directory is empty until the scripts move there, so this commit on
its own is a no-op and the wrapper keeps doing the work.
Assisted-by: claude-opus-5 (Claude Code)
The client walks `/usr/libexec/pdist/post-update.d` now, so move the
scripts there and drop the wrapper:
/usr/libexec/post-pdist/post-pdist gone
/usr/libexec/post-pdist/0001-… → /usr/libexec/pdist/post-update.d/0001-…
/usr/libexec/post-pdist/0002-… → /usr/libexec/pdist/post-update.d/0002-…
They sit next to the pdist client which runs them, under the name of the
program owning the directory rather than under the name of the wrapper
which used to. `.d` says a script can be dropped in without editing
anything else, which `install.sh` has done since commit 692fa93.
`/usr/libexec`, not `/etc`: these are executables shipped with the
distribution and not host configuration.
`pdistd@.service` no longer has to name the wrapper. The old directory
disappears from the clients on the next push, because pdist deletes
what the master does not have.
Installed into a `DESTDIR` to check the layout:
$ DESTDIR=/scratch/... ./install.sh
$ find $DESTDIR -path '*post-update*'
…/usr/libexec/pdist/post-update.d
…/usr/libexec/pdist/post-update.d/0001-convert-default-target
…/usr/libexec/pdist/post-update.d/0002-reload-systemd-260.2
$ ls -l .../usr/libexec/pdist/post-update.d/
-rwxr-xr-x 1 … 0001-convert-default-target
-rwxr-xr-x 1 … 0002-reload-systemd-260.2
Assisted-by: claude-opus-5 (Claude Code)
A user created while a host is switched off cannot log in on that host
after it comes back, although `/etc/passwd` has the entry:
$ sudo journalctl -b -u make-automaps.service
make-automaps.service skipped, unmet condition check ConditionPathExists=!/etc/automount/auto.scratch
pdist distributes `/etc/amd`, but not the autofs maps generated from it:
`/etc/automount` is excluded in `/root/Distfile` and built on the client
by `make-automaps`. The only thing calling it on a running system is
`make-automaps.service`, and its
`ConditionPathExists=!/etc/automount/auto.scratch` holds exactly once,
on a host which has never built the maps. So the host receives the new
`/etc/passwd` entry with the next update, while
`/etc/automount/auto.home` keeps the state of its last boot as an
installed system, and the home directory of the new user cannot be
mounted.
`clusterd push --post make-automaps` covers the case where the host is
up when the user is created; the host which was down is picked up by
whatever update comes later, and that is the pdist update. So rebuild
the maps from there.
Verified that the client picks the script up, with make-automaps
replaced by a stub so nothing outside the scratch directory is touched:
…: executing …/post-update.d/0001-convert-default-target
…: executing …/post-update.d/0002-reload-systemd-260.2
…: executing …/post-update.d/0005-make-automaps
make-automaps stub: rebuilding maps
exit=0
and that a failing `make-automaps` fails the update:
…/post-update.d/0005-make-automaps: exit status 2
exit=2
Not tested against a real distmaster and a real client.
Resolves: #574
Assisted-by: claude-opus-5 (Claude Code)
Both update paths call make-automaps right after asking the distmaster
for an update:
-- bash -c 'netcat $(distmaster) 237 ; make-automaps'
netcat $(distmaster) 237
make-automaps
That netcat reaches pdistd.socket on the distmaster, which pushes back
to this host, and the push runs 0005-make-automaps on the client now.
So the maps are already rebuilt when netcat returns.
Assisted-by: claude-opus-5 (Claude Code)
Paul Menzel (pmenzel)
force-pushed
the
pdist-run-post-update-scripts
branch
from
ff280e8 to
2ebb8ae
Compare
The name filter accepts any '.', so leftovers on the distmaster are
distributed and run after the original:
0005-make-automaps.orig run
0002-reload-systemd-260.2.bak run
0002-x.dpkg-old run
Debian's run-parts rejects '.' entirely, but 0002-reload-systemd-260.2
carries a version number. So accept a '.' only when digits follow it:
0001-convert-default-target run
0002-reload-systemd-260.2 run
0005-make-automaps run
0005-make-automaps.orig skip
0001-foo.bak skip
0001-foo~ skip
0002-x.dpkg-old skip
0002-reload-systemd-260.2.bak skip
0002-reload-systemd-260.2~ skip
.0001, README skip
Assisted-by: claude-opus-5 (Claude Code)
run_post_update_scripts() logs each script unless `--quiet` is given,
while `--post-command` right below it logs only with `--verbose`.
`--quiet` is documented as "do not log file operations", and running a
script is not one. So log it with `--verbose`, as `--post-command` does,
and take the hostname from prog_client() instead of looking it up again.
Tested with the sub extracted and pointed at a scratch directory:
$verbose = 0:
ran 0001-a
ran 0002-b-260.2
$verbose = 1:
testhost: executing …/0001-a
ran 0001-a
testhost: executing …/0002-b-260.2
ran 0002-b-260.2
Assisted-by: claude-opus-5 (Claude Code)
0005-make-automaps runs make-automaps on every push to every host. For
`pdist push --files=/usr/local/package --para=100 ALL` that is a hundred
runs, each forking `hostconfig --host X nfs40` per NFS server and sending
HUP to the automounter, although neither /etc/amd nor anything else
make-automaps reads has changed.
So rebuild only if
- the maps were not built since boot. That keeps what pdist-bootcheck
did before: the first update after a boot rebuilds them, which also
picks up local changes such as new disks under /amd/$hostname, or
- an input was changed after the last build: /etc/amd (and so the
removal of a map), /etc/amd/amd.*, /etc/mxpkg, and /etc/hostconfig,
which hostconfig reads for the nfs40 tag.
The time of the last build is the mtime of /etc/automount/auto.pkg,
which make-automaps writes last, so a run that dies in convert_map()
leaves it old and the next push tries again. The inputs are compared by
ctime: pdist keeps the master's mtime on the files it installs, so an
amd.home edited on the master before the last local build would look
older than the maps.
Tested with the paths pointed at a scratch directory and make-automaps
replaced by `echo REBUILD`:
a) auto.pkg missing REBUILD
b) maps older than boot REBUILD
c) built after boot, inputs unchanged skip
amd.home mtime 2020-01-01 00:00:00 ctime 2026-10-15 06:33:26
d) amd.home installed with old mtime REBUILD
e) built after removal of amd.home skip
f) amd.home removed after build (dir ctime) REBUILD
Assisted-by: claude-opus-5 (Claude Code)
Collaborator
|
I forgot why it is better if the scripts are run implicitly and not explicitly and only when ordered on the command line. |
Collaborator
|
Why did we rename the libexec directory? |
Contributor
Author
From the commit message: They sit next to the pdist client which runs them, under the name of the |
Contributor
Author
Good question. The original motivation is the resolved issue #574 to run |
Collaborator
|
If we don't have a reason to change, we shouldn't. |
Contributor
Author
Do you mean both things you raised or just one of the two? |
Collaborator
I though the first thing ("They sit next to the pdist client which runs them, under the name of the program owning the directory rather than under the name of the wrapper which used to") is a result of the second thing (use some pdist built-in directory special case instead of an explicit --post command) ? |
Contributor
Author
|
We want to run |
Collaborator
That doesn't convince me. If admins want to manually mirror what is done automatically, they can us Sometimes pdist is used for other things. I do it all the time for simple updates of life systems ( I'm on the explicit is better than implicit side here. |
Sign in
to join this conversation on GitHub.
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Run the scripts in
/usr/libexec/pdist/post-update.dfrom the pdist client after every update, not via--post-command. A failing script now fails the push; backup files are skipped. Add0005-make-automaps, which rebuilds the maps when an input changed or after boot, so hosts that were down pick up new users.pdist-bootcheckno longer calls make-automaps.Resolves: #574