-
Notifications
You must be signed in to change notification settings - Fork 0
RFC: bee-mxqsub: Run only install and packaging under fakeroot #3491
base: master
Are you sure you want to change the base?
Conversation
bee-mxqsub runs all of beesh under fakeroot, so the stat, chmod and
chown calls of every configure and build process go through the one
faked daemon. Large packages pay for this most: the Firefox build on a
full 256-processor node was killed while packaging,
14:38.60 E make[2]: *** Waiting for unfinished jobs....
/home/pmenzel/bee-files/scripts/bee-mxqsub: line 1: 222979 CPU time limit exceeded (core dumped) ${sandbox[@]+"${sandbox[@]}"} fakeroot beesh "${beesh_args[@]}" "${beefile}"
and without the preload `mach build` took 11:34 instead of 17:37.
bee’s beesh gains `--fakeroot` (branch beesh-add-fakeroot-option),
which starts faked right before the install step and stops it after
packaging. Use it when the installed beesh lists it in `--help`, and
keep calling all of beesh under fakeroot otherwise, so jobs on nodes
with an older bee behave as before.
beesh then reads its configuration with the real UID, for which beerc
picks the repository below ~/.local and the user’s package database
and cache. Export BEE_REPOSITORY_PREFIX, BEE_METADIR and BEE_CACHEDIR
from `fakeroot bee --print-config`, which the job reads anyway, so the
packages still go to /src/mariux/beeroot unless --repository says
otherwise. The overlay sandbox runs beesh as UID 0 and has its own
cache, so it needs neither.
Tested by running the job phase locally with a package whose Makefile
installs with `-o root`, mode 4755 and `-o 1234`: with the new beesh,
the build step ran with the real UID and the package has the same owners
and modes as with the installed beesh, which takes the fallback.
Assisted-by: claude-opus-5.5 (Claude Code)
|
This comment was generated by an LLM (Claude Code) on behalf of Paul Menzel (@pmenzel). Result of the MXQ check: packages built with
Details of the test are in the next comment. |
|
This comment was generated by an LLM (Claude Code) on behalf of Paul Menzel (@pmenzel). How the change was tested1. Which packages to check Packages whose CONTENT has a setuid/setgid/sticky file or a file not owned by root, among the 1076 packages installed on the submit host: cd /usr/share/bee
for p in */; do
n=$(grep -cE 'access=0?[2-7][0-7]{3}:|:uid=[1-9]|:gid=[1-9]' "$p/CONTENT")
[ "$n" -gt 0 ] && echo "$n ${p%/}"
doneThat gives 22 packages. Left out were 2. Can configure or build depend on running as root? With
3. Builds on MXQ One job per variant on a full node, both with the same # old: installed bee, all of beesh under fakeroot (fallback)
scripts/bee-mxqsub -t 4h --no-overlay -k -B -c -B --no-archive-build \
-E PATH=/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin \
--repository ~/bee-fakeroot-test/repo-old <recipes>
# new: bee from bee#76 installed with PREFIX=~/bee-fakeroot-test/prefix
scripts/bee-mxqsub -t 2h --no-overlay -k -B -c -B --no-archive-build \
-E PATH=$HOME/bee-fakeroot-test/prefix/bin:/usr/local/bin:… \
--repository ~/bee-fakeroot-test/repo-new <recipes>The job logs say Three recipes fail on master in either mode, so they were left out of the comparison:
4. Comparison The CONTENT files are compared without size, mtime and md5, so only type, mode, owner, group, nlink and path are left: norm() {
tar xOjf "$1" CONTENT |
sed -E 's/:size=[0-9]*//; s/:mtime=[0-9]*//; s/:md5=[0-9a-f]*//' | sort
}
for n in repo-new/packages/*.bee.tar.bz2; do
diff <(norm "repo-old/packages/${n##*/}") <(norm "$n")
doneResult: 8 packages are identical. In the other 9, the new package only has additional root-owned 0755 directory entries, such as 5. Local tests before that
Not covered: packages in beeroot that are not installed on the submit host; build speed. These packages are small; the Firefox timing in the commit message (17:37 vs. 11:34) is the reference. |
Use
beesh --fakeroot(mariux64/bee#76) when available, falling back tofakeroot beesh. MXQ check: special owners and modes in 17 packages unchanged.🤖 Generated with Claude Code