smoke-tests: Share code in smoketest.py #3489
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The five smoke tests carry copies of the same helpers: report() and the
PASS/FAIL summary in all of them, run() and last_line() in the llvm and
bare-metal Arm tests, the --prefix unpack and overlay re-exec in those
two and the Mesa test, start_xvfb() in the Mesa and WebKitGTK tests, and
the vm.overcommit_memory check in the llvm and WebKitGTK tests. The
copies already drifted: report() counts only fail and warn in the
FreeType test, but anything not ok elsewhere.
Move them into scripts/smoke-tests/smoketest.py. The scripts have no
.py extension, so each puts its own directory on sys.path and imports
the module, with sys.dont_write_bytecode set so that no __pycache__
shows up in the checkout. What differs between the copies becomes an
argument: the pattern last_line() looks for, the timeout of run() (120 s
for the bare-metal Arm test), the Xvfb arguments, and which directories
--prefix overlays (/usr and /etc for Mesa, which does not require usr/).
This is a refactoring only. freetype-smoke-test and webkitgtk-smoke-test
keep their own --prefix, which takes one image directory, and only share
the reporting, Xvfb, the namespace command and the overcommit check.
The one change in behaviour: webkitgtk-smoke-test without $DISPLAY and
without xdpyinfo now exits with a message instead of a traceback.
Every test, from master and from this commit, on the same MXQ node
(galah, vm.overcommit_memory=2, TMPDIR=$MXQ_JOB_TMPDIR), output compared
after normalizing the random temporary directory names:
SAME llvm: PASS (0 failed, 2 warned) exit 0
SAME llvm-prefix: […] exit 32
SAME arm: PASS (0 failed, 0 warned) exit 0
SAME arm-prefix: PASS (0 failed, 0 warned) exit 0
SAME mesa: PASS (0 failed, 2 warned) exit 0
SAME mesa-prefix: […] exit 32
DIFFER webkit-6.0
DIFFER webkit-4.1
SAME freetype: PASS (0 failed, 0 warned), files in […] exit 0
The WebKitGTK runs differ only in the free commit memory:
-OK strict overcommit, but 1039.0 GB of the 1039.2 GB commit limit is free (a session needs 22 GB at the least)
+OK strict overcommit, but 1031.1 GB of the 1039.2 GB commit limit is free (a session needs 22 GB at the least)
llvm and mesa with five --prefix packages fail on master the same way,
before any test runs:
mount: /usr: wrong fs type, bad option, bad superblock on overlay, missing codepage or helper program, or other error.
So instead, the overlay code of master and of this commit was run on
the same two dummy layers, lower/0 with usr/ and lower/1 with usr/ and
etc/, with subprocess.run replaced by a stub that only records its
command. The recorded `unshare` commands differ only in the path of the
script, also for Mesa's /etc overlay. pyflakes reports nothing, before
and after.
Assisted-by: claude-opus-5.5 (Claude Code)
Sign in
to join this conversation on GitHub.
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.