prun: `Argument list too long`

[pmenzel]

I am trying to build OpenWrt under MarIuX, but the build of the package ca-certificates fails:

$ # set -x in /usr/bin/prun to debug
$ make -j1 V=s package/ca-certificates/compile
[…]
+ exec python certdata2pem.py
/usr/bin/prun: line 11: /dev/shm/openwrt/staging_dir/host/bin/python: Argument list too long
make[4]: *** [Makefile:6: all] Error 1
make[4]: Leaving directory '/dev/shm/openwrt/build_dir/target-mips_4kec_musl/ca-certificates-20200601/work/mozilla'
make[3]: *** [Makefile:9: all] Error 2
make[3]: Leaving directory '/dev/shm/openwrt/build_dir/target-mips_4kec_musl/ca-certificates-20200601/work'
make[2]: *** [Makefile:73: /dev/shm/openwrt/build_dir/target-mips_4kec_musl/ca-certificates-20200601/.built] Error 2
make[2]: Leaving directory '/dev/shm/openwrt/package/system/ca-certificates'
time: package/system/ca-certificates/compile#120.10#54.99#285.78
    ERROR: package/system/ca-certificates failed to build.
make[1]: *** [package/Makefile:114: package/system/ca-certificates/compile] Error 1
make[1]: Leaving directory '/dev/shm/openwrt'
make: *** [/dev/shm/openwrt/include/toplevel.mk:234: package/ca-certificates/compile] Error 2

The compile script seems to run python certdata2pem.py for each certificate, and for whatever reason, PATH is only appended to by prun.

+++ export PKG_CONFIG_PATH
+++ '[' -d /pkg/python-3.7.7-0/.compatlibs ']'
+ exec python certdata2pem.py
+ '[' 3 -lt 2 ']'
+ arg1=python3
+ shift
+ set -- python3 python certdata2pem.py
+ pkg=python3
+ shift
+ . /usr/local/package/lib/python3.profile
++ . /pkg/python-3.7.7-0/profile
+++ . /pkg/cuda-10.2.89-1/profile
++++ PATH=/pkg/cuda-10.2.89-1/bin:/pkg/python-3.7.7-0/bin:/pkg/cuda-10.2.89-1/bin:/pkg/python-3.7.7-0/bin:/pkg/cuda-10.2.89-1/bin:/pkg/python-3.7.7-0/bin:/pkg/cuda-10.2.89-1/bin:/pkg/python-3.7.7-0/bin:/pkg/cuda-10.2.89-1/bin:/pkg/python-3.7.7-0/bin:/pkg/cuda-10.2.89-1/bin:/pkg/python-3.7.7-0/bin:/pkg/cuda-10.2.89-1/bin:/pkg/python-3.7.7-0/bin:/pkg/cuda-10.2.89-1/bin:/pkg/python-3.7.7-0/bin:/pkg/cuda-10.2.89-1/bin:/pkg/python-3.7.7-0/bin:/pkg/cuda-10.2.89-1/bin:/pkg/python-3.7.7-0/bin:/pkg/cuda-10.2.89-1/bin:[several hundreds]:/pkg/python-3.7.7-0/lib/pkgconfig:/pkg/python-3.7.7-0/lib/pkgconfig:/pkg/python-3.7.7-0/lib/pkgconfig:/pkg/python-3.7.7-0/lib/pkgconfig:/dev/shm/openwrt/staging_dir/target-mips_4kec_musl/usr/lib/pkgconfig:/dev/shm/openwrt/staging_dir/target-mips_4kec_musl/usr/share/pkgconfig

Feel free to use /dev/shm/openwrt on invidia.

[donald]

Hmmm. The problem is, that they create /dev/shm/openwrt/staging_dir/host/bin/python -> /usr/bin/python3 and then call python bla.py.

/usr/bin/python3 is a generic wrapper. It used (hardlinked) for several /usr/bin commands which happen to be in python3. It calls prun python3 XXXXwith XXXX being the name it was called with. It expects XXXX to be available in the package. Because of the symlink by the build system, it is called with the name 'python`. And there is no "python" command in our default python3 package.

[donald]

We could change the python wrappers so that e.g. /usr/bin/python3 would call prun python3 python3 "$@" instead of prun python3 "$(basename "$0")" "$@". This would work even if the user links that to some stupid name. Then we couldn't hardlink a single wrapper for python2 and another one for python3 but would need to install files ( 24 for python3, 9 for python2).

[donald]

Or we rebuild 'python3' and add a python command into the package. python-3.8.6 and python-3.9.0 seem to have that for some reason or another:

buczek@theinternet:~/git/pkg-scripts (master)$ ls -l /package/pkg/python-3*/bin/python
lrwxrwxrwx 1 build build 9 Nov 30 17:22 /package/pkg/python-3.8.6-0/bin/python -> python3.8
lrwxrwxrwx 1 bin bin 7 Jan 8 13:53 /package/pkg/python-3.9.0-0/bin/python -> python3

But again we have packages in /package/pkg which are not in https://github.molgen.mpg.de/mariux64/pkg-scripts so we can't even look, if the script was changed or its a new python3 default...

[pmenzel]

No idea, what the best solution is.

In the meantime, updating the OpenWRT package ca-certificates to 20210119-1 containing the change
Grab Ubuntu patch to make the package compatible and building with Python3
works around the issue.

[donald]

@pmenzel I think, you did something to invidia:/dev/shm/openwrt, because now it just works. Can you test, whether mariux64/bee-files#2031 fixes the problem?

[pmenzel]

Yes, I created my first OpenWrt patch, and updated the ca-certificates package. I am going to test it next week.