bash: update to 5.1p4

[wwwutz]

• fixed SRCURLs
• update
• static

bee update bash-5.1_p4-3

[wwwutz]

so far tested:
works fine on invidia, even deadbird on bee-queen-steroids did not show any problems so far.

anyway: yes, please test

[donald]

Error messages are gone, but the git completion doesn't work. Perhaps this needs to be updated, too.

[wwwutz]

wwwutz@eldersoftheinternet:bee-files (update-bash-51)$ . /usr/share/git-contrib/completion/git-completion.bash 
wwwutz@eldersoftheinternet:bee-files (update-bash-51)$ git 
add               cherry            dag               grep              mergetool         remote            shortlog          tag 
am                cherry-pick       describe          gui               mv                repack            show              whatchanged 
apply             citool            diff              help              notes             replace           show-branch       worktree 
archive           clang-format      difftool          init              prune             request-pull      sparse-checkout   
bisect            clean             fetch             instaweb          pull              reset             stage             
blame             clone             format-patch      lfs               push              restore           stash             
branch            cola              fsck              log               range-diff        revert            status            
bundle            commit            gc                maintenance       rebase            rm                submodule         
checkout          config            gitk              merge             reflog            send-email        switch            

works for me ... ?!??

[donald]

My test was wrong. Yes, it works now. Thank you.

[donald]

LGTM

[donald]

Static linking has disadvantages, too. Nearly all programs are linked against libc, which has a few GB (10?) of read only program data. All programs linked against the shared library share the same in-memory copy. If we link everything statically, each program would use its own in-memory copy. A few GB times the number or running programs is relevant for workstations.

Also, multiple in-memory copies might prevent optimal utilization of the processor caches, which would be very relevant to performance.

[pmenzel]

Shameless plug, that Linus also commented on remotely related issue in the LKML thread Re: Very slow clang kernel config ...

[donald]

I guess, some people might stop reading after "Shared libraries are not a good thing..."

So I'd like to stress that

• "unless it's some very core library used by a lot of things"
• "again, unless it's some big library that is typically used by lots of different programs at the same time"
• "the only case shared libraries really make sense is for truly standardized system libraries that are everywhere, and are part of the base distro."

seems to fit glibc.

[wwwutz]

strace -o bla bash --version | wc looks like:

  83  496 5692 bla-shared
  49  271 2968 bla-static

that's 1.6 times more lines of code ( if we measure this way, which is obviously not a good way to benchmark)

Make it seconds: whereas the static version does less code, it will end faster and free up memory faster.

Also hacks via LD_PRELOAD hack will fail.

Also... libc-2.33.so is 17 MB, not GB ... bash bumped up from 4.5 to 13 MB. Ha 1.6 times faster AND bigger 8-)

[wwwutz]

just for fun:

% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
 21.84    0.000257         257         1           execve
 21.41    0.000252          14        18           read
 13.76    0.000162          11        15           mmap
  8.92    0.000105          13         8         1 openat
  5.35    0.000063           9         7           newfstatat
  5.01    0.000059          10         6           write
  4.84    0.000057           8         7           close
  4.59    0.000054          11         5           mprotect
  4.16    0.000049          49         1         1 access
  3.48    0.000041          10         4           brk
  1.78    0.000021          21         1           munmap
  1.27    0.000015           8         2           pread64
  0.68    0.000008           8         1           getuid
  0.59    0.000007           7         1           rt_sigprocmask
  0.59    0.000007           7         1           getgid
  0.59    0.000007           7         1           getegid
  0.59    0.000007           7         1           arch_prctl
  0.51    0.000006           6         1           geteuid
------ ----------- ----------- --------- --------- ----------------
100.00    0.001177                    81         2 total
(END)


% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
 49.67    0.000532         532         1           execve
 14.01    0.000150          30         5           brk
 12.14    0.000130           9        15           read
  5.14    0.000055           9         6           write
  4.76    0.000051          13         4         1 openat
  2.80    0.000030          30         1           readlink
  2.52    0.000027           9         3           newfstatat
  2.43    0.000026           9         3           close
  1.12    0.000012          12         1           mprotect
  0.93    0.000010          10         1           mmap
  0.84    0.000009           9         1           arch_prctl
  0.65    0.000007           7         1           rt_sigprocmask
  0.65    0.000007           7         1           uname
  0.65    0.000007           7         1           getuid
  0.56    0.000006           6         1           getgid
  0.56    0.000006           6         1           geteuid
  0.56    0.000006           6         1           getegid
------ ----------- ----------- --------- --------- ----------------
100.00    0.001071                    47         1 total
(END)

[pmenzel]

I guess, some people might stop reading after "Shared libraries are not a good thing..."

No, the whole message and even thread was an interesting read for me.

So I'd like to stress that

* "unless it's some very core library used by a lot of things"

* "again, unless it's some big library that is typically used by lots of different programs at the same time"

* "the only case shared libraries really make sense is for truly standardized system libraries that are everywhere, and are part of the base distro."

seems to fit glibc.

Absolutely.

[donald]

Also... libc-2.33.so is 17 MB, not GB

Oops. Yes, indeed. So the number of copies is not a relevant factor.

[donald]

buczek@theinternet:~$ time for in in $(seq 1000);do ./bash.shared -c exit 0;done

real 0m3.149s
user 0m2.149s
sys 0m1.129s
buczek@theinternet:~$ time for in in $(seq 1000);do ./bash.static -c exit 0;done

real 0m2.736s
user 0m2.024s
sys 0m0.834s

[donald]

To be clear: I don't request a change here. LGTM

[pmenzel]

So, in case of security issues in glibc and other libraries, who remembers to rebuild the packages using these?

[donald]

I think, statically linked programs can't use dlopen() so bash would loose the ability to load dynamically loadable builtins (enable -f). Not sure and I don't know if anything/anybody uses this. But needs to be considered.

[donald]

Breaks 'fakeroot' (see #2133).

[wwwutz]

So, in case of security issues in glibc and other libraries, who remembers to rebuild the packages using these?

I will. I think others will join.

[wwwutz]

OK, now we have a 'gimmeyoursharedlibshackmewithld-preload' version with ncurses glued in.

bee update bash-5.1_p4-3

[donald]

LGTM

[pmenzel]

No idea if related, I noticed, Ctrl + left/right arrow now jumps words in MarIuX as I am used to from my GNU/Linux (Debian) systems. Before, it would always show some control character with [^ or so. So, thank you for fixing that.