beesh.sh: use pbzip2 for saving build environment when available

[thomas]

This will keep the mood down when building kernels :)

[mariux]

why not use -I to set compress program? so you do not need to store&read file which should give even more speedup..

[thomas]

Hi Marius :)
Good point, IIRC a similar approach using bgzip (a parallel gzip from samtools) failed, so I went for the safe way right from the start. And to be honest, I haven't even checked if pbzip2 can be feed from stdin.

[donald]

-I pbzip2 does work, so I'd vote for that.

[thomas]

Commit 6b1ee7e holds the new variant using -I.

[pmenzel]

For the record, some tests with pbzip2 and lz4.

$ more /etc/motd

  #     #                  ###         #     #  elcattivo
  ##   ##    ##    #####    #   #    #  #   #   system Supermicro 2023US-TR4
  # # # #   #  #   #    #   #   #    #   # #    cpu    128x AMD EPYC 7601
  #  #  #  #    #  #    #   #   #    #    # 64  ram    1031.7 GB
  #     #  ######  #####    #   #    #   # #    arch   x86_64 / 64-bit
  #     #  #    #  #   #    #   #    #  #   #   glibc  2.27
  #     #  #    #  #    #  ###   ####  #     #  kernel 4.14.30.mx64.211 #1

    / to bee or not to bee /   tags: amd mx64 server 
  ** Public compute server **
  ** if the system is full, please use holidayincambodia **

$ cd /dev/shm
$ ls -lh linux-4.14.56-215.x86_64.beebuild.tar.bz2
-rw-r--r-- 1 pmenzel pmenzel 1.7G Jul 20 11:28 linux-4.14.56-215.x86_64.beebuild.tar.bz2
$ time tar -I pbzip2 -xf linux-4.14.56-215.x86_64.beebuild.tar.bz2 # compressed with normal bzip2

real	3m41.398s
user	3m44.894s
sys	0m7.627s
$ time tar -I pbzip2 -cf linux-4.14.56-215.x86_64.beebuild.tar.bz2  linux-4.14.56-215.x86_64

real	0m21.735s
user	22m8.970s
sys	1m50.933s
$ time tar -I lz4 -cf linux-4.14.56-215.x86_64.beebuild.tar.lz4  linux-4.14.56-215.x86_64

real	0m19.761s
user	0m15.200s
sys	0m7.063s
$ ls -lh *lz4 *bz2
-rw-rw---- 1 pmenzel pmenzel 1.7G Aug 20 11:57 linux-4.14.56-215.x86_64.beebuild.tar.bz2
-rw-rw---- 1 pmenzel pmenzel 2.6G Aug 20 11:58 linux-4.14.56-215.x86_64.beebuild.tar.lz4
$ ls -lh /src/mariux/beeroot/build-archives/linux-4.14.56-215.x86_64.beebuild.tar.bz2
-rw-r--r-- 1 root root 1.7G Jul 20 11:28 /src/mariux/beeroot/build-archives/linux-4.14.56-215.x86_64.beebuild.tar.bz2
$ rm -rf linux-4.14.56-215.x86_64
$ time tar -I pbzip2 -xf linux-4.14.56-215.x86_64.beebuild.tar.bz2  

real	0m11.600s
user	10m33.255s
sys	0m51.540s
$ rm -rf linux-4.14.56-215.x86_64
$ time tar -I lz4 -xf linux-4.14.56-215.x86_64.beebuild.tar.lz4

real	0m9.340s
user	0m5.363s
sys	0m6.890s
$ rm -rf linux-4.14.56-215.x86_64
$ time tar -cjf linux-4.14.56-215.x86_64.beebuild.tar.bz2 linux-4.14.56-215.x86_64

real	9m0.065s
user	9m1.129s
sys	0m14.693s

[donald]

Fine with me. I suggest, you git rebase -i master, remove the first two commits and git push -f the branch.

[thomas]

Cleaned up. What are the next steps, a new bee version?

Well, the present bee file says BEE_VERSION bee-1.2.16-01, but bee --version tells me 1.2.14 -- is this an issue?

[donald]

Yes, make a release. Stick to existing name scheme for release name and tag name. Next release will include #24 as well (for the release information). The make an update mariux64/bee-files.