diff --git a/CLAUDE.md b/CLAUDE.md index f11af16..e1faeb4 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -42,7 +42,9 @@ There is no test suite. Lint shell scripts with `shellcheck --rcfile=shellcheckr | `${PROJECT}/profile` | Generated runtime env sourced by `startstop.sh`; chains all component `profile` files | | `${PROJECT}/paperless-ngx/paperless.conf` | Generated Django/paperless config | -To update a component version: change `BUILD_` and `BUILD__SRCURL` in `build.profile`, delete its `.built` tag, re-run `buildall.sh`. +To update a component version: change `BUILD_` and `BUILD__SRCURL` in `build.profile`, delete its `.built` tag, re-run `buildall.sh`. New tarballs must be mirrored first with `md5repo.sh ` (only available on the dev host; production hosts just pull the resolved URLs). + +`.built` tags live in `${BUILD_DIR}//` and are **not** per `BUILD_TAG`. When `BUILD_TAG` changes (new `PREFIX`), components whose version did not change would be skipped and missing from the new prefix — remove all tags first: `find build/ -mindepth 2 -maxdepth 2 -name .built -delete`. ## Runtime architecture @@ -71,9 +73,9 @@ PID/PGID files live in `$DEVSHM` (`/dev/shm/${USER}/${BUILD_TAG}`). Logs are in ## Key patches applied during build -- **`nginx.chicken.patch`**: Renames nginx as "chicken" in HTTP headers, error pages, and version strings. Applied in `nginx.build.sh`. The Server response header advertises `chicken/1.32.0` instead of `nginx/...`. +- **`nginx.chicken.patch`**: Renames nginx as "chicken" in HTTP headers, error pages, and version strings. Applied in `nginx.build.sh` (only if not already applied; a rejected hunk fails the build, and after install `nginx -v` must report `chicken/`). The Server response header says `chicken`. The patch carries the version lines of `src/core/nginx.h` as context, so check that it still applies on every nginx bump. The same patch is shared with immich/toss-baremetal. -- **`paperless-ngx.asgi.patch`**: Backports a fix for [Django ticket #36399](https://code.djangoproject.com/ticket/36399) into paperless-ngx's `asgi.py`. Has an `assert django.VERSION < (6,0)` guard so it will fail loudly when Django 6 ships and the upstream fix lands. Applied in `paperless-ngx.build.sh`. +The former `paperless-ngx.asgi.patch` (backport of [Django ticket #36399](https://code.djangoproject.com/ticket/36399), multiple `Cookie` headers joined with `,`) was removed: it had been disabled since 2.16.3, and the bug cannot trigger here because nginx merges HTTP/2/3 cookie headers into a single `Cookie` header and talks HTTP/1.1 to granian. Django 5.2.x still has the bug, so only revisit this if granian is ever exposed directly. ## systemd @@ -98,12 +100,12 @@ The build order in `buildall.sh` respects these dependencies (see `README.dot` f ## Source download strategy -Sources are downloaded from a private mirror at `beehive.molgen.mpg.de` (MD5 hash in the URL path) rather than upstream directly. Git-based components (imagemagick, pngquant, unpaper, tesseract, jbig2enc, jbig2dec, qpdf) are cloned from GitHub, then cached as `.tar` archives in `${BUILD_DIR}` to speed up rebuilds. +Sources are downloaded from a private mirror at `beehive.molgen.mpg.de` (MD5 hash in the URL path) rather than upstream directly. Git-based components (imagemagick, pngquant, unpaper, tesseract, jbig2enc, jbig2dec, qpdf) are cloned from GitHub, then cached as `.tar` archives in `${BUILD_DIR}` to speed up rebuilds. Tagged ones clone `--branch `; the snapshot ones (jbig2enc, pngquant, unpaper) use a `git describe`-style pin like `unpaper-7.0.0-127-g5bcef8a` and the build script checks out the `` after `-g`, so to bump one put the new `git describe --tags` output into `BUILD_`. ## Utility scripts -- `checkversions.sh` — checks installed versions against upstream releases for all components; sources `${PROJECT}/profile` to find installed binaries -- `cleanall.sh` — removes all build directories (not nginx) and logs; run from `${BUILD_DIR}` +- Version checks use the shared `../baremetals/checkversions.sh .` (run from this repo; compares `build.profile` pins against upstream). The old local `checkversions.sh` was removed. +- `cleanall.sh` — removes all component build trees (and with them their `.built` tags) and logs under `${BUILD_DIR}`, except nginx; cached source tarballs are kept. Run from the repo directory - `runifchanged.sh` — wraps `inotifywait` to re-execute a script when its file is saved (development helper) ## Management commands @@ -119,7 +121,7 @@ After build, wrapper scripts for paperless Django management commands are instal ## Data migration between versions -Set `BUILD_TAG` in `build.profile` to the new version and rebuild. Then: +Set `BUILD_TAG` in `build.profile` to the new version, remove all `.built` tags (see above) and rebuild. Then (see `UPGRADE.md` for the 2.20.x → 3.x roadmap): ```bash OLD=~/paperless-server/2.8.6 diff --git a/README.dot b/README.dot index edbd24e..b7847df 100644 --- a/README.dot +++ b/README.dot @@ -7,7 +7,6 @@ digraph depends { FF [ label="ffmpeg" ] PN [ label="pngquant" ] UN [ label="unpaper" ] - ZB [ label="zbar" ] PL [ label="paperless" ] LE [ label="leptonica" ] TE [ label="tesseract" ] @@ -39,7 +38,7 @@ digraph depends { { SQ, PY } -> NG - { PY, SQ, NO, FF, PN, ZB, UN, LE, TE, JD, JE, NG } -> PL + { PY, SQ, NO, FF, PN, UN, LE, TE, JD, JE, NG } -> PL } diff --git a/UPGRADE.md b/UPGRADE.md new file mode 100644 index 0000000..1df2718 --- /dev/null +++ b/UPGRADE.md @@ -0,0 +1,204 @@ +# Upgrade roadmap: 2.20.x → 2.20.15 → 3.3.1 + +This runbook is for production hosts. Everything they need is already in the repo: +all source tarballs are pre-mirrored on `beehive.molgen.mpg.de`, with the URLs resolved in `build.profile`. +You do **not** need `md5repo.sh`, `../baremetals` or any sibling repo. + +paperless-ngx only supports upgrading to v3 **from 2.20.15** +([upstream migration guide](https://github.com/paperless-ngx/paperless-ngx/blob/v3.3.1/docs/migration-v3.md)). +So every host goes through two hops. Each hop is a git tag: + +| Tag | `BUILD_TAG` | What it is | +|---|---|---| +| `paperless-2.20.15-4` | 2.20.15 | All dependencies at their current bugfix level, paperless 2.20.15 | +| `paperless-3.3.1-8` | 3.3.1 | paperless 3.3.1 plus the build/config changes v3 needs | + +**Use only these two tags.** All older `paperless-2.20.15*` and `paperless-3.3.1*` tags are superseded: +- libjpeg-turbo 3.2.0 fails to build on them. +- On most of them, `buildall.sh` reports success even when a component failed. +- nginx shows `Server: nginx` instead of chicken. +- The consume-test PDF is malformed. +- The `bin/` wrappers drop all their arguments. + +Both hops were tested end to end on a fresh host with these two tags: +- full build of each hop +- start, consume with OCR and login +- data migration 2.20.15 → 3.3.1, including the Tantivy reindex and search + +Do **not** skip hop 1. 2.20.15 must start once, so its DB migrations get applied, before you build 3.3.1. + +--- + +## 0. Before you start (once per host) + +- **Network.** The host needs to reach `beehive.molgen.mpg.de`, `github.com`, PyPI, the npm registry and, + for v3 only, `download.pytorch.org`. +- **Disk.** Each hop is a fresh install under `~/paperless-server/`, and `data/` + `media/` get *copied*. + You need room for roughly two times `media/` plus about 10 GB for the build (v3 pulls in torch, which is CPU-only but still large). +- **CPU (v3).** NumPy needs SSE4.2: `grep -o -m1 sse4_2 /proc/cpuinfo` must print something. + If it prints nothing, add `PAPERLESS_TRAIN_TASK_CRON=disable` to the conf (see the migration guide). +- **`build.local`.** Keep the one you already have. Check that it sets a fixed `PAPERLESS_SECRET_KEY`. + If it doesn't, copy the value out of the running install's `paperless-ngx/paperless.conf` into `build.local`. + Otherwise every rebuild generates a new key, which logs everyone out, and v3 refuses to start without a key. +- **Leftovers in `consume/`.** List them: `find -L "$OLD/consume" -type f ! -path '*/.st*'`. v2 rejected duplicates and left + the files there, so they're retried on every start. v3 would *import* them; see the duplicates row in the table under hop 2. +- **Backup.** With the old install running: + ```bash + OLD=~/paperless-server/2.20.10 # your current version + mkdir -p "$OLD/export" + cd "$OLD/paperless-ngx/src" && ( . "$OLD/profile"; python3 manage.py document_exporter "$OLD/export" ) + ls "$OLD/export" # must contain manifest.json + ``` + Call `manage.py` directly here. On installs built from older tags, `bin/document_exporter` drops its arguments + and exports into `paperless-ngx/src` instead. + +## Hop 1: 2.20.x → 2.20.15 + +```bash +cd /paperless-baremetal +git fetch --tags origin # or whichever remote this host pulls from +git checkout paperless-2.20.15-4 + +# BUILD_TAG changed, so remove all .built tags. Otherwise components whose +# version did not change get skipped and are missing from the new prefix. +find build/ -mindepth 2 -maxdepth 2 -name .built -delete + +./buildall.sh # long; log in build/logs/build.log +``` + +Make sure every component built: `grep -i error build/logs/build.log`, and check that no `build/logs/.log` ends in a failure. + +**Switch over:** +```bash +OLD=~/paperless-server/2.20.10 +NEW=~/paperless-server/2.20.15 + +cd "$OLD" && ./startstop.sh stop +rm -rf "$NEW/data" "$NEW/media" +cp -ax "$OLD/data" "$NEW/" +cp -ax "$OLD/media" "$NEW/" +# move any not-yet-consumed files +cp -ax "$OLD/consume/." "$NEW/consume/" 2>/dev/null || : +cd "$NEW" && ./startstop.sh start # runs pending migrations automatically +./startstop.sh status +``` + +If you use the systemd unit, `systemd.build.sh` printed the new `sudo install` / `systemctl` commands. +Re-install the unit, because its path contains `BUILD_TAG`. + +**Encrypted documents.** v3 removed GPG support and the `decrypt_documents` command, so check now while 2.20.15 is running: +```bash +cd "$NEW/paperless-ngx/src" && . "$NEW/profile" +./manage.py shell -c "from documents.models import Document; print(Document.objects.filter(storage_type='gpg').count())" +``` +If this prints anything other than `0`, run `"$NEW/bin/decrypt_documents" --passphrase ` before hop 2. + +**Verify hop 1:** +- Web login works; the UI footer or settings show 2.20.15. +- Drop the repo's `paperless-online-test.pdf` into `$NEW/consume/`. It should appear as a document with OCR text. +- `log/` has no tracebacks: `grep -il traceback "$NEW"/log/*`. +- Leave it running for a bit. The old install stays untouched as your rollback. + +## Hop 2: 2.20.15 → 3.3.1 + +Read the "Breaking changes" section below first, especially if you use custom pre/post-consume scripts, OIDC or PostgreSQL/MariaDB. + +```bash +cd /paperless-baremetal +git fetch --tags origin +git checkout paperless-3.3.1-8 +find build/ -mindepth 2 -maxdepth 2 -name .built -delete +./buildall.sh +``` + +Then switch over exactly as in hop 1, with `OLD=~/paperless-server/2.20.15` and `NEW=~/paperless-server/3.3.1`. +On first start the worker's migrate step also runs `document_index reindex --if-needed`. +That **rebuilds the full-text index (Whoosh → Tantivy)** and can take a while on a large archive. +Search results are incomplete until it finishes. + +**Verify hop 2:** +- Web login works and shows 3.3.1. A `403` on login points at the proxy/IP settings; see below. +- Search returns results once the reindex is done. +- The consume test with `paperless-online-test.pdf` works. +- A barcode/ASN test document still gets split/tagged, if you use barcodes. v3 uses zxing-cpp instead of zbar. +- `grep -il traceback "$NEW"/log/*` finds nothing. Exceptions: consume errors for files that also failed on v2 + (e.g. Office files without Tika, 16-bit TIFFs) and duplicate rejections. +- Retrain the classifier once: `"$NEW/bin/document_create_classifier"`. v3 clears the task history, so the status + page warns "No classifier training tasks found". A manual run doesn't clear that warning, because the page only counts + *scheduled* training tasks. It goes away after the next scheduled run (hourly by default). + +### Breaking changes in v3 and how this repo handles them + +| Change | Handled by | Action on the host | +|---|---|---| +| `PAPERLESS_SECRET_KEY` required (`change-me` rejected) | conf always sets it | keep it fixed in `build.local` (step 0) | +| GPG encryption removed, `decrypt_documents` gone | wrapper removed | decrypt during hop 1 if needed | +| NLTK removed | NLTK download and `PAPERLESS_NLTK_DIR` removed | `~/paperless-server/*/nltk` can be deleted later | +| pyzbar removed (zxing-cpp only) | zbar component removed from the build | none | +| Search index Whoosh → Tantivy | `startstop.sh` runs `document_index reindex --if-needed` | wait for the reindex; saved views using `note:`/`custom_field:` are migrated, plain terms no longer match notes | +| New LLM index | `startstop.sh` runs `document_llmindex migrate`; `bin/document_llmindex` wrapper added | none | +| Login rate limit uses client IP | conf sets `PAPERLESS_ALLAUTH_TRUSTED_CLIENT_IP_HEADER=X-Real-IP`; nginx sets this header and granian only listens on a unix socket | none | +| Consumer: `CONSUMER_POLLING*`/`INOTIFY_DELAY` renamed/removed, ignore patterns are now regex | not used here | update them if you added them by hand | +| Duplicates no longer rejected by default | `PAPERLESS_CONSUMER_DELETE_DUPLICATES` passes through from `build.local` (default `false` = v3 behaviour) | **check `consume/` for old leftover files** that v2 kept rejecting: v3 would import them as duplicates. Set `PAPERLESS_CONSUMER_DELETE_DUPLICATES=true` in `build.local` to reject them like v2; v3 then also *deletes* them from `consume/`, which a syncthing send-receive folder propagates to the sender | +| `OCR_MODE=skip/skip_noarchive` and `OCR_SKIP_ARCHIVE_FILE` removed; new `ARCHIVE_FILE_GENERATION` defaults to `auto` (no archive for born-digital PDFs) | we use `OCR_MODE=force` (still valid) and set `ARCHIVE_FILE_GENERATION=always`, which keeps the v2 behaviour of always making a PDF/A archive | set `PAPERLESS_ARCHIVE_FILE_GENERATION=auto` in `build.local` if you want the v3 default; check OCR settings in the admin UI if you changed them there | +| Pre/post-consume scripts no longer get `$1…$8` | — | switch custom scripts to the `DOCUMENT_*` env vars | +| Postgres/MariaDB need explicit `PAPERLESS_DBENGINE` | SQLite is the default here | only if you configured an external DB | +| OIDC may need `token_auth_method` | — | only if you use OIDC and get `invalid_client` | +| Task history cleared | — | none | +| `torch` (CPU) is now a hard dependency | `paperless-ngx.build.sh` installs from the PyTorch CPU index, like upstream's Dockerfile | network access to `download.pytorch.org` | + +## Host variations (check before you start) + +Real installs differ from the plain `~/paperless-server/` layout assumed above. Adapt the steps if your host has any of these: + +- **Different permissions on the old install** (e.g. `drwxrws---` for a shared project group). New builds use the + builder's umask; match the old install before switching: `chmod -R g+w,o-rwx "$NEW"` (doesn't follow the consume symlink). +- **A shared `BUILD_DIR`** (the default `build`, or `BUILD_DIR=build` in `build.local`). The `.built` markers then belong to + whatever version built last. Run the `find build/ … -name .built -delete` step before **each** hop's build, not + just once. Otherwise the hop 2 build skips almost everything, and those components are missing from the 3.3.1 install. +- **A per-version `BUILD_DIR` in `build.local`** (e.g. `BUILD_DIR=/dev/shm/${USER}/build-${BUILD_TAG}`). Every version + gets a fresh build dir, so you can skip the `find build/ … -name .built -delete` step. The repo's `build` symlink may still + point to an old, deleted build dir; that's harmless. +- **A shared `consume/` symlink** (e.g. `consume -> ../consume`, fed by syncthing or scanners). The build creates `consume/` + as a real directory in the new install. Replace it with the same symlink, and don't run the `cp … consume/.` step, + or files land where nobody watches: + ```bash + rmdir "$NEW/consume" && ln -s "$(readlink "$OLD/consume")" "$NEW/consume" + ``` +- **A `prod` symlink plus a top-level `startstop.sh` wrapper** (`cd prod; ./startstop.sh "$@"`), which is what the boot + scripts call. Switch over by re-pointing the symlink, then start through the wrapper: + ```bash + ln -sfn /path/to/prod + ``` + Everything that goes through `prod` (e.g. a `media -> prod/media` link) follows automatically. +- **Fetching with a read-only deploy key**, if the host's default ssh key has no GitHub access: + ```bash + GIT_SSH_COMMAND="ssh -i ~/.ssh/ghm_paperlessbm_ro -o IdentitiesOnly=yes" git fetch --tags origin + ``` + +**Both hops in one maintenance window.** Building doesn't touch the running install: it uses its own prefix, build dir and +`/dev/shm` paths, and it doesn't start any services. So you can build both versions ahead of time while the old one keeps +serving: +1. Check out the hop 1 tag and build. +2. Check out the hop 2 tag and build. +3. In the window: stop the old install, copy to 2.20.15, start it (this runs its migrations), verify, stop it. +4. Copy to 3.3.1, start it, verify. + +## Rollback + +Each hop leaves the previous install untouched in its own directory. To roll back, stop the new install +and start the old one: `cd $OLD && ./startstop.sh start`. Any documents added since the switch-over are +only in the new install; re-import them from its `media/` or from an export. + +## Cleanup (after a week or so of stable operation) + +```bash +rm -rf ~/paperless-server/2.20.10 ~/paperless-server/2.20.15 +``` + +## Maintainer notes (dev host only) + +- Bumping a tarball means running `md5repo.sh `, then putting the printed beehive URL into `build.profile`. +- Check versions with `../baremetals/checkversions.sh .` +- `./cleanall.sh` (from the repo dir) wipes all component build trees except nginx. It's an alternative to + the `find … -name .built -delete` step, but it also throws away the extracted sources. diff --git a/build.profile b/build.profile index 1dfcb39..bbc60af 100644 --- a/build.profile +++ b/build.profile @@ -1,6 +1,6 @@ -BUILD_TAG=2.20.10 +BUILD_TAG=3.3.1 -BUILD_paperless=paperless-ngx-2.20.10 +BUILD_paperless=paperless-ngx-3.3.1 BUILD_DIR=build BUILD_SRC="${PWD}" @@ -16,61 +16,57 @@ TMPDIR=${TMPDIR:-/tmp/${USER}/${BUILD_TAG}} LOGDIR=${PROJECT}/log # https://sqlite.org -# wget "https://sqlite.org/2026/sqlite-autoconf-3510200.tar.gz" -BUILD_sqlite=sqlite-3.51.2 -BUILD_sqlite_SRCURL="https://beehive.molgen.mpg.de/49600a5739d382c648b1a317e4b57446/sqlite-autoconf-3510200.tar.gz" +# wget "https://sqlite.org/2026/sqlite-autoconf-3530400.tar.gz" +BUILD_sqlite=sqlite-3.53.4 +BUILD_sqlite_SRCURL="https://beehive.molgen.mpg.de/ee54db5a5b04e77d4883ce0c051a1eba/sqlite-autoconf-3530400.tar.gz" # https://download.redis.io/releases # community edition ! -# wget https://download.redis.io/releases/redis-7.4.8.tar.gz -BUILD_redis=redis-7.4.8 -BUILD_redis_SRCURL="https://beehive.molgen.mpg.de/f8c77d9fa16d3da8c9f745114f4e69e3/${BUILD_redis}.tar.gz" +# wget https://download.redis.io/releases/redis-7.4.11.tar.gz +BUILD_redis=redis-7.4.11 +BUILD_redis_SRCURL="https://beehive.molgen.mpg.de/6d538d9ce2e8c91f0a875886ad550b3c/${BUILD_redis}.tar.gz" -# wget "https://www.python.org/ftp/python/3.12.13/Python-3.12.13.tgz" -BUILD_python=Python-3.12.13 -BUILD_python_SRCURL="https://beehive.molgen.mpg.de/64e26b69406a9e828dce782172f0d99c/${BUILD_python}.tgz" +# wget "https://www.python.org/ftp/python/3.12.15/Python-3.12.15.tgz" +BUILD_python=Python-3.12.15 +BUILD_python_SRCURL="https://beehive.molgen.mpg.de/dbbf38277b3579d0a53aecfd7e9d6012/${BUILD_python}.tgz" -# wget "https://nodejs.org/download/release/v24.14.0/node-v24.14.0.tar.gz" -BUILD_node=node-24.14.0 -BUILD_node_SRCURL="https://beehive.molgen.mpg.de/74be2d0f33e2fc00d3d319885ac0c6e5/${BUILD_node/-/-v}.tar.gz" +# wget "https://nodejs.org/download/release/v24.21.0/node-v24.21.0.tar.gz" +BUILD_node=node-24.21.0 +BUILD_node_SRCURL="https://beehive.molgen.mpg.de/a0d4131e52d9055f8428439542018d44/${BUILD_node/-/-v}.tar.gz" -# SRCURL="https://ffmpeg.org//releases/${BUILD_ffmpeg}.tar.gz" -BUILD_ffmpeg=ffmpeg-8.0 -BUILD_ffmpeg_SRCURL="https://beehive.molgen.mpg.de/74bb5dbf6c2b35ab50aad7a939ef3ea2/${BUILD_ffmpeg}.tar.gz" - -# SRCURL="https://github.com/mchehab/zbar/archive/refs/tags/0.23.93.tar.gz" -BUILD_zbar=zbar-0.23.93 -BUILD_zbar_SRCURL="https://beehive.molgen.mpg.de/3f69d17f6495de023b59b3539ce5e605/${BUILD_zbar}.tar.gz" +# wget "https://ffmpeg.org/releases/ffmpeg-8.0.3.tar.gz" +BUILD_ffmpeg=ffmpeg-8.0.3 +BUILD_ffmpeg_SRCURL="https://beehive.molgen.mpg.de/3057126c124c8515ff7122a6d41d609b/${BUILD_ffmpeg}.tar.gz" # wget "https://github.com/DanBloomberg/leptonica/releases/download/1.87.0/leptonica-1.87.0.tar.gz" BUILD_leptonica=leptonica-1.87.0 BUILD_leptonica_SRCURL="https://beehive.molgen.mpg.de/d523d2c883b71850a46af2517d8915bf/${BUILD_leptonica}.tar.gz" -# wget "https://nginx.org/download/nginx-1.31.0.tar.gz" -BUILD_nginx=nginx-1.31.0 -BUILD_nginx_SRCURL="https://beehive.molgen.mpg.de/40b2a7057eb55965d42d5416636c27e2/${BUILD_nginx}.tar.gz" +# wget "https://nginx.org/download/nginx-1.31.6.tar.gz" +BUILD_nginx=nginx-1.31.6 +BUILD_nginx_SRCURL="https://beehive.molgen.mpg.de/6d59f919382fae46812c44ffbc87498a/${BUILD_nginx}.tar.gz" # SRCURL="https://github.com/liberationfonts/liberation-fonts/files/7261482/liberation-fonts-ttf-2.1.5.tar.gz" BUILD_libfontttf=liberation-fonts-ttf-2.1.5 BUILD_libfontttf_SRCURL="https://beehive.molgen.mpg.de/31b453e0b77bacde410a34a725b34f8a/${BUILD_libfontttf}.tar.gz" -# wget "https://download.osgeo.org/libtiff/tiff-4.7.1.tar.gz" -BUILD_libtiff=tiff-4.7.1 -BUILD_libtiff_SRCURL="https://beehive.molgen.mpg.de/f1044dd3b4466cc53464210148e08146/${BUILD_libtiff}.tar.gz" +# wget "https://download.osgeo.org/libtiff/tiff-4.7.2.tar.gz" +BUILD_libtiff=tiff-4.7.2 +BUILD_libtiff_SRCURL="https://beehive.molgen.mpg.de/a2a65ea450b8f469c667128848ffebfc/${BUILD_libtiff}.tar.gz" -# SRCURL="https://github.com/libjpeg-turbo/libjpeg-turbo/releases/download/3.1.3/libjpeg-turbo-3.1.3.tar.gz" -BUILD_libjpegturbo=libjpegturbo-3.1.3 -BUILD_libjpegturbo_SRCURL="https://beehive.molgen.mpg.de/d23f3be7e58ad79d297845e64807472c/libjpeg-turbo-3.1.3.tar.gz" +# SRCURL="https://github.com/libjpeg-turbo/libjpeg-turbo/releases/download/3.2.0/libjpeg-turbo-3.2.0.tar.gz" +BUILD_libjpegturbo=libjpegturbo-3.2.0 +BUILD_libjpegturbo_SRCURL="https://beehive.molgen.mpg.de/a80f9f1c735b57d3cdd66daed1ba6bb4/libjpeg-turbo-3.2.0.tar.gz" # SRCURL="https://storage.googleapis.com/downloads.webmproject.org/releases/webp/libwebp-1.6.0.tar.gz" BUILD_libwebp=libwebp-1.6.0 BUILD_libwebp_SRCURL="https://beehive.molgen.mpg.de/cceb6447180f961473b181c9ef38b630/${BUILD_libwebp}.tar.gz" -# wget "https://github.com/ArtifexSoftware/ghostpdl-downloads/releases/download/gs10060/ghostscript-10.06.0.tar.gz" -BUILD_ghostscript=ghostscript-10.06.0 -BUILD_ghostscript_SRCURL="https://beehive.molgen.mpg.de/d472f289582b209731954642bd34ca86/${BUILD_ghostscript}.tar.gz" +# wget "https://github.com/ArtifexSoftware/ghostpdl-downloads/releases/download/gs10071/ghostscript-10.07.1.tar.gz" +BUILD_ghostscript=ghostscript-10.07.1 +BUILD_ghostscript_SRCURL="https://beehive.molgen.mpg.de/de0ed6fd5dc92556e79160bb5efa1f09/${BUILD_ghostscript}.tar.gz" # gits -BUILD_imagemagick=imagemagick-7.1.2-15 +BUILD_imagemagick=imagemagick-7.1.2-33 BUILD_imagemagick_SRCURL="https://github.com/ImageMagick/ImageMagick.git" # git clone / git describe @@ -80,7 +76,7 @@ BUILD_pngquant_SRCURL="https://github.com/kornelski/pngquant.git" BUILD_unpaper=unpaper-7.0.0-127-g5bcef8a BUILD_unpaper_SRCURL="https://github.com/unpaper/unpaper.git" -BUILD_tesseract=tesseract-5.5.2 +BUILD_tesseract=tesseract-5.5.3 BUILD_tesseract_SRCURL="https://github.com/tesseract-ocr/tesseract.git" BUILD_tessdata=tessdata_best-4.1.0 @@ -89,7 +85,7 @@ BUILD_tessconfigs_SRCURL="https://github.com/tesseract-ocr/tessconfigs.git" # git clone / git describe #BUILD_jbig2enc=jbig2enc-0.30-16 -BUILD_jbig2enc=jbic2enc-0.30-20-g2f7f76b +BUILD_jbig2enc=jbig2enc-0.30-20-g2f7f76b BUILD_jbig2enc_SRCURL="https://github.com/agl/jbig2enc.git" BUILD_jbig2dec=jbig2dec-0.20 diff --git a/buildall.sh b/buildall.sh index 294313c..b14493c 100755 --- a/buildall.sh +++ b/buildall.sh @@ -1,6 +1,7 @@ #!/bin/bash set -e set -u +set -o pipefail BUILD_LOCAL=build.local if [ ! -e ${BUILD_LOCAL} ]; then @@ -85,7 +86,6 @@ _pkgs=( tesseract pngquant unpaper - zbar libfontttf qpdf nginx @@ -95,13 +95,14 @@ build() { local _pkg=$1 eval "_tag=${BUILD_DIR}/\$BUILD_${_pkg}/.built" if [[ ! -e "$_tag" ]]; then - ./${_pkg}.build.sh 2>&1 | tee "${LOGS}/${_pkg}.log"; - r=${PIPESTATUS[0]} + local r=0 + # pipefail: r is the build script's status (tee itself succeeds) + ./${_pkg}.build.sh 2>&1 | tee "${LOGS}/${_pkg}.log" || r=$? if [[ $r == 0 ]]; then touch "$_tag" else echo "${_pkg}.build.sh failed exit code $r" - exit + exit "$r" fi fi } diff --git a/checkversions.sh b/checkversions.sh deleted file mode 100755 index 8a8727a..0000000 --- a/checkversions.sh +++ /dev/null @@ -1,300 +0,0 @@ -#!/bin/bash -set -u - -. build.profile - -[[ -d "${TMPDIR}" ]] || mkdir -p "${TMPDIR}" - -tmp="${TMPDIR}/cv.$$.tmp" -#tmp=/tmp/cv.tmp - -function rm_tmp() { - : - rm -f "${tmp}" -} - -trap rm_tmp EXIT - -declare -A CHECK_URL - -function parsefile() { - U=$1 - P=$2 - D=${3:-''} - - curl -s -L "${U}" -o "${tmp}" - x1=$(grep --max-count=1 -oP -e "${P}" "${tmp}") - if [[ -n "${D}" ]]; then - x1=${x1}'-'$(grep --max-count=1 -oP -e "${D}" "${tmp}") - fi - echo $x1 -} - -function parselast() { - U=$1 - P=$2 - D=${3:-''} - - curl -s -L "${U}" -o "${tmp}" - x1=$(grep -oP -e "${P}" "${tmp}"|tail -1) - echo $x1 -} - -# git_describe_remote -function gdrem() { - - G=$1 - P=$2 - C=${3:-'(?<=Release ).*?(?=<)'} - D=${4:-''} - L=$(git ls-remote --refs --sort=-v:refname "https://github.com/${G}" "${P}" | head -1) - - - _IFS=$IFS - ${IFS+':'} unset _IFS - IFS=[$'\t'/] - set -- $L - IFS=$_IFS - ${_IFS+':'} unset IFS - - SHA=$1 - TAG=$4 - URL="https://github.com/${G}/releases" - parsefile "${URL}/tag/${TAG}" "${C}" "${D}" - echo "${URL}" -} - -function parse() { - U=$1 - P=$2 - parsefile "${U}" "${P}" - echo "${U}" -} - -function bparse() { - U=$1 - P=$2 - - if [[ $(curl -s -L "${U}") =~ ${P} ]]; then - echo "${BASH_REMATCH[1]}${BASH_REMATCH[2]:+-}${BASH_REMATCH[2]:-}" - fi - echo "${U}" -} - -function mtype() { - local _a=$1 - local _p=$(type -p $_a) - echo '...'${_p#$PREFIX} -} - -# https://api.github.com/repos/paperless-ngx/paperless-ngx/releases/latest& - -if [[ -e "$PROJECT"/profile ]]; then - . "$PROJECT"/profile - function exec_BUILD_node () { - node --version - mtype node - } - function exec_BUILD_python () { - python --version|cut -d' ' -f2 - mtype python - } - function exec_BUILD_redis () { - redis-cli --version|cut -d' ' -f2 - mtype redis-cli - } - function exec_BUILD_sqlite () { - sqlite3 --version|cut -d' ' -f1 - mtype sqlite3 - } - function exec_BUILD_nginx () { - nginx -v 2>&1|cut -d/ -f2|cut -d' ' -f1 - mtype nginx - } - function exec_BUILD_ffmpeg () { - ffmpeg -version | head -1 | cut -d' ' -f3 - mtype ffmpeg - } - function exec_BUILD_imagemagick () { - magick --version | head -1 | cut -d' ' -f3 - mtype magick - } - function exec_BUILD_unpaper () { - unpaper --version | head -1 - mtype unpaper - } - function exec_BUILD_tesseract () { - tesseract --version | head -1 | cut -d' ' -f2 - mtype tesseract - } - function exec_BUILD_zbar () { - zbarimg --version - mtype zbarimg - } - function exec_BUILD_leptonica() { - pkg-config --modversion lept - echo "no-exec" - } - function exec_BUILD_jbig2enc () { - jbig2 -V 2>&1|head -1|cut -d' ' -f2 - mtype jbig2 - } - function exec_BUILD_jbig2dec () { - jbig2dec -V 2>&1|cut -d' ' -f2 - mtype jbig2dec - } - function exec_BUILD_pngquant () { - pngquant -V - mtype pngquant - } - function exec_BUILD_libwebp () { - webpinfo -version|cut -d' ' -f4 - mtype webpinfo - } - function exec_BUILD_libjpegturbo () { - re='version (\S+)\s\(build\s(\S+)\)' - [[ $(cjpeg -version 2>&1) =~ $re ]] && echo "${BASH_REMATCH[1]}-${BASH_REMATCH[2]}" - mtype cjpeg - } - function exec_BUILD_libtiff () { - re='Version\s+(\S+)' - [[ $(tiffinfo 2>&1) =~ $re ]] && echo "${BASH_REMATCH[1]}" - mtype tiffinfo - } - function exec_BUILD_ghostscript () { - gs --version - mtype gs - } - -fi - -version_BUILD_paperless() { - gdrem paperless-ngx/paperless-ngx 'refs/tags/v*'|cut -d' ' -f2 -} -version_BUILD_imagemagick() { - gdrem ImageMagick/ImageMagick 'refs/tags/[0-9]*'|cut -d' ' -f1 -} -version_BUILD_node() { - parse 'https://github.com/nodejs/node/releases' '(?<=Version ).*?(?= ).*LTS\)'|cut -d' ' -f1 -} -version_BUILD_python() { - parselast 'https://raw.githubusercontent.com/python/peps/main/peps/pep-0693.rst' '(?<=^- )[0-9].*(?=: )' -} -version_BUILD_redis() { - parse 'https://github.com/redis/redis/releases' '(?<=/redis/redis/tree/)7.*?(?=")' -} -version_BUILD_sqlite() { - parse 'https://sqlite.org/index.html' '(?<=Version ).*(?=<)' -} -version_BUILD_nginx() { - parse 'https://nginx.org/' '(?<=nginx-).*?(?=<)' -} -version_BUILD_tesseract() { - gdrem tesseract-ocr/tesseract 'refs/tags/[0-9]*'|cut -d' ' -f1 -} -version_BUILD_tessdata() { - gdrem tesseract-ocr/tessdata_best 'refs/tags/[0-9]*' '(?<=Release ).*?(?=\s)'|cut -d' ' -f2 -} -version_BUILD_jbig2dec() { - gdrem ArtifexSoftware/jbig2dec 'refs/tags/[0-9]*'|cut -d' ' -f2 -} -version_BUILD_jbig2enc() { - gdrem agl/jbig2enc 'refs/tags/[0-9]*'|cut -d' ' -f1 -} -version_BUILD_leptonica() { - gdrem DanBloomberg/leptonica 'refs/tags/[0-9]*'|cut -d' ' -f3 -} -version_BUILD_ffmpeg() { - parse 'https://git.videolan.org/?p=ffmpeg.git;a=blob_plain;f=Changelog' '(?<=^version )\d.*(?=:)' -} -version_BUILD_pngquant() { - gdrem kornelski/pngquant 'refs/tags/*'|cut -d' ' -f1 -} -version_BUILD_zbar() { - gdrem mchehab/zbar 'refs/tags/[0-9]*'|cut -d' ' -f2 -} -version_BUILD_unpaper() { - gdrem unpaper/unpaper 'refs/tags/unpaper-*' '(?<=<title>Release unpaper-).*?(?=\s)' '(?<= )[0-9]+(?= commits)'|tr '\n' ' ' -} -version_BUILD_libfontttf() { - gdrem liberationfonts/liberation-fonts 'refs/tags/[0-9]*'|cut -d' ' -f1 -} -version_BUILD_libwebp() { - bparse 'https://chromium.googlesource.com/webm/libwebp/' 'v([1-9]+\.[0-9]+\.[0-9]+)' -} -version_BUILD_libjpegturbo() { - gdrem libjpeg-turbo/libjpeg-turbo 'refs/tags/[0-9]*'|cut -d' ' -f1 -} -version_BUILD_libtiff() { - bparse 'https://download.osgeo.org/libtiff/?C=M&O=D' 'tiff-(\S+?).tar.gz' -} -version_BUILD_qpdf() { - gdrem qpdf/qpdf 'refs/tags/v[0-9]*'|cut -d' ' -f2 -} - -function version() { - Bi=$1 - B=$(eval eval echo \\$\${Bi}) - shift -# printf "%-s\n" "$Bi" - printf "#### %26s\n" "$B" - if [[ $(type -t exec_${Bi}) = 'function' ]]; then -# printf " exe %32s %s\n" "$(exec_${Bi}|tr '\n' ' ')" - v=$(exec_${Bi}|tr '\n' ' ') - printf " exe %26s %s\n" $v - else - : # printf "no exec_()\n" - fi - - if [[ $(type -t version_${Bi}) = 'function' ]]; then - v=$(version_${Bi}) - printf " ver %26s %s\n" $v - else - printf "no version_()\n" - fi - echo -} - -_parts=( -BUILD_paperless -BUILD_node -BUILD_python -BUILD_redis -BUILD_sqlite -BUILD_nginx -BUILD_tesseract -BUILD_tessdata -BUILD_libwebp -BUILD_libjpegturbo -BUILD_libtiff -BUILD_jbig2dec -BUILD_jbig2enc -BUILD_leptonica -BUILD_ffmpeg -BUILD_imagemagick -BUILD_pngquant -BUILD_zbar -BUILD_unpaper -BUILD_libfontttf -BUILD_qpdf -BUILD_ghostscript -) - -_xparts=( -BUILD_ghostscript -) - -# list of versions -for p in "${_parts[@]}"; do - B=$(eval eval echo \\$\${p}) - echo -n "$B - " -done -echo - -# check versions -for p in "${_parts[@]}"; do - version $p -done - -# apptainer build --sandbox xxx docker://ghcr.io/paperless-ngx/paperless-ngx:latest -# grep PYTHON xxx/.singularity.d/env/10-docker2singularity.sh diff --git a/cleanall.sh b/cleanall.sh old mode 100644 new mode 100755 index 5588e7a..b84c998 --- a/cleanall.sh +++ b/cleanall.sh @@ -3,26 +3,34 @@ set -e set -x set -u +# run from the repo directory; removes build trees (incl. their .built tags) +# and logs below ${BUILD_DIR}. cached source tarballs (*.tar*) are kept. + . build.profile declare -a RMRF RMRF=( + "${BUILD_sqlite}" + "${BUILD_redis}" + "${BUILD_python}" + "${BUILD_node}" + "${BUILD_libjpegturbo}" + "${BUILD_libtiff}" + "${BUILD_libwebp}" "${BUILD_ffmpeg}" "${BUILD_jbig2dec}" - "${BUILD_jbig2enc}" + "${BUILD_ghostscript}" + "${BUILD_imagemagick}" "${BUILD_leptonica}" + "${BUILD_jbig2enc}" + "${BUILD_tesseract}" + "${BUILD_pngquant}" + "${BUILD_unpaper}" "${BUILD_libfontttf}" + "${BUILD_qpdf}" # "${BUILD_nginx}" - "${BUILD_node}" "${BUILD_paperless}" - "${BUILD_pngquant}" - "${BUILD_python}" - "${BUILD_redis}" - "${BUILD_sqlite}" - "${BUILD_tesseract}" - "${BUILD_unpaper}" - "${BUILD_zbar}" ./logs ) @@ -35,4 +43,6 @@ function removeall { set -e } +pushd "${BUILD_DIR}" removeall +popd diff --git a/ffmpeg.build.sh b/ffmpeg.build.sh index ab1335f..ab918ab 100755 --- a/ffmpeg.build.sh +++ b/ffmpeg.build.sh @@ -26,7 +26,7 @@ EOF test -e "${BUILD_PKG}".tar.gz || wget -nv "${SRCURL}" -O "${BUILD_PKG}".tar.gz -test -d "${BUILD_PKG}" || mkdir -pv "${BUILD_PKG}" && tar -xf "${BUILD_PKG}".tar.gz --strip-components=1 -C "${BUILD_PKG}" +test -d "${BUILD_PKG}" || { mkdir -pv "${BUILD_PKG}" && tar -xf "${BUILD_PKG}".tar.gz --strip-components=1 -C "${BUILD_PKG}"; } cd "${BUILD_PKG}" diff --git a/ghostscript.build.sh b/ghostscript.build.sh index 9c43ee5..ff34fb1 100755 --- a/ghostscript.build.sh +++ b/ghostscript.build.sh @@ -28,7 +28,7 @@ EOF test -e "${BUILD_PKG}".tar.gz || wget -nv "${SRCURL}" -O "${BUILD_PKG}".tar.gz -test -d "${BUILD_PKG}" || mkdir -pv "${BUILD_PKG}" && tar -xf "${BUILD_PKG}".tar.gz --strip-components=1 -C "${BUILD_PKG}" +test -d "${BUILD_PKG}" || { mkdir -pv "${BUILD_PKG}" && tar -xf "${BUILD_PKG}".tar.gz --strip-components=1 -C "${BUILD_PKG}"; } cd "${BUILD_PKG}" diff --git a/jbig2enc.build.sh b/jbig2enc.build.sh index c589031..f49420a 100755 --- a/jbig2enc.build.sh +++ b/jbig2enc.build.sh @@ -26,16 +26,11 @@ fi if [[ ! -d ${BUILD_PKG} ]]; then git clone "${SRCURL}" --single-branch "${BUILD_PKG}" - # check version - pushd "${BUILD_PKG}" - GITVER=$(git describe --tags) - popd - - if [ "${GITVER}" != "${BUILD_PKG#*-}" ]; then - echo "${BUILD_PKG} should be updated to ${GITVER} update!" - fi fi +# pin to the exact commit encoded in BUILD_jbig2enc (...-g<sha>) +git -C "${BUILD_PKG}" checkout --detach "${BUILD_PKG##*-g}" + if [[ ! -e ${BUILD_PKG}.tar ]]; then tar cf "${BUILD_PKG}".tar "${BUILD_PKG}" fi diff --git a/leptonica.build.sh b/leptonica.build.sh index 841592a..573db24 100755 --- a/leptonica.build.sh +++ b/leptonica.build.sh @@ -28,7 +28,7 @@ LD_LIBRARY_PATH=${PREFIX}/lib\${LD_LIBRARY_PATH:+:}\${LD_LIBRARY_PATH:-} EOF test -e ${BUILD_PKG}.tar.gz || wget -nv ${SRCURL} -O ${BUILD_PKG}.tar.gz -test -d ${BUILD_PKG} || mkdir -pv ${BUILD_PKG} && tar -xf ${BUILD_PKG}.tar.gz --strip-components=1 -C ${BUILD_PKG} +test -d ${BUILD_PKG} || { mkdir -pv ${BUILD_PKG} && tar -xf ${BUILD_PKG}.tar.gz --strip-components=1 -C ${BUILD_PKG}; } cd ${BUILD_PKG} diff --git a/libfontttf.build.sh b/libfontttf.build.sh index 8546ade..08da260 100755 --- a/libfontttf.build.sh +++ b/libfontttf.build.sh @@ -19,7 +19,7 @@ PREFIX="${PREFIX}/${BUILD_PKG}" mkdir -p "${PREFIX}" test -e "${BUILD_PKG}.tar.gz" || wget -nv "${SRCURL}" -O "${BUILD_PKG}.tar.gz" -test -d "${BUILD_PKG}" || mkdir -pv "${BUILD_PKG}" && tar -xf "${BUILD_PKG}.tar.gz" --strip-components=1 -C "${BUILD_PKG}" +test -d "${BUILD_PKG}" || { mkdir -pv "${BUILD_PKG}" && tar -xf "${BUILD_PKG}.tar.gz" --strip-components=1 -C "${BUILD_PKG}"; } cd "${BUILD_PKG}" diff --git a/libjpegturbo.build.sh b/libjpegturbo.build.sh index e836b8f..60e101f 100755 --- a/libjpegturbo.build.sh +++ b/libjpegturbo.build.sh @@ -24,7 +24,7 @@ EOF test -e "${BUILD_PKG}".tar.gz || wget -nv "${SRCURL}" -O "${BUILD_PKG}".tar.gz -test -d "${BUILD_PKG}" || mkdir -pv "${BUILD_PKG}" && tar -xf "${BUILD_PKG}".tar.gz --strip-components=1 -C "${BUILD_PKG}" +test -d "${BUILD_PKG}" || { mkdir -pv "${BUILD_PKG}" && tar -xf "${BUILD_PKG}".tar.gz --strip-components=1 -C "${BUILD_PKG}"; } pushd "${BUILD_PKG}" @@ -33,6 +33,7 @@ local _conf=( -DCMAKE_INSTALL_LIBDIR=lib -DCMAKE_COLOR_MAKEFILE=OFF -DCMAKE_VERBOSE_MAKEFILE=ON + -DCMAKE_C_FLAGS='-O2 -fPIC' ) rm -rf b diff --git a/libtiff.build.sh b/libtiff.build.sh index 3b86eaf..824be2b 100755 --- a/libtiff.build.sh +++ b/libtiff.build.sh @@ -26,7 +26,7 @@ EOF test -e "${BUILD_PKG}".tar.gz || wget -nv "${SRCURL}" -O "${BUILD_PKG}".tar.gz -test -d "${BUILD_PKG}" || mkdir -pv "${BUILD_PKG}" && tar -xf "${BUILD_PKG}".tar.gz --strip-components=1 -C "${BUILD_PKG}" +test -d "${BUILD_PKG}" || { mkdir -pv "${BUILD_PKG}" && tar -xf "${BUILD_PKG}".tar.gz --strip-components=1 -C "${BUILD_PKG}"; } cd "${BUILD_PKG}" diff --git a/libwebp.build.sh b/libwebp.build.sh index 90d925c..1af37f0 100755 --- a/libwebp.build.sh +++ b/libwebp.build.sh @@ -24,7 +24,7 @@ PKG_CONFIG_PATH=${PREFIX}/lib/pkgconfig\${PKG_CONFIG_PATH:+:}\${PKG_CONFIG_PATH: EOF test -e "${BUILD_PKG}".tar.gz || wget -nv "${SRCURL}" -O "${BUILD_PKG}".tar.gz -test -d "${BUILD_PKG}" || mkdir -pv "${BUILD_PKG}" && tar -xf "${BUILD_PKG}".tar.gz --strip-components=1 -C "${BUILD_PKG}" +test -d "${BUILD_PKG}" || { mkdir -pv "${BUILD_PKG}" && tar -xf "${BUILD_PKG}".tar.gz --strip-components=1 -C "${BUILD_PKG}"; } cd "${BUILD_PKG}" diff --git a/nginx.build.sh b/nginx.build.sh index 1f7284d..4585044 100755 --- a/nginx.build.sh +++ b/nginx.build.sh @@ -29,7 +29,7 @@ if [ -v NGINX_SSL_DHPARAM ]; then fi test -e ${BUILD_PKG}${BUILD_EXT} || wget -nv "${SRCURL}" -O ${BUILD_PKG}${BUILD_EXT} -test -d ${BUILD_PKG} || mkdir -pv ${BUILD_PKG} && tar -xf ${BUILD_PKG}${BUILD_EXT} --strip-components=1 -C ${BUILD_PKG} +test -d ${BUILD_PKG} || { mkdir -pv ${BUILD_PKG} && tar -xf ${BUILD_PKG}${BUILD_EXT} --strip-components=1 -C ${BUILD_PKG}; } pushd ${BUILD_PKG} @@ -37,7 +37,11 @@ pushd ${BUILD_PKG} # What you see here is the representation of the string "nginx" compressed with HPACK's Huffman encoding. # ~/go/src/github.com/wwwutz/hpack -patch -p1 --silent --backup --forward --input=${BUILD_SRC}/nginx.chicken.patch || true +# The patch carries nginx_version/NGINX_VERSION as context lines: check it on +# every nginx bump. Idempotent (already-applied is fine), anything else fails. +if ! patch -p1 --dry-run --silent --reverse --input=${BUILD_SRC}/nginx.chicken.patch >/dev/null; then + patch -p1 --silent --forward --input=${BUILD_SRC}/nginx.chicken.patch +fi local _conf=( --prefix="${PREFIX}" @@ -63,6 +67,9 @@ CFLAGS='-O2 -fPIC' \ make -j "${NPROC}" install +# a rejected/missing patch hunk must not go unnoticed +"${PREFIX}"/sbin/nginx -v 2>&1 | grep -q "^nginx version: chicken/" || { echo "chicken patch not effective"; exit 1; } + popd } diff --git a/nginx.chicken.patch b/nginx.chicken.patch index e05d2b2..10cf1b7 100644 --- a/nginx.chicken.patch +++ b/nginx.chicken.patch @@ -19,7 +19,7 @@ index a57c2f9..6ba724b 100644 </body> </html> diff --git a/html/index.html b/html/index.html -index 75bcddc..b6668f7 100644 +index e8f5622..b6668f7 100644 --- a/html/index.html +++ b/html/index.html @@ -1,7 +1,7 @@ @@ -41,7 +41,7 @@ index 75bcddc..b6668f7 100644 -API gateway, load balancer, content cache, or other features.</p> +<h1>Chicken!</h1> +<p>Bwok Puk bok pukak bwak-bwak bwok Cluck bwak-bwak waaaaak bwwwaak-bwwwaaaaak.</p> - + -<p>For online documentation and support please refer to -<a href="https://nginx.org/">nginx.org</a>.<br/> -To engage with the community please visit @@ -50,35 +50,32 @@ index 75bcddc..b6668f7 100644 -security features and capabilities please refer to -<a href="https://f5.com/nginx">f5.com/nginx</a>.</p> +<p>Cluckity... Cluckity Puk-puk Waaaaak cock-a-doodle-doo bwak-bwak-bwak cluckity bwwwaak! Cluckity bok-bok Bok bwok-bwok-bwok bwak-bwak-bwak cluck. Cluck wak-waaaak pukaaaak-pukaaaak-pukaaaaak Puk-puk-puk Puk Gobble-gobble bok Pukaaaaaaak-pukaaaaak bwwwak-bwwwak-bwwwaaaaak Puk bwak bwak-bwak waak-waaaaaaak cluck. Pukak Bwok?</p> - + -<p><em>Thank you for using nginx.</em></p> +<p><em>Cluck-cluck-cluck gobble-gobble-gobble honk Gobble-gobble bwwwaaaaaak.</em></p> </body> </html> diff --git a/src/core/nginx.h b/src/core/nginx.h -index f4ccc61..8ff917c 100644 +index 088aa1e..0e1632f 100644 --- a/src/core/nginx.h +++ b/src/core/nginx.h -@@ -9,9 +9,9 @@ - #define _NGINX_H_INCLUDED_ - - --#define nginx_version 1031000 --#define NGINX_VERSION "1.31.0" +@@ -10,8 +10,8 @@ + + + #define nginx_version 1031006 + #define NGINX_VERSION "1.31.6" -#define NGINX_VER "nginx/" NGINX_VERSION -+#define nginx_version 1032000 -+#define NGINX_VERSION "1.32.0" +#define NGINX_VER "chicken/" NGINX_VERSION - + #ifdef NGX_BUILD #define NGINX_VER_BUILD NGINX_VER " (" NGX_BUILD ")" diff --git a/src/http/ngx_http_special_response.c b/src/http/ngx_http_special_response.c -index 2984ffe..cc40bc6 100644 +index eaf42e3..fd682a3 100644 --- a/src/http/ngx_http_special_response.c +++ b/src/http/ngx_http_special_response.c @@ -33,7 +33,7 @@ static u_char ngx_http_error_build_tail[] = - - + + static u_char ngx_http_error_tail[] = -"<hr><center>nginx</center>" CRLF +"<hr><center>chicken</center>" CRLF @@ -92,33 +89,46 @@ index 6b73b1e..a7b8094 100644 @@ -120,7 +120,7 @@ ngx_http_v2_header_filter(ngx_http_request_t *r) ngx_http_core_srv_conf_t *cscf; u_char addr[NGX_SOCKADDR_STRLEN]; - + - static const u_char nginx[5] = { 0x84, 0xaa, 0x63, 0x55, 0xe7 }; + static const u_char nginx[6] = { 0x85, 0x24, 0xe6, 0x27, 0x52, 0xd5 }; #if (NGX_HTTP_GZIP) static const u_char accept_encoding[12] = { 0x8b, 0x84, 0x84, 0x2d, 0x69, 0x5b, 0x05, 0x44, 0x3c, 0x86, 0xaa, 0x6f diff --git a/src/http/v3/ngx_http_v3_filter_module.c b/src/http/v3/ngx_http_v3_filter_module.c -index e3f1536..ff97b0f 100644 +index 4d2276d..7b156a6 100644 --- a/src/http/v3/ngx_http_v3_filter_module.c +++ b/src/http/v3/ngx_http_v3_filter_module.c @@ -169,7 +169,7 @@ ngx_http_v3_header_filter(ngx_http_request_t *r) n = sizeof(NGINX_VER_BUILD) - 1; - + } else { - n = sizeof("nginx") - 1; + n = sizeof("chicken") - 1; } - + len += ngx_http_v3_encode_field_lri(NULL, 0, @@ -352,8 +352,8 @@ ngx_http_v3_header_filter(ngx_http_request_t *r) n = sizeof(NGINX_VER_BUILD) - 1; - + } else { - p = (u_char *) "nginx"; - n = sizeof("nginx") - 1; + p = (u_char *) "chicken"; + n = sizeof("chicken") - 1; } - + ngx_log_debug2(NGX_LOG_DEBUG_HTTP, c->log, 0, +diff --git a/src/http/ngx_http_header_filter_module.c b/src/http/ngx_http_header_filter_module.c +index 7899383..b8fcb8f 100644 +--- a/src/http/ngx_http_header_filter_module.c ++++ b/src/http/ngx_http_header_filter_module.c +@@ -47,7 +47,7 @@ ngx_module_t ngx_http_header_filter_module = { + }; + + +-static u_char ngx_http_server_string[] = "Server: nginx" CRLF; ++static u_char ngx_http_server_string[] = "Server: chicken" CRLF; + static u_char ngx_http_server_full_string[] = "Server: " NGINX_VER CRLF; + static u_char ngx_http_server_build_string[] = "Server: " NGINX_VER_BUILD CRLF; + diff --git a/node.build.sh b/node.build.sh index ee26fb1..aa6a655 100755 --- a/node.build.sh +++ b/node.build.sh @@ -27,7 +27,7 @@ npm_config_cache=${PREFIX}/cache EOF test -e "${BUILD_PKG}.tar.gz" || wget -nv "${SRCURL}" -O "${BUILD_PKG}.tar.gz" -test -d "${BUILD_PKG}" || mkdir -pv "${BUILD_PKG}" && tar -xf "${BUILD_PKG}.tar.gz" --strip-components=1 -C "${BUILD_PKG}" +test -d "${BUILD_PKG}" || { mkdir -pv "${BUILD_PKG}" && tar -xf "${BUILD_PKG}.tar.gz" --strip-components=1 -C "${BUILD_PKG}"; } . "${PREFIX}/profile" diff --git a/paperless-ngx.asgi.patch b/paperless-ngx.asgi.patch deleted file mode 100644 index 23fec7c..0000000 --- a/paperless-ngx.asgi.patch +++ /dev/null @@ -1,141 +0,0 @@ -diff --git a/src/paperless/asgi.py b/src/paperless/asgi.py -index 8d63c347a..45d8daa3a 100644 ---- a/src/paperless/asgi.py -+++ b/src/paperless/asgi.py -@@ -1,13 +1,134 @@ - import os - --from django.core.asgi import get_asgi_application -+# from django.core.asgi import get_asgi_application -+ -+import django -+from django.core.handlers.asgi import ASGIHandler -+ -+# https://code.djangoproject.com/ticket/36399 -+assert django.VERSION < (6,0), "Remove ASGIRequest backport." -+ -+class XXXASGIRequest(HttpRequest): -+ """ -+ Custom request subclass that decodes from an ASGI-standard request dict -+ and wraps request body handling. -+ """ -+ -+ # Number of seconds until a Request gives up on trying to read a request -+ # body and aborts. -+ body_receive_timeout = 60 -+ -+ def __init__(self, scope, body_file): -+ self.scope = scope -+ self._post_parse_error = False -+ self._read_started = False -+ self.resolver_match = None -+ self.path = scope["path"] -+ self.script_name = get_script_prefix(scope) -+ if self.script_name: -+ # TODO: Better is-prefix checking, slash handling? -+ self.path_info = scope["path"].removeprefix(self.script_name) -+ else: -+ self.path_info = scope["path"] -+ # HTTP basics. -+ self.method = self.scope["method"].upper() -+ # Ensure query string is encoded correctly. -+ query_string = self.scope.get("query_string", "") -+ if isinstance(query_string, bytes): -+ query_string = query_string.decode() -+ self.META = { -+ "REQUEST_METHOD": self.method, -+ "QUERY_STRING": query_string, -+ "SCRIPT_NAME": self.script_name, -+ "PATH_INFO": self.path_info, -+ # WSGI-expecting code will need these for a while -+ "wsgi.multithread": True, -+ "wsgi.multiprocess": True, -+ } -+ if self.scope.get("client"): -+ self.META["REMOTE_ADDR"] = self.scope["client"][0] -+ self.META["REMOTE_HOST"] = self.META["REMOTE_ADDR"] -+ self.META["REMOTE_PORT"] = self.scope["client"][1] -+ if self.scope.get("server"): -+ self.META["SERVER_NAME"] = self.scope["server"][0] -+ self.META["SERVER_PORT"] = str(self.scope["server"][1]) -+ else: -+ self.META["SERVER_NAME"] = "unknown" -+ self.META["SERVER_PORT"] = "0" -+ # Headers go into META. -+ for name, value in self.scope.get("headers", []): -+ name = name.decode("latin1") -+ if name == "content-length": -+ corrected_name = "CONTENT_LENGTH" -+ elif name == "content-type": -+ corrected_name = "CONTENT_TYPE" -+ else: -+ corrected_name = "HTTP_%s" % name.upper().replace("-", "_") -+ # HTTP/2 say only ASCII chars are allowed in headers, but decode -+ # latin1 just in case. -+ value = value.decode("latin1") -+ if corrected_name == "HTTP_COOKIE": -+ existing = self.META.get("HTTP_COOKIE") -+ if existing is not None: -+ value = existing + value -+ self.META["HTTP_COOKIE"] = value -+ else: -+ if corrected_name in self.META: -+ value = self.META[corrected_name] + "," + value -+ self.META[corrected_name] = value -+ # Pull out request encoding, if provided. -+ self._set_content_type_params(self.META) -+ # Directly assign the body file to be our stream. -+ self._stream = body_file -+ # Other bits. -+ self.resolver_match = None -+ -+ @cached_property -+ def GET(self): -+ return QueryDict(self.META["QUERY_STRING"]) -+ -+ def _get_scheme(self): -+ return self.scope.get("scheme") or super()._get_scheme() -+ -+ def _get_post(self): -+ if not hasattr(self, "_post"): -+ self._load_post_and_files() -+ return self._post -+ -+ def _set_post(self, post): -+ self._post = post -+ -+ def _get_files(self): -+ if not hasattr(self, "_files"): -+ self._load_post_and_files() -+ return self._files -+ -+ POST = property(_get_post, _set_post) -+ FILES = property(_get_files) -+ -+ @cached_property -+ def COOKIES(self): -+ return parse_cookie(self.META.get("HTTP_COOKIE", "")) -+ -+ def close(self): -+ super().close() -+ self._stream.close() -+ -+ -+class BackportASGIHandler(ASGIHandler): -+ request_class = XXXASGIHandler -+ - - # Fetch Django ASGI application early to ensure AppRegistry is populated - # before importing consumers and AuthMiddlewareStack that may import ORM - # models. - - os.environ.setdefault("DJANGO_SETTINGS_MODULE", "paperless.settings") --django_asgi_app = get_asgi_application() -+# django_asgi_app = get_asgi_application() -+ -+django.setup(set_prefix=False) -+ -+application = BackportASGIHandler() - - from channels.auth import AuthMiddlewareStack # noqa: E402 - from channels.routing import ProtocolTypeRouter # noqa: E402 diff --git a/paperless-ngx.build.sh b/paperless-ngx.build.sh index 27e492e..64dd899 100755 --- a/paperless-ngx.build.sh +++ b/paperless-ngx.build.sh @@ -13,7 +13,6 @@ set -u . ${PREFIX}/${BUILD_unpaper}/profile . ${PREFIX}/${BUILD_sqlite}/profile . ${PREFIX}/${BUILD_imagemagick}/profile -. ${PREFIX}/${BUILD_zbar}/profile . ${PREFIX}/${BUILD_python}/profile . ${PREFIX}/${BUILD_node}/profile . ${PREFIX}/${BUILD_leptonica}/profile @@ -69,8 +68,6 @@ pushd $PREFIX pip install uv -pipenv install - pushd src-ui npm update -g pnpm npm install -g corepack@latest @@ -86,28 +83,41 @@ pushd src echo "Installing Python requirements" uv export --quiet --no-dev --all-extras --format requirements-txt --output-file requirements.txt -uv pip install --system --no-python-downloads --python-preference system --requirements requirements.txt +# torch must come from the CPU-only index (as in upstream's Dockerfile), otherwise PyPI's CUDA build is pulled +uv pip install --system --no-python-downloads --python-preference system \ + --index https://pypi.org/simple \ + --index https://download.pytorch.org/whl/cpu \ + --index-strategy unsafe-best-match \ + --requirements requirements.txt ./manage.py collectstatic --clear --no-input --link ./manage.py compilemessages -# migration done by startstop.sh -# ./manage.py migrate +# the admin user needs the auth tables: migrate first (no-op on an existing, +# current DB; startstop.sh also migrates on every start). On an upgrade the +# fresh data/ created here is replaced by the copied one anyway. +# redis isn't running at build time, but v3's post_migrate (django-treenode) +# writes to the Django cache, which is redis-backed: use an in-process cache +# for these two build-time commands only. +export PAPERLESS_CACHE_BACKEND=django.core.cache.backends.locmem.LocMemCache +./manage.py migrate --no-input PAPERLESS_ADMIN_USER=${PAPERLESS_ADMIN_USER:-admin} PAPERLESS_ADMIN_PASSWORD=${PAPERLESS_ADMIN_PASSWORD:-$(</dev/urandom tr -dc '[:lower:][:upper:][:digit:]' | head -c64)} PAPERLESS_ADMIN_MAIL=${PAPERLESS_ADMIN_MAIL:-admin@localhost.lan} -./manage.py shell -c "from django.contrib.auth.models import User; User.objects.create_superuser('${PAPERLESS_ADMIN_USER}', '${PAPERLESS_ADMIN_MAIL}', '${PAPERLESS_ADMIN_PASSWORD}')" || true +# create only if missing (rebuilds keep the existing admin and its password) +./manage.py shell -c "from django.contrib.auth.models import User; User.objects.filter(username='${PAPERLESS_ADMIN_USER}').exists() or User.objects.create_superuser('${PAPERLESS_ADMIN_USER}', '${PAPERLESS_ADMIN_MAIL}', '${PAPERLESS_ADMIN_PASSWORD}')" +unset PAPERLESS_CACHE_BACKEND -for command in decrypt_documents \ - document_archiver \ +for command in document_archiver \ document_exporter \ document_importer \ mail_fetcher \ document_create_classifier \ document_index \ + document_llmindex \ document_renamer \ document_retagger \ document_thumbnails \ @@ -122,22 +132,13 @@ tee ${PROJECT}/bin/${command} <<_EOSH_ #!/bin/bash . ${PROJECT}/profile cd ${PREFIX}/src -python3 manage.py ${command} "$@" +python3 manage.py ${command} "\$@" _EOSH_ chmod u=rwx,g=rwx,o=rx ${PROJECT}/bin/${command} done popd -echo "Installing NLTK data" - -mkdir -pv ${PROJECT}/nltk - -python3 -W ignore::RuntimeWarning -m nltk.downloader -d "${PROJECT}/nltk" snowball_data -python3 -W ignore::RuntimeWarning -m nltk.downloader -d "${PROJECT}/nltk" stopwords -python3 -W ignore::RuntimeWarning -m nltk.downloader -d "${PROJECT}/nltk" punkt -python3 -W ignore::RuntimeWarning -m nltk.downloader -d "${PROJECT}/nltk" punkt_tab - popd } diff --git a/paperless-online-test.pdf b/paperless-online-test.pdf index 9428352..0542817 100644 Binary files a/paperless-online-test.pdf and b/paperless-online-test.pdf differ diff --git a/paperless.conf.build.sh b/paperless.conf.build.sh index 0275b9b..3afcde0 100755 --- a/paperless.conf.build.sh +++ b/paperless.conf.build.sh @@ -26,15 +26,22 @@ PAPERLESS_CONSUMPTION_DIR=${PROJECT}/consume PAPERLESS_DATA_DIR=${PROJECT}/data PAPERLESS_LOGGING_DIR=${LOGDIR} PAPERLESS_MEDIA_ROOT=${PROJECT}/media -PAPERLESS_NLTK_DIR=${PROJECT}/nltk PAPERLESS_SCRATCH_DIR=${TMPDIR} PAPERLESS_STATICDIR=${PREFIX}/static PAPERLESS_URL=${PAPERLESS_URL} PAPERLESS_CSRF_TRUSTED_ORIGINS=${PAPERLESS_CSRF_TRUSTED_ORIGINS:-${PAPERLESS_URL}} PAPERLESS_COOKIE_PREFIX=ngx-${BUILD_TAG}- +# nginx sets X-Real-IP; granian is only reachable via its unix socket, so the header is trustworthy +PAPERLESS_ALLAUTH_TRUSTED_CLIENT_IP_HEADER=X-Real-IP PAPERLESS_OCR_MODE=${PAPERLESS_OCR_MODE:-force} +# v2 behaviour (OCR_SKIP_ARCHIVE_FILE=never): always keep a PDF/A archive. +# v3's default "auto" would skip it for born-digital PDFs. Override in build.local. +PAPERLESS_ARCHIVE_FILE_GENERATION=${PAPERLESS_ARCHIVE_FILE_GENERATION:-always} +# v3 imports duplicates by default (v2 rejected them and left the file in consume/). +# true = reject like v2, but v3 also deletes the duplicate file from consume/. +PAPERLESS_CONSUMER_DELETE_DUPLICATES=${PAPERLESS_CONSUMER_DELETE_DUPLICATES:-false} PAPERLESS_OCR_DESKEW=false PAPERLESS_OCR_LANGUAGE=deu+eng # https://github.com/paperless-ngx/paperless-ngx/issues/416 diff --git a/pngquant.build.sh b/pngquant.build.sh index 1d56950..a64e58a 100755 --- a/pngquant.build.sh +++ b/pngquant.build.sh @@ -20,15 +20,12 @@ fi if [[ ! -d ${BUILD_PKG} ]]; then git clone --recursive https://github.com/kornelski/pngquant.git --single-branch ${BUILD_PKG} - # check version - pushd ${BUILD_PKG} - GITVER=pngquant-$(git describe --tags) - popd - if [ "${GITVER}" != "${BUILD_PKG}" ]; then - echo "${BUILD_PKG} should be updated to ${GITVER} update!" - fi fi +# pin to the exact commit encoded in BUILD_pngquant (...-g<sha>), incl. the libimagequant submodule +git -C "${BUILD_PKG}" checkout --detach "${BUILD_PKG##*-g}" +git -C "${BUILD_PKG}" submodule update --init --recursive + if [[ ! -e "${BUILD_PKG}".tar ]]; then tar cf "${BUILD_PKG}".tar "${BUILD_PKG}" fi diff --git a/profile.build.sh b/profile.build.sh index 55ec44e..41bc786 100755 --- a/profile.build.sh +++ b/profile.build.sh @@ -12,7 +12,6 @@ BUILD_TAG=${BUILD_TAG} PATH=$HOME/bin:${PROJECT}/bin:/bin:/sbin:/usr/bin:/usr/sbin -. ${PROJECT}/${BUILD_zbar}/profile . ${PROJECT}/${BUILD_jbig2enc}/profile . ${PROJECT}/${BUILD_jbig2dec}/profile . ${PROJECT}/${BUILD_sqlite}/profile diff --git a/python.build.sh b/python.build.sh index 6685406..aa72432 100755 --- a/python.build.sh +++ b/python.build.sh @@ -23,7 +23,7 @@ PATH="${PREFIX}"/bin:\${PATH} EOF test -e "${BUILD_PKG}.tar.gz" || wget -nv "${SRCURL}" -O "${BUILD_PKG}.tar.gz" -test -d "${BUILD_PKG}" || mkdir -pv "${BUILD_PKG}" && tar -xf "${BUILD_PKG}.tar.gz" --strip-components=1 -C "${BUILD_PKG}" +test -d "${BUILD_PKG}" || { mkdir -pv "${BUILD_PKG}" && tar -xf "${BUILD_PKG}.tar.gz" --strip-components=1 -C "${BUILD_PKG}"; } pushd "${BUILD_PKG}" @@ -37,7 +37,7 @@ export PKG_CONFIG_PATH SQLITE_PREFIX=$(pkg-config sqlite3 --variable=prefix) -LDFLAGS="-Wl,-rpath -Wl,-L${SQLITE_PREFIX}/lib" \ +LDFLAGS="-Wl,-rpath,${SQLITE_PREFIX}/lib" \ ./configure "${_conf[@]}" make -j "${NPROC}" @@ -52,7 +52,6 @@ ln -vfs pip3 "${PREFIX}/bin/pip" . "${PREFIX}/profile" pip3 install --prefix="${PREFIX}" --upgrade pip -python3 -m pip install --no-cache-dir --upgrade pipenv==2024.0.1 # for unpaper pip3 install meson # for paperless-ngx diff --git a/redis.build.sh b/redis.build.sh index 0c46112..507332d 100755 --- a/redis.build.sh +++ b/redis.build.sh @@ -23,7 +23,7 @@ REDISCLI_HISTFILE=${PREFIX}/.rediscli_history EOF test -e ${BUILD_PKG}.tar.gz || wget -nv ${SRCURL} -O ${BUILD_PKG}.tar.gz -test -d ${BUILD_PKG} || mkdir -pv ${BUILD_PKG} && tar -xf ${BUILD_PKG}.tar.gz --strip-components=1 -C ${BUILD_PKG} +test -d ${BUILD_PKG} || { mkdir -pv ${BUILD_PKG} && tar -xf ${BUILD_PKG}.tar.gz --strip-components=1 -C ${BUILD_PKG}; } pushd ${BUILD_PKG} diff --git a/sqlite.build.sh b/sqlite.build.sh index 82a9cc3..d19ba11 100755 --- a/sqlite.build.sh +++ b/sqlite.build.sh @@ -25,7 +25,7 @@ LD_LIBRARY_PATH=${PREFIX}/lib\${LD_LIBRARY_PATH:+:}\${LD_LIBRARY_PATH:-} EOF test -e ${BUILD_PKG}${BUILD_EXT} || wget -nv ${SRCURL} -O ${BUILD_PKG}${BUILD_EXT} -test -d ${BUILD_PKG} || mkdir -pv ${BUILD_PKG} && tar -xf ${BUILD_PKG}${BUILD_EXT} --strip-components=1 -C ${BUILD_PKG} +test -d ${BUILD_PKG} || { mkdir -pv ${BUILD_PKG} && tar -xf ${BUILD_PKG}${BUILD_EXT} --strip-components=1 -C ${BUILD_PKG}; } pushd ${BUILD_PKG} diff --git a/startstop.build.sh b/startstop.build.sh index 1c2e349..5063c43 100755 --- a/startstop.build.sh +++ b/startstop.build.sh @@ -203,6 +203,10 @@ srv_migrate() { ./manage.py migrate "\${_opts[@]}" fi + # both are no-ops when already current (first v3 start: Whoosh -> Tantivy rebuild) + ./manage.py document_index reindex --if-needed --no-progress-bar + ./manage.py document_llmindex migrate + popd } diff --git a/unpaper.build.sh b/unpaper.build.sh index 5053d22..f0dfef0 100755 --- a/unpaper.build.sh +++ b/unpaper.build.sh @@ -23,16 +23,11 @@ fi if [[ ! -d ${BUILD_PKG} ]]; then git clone "${SRCURL}" --single-branch "${BUILD_PKG}" - # check version - pushd "${BUILD_PKG}" - GITVER=$(git describe --tags) - popd - if [ "${GITVER}" != "${BUILD_PKG}" ]; then - echo "${BUILD_PKG} should be updated to ${GITVER} update!" - exit 1 - fi fi +# pin to the exact commit encoded in BUILD_unpaper (...-g<sha>) +git -C "${BUILD_PKG}" checkout --detach "${BUILD_PKG##*-g}" + if [[ ! -e ${BUILD_PKG}.tar ]]; then tar cf "${BUILD_PKG}".tar "${BUILD_PKG}" fi @@ -58,6 +53,9 @@ local _conf=( --pkg-config-path="${PKG_CONFIG_PATH}" ) +# a builddir left from an earlier build (shared BUILD_DIR, same unpaper +# version) is not reconfigured and fails after a meson upgrade +rm -rf builddir CFLAGS="${_cflags[*]}" meson setup "${_conf[@]}" builddir meson install -C builddir diff --git a/zbar.build.sh b/zbar.build.sh deleted file mode 100755 index 2c79771..0000000 --- a/zbar.build.sh +++ /dev/null @@ -1,66 +0,0 @@ -#!/bin/bash -{ -set -x -set -e -set -u - -. build.profile - -. ${PREFIX}/${BUILD_libjpegturbo}/profile -. ${PREFIX}/${BUILD_imagemagick}/profile - - -function B_ZBAR { - -SRCURL=${BUILD_zbar_SRCURL} - -BUILD_PKG=${BUILD_zbar} -BUILD_EXT=${SRCURL/${SRCURL%.*.*}/} - -PREFIX=${PREFIX}/${BUILD_PKG} - -mkdir -p $PREFIX - -cat >${PREFIX}/profile <<-EOF -PATH=${PREFIX}/bin:\$PATH -PKG_CONFIG_PATH=${PREFIX}/lib/pkgconfig\${PKG_CONFIG_PATH:+:}\${PKG_CONFIG_PATH:-} -LD_LIBRARY_PATH=${PREFIX}/lib\${LD_LIBRARY_PATH:+:}\${LD_LIBRARY_PATH:-} -EOF - - -test -e ${BUILD_PKG}${BUILD_EXT} || wget -nv ${SRCURL} -O ${BUILD_PKG}${BUILD_EXT} -test -d ${BUILD_PKG} || mkdir -pv ${BUILD_PKG} && tar -xf ${BUILD_PKG}${BUILD_EXT} --strip-components=1 -C ${BUILD_PKG} - -cd ${BUILD_PKG} - -autoreconf -vfi - -local _conf=( - --prefix ${PREFIX} - --disable-video - --disable-nls - --disable-doc - --enable-introspection=no - --with-gir=no - --with-x=no - --with-qt=no - --with-gtk=no - --with-dbus=no - --with-python=no - --with-xshm=no -) - -./configure "${_conf[@]}" - -make -j $NPROC -make install - -} - -pushd "${BUILD_DIR}" - -B_ZBAR - -popd - -}