Faster program starts, sudo/su without root, Rust, redis and nginx working, Go patch race fixed; 1.26.40

.github/workflows/build-release.yml

@@ -131,8 +131,8 @@ jobs:
           mkdir -p hook-libs
           # The code that is added to static Go programs (gosys/gosys.c), which the 64-bit hook carries
           scripts/build-gosys.sh
-          aarch64-linux-gnu-gcc -std=gnu11 -shared -fPIC -O3 -Wall -Wno-nonnull-compare -Wl,-z,max-page-size=16384 -o hook-libs/libcortex-hook-arm64.so app/src/main/cpp/cortex-hook.c -ldl
-          arm-linux-gnueabihf-gcc -std=gnu11 -shared -fPIC -O3 -Wall -Wno-nonnull-compare -Wl,-z,max-page-size=16384 -o hook-libs/libcortex-hook-arm.so app/src/main/cpp/cortex-hook.c -ldl
+          aarch64-linux-gnu-gcc -std=gnu11 -shared -fPIC -O3 -fno-delete-null-pointer-checks -Wall -Wno-nonnull-compare -Wl,-z,max-page-size=16384 -o hook-libs/libcortex-hook-arm64.so app/src/main/cpp/cortex-hook.c -ldl
+          arm-linux-gnueabihf-gcc -std=gnu11 -shared -fPIC -O3 -fno-delete-null-pointer-checks -Wall -Wno-nonnull-compare -Wl,-z,max-page-size=16384 -o hook-libs/libcortex-hook-arm.so app/src/main/cpp/cortex-hook.c -ldl
           # Root mode's helper: a static executable, run as root before any chroot exists
           aarch64-linux-gnu-gcc -std=gnu11 -static -O2 -Wall -Wextra -Werror -Wl,-z,max-page-size=16384 -o hook-libs/cortex-chroot-arm64.elf app/src/main/cpp/root/cortex-chroot.c
           arm-linux-gnueabihf-gcc -std=gnu11 -static -O2 -Wall -Wextra -Werror -Wl,-z,max-page-size=16384 -o hook-libs/cortex-chroot-arm.elf app/src/main/cpp/root/cortex-chroot.c
@@ -341,7 +341,8 @@ jobs:
 
       # Static Go programs (hook/go_programs.c, gosys/): built for AArch64, patched like the hook does
       # and run with qemu-user on a root file system of Ubuntu's own arm64 packages. It also makes
-      # the added code's header, which the native tests below compile the hook with.
+      # the added code's header, which the native tests below compile the hook with. The hook's own
+      # programs under qemu-user (test_hook_qemu.sh) run here too.
       - name: Test static Go programs
         if: github.event_name == 'pull_request' || github.event.inputs.dry_run == 'true'
         env:
@@ -351,6 +352,7 @@ jobs:
           sudo apt-get update -o Acquire::Retries=3 -qq
           sudo apt-get install -y --no-install-recommends qemu-user gcc-aarch64-linux-gnu libc6-dev-arm64-cross binutils-aarch64-linux-gnu
           bash tests/scripts/test_go_programs.sh
+          CORTEX_REQUIRE_QEMU_TESTS=1 bash tests/scripts/test_hook_qemu.sh
 
       - name: Run Native & Shell Script Tests and Verify PR Debug Build
         if: github.event_name == 'pull_request'

.github/workflows/compat-nightly.yml

@@ -13,9 +13,18 @@ on:
         description: "The release to test (its tag; empty: the latest)"
         type: string
         default: ""
+      only:
+        description: "Only these packages (names of the table, separated by spaces; empty: all)"
+        type: string
+        default: ""
+      apk_run:
+        description: "Test the APKs of this run of Build and Release (a dry run's artifact) instead of a release"
+        type: string
+        default: ""
 
 permissions:
   contents: read
+  actions: read
 
 concurrency:
   group: package-compatibility
@@ -34,12 +43,20 @@ jobs:
         env:
           GH_TOKEN: ${{ github.token }}
           TAG: ${{ inputs.release }}
+          APK_RUN: ${{ inputs.apk_run }}
         run: |
           set -euo pipefail
           mkdir -p apks
-          # Without a tag, gh takes the latest release ($TAG is left unquoted for that)
-          gh release download $TAG --repo "$GITHUB_REPOSITORY" --pattern 'Cortex-arm64-v8a-signed.apk' --dir apks
-          gh release view $TAG --repo "$GITHUB_REPOSITORY" --json tagName --jq .tagName > tag.txt
+          if [ -n "$APK_RUN" ]; then
+            # The APKs a run of Build and Release made (a dry run keeps them for a day)
+            gh run download "$APK_RUN" --repo "$GITHUB_REPOSITORY" --name cortex-release-apks --dir apk-run
+            cp "$(find apk-run -name 'Cortex-arm64-v8a-signed.apk' | head -n 1)" apks/
+            echo "run $APK_RUN" > tag.txt
+          else
+            # Without a tag, gh takes the latest release ($TAG is left unquoted for that)
+            gh release download $TAG --repo "$GITHUB_REPOSITORY" --pattern 'Cortex-arm64-v8a-signed.apk' --dir apks
+            gh release view $TAG --repo "$GITHUB_REPOSITORY" --json tagName --jq .tagName > tag.txt
+          fi
           echo "Testing $(cat tag.txt)"
           ls -l apks
 
@@ -51,6 +68,8 @@ jobs:
 
       - name: Install and try the packages on the emulator
         uses: reactivecircus/android-emulator-runner@a421e43855164a8197daf9d8d40fe71c6996bb0d # v2.38.0
+        env:
+          E2E_COMPAT_ONLY: ${{ inputs.only }}
         with:
           api-level: 34
           arch: x86_64

CHANGELOG.md

@@ -4,6 +4,85 @@ What changed in each version of Cortex Terminal, newest first. The release workf
 of a GitHub release from the section of its version here. Older versions are described on the
 [GitHub releases](https://github.com/ssfuisu/cortex/releases) page.
 
+## 1.26.40 (2026-10-25)
+
+### Fixed
+
+- **redis, nginx and PostgreSQL install.** The Ubuntu image kept only `root`, `daemon` and `cortex` in
+  `/etc/passwd` and `/etc/group`, but the scripts of packages use the system users and groups every
+  Ubuntu has (`man`, `www-data`, `adm`, `tty`, `staff`...): `apt install redis-server`, `nginx` or
+  `postgresql` (and any package whose man page the man-db trigger makes, with `setpriv --reuid=man`)
+  stopped halfway. Cortex now adds what is missing from the list Ubuntu's own `base-passwd` keeps in
+  the image (`/usr/share/base-passwd`), with locked entries in `/etc/shadow` and `/etc/gshadow`, and
+  changes nothing that is there. redis then starts and answers (`PONG`) in the emulator test, nginx
+  installs and serves a page. PostgreSQL installs too, but its server refuses to run as root and an
+  app cannot become another user, so the cluster the package would make is not made outside root
+  mode (`psql` works). MariaDB (`mariadb-server`) installs, starts and answers queries.
+- **ripgrep, `rustc` and `cargo` crashed at the first file they opened.** Rust asks whether `statx`
+  exists by calling it with a NULL path; glibc's header says the path is never NULL, so the compiler
+  deleted the hook's check. Both programs now work (emulator test: ripgrep and Rust's hello world).
+- **`chfn` and `adduser --gecos` (which PostgreSQL's installation runs) failed** ("Authentication
+  failure"): Ubuntu's libraries found Android's SELinux and PAM's root check then refused root.
+  SELinux is off for Ubuntu's programs, and the shadow database is read from the image's
+  `/etc/shadow` (`getspnam`).
+- **`go run` and `go build` could crash at random** (a segmentation fault in the Go tool): the
+  scratch memory that Cortex's change in a Go program uses for the paths it rewrites was shared by
+  goroutines that ran at the same time. Each slot now belongs to the goroutine that took it until
+  it is done (the build runs a stress test: 96 goroutines read 200 files 300 times each). And the
+  slot of a program that was started (by a vfork child, whose execve does not return) is given back
+  when the goroutine goes on; before, a program that started programs from more than 64 goroutines
+  at once could wait for a slot for good.
+- **`sudo` and `su` on a phone without root said "root not found".** `sudo apt install x`, the first
+  line of nearly every guide, now runs the command as it is (Cortex's shell is root inside Ubuntu as
+  far as the programs can tell); `sudo -u USER`, `sudo -i`, `su USER` (in the shell that user has),
+  `su -c CMD` and `su -` work through `runuser` and the shell. On a phone that gives Cortex real root they still ask Android for it.
+- **After another color scheme was chosen, the old one came back as a block** (#29): close the
+  keyboard and the rows the terminal grew into showed the previous scheme's background, in the size
+  of the keyboard. The cells of a session keep the colors the scheme had when they were written, and
+  a change of scheme only changed what was drawn next. Cortex now carries the new scheme into the
+  screen, the scrollback and the other screen of every session when it is chosen (the default
+  colors and the 16 ANSI ones; the 256-color and 24-bit colors of programs are the same in every
+  scheme).
+- A program that called `putenv()` or `setenv()` and then `execv()` before it used any file could
+  crash in the exec (the hook set its variables only then, and the block the program had passed was
+  freed by that). The hook now starts before the program does.
+
+### Faster
+
+- **Starting programs.** What the library adds to starting a program is about half of what it was.
+  The look for the ARM translator's memory files (x86 emulators and Chromebooks; a phone has none)
+  asked about each descriptor from 3 to 63, 61 system calls in every program; it now reads
+  `/proc/self/fd` once, and only in the first program of a session: the ones it starts are told
+  there is nothing to find (`CORTEX_NT=0`). `ld.so` is given every library directory once instead
+  of twice, and the environment is prepared with a fifth of the comparisons. Measured against 1.26.39
+  with the same benchmark on one x86 machine: the library's share of starting `ld.so` 457 µs ->
+  189 µs, starting a program as a shell does 16% faster, a script 11%. Counted with qemu-user a
+  program that does nothing makes 177 system calls instead of 236 (170 without the library), a
+  shell starting a program five times 548 instead of 916, a login shell 1,111 instead of 1,357,
+  `dpkg -l` 569 instead of 752, `ls -l` 713 instead of 837.
+- **Less waking of the phone.** The watch for local web servers looked every 3 seconds even when
+  nothing was listening, and on Android 10 and newer every look asked `/proc/net`, which they
+  refuse (each refusal a line in the system log). There it now looks every 15 seconds when there is
+  nothing to follow and asks `/proc/net` again only every 10 minutes; the hook's own reports still
+  wake it at once, and a server that does not answer yet is looked at again within 3 seconds.
+- No warning from the compiler or the Android SDK that the code can avoid: the unused function and
+  variable, `stopForeground(true)` and `scaledDensity` are replaced.
+
+### For developers
+
+- `tests/scripts/test_hook_qemu.sh` runs the hook under qemu-user with small programs (statx with a
+  NULL path, SELinux, the shadow file, the translator's descriptors, `CORTEX_NT`, `putenv` +
+  `execv`); the build runs it. `tests/scripts/test_go_programs.sh` has a stress test of the Go
+  change. `tests/scripts/test_shell_scripts.sh` covers `sudo` and `su` without root.
+- The package compatibility run takes the packages to test (`only`; a name that is not in the table,
+  or a list that is not names, fails the run instead of testing nothing or everything) and the APKs
+  of a dry run (`apk_run`); the nginx test installs the `curl` it needs, and PostgreSQL has a client row (passes)
+  next to the server row (a known gap outside root mode: the server refuses to run as root, and an
+  app cannot become another user).
+- Known gaps, seen in the x86 emulator only: Go programs and the JVM's compiler crash inside the
+  emulator's ARM translator (a Go program also when it is not changed by Cortex; they run under
+  qemu-user and on phones).
+
 ## 1.26.39 (2026-10-25)
 
 ### New

README.md

@@ -122,6 +122,8 @@ Measured on one 8-core Android phone (Linux 4.14, one big core, median of 9 inte
 
 **Starting a program** (what a shell does for every command): the library made 19 system calls before the program started and now makes 7, no longer changes the permissions of `ld.so` each time, and lists the library directories that hold nearly every library first, so a program no longer pays two failed lookups per library (`apt-cache` made 34 of them, `bash` 4, now none). Starting a program through the library takes 769 µs instead of 903 µs; starting `ld.so` by hand without the library takes 491 µs.
 
+**Starting a program in 1.26.40**: measured against the previous release on one x86 Linux machine with the same benchmark (`tests/bench/run-micro.sh`, 25 interleaved runs each): what the library adds to starting `ld.so` with a program fell from 457 µs to 189 µs, and starting a program the way a shell does is 16% faster (1,253 µs to 1,053 µs; a script 11%). The library no longer looks into `/proc/self/fd` in every program (the first program of a session looks; those it starts are told by `CORTEX_NT=0` that there is nothing to find, as on any phone that runs ARM code itself), no longer repeats the library directories in the `--library-path` it gives `ld.so` (which went through each of them twice), and compares only the names that begin with the same letter when it prepares the environment (this was 81,000 instructions before every start, now 17,000). Counted with qemu-user, a program that does nothing makes 177 system calls instead of 236 (170 without the library), a shell that starts a program five times 548 instead of 916, a login shell 1,111 instead of 1,357, `dpkg -l` 569 instead of 752. These figures come from an x86 machine and from qemu-user; a phone has not been measured, and shows other times.
+
 **Whole commands** (`find`, `ls -lR`, `dpkg`, `tar`, `grep -r`, compiling a C file, reinstalling 36 packages) differ from the previous version by less than the up to ±10% variation of the phone from run to run, with two exceptions that held in two separately designed measurements: `ls -lR` is about 20% faster (it no longer prints an error line for every symbolic link) and a shell that starts a program a hundred times is 5 to 14% faster. The library was a small part of the other commands to begin with. Numbers differ from phone to phone, so compare builds on the same device.
 
 To measure your own device (inside Cortex, with a C compiler installed):
@@ -286,7 +288,10 @@ Use `apt` like on any Ubuntu system, for example `apt install git python3 build-
 Not yet. Cortex is a terminal; there is no X11 or Wayland server.
 
 **Will every Ubuntu package work?**
-Most command-line tools do. Packages that depend on kernel features Android does not allow for apps (for example systemd services or privileged device access) cannot work. On a rooted phone, [root mode](#root-mode) removes the limits that come from running as an app; systemd cannot be the init system in either mode.
+Most command-line tools do. Packages that depend on kernel features Android does not allow for apps (for example systemd services or privileged device access) cannot work. An app cannot become another user, so a server that refuses to run as root cannot run in normal mode: PostgreSQL's packages install (`psql` works against a server elsewhere), but its own cluster is not created and the server not started; use [root mode](#root-mode) for that. MariaDB (`apt install mariadb-server`, then `mariadbd-safe &` and `mariadb`) and redis run in normal mode. On a rooted phone, [root mode](#root-mode) removes the limits that come from running as an app; systemd cannot be the init system in either mode.
+
+**Does `sudo` work?**
+Yes. In normal mode Cortex's shell is root as far as Ubuntu's programs can tell, so `sudo apt install x` and the other commands from guides run the command as it is (`sudo -u USER` and `su USER` go through `runuser`). On a rooted phone that gives Cortex root, `sudo` and `su` still ask Android for real root, as before; in [root mode](#root-mode) they work as on a Linux PC.
 
 **Can I compile C and C++ programs?**
 Yes. `apt install build-essential` installs gcc, g++ and make, and the C library headers and libraries (`libc6-dev`, `linux-libc-dev`) come with the app, so `-dev` packages such as `zlib1g-dev` or `python3-dev` install too. Statically linked programs (`gcc -static`) run as well: before such a program starts for the first time, Cortex patches the system calls that Android's seccomp filter forbids for apps in its file (once; the file keeps its times and mode). One limit: Cortex keeps the C library itself (`libc6`, `libc-bin`) at the version it ships, because Ubuntu's own build does not start under Android's filter, so `apt upgrade` leaves those packages alone.

app/build.gradle.kts

@@ -16,8 +16,8 @@ android {
         // from the app's writable data directory (filesDir). Android 10+ (targetSdk >= 29)
         // enforces SELinux W^X restrictions preventing execution from writable app data storage.
         targetSdk = 28
-        versionCode = 12639
-        versionName = "1.26.39"
+        versionCode = 12640
+        versionName = "1.26.40"
         // UbuntuOnDeviceTest (app/src/androidTest), on Firebase Test Lab
         testInstrumentationRunner = "androidx.test.runner.AndroidJUnitRunner"