Skip to content

fix(images): one router's extras must not cancel the whole lab - #14

Merged
VizzleTF merged 1 commit into
mainfrom
fix/12-extras-should-not-cancel-the-lab
Sep 4, 2026
Merged

fix(images): one router's extras must not cancel the whole lab#14
VizzleTF merged 1 commit into
mainfrom
fix/12-extras-should-not-cancel-the-lab

Conversation

@VizzleTF

@VizzleTF VizzleTF commented Sep 4, 2026

Copy link
Copy Markdown
Member

docker compose build puts every router in one buildkit solve, and buildkit cancels the
solve on the first target that fails. The extras step ran under set -eu with no guard,
so one unusable staged file failed the whole lab.

Measured on a three-router stand — an .ipk with an unsatisfiable dependency:

#14 ERROR: … did not complete successfully: exit code: 255
#16 CANCELED
#13 CANCELED
target bad2410: failed to solve: …
owlab: build failed: exit status 1

Both release lines do it: the same stand with an .apk that apk refuses
(unable to select packages: breaks: luci-base-…[ucode>=2026.02.27]) exited 27 and
cancelled its neighbour. It is set -eu, not the package manager.

Neither docker compose build (v5.1.2) nor docker buildx bake (v0.33.0) has a
--keep-going, so tolerance has to live in the RUN itself.

Tolerance is not silence

The step installs the set in one command as before — that is what lets the package
manager resolve among the staged files. Only if that fails does it retry them one at a
time, in staged order, which is the order the developer wrote them in: a dependent
package staged after the one it needs installs second and resolves against a router that
already has it.

What still will not install is named, not swallowed:

  • in the build log: owlab: FAILED to install: <file>;
  • recorded in /etc/owlab/extras-failed inside the image;
  • under the owlab up ready table: ! bad2410 is running WITHOUT owlab-bad-extra_1.0_all.ipk,
    with owlab up --rebuild bad2410 as the next step;
  • owlab test fails the router on a new extra_packages step, before any assertion runs.

Extras are files the developer staged and explicitly asked for. Skipping one quietly and
returning a green build would be worse than failing — a check that could not run counts
as failed here, not as skipped.

After

Image owlab-lab12-bad2410:latest  Built
Image owlab-lab12-ok2512:latest   Built
Image owlab-lab12-ok2410:latest   Built
#11 4.707 owlab: FAILED to install: owlab-bad-extra_1.0_all.ipk
! bad2410 is running WITHOUT owlab-bad-extra_1.0_all.ipk
EXIT=1

No CANCELED. The healthy path is unchanged: the same stand with working extras exits 0
with no ! line, the packages are installed, and a stale extras-failed from an earlier
build is cleared at the top of the layer — this Dockerfile also runs on top of an
already-built owlab image.

Contract change

owlab up can now exit 1 with the lab running. That is deliberate and documented in
docs/reference*.md, docs/troubleshooting*.md and the changelog: a router missing a
package the developer asked for is not a successful run, and the lab is still up so it
can be inspected.

Fixes #12

`docker compose build` puts every router in one buildkit solve, and buildkit
cancels the solve on the first target that fails. The extras step ran under
`set -eu` with no guard, so one unusable staged file turned into a failed lab:
measured on a three-router stand, an .ipk with an unsatisfiable dependency
exited 255 and took `#16 CANCELED`, `#13 CANCELED` and `owlab: build failed`
with it. Both release lines did it -- it is `set -eu`, not the package manager:
the same stand with a .apk apk answers `unable to select packages` exited 27
and cancelled its neighbour. Neither `docker compose build` nor
`docker buildx bake` has a --keep-going, so the tolerance has to live in the
RUN.

The extras step now installs the set as before and, only if that fails,
installs each file on its own in the staged order, so the good ones still land.
The successful path is unchanged.

Tolerance without a report would be worse than the cancelled builds it
replaces, so what still fails is named once per package and recorded in
/etc/owlab/extras-failed. `owlab up` reads that back off each running router
and prints it under the ready table, exiting non-zero; `owlab test` gains an
`extra_packages` step that fails the router before any assertion runs against a
box missing what it was told to have.

Fixes #12
@VizzleTF
VizzleTF merged commit 417d71d into main Sep 4, 2026
14 checks passed
@VizzleTF
VizzleTF deleted the fix/12-extras-should-not-cancel-the-lab branch September 4, 2026 16:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

owlab up: a stale opkg index in a cached layer makes one router's extras kill the whole build

1 participant