Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,29 @@ Versioning: [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
today keeps working across minor and patch releases; a change that would break
one waits for a major.

## [Unreleased]

### Fixed

- **A 24.10 image build no longer fails on a package nobody asked for.** `opkg update`
ran in an earlier layer than the install of the `extra_packages:` files, and buildkit
keeps that layer for as long as the release and the feed package list hold — while the
install layer re-runs on every change to the staged files. The index the install read
could therefore be months old, and a pin does not make that safe: OpenWrt rebuilds the
packages inside `releases/24.10.8/` in place, under the same version string. Measured
on 2026-09-04 against an image built 2026-07-29, `bash 5.2.37-r1` was 473650 bytes /
`f1872e60…` in the image's index and 473647 bytes / `20eaa220…` on the server, which
opkg reports as `Checksum or size mismatch for package bash` and the build as
`exit code: 255`. `opkg update` now runs in the same layer as the install, and
`owlab test` refreshes the index before installing a local file too — a package
installed by path still resolves its dependencies out of the index. Nothing that
cached before stops caching: the refresh is inside the branch that has something to
install, in the layer that was re-running anyway, and costs about 1.6 s and a megabyte
of gzipped indexes per opkg router. The apk line needs none of this, measured:
apk-tools 3.0.5 revalidates a cached index older than `--cache-max-age` (4 hours by
default) on its own, and re-downloaded every APKINDEX before resolving in a month-old
image.

## [0.5.5] - 2026-09-04

### Fixed
Expand Down
15 changes: 14 additions & 1 deletion cmd/owlab/test.go
Original file line number Diff line number Diff line change
Expand Up @@ -426,7 +426,20 @@ func (a *app) testInstall(ctx context.Context, r *config.Router, run syncpkg.Exe
}

cmd := pre + pkgmgr.Install(pm, installArgs, pkgmgr.Options{
Update: len(feed) > 0 || feedSrc != nil,
// Unconditional, not just when a feed was added. A package installed
// by path still resolves its DEPENDENCIES out of the index, and the
// index a router boots with is as old as the image layer that ran
// `opkg update` — which is cached, and on a reused image is whatever
// the feed looked like the week it was built. OpenWrt rebuilds the
// packages inside a pinned point release without bumping a version,
// so an index that old names bytes the server no longer serves:
//
// opkg_install_pkg: Checksum or size mismatch for package bash.
//
// Free on the apk line, which revalidates a cached index older than
// four hours by itself; the cost on the opkg line is one index fetch
// per router per run.
Update: true,
// A locally built package carries no signature the router's keyring
// knows, and there is no key it could carry that would.
Untrusted: len(files) > 0,
Expand Down
52 changes: 51 additions & 1 deletion docs/troubleshooting.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,48 @@ when you hit them, because almost none of them presents as its cause.

---

## "The image build fails"

### The index in the image is older than the layer that uses it

**Symptom.** `owlab up` fails while installing an `extra_packages:` file on a
24.10 router, and names a package nobody asked for:

```
opkg_install_pkg: Checksum or size mismatch for package bash. Either the opkg
or the package index are corrupt. Try 'opkg update'.
owlab: build failed: exit status 1
```

`bash` here is a **dependency** of the staged package, not the staged package.
Only the opkg line does this.

**Cause.** `opkg update` ran in an earlier layer than the install. Buildkit
keeps that layer for as long as the release and the feed package list hold,
while the install layer re-runs on every change to the staged files — so the
index can be months older than the install reading it.

A pinned point release does not save you, because it is not frozen: OpenWrt
rebuilds the packages inside `releases/24.10.8/` in place, without bumping a
version. Measured on 2026-09-04 against an image built 2026-07-29:

```
index in the image bash 5.2.37-r1 Size 473650 SHA256 f1872e60...
downloads.openwrt.org bash 5.2.37-r1 Size 473647 SHA256 20eaa220...
```

Same version, different bytes — which is exactly what opkg reports as a
checksum mismatch.

**Fix.** `opkg update` runs in the same layer as the install, and `owlab test`
refreshes the index before installing a local file too. apk needs neither:
apk-tools 3.0.5 revalidates a cached index older than `--cache-max-age`
(4 hours by default) on its own, so `apk add` in a month-old image
re-downloads every APKINDEX before it resolves anything. opkg has no such
policy — it reads whatever the last `opkg update` left and never asks.

---

## "The router never answers on its published port"

### The container's interface is in no firewall zone
Expand Down Expand Up @@ -365,7 +407,7 @@ looking at is worth doing before debugging anything else.

---

## Two upstream behaviours worth knowing
## Three upstream behaviours worth knowing

### `kmods` must stay in the feed list

Expand All @@ -381,3 +423,11 @@ apk records hard pins in `/etc/apk/world` (`base-files=1707~4ccb782af7`).
Pointing a 25.12.4 rootfs at the 25.12.5 feed fails every install with
`breaks: world[...]`. owlab does not rewrite the feeds at all, which is the
simplest way to keep this right.

### A pinned point release is still a moving target

The pin fixes which release you install from, not which bytes that release
serves. `releases/24.10.8/packages/` is rebuilt in place — the `Packages`
index for a release cut months ago was last modified yesterday — and a package
can be replaced under the same version string. So an index is only good for as
long as it is fresh, and every place that reads one has to refresh it first.
54 changes: 53 additions & 1 deletion docs/troubleshooting_ru.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,50 @@

---

## «Сборка образа падает»

### Индекс в образе старше слоя, который им пользуется

**Симптом.** `owlab up` падает на установке файла из `extra_packages:` на
роутере 24.10 и называет пакет, который никто не просил:

```
opkg_install_pkg: Checksum or size mismatch for package bash. Either the opkg
or the package index are corrupt. Try 'opkg update'.
owlab: build failed: exit status 1
```

`bash` здесь — **зависимость** установленного пакета, а не он сам. Происходит
только на линии opkg.

**Причина.** `opkg update` выполнялся в более раннем слое, чем установка.
Buildkit держит тот слой ровно столько, сколько не меняются релиз и список
пакетов из фида, а слой установки перезапускается при любом изменении
подготовленных файлов — так что индекс бывает на месяцы старше установки,
которая его читает.

Пин на точечный релиз не спасает: релиз не заморожен. OpenWrt пересобирает
пакеты внутри `releases/24.10.8/` на месте, не меняя версию. Замерено
2026-09-04 против образа, собранного 2026-07-29:

```
индекс в образе bash 5.2.37-r1 Size 473650 SHA256 f1872e60...
downloads.openwrt.org bash 5.2.37-r1 Size 473647 SHA256 20eaa220...
```

Одна и та же версия, разные байты — ровно то, о чём opkg говорит как о
несовпадении контрольной суммы.

**Что сделано.** `opkg update` выполняется в том же слое, что и установка, а
`owlab test` обновляет индекс и перед установкой локального файла. apk не
нужно ни то, ни другое: apk-tools 3.0.5 сам перепроверяет кэшированный индекс
старше `--cache-max-age` (по умолчанию 4 часа), поэтому `apk add` в
месячном образе перекачивает все APKINDEX прежде, чем что-либо разрешать. У
opkg такой политики нет — он читает то, что оставил последний `opkg update`, и
ничего не спрашивает.

---

## «Роутер вообще не отвечает на опубликованном порту»

### Интерфейс контейнера не попал ни в одну зону firewall
Expand Down Expand Up @@ -360,7 +404,7 @@ VM не грузится или виснет на середине, тогда

---

## Два поведения апстрима, которые стоит знать
## Три поведения апстрима, которые стоит знать

### `kmods` должен остаться в списке фидов

Expand All @@ -376,3 +420,11 @@ apk записывает жёсткие пины в `/etc/apk/world` (`base-file
Направив rootfs 25.12.4 на фид 25.12.5, вы получите падение любой установки с
`breaks: world[...]`. owlab не переписывает фиды вообще — самый простой способ
не сломать это.

### Пин на точечный релиз всё равно движется

Пин фиксирует, из какого релиза вы ставите, но не то, какие байты этот релиз
отдаёт. `releases/24.10.8/packages/` пересобирается на месте — индекс
`Packages` релиза, вышедшего месяцы назад, изменён вчера, — и пакет может быть
заменён под той же строкой версии. Значит, индекс годен ровно пока он свежий, и
каждое место, которое его читает, обязано сначала его обновить.
29 changes: 29 additions & 0 deletions images/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -135,6 +135,34 @@ COPY extra/ /tmp/owlab-extra/
# installing one at a time makes that a hard error ("unable to select
# packages") for anything but the luckiest ordering. Handed the whole set, the
# package manager resolves among them.
#
# opkg's index is refreshed HERE, in the layer that uses it, and that is the
# whole reason the line exists. The `opkg update` above runs in a different
# RUN, whose inputs are the release and the feed package list; buildkit keeps
# that layer for as long as those hold, while this one re-runs on every change
# to the staged files. The two are routinely months apart.
#
# That is fatal because a pinned point release is not frozen. OpenWrt rebuilds
# the packages inside `releases/24.10.8/` in place, without bumping a version,
# so an index from an older build names bytes the server no longer serves:
#
# image built 2026-07-29 bash 5.2.37-r1 Size 473650 SHA256 f1872e60...
# downloads.openwrt.org bash 5.2.37-r1 Size 473647 SHA256 20eaa220...
#
# opkg_install_pkg: Checksum or size mismatch for package bash. Either the
# opkg or the package index are corrupt. Try 'opkg update'.
#
# It bites here rather than above because a local package is installed by
# path, but its DEPENDENCIES still come out of the index.
#
# apk gets no equivalent line, measured: apk-tools 3.0.5 revalidates a cached
# index older than --cache-max-age (4 hours by default) on its own, so an
# `apk add` inside a month-old image re-downloads every APKINDEX before it
# resolves anything, and a second one seconds later does not. opkg has no such
# policy — it reads whatever the last `opkg update` left and never asks.
#
# Inside the `if`, so a build with no out-of-feed package fetches nothing, and
# in this layer only, so nothing that cached before still caches.
RUN set -eu; \
dir="/tmp/owlab-extra/$ROUTER_ID"; \
set -- ; \
Expand All @@ -144,6 +172,7 @@ RUN set -eu; \
if [ "$PKG_MANAGER" = "apk" ]; then \
apk add --allow-untrusted --force-overwrite "$@"; \
else \
opkg update; \
opkg install --force-overwrite "$@"; \
fi; \
fi; \
Expand Down