Skip to content

1.9.0: advert suppression you can see, undo and teach — and faster #10

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
antoine-debroye merged 1 commit into from
Sep 16, 2026
Merged

1.9.0: advert suppression you can see, undo and teach — and faster #10

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
58 changes: 58 additions & 0 deletions .github/workflows/docker.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -159,3 +159,61 @@ jobs:

- name: Show what was published
run: docker buildx imagetools inspect ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:latest

# The NVIDIA GPU image (Dockerfile.cuda): amd64 only, published under its own tags
# (`1.9.0-cuda`, `1.9-cuda`, `latest-cuda`) and never part of the multi-arch manifest
# above. It depends on the tests passing but nothing depends on it: compiling CUDA
# takes a while, and a failure here must not stop the regular image being published.
# No runner here has a GPU, so this proves the image builds and its CPU fallback works;
# it cannot prove the GPU path. That is checked on real hardware (README, GPU section).
build-cuda:
needs: test
runs-on: ubuntu-latest
continue-on-error: true
permissions:
contents: read
packages: write
steps:
- uses: actions/checkout@v5

# The CUDA toolchain and build cache need more room than a runner has free.
- name: Free disk space
run: |
sudo rm -rf /usr/share/dotnet /usr/local/lib/android /opt/ghc /opt/hostedtoolcache/CodeQL
df -h /

- uses: docker/setup-buildx-action@v3

- name: Log in to GHCR
if: github.event_name != 'pull_request'
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Work out the tags
id: meta
uses: docker/metadata-action@v5
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
flavor: |
latest=false
suffix=-cuda,onlatest=true
tags: |
type=raw,value=latest,enable={{is_default_branch}}
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
type=sha,format=short

- name: Build and push
uses: docker/build-push-action@v6
with:
context: .
file: Dockerfile.cuda
platforms: linux/amd64
push: ${{ github.event_name != 'pull_request' }}
provenance: false
tags: ${{ steps.meta.outputs.tags }}
cache-from: type=gha,scope=cuda
cache-to: type=gha,mode=max,scope=cuda
104 changes: 104 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,110 @@ Updating is changing the image tag and redeploying. The database migrates itself
forward on start, and no release so far has needed anything else — where a release
changes what your listeners see, it says so.

## 1.9.0 — 2026-09-16

### Changed — adverts you can see, and put back

- **One page shows what was cut, from which episode, and why.** The Adverts page is now
three things top to bottom: four figures (episodes cut, minutes removed, waiting, held);
*What SelfPod cuts* — every rule in force, each with how many episodes it was heard in and
one button to forget it; and *Episode by episode* — each episode as a bar with what was cut
and what is waiting marked on it, one sentence of reason per stretch, and a ▶ that plays the
original from three seconds before to three seconds after. The ~50 status phrases and 12
button labels of 1.8 are now four states for a stretch and six buttons.
- **Put a cut back in one episode, or everywhere.** *Restore here* leaves that stretch in that
episode only; the rule goes on cutting the others. *Restore everywhere and stop* undoes the
rule itself. Both take seconds: a decision no longer waits for a pass.
- **Teach it directly.** Type the words the programme starts or ends with and press Teach —
no more hunting for them three menus deep in a transcript. On any episode, give a time range
and say whether it is an advert or the station jingle; tap the bar, or take the position from
the player. Words heard there are remembered for later episodes; without words, the stretch is
cut from that episode.
- **Hear what was removed.** The episode page plays the original as well as the published copy.
- **In automatic mode the station jingle is cut without asking** when SelfPod hears it in every
recent episode — measured on the show this was built for, six out of six. It is listed as an
automatic cut with Forget; forgetting it means it is not proposed again. Review mode still asks.
- The episode table has an *Adverts* column (cut 0:39 / waiting / held …) and the dashboard card
says how much was cut, so nothing needs opening to see where things stand.
- "Sounds like a sponsor read, heard once" is no longer a row to decide about. It is highlighted
in the episode's words, where it can be taught from.

### Changed — faster, and without waiting for the clock

- **A new episode is looked at as soon as it arrives.** A pass used to start only on the
scheduler's tick, so a new episode sat out of the feed for up to a whole rescan interval
before anything read it. A scan that finds a new or changed episode now starts that
show's pass at once, and a pass runs after the scan at startup. The tick stays, as the
fallback. Asking for a show's pass again while one is still waiting to start is the same
pass, so a subscription backfill of a hundred downloads is not a hundred passes.
- **An unchanged file is not read.** Every pass used to read and hash every episode of
every show to find out that nothing had changed — the whole library off the disk every
few minutes. The size and modification time now answer that; the digest is still the
authority when they disagree.
- **The comparing is done off the thread that serves listeners.** The searches that
compare episodes with each other are quadratic and ran where downloads and the health
check are answered; on a show a few months old that was long enough for a NAS to
restart the app mid-pass. They now run in a worker thread.
- **Decisions no longer wait for a pass.** Removing, keeping, restoring and forgetting
change only what is already known, so they cut and publish straight away — seconds —
instead of running a whole pass first, and never queue behind another show being heard.
- **The recogniser uses the machine it is on.** `WHISPER_THREADS` now defaults to the
physical cores less one (seven on an eight-core i7) rather than two, and it runs at nice
10 rather than 15 (`WHISPER_NICE`). Up to eight already-published episodes are read again
per pass after a change of model (`WHISPER_BACKFILL_PER_RUN`), rather than two.
- **An NVIDIA GPU can do the listening.** A second image, `ghcr.io/antoine-debroye/selfpod:1.9.0-cuda`
(amd64), carries a CUDA build of whisper.cpp and uses `small` by default. It uses the GPU
only if whisper reports that it did, and otherwise falls back to the processor and says
so in the health banner. **Experimental:** no machine the project is built on has a GPU,
so this image is proved to build and to fall back, and on a GPU only once it has run on
real hardware. README, *Listening on an NVIDIA GPU*.
- An episode held out of the feed for more than half an hour is said out loud in the
health banner, and every hold is dated from when it began.

### Fixed

- **A read in every episode of a longer show could never be found.** Both searches leave
out anything that recurs too often — that is what keeps silence from swamping them — and
both searched the whole show, so a sponsor read in every one of more than 64 episodes (by
its words) or 32 (by its sound) had every trace of itself left out. Measured: found in 64
episodes, nothing in 65; found in 32, nothing in 33. On a show published twice a day that
was about a month. New repeated stretches are now looked for among the newest 24 episodes
(`AD_CORPUS_WINDOW`, at most 32); what was found in older episodes stays cut, and reads
already decided about are still matched in every episode.
- **The same advert under a new name.** A stretch found by its sound was recognised by a
signature taken from whichever episode the search started from, and that changed as
episodes arrived — the same read turned up as a new question while the new episodes it
was in went uncut. Finds are now matched to what is already known by where they are.
- **A ten-minute "difference" could be cut unasked.** What differs between two downloads
of one episode was approved on sight in automatic mode whatever its length; the limit
written for exactly that case was never consulted. Longer than two and a half minutes, it
is now offered, not cut.
- **A file replaced during a pass could be cut in the wrong places.** The cut is made by
frame positions measured on the file that was read; a file replaced since — a re-download,
an edit over the share — is now left until the next pass reads it again.
- **Two cuts of one episode at once.** An undo arriving while a pass was cutting the same
episode could publish the older cut over the newer decision. Cuts of one episode now wait
for each other.
- Changing where SelfPod listens left the old transcripts on disk; they are now removed.
- Moving the edges of a read left its sponsor cues describing the old words.
- **A closing tag could ask for a whole episode to be cut.** A boundary taught as "cut from these
words to the end" was matched anywhere in a short episode heard as one window, so where the host
read the same sponsor tag to *open* an episode it matched at 0:00. Found on real audio: four
episodes of six refused to trim. An end now belongs in an episode's second half (or last minute),
a start in its first, and no boundary may cut more than four fifths of an episode.
- The station jingle's own sound was offered back as "the same 14 seconds of sound in 4 episodes"
beside the cut it explained.
- Acceptance step 20 still expected a superseded enclosure address to be refused, which
1.8.2 removed on purpose; it now checks what the media route actually does.

### Upgrading

- The database migrates itself, additively: 1.8.8 still runs against it. Two things do not
survive a rollback to 1.8: a *Restore here* (1.8 cuts that stretch again), and the removal
of undecided "heard once" rows, which 1.8 would not recreate until it hears them again.
- **Undecided "sounds like a sponsor read, heard once" rows are removed by the migration.**
They could never be cut on their own. Ones you had decided about are kept.

## 1.8.8 — 2026-09-16

### Added — the jingle's sound, not its words
Expand Down
137 changes: 137 additions & 0 deletions Dockerfile.cuda
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,137 @@
# syntax=docker/dockerfile:1

# ---------------------------------------------------------------------------
# SelfPod with an NVIDIA GPU for listening (spec §19.9). amd64 only.
#
# The same app as Dockerfile, on a different base: NVIDIA's container runtime hands a
# GPU to glibc userspace, and the regular image is Alpine (musl). So this is Ubuntu,
# with Node from its official tarball, and whisper.cpp built three ways — for CUDA,
# and the same two CPU builds as the regular image. The start-up check uses the CUDA
# build only if it actually ran on a GPU, and otherwise falls back to a CPU build and
# says so in the health banner (src/services/transcriber.js).
#
# What cannot be proved here: that it runs on a GPU. Build machines have none, and the
# CUDA build cannot even load without the driver's libcuda, which the NVIDIA runtime
# mounts into the container at start. The build proves the CPU builds on real audio,
# and that the CUDA build links against everything but that one library.
# ---------------------------------------------------------------------------
ARG CUDA_VERSION=12.6.3
ARG UBUNTU=ubuntu22.04

FROM nvidia/cuda:${CUDA_VERSION}-devel-${UBUNTU} AS build

ARG NODE_VERSION=22.23.2
ARG NODE_SHA256=d60acfe00a2932254bb0ad20e01b0d74397a0875595de719654b214f4b03f307
# The RTX 3050 is Ampere, compute capability 8.6. Building for every architecture takes
# several times as long and is several times the size; override for other cards.
ARG CUDA_ARCHITECTURES=86
ARG WHISPER_CPP_TAG=b4938
ARG WHISPER_BASE_SHA256=422f1ae452ade6f30a004d7e5c6a43195e4433bc370bf23fac9cc591f01a8898
ARG WHISPER_SMALL_SHA256=ae85e4a935d7a567bd102fe55afc16bb595bdb618e11b2fc7591bc08120411bb

ENV DEBIAN_FRONTEND=noninteractive
RUN apt-get update \
&& apt-get install -y --no-install-recommends ca-certificates curl xz-utils git cmake make g++ python3 \
&& rm -rf /var/lib/apt/lists/*

RUN curl -fsSLo /tmp/node.tar.xz "https://nodejs.org/dist/v${NODE_VERSION}/node-v${NODE_VERSION}-linux-x64.tar.xz" \
&& echo "${NODE_SHA256} /tmp/node.tar.xz" | sha256sum -c - \
&& mkdir -p /opt/node && tar -xJf /tmp/node.tar.xz -C /opt/node --strip-components=1 \
&& rm /tmp/node.tar.xz
ENV PATH=/opt/node/bin:$PATH

WORKDIR /app
COPY package.json package-lock.json ./
# glibc prebuilds for better-sqlite3 and sharp; never --omit=optional (sharp's binaries are optional deps).
RUN npm ci --omit=dev
RUN node -e "require('better-sqlite3'); require('sharp'); console.log('native modules OK')"

RUN git clone --depth 1 --branch "$WHISPER_CPP_TAG" https://github.com/ggml-org/whisper.cpp /tmp/whisper.cpp

ENV WHISPER_CMAKE="-DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=OFF -DGGML_NATIVE=OFF -DGGML_OPENMP=OFF \
-DWHISPER_BUILD_EXAMPLES=ON -DWHISPER_BUILD_TESTS=OFF -DWHISPER_BUILD_SERVER=OFF -DWHISPER_SDL2=OFF -DWHISPER_CURL=OFF"

# ggml is linked statically; the CUDA libraries stay shared (cuBLAS alone is hundreds of
# megabytes) and come from the runtime image below.
RUN set -e; mkdir -p /app/whisper; \
cmake -S /tmp/whisper.cpp -B /tmp/wbc $WHISPER_CMAKE -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES="$CUDA_ARCHITECTURES" \
-DGGML_SSE42=ON -DGGML_AVX=ON -DGGML_AVX2=ON -DGGML_FMA=ON -DGGML_F16C=ON \
&& cmake --build /tmp/wbc --target whisper-cli -j"$(nproc)" \
&& cp /tmp/wbc/bin/whisper-cli /app/whisper/whisper-cli-cuda; \
cmake -S /tmp/whisper.cpp -B /tmp/wb3 $WHISPER_CMAKE -DGGML_STATIC=ON -DGGML_SSE42=ON -DGGML_AVX=ON -DGGML_AVX2=ON -DGGML_FMA=ON -DGGML_F16C=ON \
&& cmake --build /tmp/wb3 --target whisper-cli -j"$(nproc)" \
&& cp /tmp/wb3/bin/whisper-cli /app/whisper/whisper-cli-v3; \
cmake -S /tmp/whisper.cpp -B /tmp/wb2 $WHISPER_CMAKE -DGGML_STATIC=ON -DGGML_SSE42=ON -DGGML_AVX=OFF -DGGML_AVX2=OFF -DGGML_FMA=OFF -DGGML_F16C=OFF \
&& cmake --build /tmp/wb2 --target whisper-cli -j"$(nproc)" \
&& cp /tmp/wb2/bin/whisper-cli /app/whisper/whisper-cli-v2; \
rm -rf /tmp/wbc /tmp/wb3 /tmp/wb2 /tmp/whisper.cpp

RUN curl -fsSLo /app/whisper/ggml-base-q5_1.bin https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml-base-q5_1.bin \
&& echo "$WHISPER_BASE_SHA256 /app/whisper/ggml-base-q5_1.bin" | sha256sum -c - \
&& curl -fsSLo /app/whisper/ggml-small-q5_1.bin https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml-small-q5_1.bin \
&& echo "$WHISPER_SMALL_SHA256 /app/whisper/ggml-small-q5_1.bin" | sha256sum -c -

# ---------------------------------------------------------------------------
# Runtime stage: CUDA runtime libraries (cuBLAS, the CUDA runtime), no compilers.
# ---------------------------------------------------------------------------
FROM nvidia/cuda:${CUDA_VERSION}-runtime-${UBUNTU}

ENV DEBIAN_FRONTEND=noninteractive
# util-linux: setpriv, which drops privileges as su-exec does on Alpine.
# passwd: usermod/groupmod for PUID/PGID. tzdata: TZ.
RUN apt-get update \
&& apt-get install -y --no-install-recommends ca-certificates tzdata util-linux passwd \
&& rm -rf /var/lib/apt/lists/*

# The same app user as the regular image, at the default PUID/PGID. Ubuntu 22.04's
# CUDA images have no user at 1000 to move out of the way (24.04's "ubuntu" user would).
RUN groupadd -g 1000 selfpod \
&& useradd -u 1000 -g selfpod -d /app -s /usr/sbin/nologin -M selfpod

COPY --from=build /opt/node /opt/node
ENV PATH=/opt/node/bin:$PATH

WORKDIR /app
COPY --from=build /app/node_modules ./node_modules
COPY --from=build /app/package.json ./package.json
COPY --from=build /app/whisper ./whisper
COPY src ./src
COPY docker ./docker
COPY scripts ./scripts

# The CPU builds are proved on real audio against this image's libraries. The CUDA build
# is proved to link against everything except libcuda.so.1, which only the NVIDIA runtime
# provides, at start.
RUN WHISPER_SMOKE_SKIP=whisper-cli-cuda node docker/whisper-smoke.mjs \
&& missing="$(ldd /app/whisper/whisper-cli-cuda | grep 'not found' | grep -v 'libcuda.so.1' || true)" \
&& if [ -n "$missing" ]; then echo "whisper-cli-cuda is missing libraries: $missing"; exit 1; fi \
&& echo "whisper-cli-cuda links (libcuda.so.1 comes from the driver at run time)"

RUN chmod +x docker/entrypoint.sh \
&& mkdir -p /data \
&& chown selfpod:selfpod /app

# `small` by default here: the GPU makes it cheap, and it hears French markedly better.
ENV NODE_ENV=production \
PORT=8080 \
DATA_DIR=/data \
PUID=1000 \
PGID=1000 \
WHISPER_MODEL=small \
NVIDIA_VISIBLE_DEVICES=all \
NVIDIA_DRIVER_CAPABILITIES=compute,utility

EXPOSE 8080
VOLUME ["/data"]

HEALTHCHECK --interval=30s --timeout=5s --start-period=60s --retries=3 \
CMD ["node", "/app/docker/healthcheck.mjs"]

ENTRYPOINT ["/app/docker/entrypoint.sh"]

LABEL org.opencontainers.image.title="SelfPod (NVIDIA GPU)" \
org.opencontainers.image.description="Self-hosted podcast server for private feeds, listening for spoken adverts on an NVIDIA GPU." \
org.opencontainers.image.source="https://github.com/antoine-debroye/selfpod" \
# As the regular image, plus NVIDIA's CUDA runtime libraries from the base image,
# redistributed under the CUDA EULA. THIRD-PARTY-LICENSES.md says so.
org.opencontainers.image.licenses="MIT AND LGPL-2.1-or-later AND LicenseRef-NVIDIA-CUDA-EULA"
Loading
Loading