local-ai-monthly: the monthly digest has never completed a run, and a failure keeps no stderr

[mecattaf]

What is true today

MEASURED (2026-09-19, read-only):

• home/tally.nix:450 declares the calendar producer monthly-local-ai-review, onCalendar = "*-*-01 00:30:00", pool local-ai-review, dedupKey monthly-local-ai-review-%Y-%m, runtimeMaxSec = 43200, evidence exit:0 + artifact:/home/tom/.local/state/local-ai-monthly/last-run.json + hash:sha256. The same producer is present in the live config at ~/.config/tally/config.json, so it is wired and enabled, not dormant.
• ~/.local/state/local-ai-monthly/last-run.json holds exactly one record:

{"schema_version": 2, "period": "2026-09",
 "run_dir": "/run/user/1000/local-ai-monthly/2026-09.bzi0ya",
 "started_at": "2026-09-10T10:56:14Z", "completed_at": "2026-09-10T10:56:59Z",
 "status": "failed", "error": "workflow exited with status 1",
 "sources": [], "pr_url": null}

45 seconds, zero sources evaluated, no PR.

REPORTED (~/today/review/2026-09-19/DOTFILES-SCHEDULED-RUNS-ARCHAEOLOGY.md, read-only archaeology pass of the same night):

• No automation/local-ai-review-* branch exists on the remote.
• No merged or open PR with local-ai in its title or branch exists in the last 300 PRs.
• docs/local-ai/tallies/ holds one file, 2026-07-29.md, which opens with "Superseded 2026-08-29".
• The pipeline was born without a live run: PR feat(local-ai): add Tally monthly review bot #91's own validation notes say "The live Pi call was intentionally not bypassed when worker-tb:9292 was unavailable ... no runtime review PR was published during validation."

The gap

Two distinct problems, one issue because the first hides the second.

1. A failure keeps nothing. run_dir lives under /run/user/1000, which is cleared. supervisor.sh writes no durable stderr path into state. The 2026-09-10 failure is therefore unresolvable from disk: exit status 1 and nothing else. Every future failure will be equally opaque.
2. The deterministic legs have never been separated from the judged leg. --prepare-only exists and is documented as a no-GPU, no-publish dry path, but nothing runs it, so a failure in the git or Hugging Face capture stages is indistinguishable from a failure in the Halogen judge stage.

Proposed change

• pkgs/local-ai-monthly/supervisor.sh redirects stderr to ~/.local/state/local-ai-monthly/<period>.stderr.log (rotating, last three kept) and writes that path into last-run.json as stderr_path, on success and on failure alike. A failure whose stderr path is absent or empty is itself the bug.
• Keep last-run.json as the evidence artifact, but make it append-only per period (<period>.json plus a last-run.json symlink), so a manual run on the 10th no longer overwrites the scheduled fire of the 1st. Today those two are indistinguishable.
• One --prepare-only run recorded before the next scheduled fire, its receipt banked.

Acceptance

• A deliberately failed run (point the judge at an unreachable URL) leaves a non-empty stderr log at the path named in last-run.json, and the tally job still grades exit:0 false, not silently green.
• ls ~/.local/state/local-ai-monthly/ shows one JSON per period attempted, not one slot.
• No change to home/tally.nix. The scheduling surface is already correct: local-ai-review mutex on the parent, coordinator-gpu only on the nested child, 12 hour ceiling, monthly dedup key.

Not in scope

Merging its own PR (it never has and must not), editing lib/local-models.nix, downloading weights, and any nixos-rebuild. Ruling 27 keeps the switch and the merges with Tom.

Related

Nothing open covers this file. Adjacent, not duplicate: #423 and #424 (pi adapter model override and --thinking, which the judge stage would inherit once it runs), #428 (the Qwen token plan catalogue).