Phases 1 & 5: mapping parser + pluggable corpus (incl. Ollama)

[christiaanjs]

Implements Phase 1 (mapping parser) and Phase 5 (pluggable corpus, incl. a local Ollama source) from PLAN.md, then wires both into the two frontends. The keymap is now parsed from mappings/, and practice content can come from a static bank, a text file, a codebase, or a local Ollama model — all behind the existing Mapping/Corpus seam interfaces, so the drill and frontends are unchanged.

Phase 1 — Mapping parser

• core.ParseMapping(fsys fs.FS, dir string) (Mapping, error) (core/mapping_parse.go) reads mappings/*.json, extracts the alt-layer (space-held) mirror pairs, and builds a mapping. It takes an fs.FS (never os) so core stays wasm-safe; the JSON is embedded via mappings/mappings.go (embed.FS).
• Unified the static and parsed mappings under one shared mirrorTable type (core/mirror_table.go) implementing Hint/Supported/Diagnose/Reference, so parsed and static behave identically by construction. staticMapping is refactored to build the same type; NewStaticMapping is exported as the fallback.
• Errors on unknown key codes and on conflicting duplicate mappings across files.
• core/mapping_parse_test.go: oracle tests asserting the parsed mapping equals the static one across all interface methods, plus fstest.MapFS edge cases.

Phase 5 — Corpus subsystem

New corpus/ package (all I/O lives here, outside the pure core), each constructor returning a Source that implements core.Corpus:

• FromText / FromReader / FromFile — plaintext.
• FromCodebase — tokenizes source identifiers (camel/identifier splitting) into practice words/sentences.
• FromOllama — queries a local Ollama server via the official github.com/ollama/ollama/api client (non-streaming); errors on too-few results so the caller falls back to static.
• Selection logic (deterministic-by-seed) is ported from staticCorpus. NewStaticCorpus is exported as the fallback.

Frontend wiring

• TUI (cmd/tui): new flags -corpus (static|file|code|ollama), -corpus-path, -ollama-model, -ollama-host. Parses the embedded mapping and builds the selected corpus, each falling back to static with a stderr warning printed before Bubble Tea takes the screen.
• Web (cmd/web): parses the embedded mapping (static fallback); corpus stays static — a browser sandbox can reach neither the filesystem nor a local Ollama server.

Streaming corpus (replaces the batch Ollama path)

The first cut of -corpus=ollama blocked startup fetching 200 words + 40 sentences, then never generated again. It is now streaming and unbounded:

• corpus.Producer — a vendor-neutral seam. Produce(ctx, kind, n, emit) emits each item the moment its line completes, not batched at the end. This is exactly the seam Phase 7's Anthropic provider plugs into, so that phase no longer needs a new abstraction.
• corpus.Stream — a core.Corpus that never blocks. It serves the static fallback while cold and generated text once warm, so the app starts instantly and upgrades in place. Buffers are rings (2000 words / 500 sentences, oldest evicted): content is unbounded, memory is not. The producer goroutine sleeps unless woken by demand or a backoff timer, so an idle trainer generates nothing.
• Soft failure — a dead server, or a model that repeats itself, backs off and retries while the drill keeps running on the fallback. Ollama being down never breaks the trainer.

Visible, not silent

Content now arrives after startup, so the core surfaces it: State.Corpus (CorpusStatus/CorpusPhase), populated via an optional StatusReporter interface so fixed banks are unaffected. The TUI renders it and ticks to re-render while it changes (Bubble Tea only redraws on messages, so background arrivals would otherwise be invisible until the next keystroke):

corpus: ollama ⠹ generating (qwen3:8b) — drilling on static text meanwhile
corpus: ollama ● streaming (qwen3:8b) — 240 words, 60 sentences so far
corpus: ollama failed — drilling on static text; retrying. (connection refused)

The Corpus contract is now explicit about the rule that makes this possible: Word/Sentence run on the frontend event loop and must never block.

Bugs found and fixed while reviewing the producer loop

Both broke the "idle ⇒ no generation" property, and both now have regression tests (the second was verified to fail before the fix):

1. A pending demand signal could short-circuit the backoff window — so with Ollama down, every drill item re-hammered the dead server, ignoring the 2s→30s backoff.
2. A round that succeeded but added nothing new reset the backoff. Since the ring dedups, a model repeating itself can never reach the low-water mark — so every drill item re-signalled and the producer regenerated forever in a tight loop at full GPU.

Thinking models

Reasoning models (qwen3, deepseek-r1) emit their chain-of-thought to GenerateResponse.Thinking, not .Response — so the corpus sat at warming with 0 words for 2+ minutes while the GPU churned. Thinking is now disabled, conditionally: the capability is resolved once from the server, because sending think to a model that lacks it is rejected.

Live verification

Not just unit-tested — exercised against a real qwen3:8b:

• Before: timed out at 2 minutes, 0 words.
• After: reached streaming in ~1s; 42 generated words + 2 generated sentences within 6.5s; TUI startup with -corpus=ollama returns in 605ms instead of blocking.

OLLAMA_LIVE_TEST=1 OLLAMA_TEST_MODEL=qwen3:8b go test ./corpus -run TestFromOllamaLive -v runs it; it is skipped by default and in CI. Full suite passes under go test -race ./....

Docs

PLAN.md, README.md, and CLAUDE.md updated to mark phases 1 & 5 done and document the corpus flags / Ollama usage.

Verification

gofmt, go vet ./..., go test ./..., native go build, and GOOS=js GOARCH=wasm go build -o web/app.wasm ./cmd/web all pass; CI green on the branch.

🤖 Generated with Claude Code