docs: post-completion portfolio polish (tags, topics, CHANGELOG, CONTRIBUTING)

CHANGELOG.md

@@ -7,10 +7,49 @@ Every number here is measured. Where one is later found wrong, it gets
 corrected **in place with the original kept**, because a retraction that
 deletes its own evidence is not a retraction.
 
-## Unreleased
+Every phase below has a matching git tag (`phase-0` through `phase-7`);
+`v1.0.0` marks the full 8-phase system design complete. Compare any two
+points in the project's history directly:
+[`phase-0...v1.0.0`](https://github.com/bsreecharanreddy/dispatch/compare/phase-0...v1.0.0).
 
-No phase shipped yet (`v0.1.0` is still the scaffold version; a tag lands
-once a phase's exit criteria are actually met).
+## [v1.0.0] / [phase-7] - 2026-09-21
+
+**Phase 7 (productionization) complete.** This closes the system
+design's entire 8-phase plan, Phase 0 through 7, merged to `main` via
+[#9](https://github.com/bsreecharanreddy/dispatch/pull/9)
+(`docs/plans/2026-09-20-phase-7-productionization-plan.md`). A Rust
+router (tonic gRPC client, axum HTTP, Prometheus metrics) in front of
+the existing Python model server, per ADR-0003's TGI-shaped split,
+containerized (Docker multi-stage builds), deployed to a local `kind`
+Kubernetes cluster with Prometheus/Grafana observability, demonstrated
+once against a real rented GPU over an SSH tunnel — `make check` green
+throughout (Python and the Rust router's own `fmt`/`clippy`/`test`
+gate). The one real GPU session found and fixed two real bugs neither
+anticipated by the plan: `python scripts/run_model_server.py` run
+directly can't resolve its own lazy `scripts.gpu.*` import (`sys.path[0]`
+is the script's directory, not the repo root — fixed by invoking
+`python -m scripts.run_model_server` instead), and a genuine defect in
+`deepseek-ai/deepseek-moe-16b-base`'s own shipped tokenizer — a
+byte-level BPE vocabulary (47,723 of 100,000 entries carry the
+GPT-2-style `Ġ` marker) wired to a SentencePiece `Metaspace`
+pre-tokenizer/decoder expecting a `▁` marker that appears in zero vocab
+entries, silently dropping every space in generated text. Root-caused
+and fixed for real (not a pod-local shim) with
+`fix_tokenizer_byte_level()` in `scripts/gpu/phase7_kernel_responder.py`,
+verified by round-trip token-id identity — invisible to Phases 0-6
+because they only ever compared logits/token ids, never decoded text for
+a human to read. Real demo request through the full stack (dev machine
+-> `kind` router pod -> gRPC -> SSH tunnel -> real GPU inference ->
+back): 136ms TTFT, correctly-spaced generated text, real Grafana
+metrics. Cost: **$6.8563** of a $5 cap, exceeded and disclosed
+mid-session, continued with explicit go-ahead. Full account:
+`docs/findings/phase-7/2026-09-21-phase-7-productionization-run.md`;
+runbook: `docs/runbooks/phase-7-productionization.md`.
+
+Project total, all eleven rented-GPU sessions across all nine measured
+phases (0, 1, 2, 3, 4, 5a, 5b, 6, 7): **$51.35**.
+
+## [phase-6] - 2026-09-20
 
 - **Phase 6 (final benchmark vs. vLLM and SGLang) complete**, 2026-09-20,
   merged to `main` via
@@ -61,6 +100,9 @@ once a phase's exit criteria are actually met).
   than rate x duration. Full account:
   `docs/findings/phase-6/2026-09-20-phase-6-final-benchmark-run.md`;
   runbook: `docs/runbooks/phase-6-final-benchmark.md`.
+
+## [phase-5b] - 2026-09-18
+
 - **Phase 5b (speculative decoding) complete**, 2026-09-18, merged via PR #6
   (`docs/plans/2026-09-17-phase-5b-speculative-decoding-plan.md`). A shared
   propose/verify/accept/rollback loop with two drafters (a 7B draft model,
@@ -94,6 +136,9 @@ once a phase's exit criteria are actually met).
   and that was raised to $20 with disclosure. Also: `docs/findings/` split
   into one subfolder per phase. Full account:
   `docs/findings/phase-5b/2026-09-18-phase-5b-speculative-decoding-run.md`.
+
+## [phase-5a] - 2026-09-17
+
 - **Phase 5a (int8 weight-only quantization) complete**, 2026-09-17
   (`docs/plans/2026-09-16-phase-5a-quantization-plan.md`). Self-computed,
   per-output-channel int8 quantization extending the Triton kernel
@@ -114,6 +159,9 @@ once a phase's exit criteria are actually met).
   cap, including two RunPod Community Cloud pods that hit a real
   host-level GPU passthrough bug. Full account:
   `docs/findings/phase-5a/2026-09-17-phase-5a-quantization-run.md`.
+
+## [phase-4] - 2026-09-17
+
 - **Phase 4 (disaggregated prefill/decode) complete**, 2026-09-17
   (`docs/plans/2026-09-16-phase-4-disaggregated-prefill-decode-plan.md`).
   Continuous-batching prefill/decode workers across two real 4-GPU H100
@@ -127,6 +175,9 @@ once a phase's exit criteria are actually met).
   reported as genuinely inconclusive. Cost: **$10.94** of a $40 cap.
   Full account:
   `docs/findings/phase-4/2026-09-17-phase-4-disaggregated-prefill-decode-run.md`.
+
+## [phase-3] - 2026-09-16
+
 - **Phase 3 (multi-GPU expert-parallel serving) complete**, 2026-09-16
   (`docs/plans/2026-09-15-phase-3-multi-gpu-expert-parallel-serving-plan.md`).
   Real 2x H200 SXM expert-parallel serving over DeepSeek's own DeepEP
@@ -140,6 +191,9 @@ once a phase's exit criteria are actually met).
   median 2 max ~20, sit below Phase 1's smallest tested point of 16).
   Cost: **$13.03** of a $25 cap. Full account:
   `docs/findings/phase-3/2026-09-16-phase-3-multi-gpu-ep-run.md`.
+
+## [phase-2] - 2026-09-15
+
 - **Phase 2 (vLLM benchmark contribution) complete**, 2026-09-15
   (`docs/plans/2026-09-15-phase-2-vllm-benchmark-contribution-plan.md`).
   Neither vLLM's nor SGLang's official MoE benchmark modeled skewed
@@ -151,6 +205,9 @@ once a phase's exit criteria are actually met).
   Cost: **$0.77**. This phase landed no code in `dispatch` itself, only
   docs — its actual contribution is the upstreamed PR. Full account:
   `docs/findings/phase-2/2026-09-15-phase-2-vllm-benchmark-run.md`.
+
+## [phase-1] - 2026-09-15
+
 - **Phase 1 (custom Triton grouped-GEMM kernel) complete**, 2026-09-15
   (`docs/plans/2026-09-15-phase-1-grouped-gemm-plan.md`). Built and
   tested: a CPU-only MoE reference and grouped-GEMM contract, a naive and
@@ -180,6 +237,9 @@ once a phase's exit criteria are actually met).
     512-2048 — a workload-specific result recorded rather than buried.
     Total GPU cost across both sessions: **$0.65**.
     Full account: `docs/findings/phase-1/2026-09-15-phase-1-grouped-gemm-run.md`.
+
+## [phase-0] - 2026-09-14
+
 - **Phase 0 (baseline) complete**, 2026-09-14
   (`docs/plans/2026-09-14-phase-0-baseline-plan.md`). Built and tested: a
   RunPod REST client and provisioning CLI, a token-by-token-timed

CLAUDE.md

@@ -209,9 +209,10 @@ duration. Full account:
 `docs/findings/phase-6/2026-09-20-phase-6-final-benchmark-run.md`; runbook:
 `docs/runbooks/phase-6-final-benchmark.md`.
 
-**Phase 7 (productionization) is complete, 2026-09-21**, on branch
-`phase-7-productionization`, PR not yet opened. **This closes the system
-design's entire 8-phase plan, Phase 0 through 7.** A Rust router (tonic
+**Phase 7 (productionization) is complete, 2026-09-21**, merged to `main`
+via [PR #9](https://github.com/bsreecharanreddy/dispatch/pull/9). **This
+closes the system design's entire 8-phase plan, Phase 0 through 7.** A
+Rust router (tonic
 gRPC client, axum HTTP, Prometheus metrics) in front of the existing
 Python model server, per ADR-0003's TGI-shaped split, containerized
 (Docker multi-stage builds), deployed to a local `kind` Kubernetes

CONTRIBUTING.md

@@ -0,0 +1,106 @@
+# Contributing
+
+This is a solo portfolio project, not one actively seeking outside
+contributions — but it's built with real conventions, and this doc exists
+so anyone reading the code (or considering a PR) can see how it's meant
+to be worked in.
+
+## Getting set up
+
+```bash
+uv sync --all-extras --dev   # Python
+cd router && cargo build     # Rust router (needs a stable toolchain)
+```
+
+## Running the checks
+
+```bash
+make check       # the full gate: lint, mypy --strict, pytest, and the
+                  # Rust router's own fmt/clippy/test -- same steps CI runs
+make check-fast  # inner loop: skips slow-marked tests
+make coverage    # pytest --cov, term + xml report
+```
+
+`make check` runs before every push in this repo's own workflow, and
+matches CI exactly (`.github/workflows/ci.yml`).
+
+## GPU-marked tests
+
+Any test that needs a real CUDA device is marked `gpu` and excluded from
+both `make check` and CI — this repo's CI has no GPU runner, and marking
+is the explicit, visible boundary of what CI can verify (never a silent
+skip). GPU-marked tests are run for real, on rented hardware, once per
+phase; each phase's findings doc under
+[`docs/findings/`](docs/findings/) records exactly which GPU, for how
+long, and at what cost.
+
+## Testing policy
+
+No implementation code is committed without tests. The testing table this
+project settled on (from Phase 1's actual kernel work):
+
+| Layer | What must be covered |
+|---|---|
+| Kernel contract (CPU) | eager backend == `ReferenceMoE`; every row covered by exactly one tile; no tile crosses an expert boundary; experts that receive zero tokens |
+| Kernels (`gpu`) | each kernel meets `assert_matches_reference` against an fp32 reference at toy, decode- and prefill-shaped dims, fp16 and bf16; a mutation must turn the suite red |
+| End to end (`gpu`, paid) | mutual top-5 logit agreement with a same-session stock run; a kernel run that patches no layers refuses to run |
+| Benchmarks | `triton.testing.do_bench`; a backend that disagrees with the eager one is refused, not timed; every JSON carries its full config |
+
+**Correctness before speed** governs all of it: a kernel isn't "fast"
+until it's been proven numerically correct against a reference
+implementation within a stated tolerance. A kernel that's fast because
+it's silently wrong is a bug, not a result.
+
+**Never quote a benchmark number that wasn't measured** on the exact
+hardware, at the exact config (batch size, sequence length, quantization
+level, warm-up handled), by this repo. A benchmark claim states its
+config alongside the number — "40% faster" isn't a result; "40% faster
+at batch size 32, seq len 2048, on a single A100 80GB, vLLM 0.x.y as
+baseline" is.
+
+## Conventions
+
+- **One branch per phase**, carrying the whole phase, pushed once as a
+  single PR.
+- **One commit per completed task**, not one bundled commit per phase.
+- **`docs/STATUS.md` updates in the same commit as the work it
+  describes.**
+- Conventional commit prefixes (`feat:`, `fix:`, `docs:`, `test:`,
+  `refactor:`, `build:`).
+- **Commit messages are plain ASCII** — `--`, never an em-dash. Docs use
+  `—` freely; the git log does not.
+- Every phase gets a `docs/design/` entry (for a new subsystem) or slots
+  into the existing [system design](docs/design/2026-09-14-dispatch-system-design.md),
+  then a `docs/plans/` implementation plan, written before any code.
+
+## Language and tooling
+
+Python 3.12+ (`uv`, `ruff`, `mypy --strict`, `pytest`), Rust (stable,
+`cargo fmt`, `cargo clippy -- -D warnings`, `cargo test`) for the Phase 7
+router. GPU provisioning (`scripts/gpu/`) calls RunPod's REST API
+directly rather than through Terraform — the actual workflow (rent, run
+a benchmark, tear down) fits a script better than Terraform state
+management, and RunPod's own Terraform provider is much less mature than
+AWS/Azure's.
+
+## Cost discipline
+
+GPU rental is real money. If you're running the paid (`gpu`-marked) test
+suite or a benchmark script yourself:
+
+- Use marketplace/spot instances (RunPod Community Cloud, Vast.ai), not
+  dedicated on-demand.
+- Set a budget cap before the first rental, not after.
+- Spin up, run the measured session, capture the evidence, tear down
+  immediately — never leave a rented GPU idle between sessions or across
+  an unbounded wait.
+- Pull every evidence file off the pod before `stop`, not after; a
+  stopped pod's disk is not guaranteed to come back.
+
+## Full picture
+
+[`CLAUDE.md`](CLAUDE.md) carries the complete set of engineering
+patterns, phase-by-phase history, and the project's own written record of
+what it got wrong and fixed along the way — written for an AI coding
+assistant working in this repo, but equally readable as the fuller
+version of this document.

README.md

@@ -22,25 +22,39 @@ result was a null or a mixed one, it's reported that way.
 
 ---
 
-> **Status:** Phases 0-6 merged to `main` across seven PRs
+> **Status:** All 8 phases (0-7) merged to `main` across eight PRs
 > ([#1](https://github.com/bsreecharanreddy/dispatch/pull/1) Phase 0,
 > [#2](https://github.com/bsreecharanreddy/dispatch/pull/2) Phase 1,
 > [#3](https://github.com/bsreecharanreddy/dispatch/pull/3) Phases 2-3,
 > [#4](https://github.com/bsreecharanreddy/dispatch/pull/4) Phase 4,
 > [#5](https://github.com/bsreecharanreddy/dispatch/pull/5) Phase 5a,
 > [#6](https://github.com/bsreecharanreddy/dispatch/pull/6) Phase 5b,
-> [#8](https://github.com/bsreecharanreddy/dispatch/pull/8) Phase 6) —
+> [#8](https://github.com/bsreecharanreddy/dispatch/pull/8) Phase 6,
+> [#9](https://github.com/bsreecharanreddy/dispatch/pull/9) Phase 7) —
 > Phase 2 landed as docs only, its actual contribution being the
-> upstreamed vLLM PR below, not a dispatch-repo code change. **Phase 7
-> (productionization) is complete**, PR pending: a real Rust router in
-> front of the Python model server, containerized, deployed to a local
-> Kubernetes cluster, demonstrated once against a real rented GPU over an
-> SSH tunnel — see [below](#phase-7-seeing-it-run). **This closes the
-> system design's entire phase plan, Phase 0 through 7.** Full
-> task-by-task record: [`docs/STATUS.md`](docs/STATUS.md). Design and
-> phasing:
+> upstreamed vLLM PR below, not a dispatch-repo code change. **This
+> closes the system design's entire 8-phase plan.** Phase 7
+> (productionization) added a real Rust router in front of the Python
+> model server, containerized, deployed to a local Kubernetes cluster,
+> demonstrated once against a real rented GPU over an SSH tunnel — see
+> [below](#phase-7-seeing-it-run). Every phase has its own git tag
+> (`phase-0` through `phase-7`); `v1.0.0` marks this completed state.
+> Full task-by-task record: [`docs/STATUS.md`](docs/STATUS.md); phase
+> history: [`CHANGELOG.md`](CHANGELOG.md). Design and phasing:
 > [`docs/design/2026-09-14-dispatch-system-design.md`](docs/design/2026-09-14-dispatch-system-design.md).
 
+**Jump to a section by interest:**
+
+| If you care about... | Start here |
+|---|---|
+| Inference / GPU-kernel engineering | [Phase 1 kernel](src/dispatch/kernels/grouped_gemm.py) + [findings](docs/findings/phase-1/2026-09-15-phase-1-grouped-gemm-run.md) |
+| Multi-GPU / distributed serving | [Phase 3](docs/findings/phase-3/2026-09-16-phase-3-multi-gpu-ep-run.md) + [Phase 4](docs/findings/phase-4/2026-09-17-phase-4-disaggregated-prefill-decode-run.md) findings |
+| Production serving / platform engineering | [Phase 7: seeing it run](#phase-7-seeing-it-run) |
+| The vLLM/SGLang head-to-head | [Phase 6 findings](docs/findings/phase-6/2026-09-20-phase-6-final-benchmark-run.md) |
+| Whether the claims actually hold up | [`docs/STATUS.md`](docs/STATUS.md) + [`docs/findings/`](docs/findings/) |
+
+(Full breakdown, one line per audience: [Who should look at what](#who-should-look-at-what).)
+
 ## In sixty seconds
 
 **What it does.** A pure-PyTorch reference MoE forward pass, a from-scratch
@@ -50,9 +64,10 @@ DeepEP library, and a disaggregated prefill/decode scheduler — each phase's
 thesis stated up front and reported honestly, including the phases where
 the answer was a null or mixed result.
 
-**Built solo, start to finish** — design docs, the kernel, the multi-GPU
-serving path, an upstreamed open-source benchmark contribution, every
-rented-GPU session, and the write-ups of what broke along the way.
+**Built entirely solo** — every Triton kernel, the multi-GPU serving path,
+the Rust router, the Kubernetes manifests, an upstreamed open-source
+benchmark contribution, every rented-GPU session, and every write-up of
+what broke along the way, start to finish, no team.
 
 **Measured across eleven rented-GPU sessions (nine phases), $51.35 total,
 every phase within its stated cap except Phase 5b (its original $10 cap
@@ -88,9 +103,13 @@ defect — the model's shipped vocabulary is byte-level BPE but wired to
 the wrong pre-tokenizer/decoder pair — found, root-caused, and fixed
 (Phase 7).
 
-**262 tests, `make check` green throughout Phase 6** (lint, `mypy --strict`,
-and the full non-GPU suite) — GPU-dependent tests are marked and excluded
-from CI by design, then run for real on rented hardware every phase.
+**277 tests** (268 Python + 9 Rust), `make check` green throughout —
+lint, `mypy --strict`, the Rust router's own `fmt`/`clippy`/`test` gate,
+and the full non-GPU suite. **86% coverage on CPU-testable logic**
+(`pytest --cov`; the kernels themselves are GPU-only and covered by the
+`gpu`-marked correctness suite instead, not by this number).
+GPU-dependent tests are marked and excluded from CI by design, then run
+for real on rented hardware every phase.
 
 ## Why this project
 
@@ -289,6 +308,10 @@ They're run for real on rented hardware every phase; each phase's findings
 doc in [`docs/findings/`](docs/findings/) records exactly which GPU, for
 how long, and at what cost.
 
+The full testing policy, conventions, and cost discipline this project
+follows: [`CONTRIBUTING.md`](CONTRIBUTING.md). Phase-by-phase history,
+one entry per release: [`CHANGELOG.md`](CHANGELOG.md).
+
 ## Design and the paper trail
 
 [`docs/design/2026-09-14-dispatch-system-design.md`](docs/design/2026-09-14-dispatch-system-design.md)
@@ -315,6 +338,12 @@ is the authoritative architecture, phasing, and scope document.
   [Phase 4's](docs/findings/phase-4/2026-09-17-phase-4-disaggregated-prefill-decode-run.md)
   findings docs for two results — one null, one mixed — found by measuring
   on real hardware rather than assumed.
+- **Production serving / platform engineering** — `router/` (the Rust
+  gRPC/HTTP router) and `k8s/`, then
+  [Phase 7's findings doc](docs/findings/phase-7/2026-09-21-phase-7-productionization-run.md):
+  a real router in front of a real GPU-backed model server, containerized
+  and deployed to Kubernetes, including a genuine tokenizer bug found and
+  fixed on the one real demo run.
 - **ML infra / open-source contribution** —
   [Phase 2's findings doc](docs/findings/phase-2/2026-09-15-phase-2-vllm-benchmark-run.md)
   and the open [vLLM PR](https://github.com/vllm-project/vllm/pull/57100): a