docs: tighten setup and operations docs

[flamboh]

Note

🤖 Claude Opus 5.5 on behalf of Oliver

Explain Like I'm Lost

The last 13 PRs added a lot of how-it-works detail to docs/user. This PR cuts those guides back to what an operator needs (prerequisites, commands, what success looks like, and common failures) and fixes stale commands and links. It does not change any code.

Why

• The user guides explained internals: merge-shard failure recovery, how the launcher assigns slots, active-source thresholds, and the MAAD encoding. They also repeated the same flag lists in several places.
• Stale or wrong content:
  • Troubleshooting said to install Node.js 22 (the pinned version is 24.18.1).
  • Troubleshooting linked to a missing #manage-d1-migrations anchor.
  • Troubleshooting described a local-D1 dev path that vite dev now rejects.
  • development.md linked to a missing #deploy-the-dashboard anchor.
  • datasets.md said the web app reads DATASETS_CONFIG_PATH. It does not.
  • setup-pipeline.md said coordinated subsets need identical locality rules. The code does not require that.
• The docs named a real dataset (uoregon) and used a real-looking path (/data/netflow/...).

Fix

Reviewer fast path: skim docs/user/setup-pipeline.md, operations.md, and datasets.md. Those three hold most of the change. Check that no procedure you rely on is gone.

• setup-pipeline.md (−156 net): each section is now one working example plus a few bullets. The cluster section keeps the prerequisites, the command, resume by rerunning, the two common failures, and cleanup. Details about merge consumption and failure modes were already in pipeline-contract.md, so this PR removes them from the user guide.
• operations.md (−99): the self-hosted container is now one export + deploy block with its two common failures. Publish and rollback are merged into one section. The Cloudflare and D1 steps are shorter.
• datasets.md (−93): shorter examples, one fields table, and neutral placeholders. The locality internals (digest identity and IPv4-mapped handling) stay in pipeline-contract.md only.
• setup-web, requirements, troubleshooting, querying, README, and user/README: cut repeated text and fixed the stale items listed above.
• docs/code:
  • pipeline-contract.md gets a short "Cluster launcher" subsection. It is the only architecture material moved out of the user docs.
  • pipeline-contract.md now describes daily_active_sources and tos_anonymized in neutral terms, and the extract-window example uses data/example.
  • documentation.md gets a "User documents" rule block: task-first writing and neutral placeholders.
• docs/agent is untouched. It contained no real hostnames.

Verification

• I checked every command and flag against a fresh netflow-db <cmd> --help build of dd25082, scripts/netflow-db-cluster.sh --help, infra/self-hosted.ts, package.json scripts, apps/web/src/env.ts, and .node-version.
• A script checked every relative link and anchor in README.md, CONTEXT.md, AGENTS.md, and docs/**. All resolve.
• I grepped the docs for real hostnames and internal URLs and found none.
• bun run format, bun run lint, and bun run typecheck pass.
• Manual check still needed: read the self-hosted deploy section against a real deploy.

Made by Claude Opus 5.5 in Claude Code.