Skip to content

feat: Go→Rust migration — m0..m3, m5 W1/W2.T1 landed (210 tests, parity green) - #29

Merged
tinhtute merged 36 commits into
masterfrom
rust-rewrite
Sep 6, 2026
Merged

tinhtute merged 36 commits into
masterfrom
rust-rewrite

Conversation

@tinhtute

@tinhtute tinhtute commented Sep 4, 2026

Copy link
Copy Markdown
Owner

What is this

Incremental landing of the Go→Rust big-bang migration (plan: docs/plans/active/go-to-rust-migration.md). This PR is the living migration branch — each milestone merges here as it lands; the single cutover commit (delete Go tree, flip CI) comes at m8 and is the only part that touches master authority.

Landed so far

Phase Scope Status
m0-foundation cargo workspace, paths/perms (0700/0600/0400), config, workspace, flock, Clock trait, differential parity harness ✅ checked
m1-assets-setup embedded assets (include_dir, go:embed pattern filter), setup, parity --op setup ✅ checked
m2-claims-evidence YAML layer (hand-rolled yaml.v3-compatible emitter + 150-case oracle corpus), claims, evidence (SHA-256 skip, 0400 immutability), manifest, 3-way claims parity ✅ checked
m3-lifecycle-trust transitions (byte-compatible journal, hard_link exclusive publish, recovery), approve/supersede/revoke, TrustValidator, lint ✅ done
m5-mcp-gateway W1 stdio transport, JSON-RPC framing, protocol revisions (wire-captured from real Go binary) ✅ done
m5-mcp-gateway W2.T1 claim resource + evidence_capture tool — fence-shape byte-identity locked with golden tests ✅ done

Remaining

Phase Scope Status
m4-index-query FTS5 (rusqlite bundled), query/ask, temporal recall, loopback embedder, p95@100K bench planned
m5-mcp-gateway W2.T2 memory_ask/memory_status/evidence_check/claim_lifecycle/campaign/doctor tools planned (gated on m4)
m6-approval-campaign challenge ceremony, batch approval, campaign planned
m7-cli-view-eval CLI surface, viewer, eval suite, smoke planned
m8-cutover CI flip, delete Go tree, single cutover commit planned

Verification (current HEAD 4053873)

  • cargo test -p zbrain --lib → 210 passed, 0 failed
  • cargo clippy --workspace --all-targets -- -D warnings → clean
  • ./scripts/parity.sh → parity: OK (workspace / claims / lifecycle)
  • Go oracle: go test ./..., go vet, CGO_ENABLED=0 go build → all green, zero regressions

…qlite

- index.rs: IndexStore (Rebuild, CheckFresh, MarkDirty, Search, claims catalog,
  trust-input mtime/change-token freshness, symlink boundary validation,
  FTS5 query builder with Go byte-scan semantics)
- index_state.rs: published manifest + rebuild state read/write with schema
  validation (user_version 3)
- search.rs: legacy workspace markdown search (token scoring, snippets)
- 1:1 ports of index_test.go, index_state_test.go, search_test.go
- manifest.rs/coordination.rs: pub(crate) visibility for reused helpers
- index_fts5_probe.rs: import moved into test fn
… embedder

- query.rs: TrustedQuery fail-closed semantics (missing/dirty/stale/rejected),
  canonical index binding, contradiction surfacing, --include scopes,
  after/before/as_of temporal recall, RRF hybrid merge, Go-exact JSON mirror
- embedder.rs: LoopbackEmbedder (deterministic token vectors, no network),
  SQLite embedding sidecar, RebuildWithOptions
- 1:1 ports of query_test.go, hybrid_test.go, embedder_test.go (304 lib tests)
…fact

- tests/bench_100k.rs: TestAskP95At100K port gated by ZBRAIN_BENCH_100K=1,
  corpus size overridable via ZBRAIN_BENCH_CORPUS (default 100000); writes
  docs/proofs/rust-bench-100k.json in the eval-perf-100k/v1 proof schema
- 10K-corpus local run: p95 384.8ms <= 2s target (Go baseline 498ms @10k)
…s-read proofs

- fixture-gen --op index/--op ask: Go oracle emits IndexSummary, FTS search
  results (6-decimal score rounding, documented tolerance), index state,
  claims rows, generation, derived tree (index binaries excluded from hashing)
- --op index-verify/--op ask-verify: read-only re-read of either runtime's
  tree; parity.sh proves Go opens the Rust index file and vice versa
- parity.rs mirrors all four ops; ask JSON byte-identical on deterministic
  fields (rebuilt_at normalized, home paths normalized, scores rounded)
- serde_json raw_value feature for Go-style float formatting
RUSTSEC-2025-0067/0068 flagged serde_yml as unsound and unmaintained.
Parse-only surface (claims/evidence frontmatter), swap is API-identical;
150-case oracle corpus + full suite still green. Also cargo fmt normalization.
The Go oracle is preserved under tag v0.2.0-go-final and in git history.
Differential parity (fixture-gen, parity.sh, cli-parity.sh, parity bin,
bench-fts5.go) retires with the oracle; committed fixtures and golden
capture tests keep the regression value.
@tinhtute
tinhtute merged commit 3c3808f into master Sep 6, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant