Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 27 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -77,3 +77,30 @@ jobs:
}'
)"
cargo run -p orbit-cli --bin orbit -- tool run orbit.task.add --input "$payload"

graph-equiv:
name: Graph Equivalence
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4.3.1
with:
ref: ${{ github.event_name == 'pull_request' && github.head_ref || github.ref_name }}
token: ${{ secrets.GITHUB_TOKEN }}

- uses: dtolnay/rust-toolchain@29eef336d9b2848a0b548edc03f92a220660cdb8 # stable (2026-05-07)
with:
components: rustfmt, clippy

- uses: actions/cache@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
with:
path: |
~/.cargo/registry
~/.cargo/git
target
key: ${{ runner.os }}-cargo-equiv-${{ hashFiles('**/Cargo.lock') }}
restore-keys: |
${{ runner.os }}-cargo-equiv-
${{ runner.os }}-cargo-

- name: Run graph equivalence harness
run: make ci-equiv
Empty file.
18 changes: 18 additions & 0 deletions .orbit/learnings/L-0054/learning.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
schema_version: 1
id: L-0054
status: active
scope:
paths:
- tools/graph-equiv/**
tags:
- graph-equiv
- ci
summary: Keep graph-equiv v1 checks fixture-scoped; full orbit-knowledge refresh is too slow for the CI gate
body: While wiring ORB-00320, running the equivalence harness through the persisted orbit-knowledge command graph caused the process to spend over a minute in v1 refresh/query work before any useful diff report. The CI gate needs a small, deterministic corpus, so v1-side checks in `tools/graph-equiv` should stay fixture-scoped and use the orbit-knowledge extraction compatibility layer unless a later task deliberately budgets and optimizes full-workspace v1 graph refresh. v2 should still be exercised through `orbit-graph-cli`, because that is the user-facing surface under test.
evidence:
- kind: task
reference: ORB-00320
created_at: 2026-05-25T06:09:14.226657Z
updated_at: 2026-05-25T06:09:14.226657Z
created_by: codex
priority: 120
Empty file.
3 changes: 3 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

7 changes: 6 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
.PHONY: help build release run check test fmt fmt-check clippy clean install uninstall dev watch audit tree ci ci-fast bench stability release-check cleanup-branches
.PHONY: help build release run check test fmt fmt-check clippy clean install uninstall dev watch audit tree ci ci-fast ci-equiv bench stability release-check cleanup-branches

# ------------------------------------------------------------
# Config
Expand Down Expand Up @@ -50,6 +50,7 @@ help:
@echo " make tree Print dependency tree"
@echo " make ci Full CI pass (clippy + tests + doc + guardrails; also runs on PRs)"
@echo " make ci-fast Pre-handoff gate for agents (fmt-check + guardrail scripts; no compile)"
@echo " make ci-equiv Run v1/v2 graph equivalence harness"
@echo " make stability Verify per-crate stability tier markers"
@echo " make release-check Verify /plugin install orbit version lockstep (see docs/RELEASE.md)"
@echo " make install Install CLI locally (INSTALL_PROFILE=debug optional)"
Expand Down Expand Up @@ -121,6 +122,10 @@ ci-fast:
./scripts/check-learning-layout.sh
./scripts/check-artifact-redaction-guardrail.sh

ci-equiv:
$(CARGO) build -p orbit-graph-cli -p graph-equiv
$(CARGO) run -p graph-equiv -- check --workspace .

# Verify every workspace crate declares its stability tier
stability:
./scripts/check-stability.sh
Expand Down
8 changes: 8 additions & 0 deletions bench/equiv-waivers.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# graph-equiv Waivers

No active waivers.

When a specific selector cannot meet the GRAPH_SPEC §16 tolerance immediately,
add a proposed waiver here with the query, selector, observed diff, rationale,
owner, review link, and removal condition. A waiver blocks until reviewed; it is
not a free pass to ignore the equivalence gate.
3 changes: 3 additions & 0 deletions tools/graph-equiv/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -12,3 +12,6 @@ workspace = true

[dependencies]
orbit-knowledge = { path = "../../crates/orbit-knowledge" }
serde.workspace = true
serde_json.workspace = true
sha2.workspace = true
75 changes: 53 additions & 22 deletions tools/graph-equiv/README.md
Original file line number Diff line number Diff line change
@@ -1,33 +1,64 @@
# graph-equiv

`graph-equiv` is the scaffold for the `orbit-knowledge` v1 to `orbit-graph`
v2 equivalence harness described in `docs/design/orbit-graph/specs/GRAPH_SPEC.md`
§16.
`graph-equiv` is the CI equivalence harness for the `orbit-knowledge` v1 to
`orbit-graph` v2 migration described in
`docs/design/orbit-graph/specs/GRAPH_SPEC.md` §16.

This crate intentionally lands only the harness shape:
The harness reads a frozen corpus from `tools/graph-equiv/corpus/`. There is one
line-oriented selector list per language:

- a backend trait with `search`, `show`, `refs`, `callees`, and `impact`
- a v1 backend wired to the current `orbit-knowledge` command surface for
`search`, `show`, and `refs`
- a v2 backend whose methods are `unimplemented!("orbit-graph not yet wired")`
- a local smoke command for checking that v1 can query a knowledge graph
- `rust.txt`
- `typescript.txt`
- `python.txt`
- `go.txt`

`callees` and `impact` are present on the backend trait so P6.1 can fill the
equivalence table without changing the dispatch shape; the v1 backend returns
an unsupported error for them until the exact v1 adapter is added. This scaffold
does not include the frozen selector corpus, per-query diff logic, waivers, a
Make target, or CI enforcement. The harness is not CI-enforced yet. P6.1 is the
follow-on that will add the corpus, equivalence comparison, and CI wiring.
Each non-empty, non-comment line starts with one query kind followed by its
argument, for example:

Local checks:

```sh
cargo build -p graph-equiv
cargo test -p graph-equiv
```text
search rust_helper
show symbol:tools/graph-equiv/fixtures/rust/sample.rs#rust_entry:function
refs symbol:tools/graph-equiv/fixtures/rust/sample.rs#rust_helper:function
callees symbol:tools/graph-equiv/fixtures/rust/sample.rs#rust_entry:function
impact symbol:tools/graph-equiv/fixtures/rust/sample.rs#rust_isolated:function
```

Optional v1 smoke check against an existing graph:
At startup the runner checks the committed corpus checksum. If a selector list
changes without updating the expected checksum in code, the run exits before any
backend query. This keeps corpus drift explicit in review.

## Tolerances

The diff logic implements the five GRAPH_SPEC §16 rules:

- `search <q>` compares the unordered set of `(kind, file, name)` triples. v2
`string` and `config` extras are ignored; missing v1 symbol matches and extra
v2 symbol matches fail.
- `show <sel>` compares source bytes byte-for-byte.
- `refs <sym>` compares `(file, line, kind)` triples after v2 is queried at the
`same_module` confidence floor.
- `callees <sym>` compares `(file, line, target_name)` triples.
- `impact <sym>` compares the depth-3 set of touched symbol qualified names.

Output is a structured JSON report. Any out-of-tolerance diff exits non-zero and
includes the language, corpus file line, query kind, selector, tolerance, and
offending rows.

## Running

`make ci-equiv` builds `orbit-graph-cli`, builds `graph-equiv`, then runs:

```sh
cargo run -p graph-equiv -- smoke --workspace . --query GraphCommandContext
cargo run -p graph-equiv -- check --workspace .
```

The v2 backend invokes `orbit-graph-cli` as a subprocess. Set
`ORBIT_GRAPH_CLI=/path/to/orbit-graph-cli` or pass `--orbit-graph-cli PATH` to
test a specific binary.

## Waivers

Per-query waivers live in `bench/equiv-waivers.md`. A waiver is a reviewed
blocker with rationale, owner, selector, query, and planned removal criteria. It
is not a free pass: CI should continue to fail until the waiver has been reviewed
and the follow-up disposition is explicit.
8 changes: 8 additions & 0 deletions tools/graph-equiv/corpus/go.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# Frozen graph-equiv selectors for Go fixture coverage.
search goHelper
search GoWidget
show symbol:tools/graph-equiv/fixtures/go/sample.go#goEntry:function
refs symbol:tools/graph-equiv/fixtures/go/sample.go#goHelper:function
refs symbol:tools/graph-equiv/fixtures/go/sample.go#goIsolated:function
callees symbol:tools/graph-equiv/fixtures/go/sample.go#goEntry:function
impact symbol:tools/graph-equiv/fixtures/go/sample.go#goIsolated:function
8 changes: 8 additions & 0 deletions tools/graph-equiv/corpus/python.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# Frozen graph-equiv selectors for Python fixture coverage.
search py_helper
search PyWidget
show symbol:tools/graph-equiv/fixtures/python/sample.py#py_entry:function
refs symbol:tools/graph-equiv/fixtures/python/sample.py#py_helper:function
refs symbol:tools/graph-equiv/fixtures/python/sample.py#py_isolated:function
callees symbol:tools/graph-equiv/fixtures/python/sample.py#py_entry:function
impact symbol:tools/graph-equiv/fixtures/python/sample.py#py_isolated:function
9 changes: 9 additions & 0 deletions tools/graph-equiv/corpus/rust.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# Frozen graph-equiv selectors for Rust fixture coverage.
search rust_helper
search RustWidget
show symbol:tools/graph-equiv/fixtures/rust/sample.rs#rust_entry:function
show file:tools/graph-equiv/fixtures/rust/sample.rs
refs symbol:tools/graph-equiv/fixtures/rust/sample.rs#rust_helper:function
refs symbol:tools/graph-equiv/fixtures/rust/sample.rs#rust_isolated:function
callees symbol:tools/graph-equiv/fixtures/rust/sample.rs#rust_entry:function
impact symbol:tools/graph-equiv/fixtures/rust/sample.rs#rust_isolated:function
9 changes: 9 additions & 0 deletions tools/graph-equiv/corpus/typescript.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# Frozen graph-equiv selectors for TypeScript fixture coverage.
search tsHelper
search TsWidget
show symbol:tools/graph-equiv/fixtures/typescript/sample.ts#tsEntry:function
show file:tools/graph-equiv/fixtures/typescript/sample.ts
refs symbol:tools/graph-equiv/fixtures/typescript/sample.ts#tsHelper:function
refs symbol:tools/graph-equiv/fixtures/typescript/sample.ts#tsIsolated:function
callees symbol:tools/graph-equiv/fixtures/typescript/sample.ts#tsEntry:function
impact symbol:tools/graph-equiv/fixtures/typescript/sample.ts#tsIsolated:function
17 changes: 17 additions & 0 deletions tools/graph-equiv/fixtures/go/sample.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
package graphfixtures

type GoWidget struct {
Name string
}

func goHelper() string {
return "go"
}

func goEntry() string {
return goHelper()
}

func goIsolated() int {
return 7
}
15 changes: 15 additions & 0 deletions tools/graph-equiv/fixtures/python/sample.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
class PyWidget:
def render(self) -> str:
return "python"


def py_helper() -> str:
return "python"


def py_entry() -> str:
return py_helper()


def py_isolated() -> int:
return 7
15 changes: 15 additions & 0 deletions tools/graph-equiv/fixtures/rust/sample.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
pub struct RustWidget {
pub label: &'static str,
}

pub fn rust_helper() -> &'static str {
"rust"
}

pub fn rust_entry() -> &'static str {
rust_helper()
}

pub fn rust_isolated() -> usize {
7
}
15 changes: 15 additions & 0 deletions tools/graph-equiv/fixtures/typescript/sample.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
export interface TsWidget {
render(): string;
}

export function tsHelper(): string {
return "typescript";
}

export function tsEntry(): string {
return tsHelper();
}

export function tsIsolated(): number {
return 7;
}
Loading
Loading