Skip to content

feat: answer who depends on this, and what must rebuild - #164

Open
evanbrobertson wants to merge 7 commits into
masterfrom
feat/reverse-dependency-edges
Open

feat: answer who depends on this, and what must rebuild#164
evanbrobertson wants to merge 7 commits into
masterfrom
feat/reverse-dependency-edges

Conversation

@evanbrobertson

Copy link
Copy Markdown
Contributor

Closes the reverse-edge half of #103. Comments the six edge kinds and the CRC ruling on #56.

oxabl reads ABL well but cannot answer the question in reverse. Its index exposes four forward lookups and nothing asks "who depends on me". The raw material was already there — each analysis run records the files it consulted, plus rows for references it could not resolve — but nobody had inverted it, and the edges did not record why one file depends on another.

This adds that, in five steps.

The preprocessor now says which includes a file names itself. SpanNode::Include nests, so the nesting was visible while the span tree existed — but the variant never recorded which file it expanded, and it is shared with {&var} and {N} substitutions that have no file at all. Below that, ExpandedFile deliberately keeps no rich tree and its dependency list is flat and transitive. So the node carries its resolved path now, None marking a substitution, and ExpandedFile captures the depth-1 set at its own construction site — the last place the nesting can be read. A resolved include also emits a node when its content is empty: "this file includes that one" is a fact about the source, not about how much text arrived.

oxabl_index gains a typed edge set. For one file: every file it depends on, the reason, and the span where the dependency is written. Six reasons — direct include, transitive include, schema table, class reference, program reference, shared producer. A new type rather than methods on WorkspaceIndex: that seam is consulted during resolve, per name, per file, while this is a workspace-level question asked once the facts exist. The builder takes the model, the schema, and the include sets as arguments, because the crate depends on neither the preprocessor nor the analysis crate and must not learn to — that direction is what keeps the reverse query clear of salsa.

oxabl_pipeline inverts them into a workspace graph. Dependents of a file or a schema table, grouped by cause, and the rebuild set as the transitive closure. The two are separate answers: "files that mention this" and "files a compiler must touch again" are different numbers. This layer owns the pass because it is the only one where the builder’s three inputs exist together. Memoization stays out — this crate must never depend on salsa.

The analyze envelope reports the edges, dependencies section 2 → 3. via widens from three values to six, name becomes target, file becomes optional, and rows carry the dependent file’s own offsets rather than post-expansion ones. The section arrives as data: the derivation lives in oxabl_index, above the analysis crate, so deriving it twice would put two answers to one question in the workspace.

The parity table can finally observe edges. It watched diagnostics and the format decision, so two clients could disagree about a dependency graph with every leg green. Each claim states what withholding a fixture’s siblings does, because that half is load-bearing.

Honesty, structurally

Under-reporting impact is the failure mode that ends trust, so the gaps are data rather than silence:

  • An include the preprocessor cannot locate is carried out of the expansion as an unresolved row. Previously it was only a PREPROC007 diagnostic, which means a missing edge nobody mentions.
  • Unresolved references live in their own collection with no accessor that merges them, so a caller cannot count a gap as a dependency.
  • A file the pass cannot read, preprocess, or analyse is recorded as unanalysed, never folded into "depends on nothing".
  • A schema edge carries no CRC and no TableId; a revision mismatch yields no schema edges rather than wrong ones.
  • The workspace unresolved ratio is reported, so an answer’s trustworthiness is legible instead of assumed.

Two known fidelity gaps are pinned by fixtures rather than papered over — a class named only by USING/NEW/AS CLASS, and a missed RUN target or SHARED producer, leave no unresolved row. Both are commented on #56.

Notes

  • Neither oxabl_index nor oxabl_pipeline names salsa or the preprocessor. The only mentions are the prohibition comments.
  • The pass is sequential on purpose: a salsa database is Send but not Sync, so there is no bound to parallelise behind.
  • oxabl analyze now runs its two phases by hand instead of through run, so the edges come from the expansion and model it already has rather than costing a second analysis.
  • Whole-workspace build time and graph size on the benchmark codebase are still to be measured; they need a driver, which arrives with the daemon’s method surface.

Verification

cargo check --workspace, cargo test --workspace (86 test binaries), cargo clippy --workspace --all-targets -- -D warnings, and cargo fmt --all -- --check all pass.

A dependency edge cannot say "direct include" versus "transitive include"
today, because the fact is destroyed on the way out of the preprocessor.
`SpanNode::Include` nests, so the nesting is visible while the span tree
exists — but the variant never recorded *which* file it expanded, and it is
shared with `{&var}` and `{N}` substitutions that have no file at all. Below
that, `ExpandedFile` deliberately keeps no rich tree and its dependency list
is flat and transitive.

So the node now carries the resolved include path, `None` marking a
substitution, and `ExpandedFile` captures the depth-1 include set at its own
construction site — the last place the nesting can still be read. A resolved
include also emits a node when its content is empty: "this file includes that
one" is a fact about the source, not about how much text arrived.

`PartialEq`/`Eq` stay intact on both, so salsa still backdates an edit whose
expansion is unchanged and cuts off the downstream diagnostics query.
The index answers four forward lookups and nothing asks the reverse. This adds
the per-file half of that: a typed edge set naming each dependency and the
reason for it — direct include, transitive include, schema table, class
reference, program reference, shared producer — plus the referencing span in
the dependent file, so a client can open an editor where the dependency is
written.

A new type rather than methods on `WorkspaceIndex`: that seam is consulted
during resolve, per name, per file, while this is a workspace-level question
asked once the facts exist. The builder takes the semantic model, the schema,
and the include sets as arguments, because this crate depends on neither the
preprocessor nor the analysis crate and must not learn to — that direction is
what keeps the reverse query clear of salsa.

Unresolved references are members of the set, never folded into a resolved
count: an impact answer that absorbs them under-reports the blast radius while
looking more confident. Includes the preprocessor could not locate are carried
out of the expansion for the same reason. A schema edge names the folded table
and no CRC — a CRC needs a compiler, and a `TableId` means nothing outside the
`Schema` that minted it.

One edge per reason per target rather than per mention, so a table named in
fifty places does not read as fifty times the dependency; the surviving span is
the earliest site, deterministically.
Inverts the per-file edge sets into a workspace graph and exposes the two
questions the product is named after: the dependents of a file or a schema
table, grouped by the reason each one depends on it, and the rebuild set as the
transitive closure. The two are separate answers because "files that mention
this" and "files a compiler has to touch again" are different numbers, and
conflating them is how an impact answer stops being trusted.

This layer owns the pass because it is the only one where the edge builder's
three inputs exist together — the semantic model, the real schema, and the
expansion's include sets. Memoization stays out: this crate must never depend on
salsa, so the graph is built once and a client that wants it warm holds it.

Honesty is structural rather than documented. A file that cannot be read,
preprocessed, or analysed is recorded as unanalysed, never folded into "depends
on nothing". Unresolved references are carried out of the pass, matched to the
subject they named, and reported beside a dependent set rather than inside it;
the workspace-level ratio makes an answer's trustworthiness legible instead of
assumed. A dependent carries the offsets where it writes the reference, so a
client can open an editor at the line that creates the dependency.

Cycles terminate, each file appears once, and the pass is sequential on purpose:
a salsa database is Send but not Sync, so there is no bound to parallelise
behind. Measure the cost rather than adding threads.
The `dependencies` section reported which files a run consulted and, for a class
lookup, which came back empty. It now reports the typed edge set: `via` widens
from three values to the six edge kinds, `name` becomes `target` — a resolved
path for an include, a folded name otherwise — `file` becomes optional, since an
include mints no index id and a schema table has no file at all, and every row
carries the offsets where the analysed file writes the reference, in that file's
own bytes rather than the post-expansion coordinates the unresolved rows used to
report. Section version 2 to 3; no other section moves.

The section now arrives as data rather than being derived here. The derivation
lives in `oxabl_index`, which sits above this crate and must never become a
normal dependency of it, so deriving the edges a second time here would put two
answers to "what does this file depend on" in the workspace. `oxabl_pipeline`
owns both sides and does the conversion; this crate serialises what it is handed,
and its section tests moved to where the converter is — which is also the only
place all six kinds can be produced.

`oxabl analyze` runs its two phases by hand rather than through `run`, so the
edges come from the expansion and model it already has instead of costing a
second analysis of the same file.
The parity table observed diagnostics and the format decision, so a reverse
dependency query was invisible to it: two clients could disagree about what a
file depends on and every leg would stay green. The table now carries an
expected-edge channel, and the pipeline leg asserts it exactly.

Each claim states what withholding the fixture's siblings does, because the
withheld half is the load-bearing one — a row that only pinned the supplied
answer would pass just as well if the edges arrived unconditionally. Two new
rows cover the kinds no existing fixture reached: real nested includes, where
direct and transitive must not be confused, and a `RUN` target beside the file
whose `DEFINE NEW SHARED` the root consumes. All six kinds are now claimed by
some row, and a test says so, so a kind cannot quietly become unobservable.

Two claims deliberately pin a *gap* rather than the answer we would prefer.
Withholding a class named only by `USING`, `NEW`, or a declared type leaves no
unresolved row, and neither does a missed `RUN` target or `SHARED` producer:
those links are recorded only when the index answers. The findings still reach
the diagnostic channel, so nothing is under-reported to a user — but the edge
set is silent, and writing that down is what will make the fix visible when
edge-kind fidelity work lands.

A cross-file row can now need a capability the browser lacks, so the browser's
cross-file comparisons filter on comparability the way its other ones already do.
@codspeed-hq

codspeed-hq Bot commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will degrade performance by 14.35%

❌ 1 regressed benchmark
✅ 41 untouched benchmarks

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Benchmark BASE HEAD Efficiency
micro/var_substitution 180.2 µs 210.4 µs -14.35%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing feat/reverse-dependency-edges (43df518) with master (a1e03e6)

Open in CodSpeed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant