Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,12 @@ All notable changes to this project will be documented in this file.

## [Unreleased]

### Changed
- `mex check` no longer switches grounding off after a source edit. When the only reason the graph is stale is that source files changed, and the complete list of changed files is known, groundings in unchanged files are checked against the last snapshot. A node in an edited tree-sitter file (Python, Go and the other non-TypeScript languages) is re-extracted from that file with the same extractor and body hash a refresh uses, so a changed body is a real `GROUNDING_DRIFT` without a refresh. What cannot be settled that way — a deleted file, an edited TypeScript or JavaScript file (its spans come from a whole compiler program), or a node no longer found in its file under the same identity — is the new `GROUNDING_UNVERIFIED` warning, counted in the score, with a message to run `mex graph refresh`. Nothing is reported `GROUNDING_GONE` or moved from a stale snapshot, and the freshness warning stays. Config, semantic-input, branch, grammar, schema or parse-health staleness still skips grounding exactly as before. Previously the score did not move after the very edit grounding exists to catch until a full refresh. The accepted cost: every grounding in an edited TypeScript or JavaScript file reads unverified rather than clean or drifted until the next refresh (#228).

### Fixed
- Closing a read-only grounding runtime now releases the descriptor that binds `graph.db`, not only its SQLite reader. On Windows the open handle made the next refresh in the same process, after any check that opened grounding readers, fail with "The live graph changed before candidate publication" (#228).

## [0.8.2] - Unreleased

### Added
Expand Down
45 changes: 45 additions & 0 deletions src/drift/checkers/grounding.ts
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,26 @@ interface GroundingReconcilerCapabilities {
getFingerprint?(nodeId: string): Fingerprint | null;
}

/** What a snapshot stale only by changed source can still say about one node (#228). */
export type SourceDriftResolution =
/** The node's body as a refresh would record it: from the snapshot when its
* file is unchanged, or re-derived exactly from the edited file. */
| { kind: "current"; bodyHash: string | undefined }
/** Only a refresh can settle this node; `reason` says why. */
| { kind: "unverified"; reason: string };

/**
* Grounding against a graph whose only fault is that source files changed.
* Supplied by the read-only runtime, which owns the exhaustive drifted-path set.
*/
export interface SourceDriftGrounding {
resolve(nodeId: string): SourceDriftResolution;
}

export function makeGroundingChecker(
graph: GraphEngine,
reconciler: Reconciler,
sourceDrift?: SourceDriftGrounding,
): GroundingChecker {
const capabilities = reconciler as Reconciler & GroundingReconcilerCapabilities;

Expand Down Expand Up @@ -52,6 +69,26 @@ export function makeGroundingChecker(
if (!isGrounding(grounding)) continue;
const current = graph.getNode(grounding.node);
const baselineSource = capabilities.getGroundedSource?.(scaffoldFile, grounding.node) ?? null;
if (sourceDrift) {
// **A stale snapshot never reconciles.** Rename and move detection
// compares fingerprints across the whole corpus, and the corpus this
// snapshot describes is no longer the working tree. A node that looks
// gone may have moved into an edited file, so nothing here is reported
// GONE, MOVED or AMBIGUOUS: what cannot be settled is UNVERIFIED, and
// a definite DRIFT comes only from a body hash a refresh would record.
const resolution = sourceDrift.resolve(grounding.node);
if (resolution.kind === "unverified") {
issues.push(issue("GROUNDING_UNVERIFIED", "warning", source,
`Grounded node cannot be verified until \`mex graph refresh\`: ${grounding.node} (${resolution.reason})`));
continue;
}
const baselineBodyHash = grounding.bodyHash ?? baselineSource?.bodyHash;
if (baselineBodyHash !== undefined && resolution.bodyHash !== baselineBodyHash) {
issues.push(issue("GROUNDING_DRIFT", "warning", source,
`Grounded node body changed: ${grounding.node}`));
}
continue;
}
if (current) {
// **The committed hash wins, and the cached one is only a fallback.**
//
Expand Down Expand Up @@ -99,6 +136,14 @@ export function makeGroundingChecker(

if (content === null) return issues;
for (const anchor of findMexAnchors(content)) {
if (sourceDrift) {
const resolution = sourceDrift.resolve(anchor.nodeId);
if (resolution.kind === "unverified") {
issues.push(issue("GROUNDING_UNVERIFIED", "warning", source,
`Inline anchor cannot be verified until \`mex graph refresh\`: ${anchor.nodeId} (${resolution.reason})`));
}
continue;
}
if (graph.getNode(anchor.nodeId)) continue;
const baselineSource = capabilities.getGroundedSource?.(scaffoldFile, anchor.nodeId) ?? null;
const baseline = capabilities.getFingerprint?.(anchor.nodeId)
Expand Down
34 changes: 28 additions & 6 deletions src/drift/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ export type GraphAwareDriftReport = DriftReport & { graphStatus: GraphStatus };
export interface GraphAwareRunDriftCheckOpts extends RunDriftCheckOpts {
readOnlyGroundingRuntimeLoader?: (
config: MexConfig,
options?: { loadRuntime?: boolean },
options?: { loadRuntime?: boolean; allowSourceDrift?: boolean },
) => Promise<ReadOnlyGroundingRuntimeResult>;
}

Expand Down Expand Up @@ -124,13 +124,25 @@ export async function runDriftCheckWithGraphStatus(
const groundingRelevant = hasGroundings || needsGroundingMigration;
let groundingRuntime: GroundingRuntime | null = null;
let graphStatus: GraphStatus | undefined;
// True when grounding runs against a snapshot stale only by changed source
// (#228). Before, any source edit switched grounding off until a full
// refresh, so the score could not move for the very edit it exists to catch.
// The runtime now checks unchanged files against the snapshot, re-derives
// nodes in edited files exactly or reports them GROUNDING_UNVERIFIED, and
// never reconciles. Every other non-fresh reason still skips grounding.
let sourceDriftGrounding = false;
try {
try {
const loadRuntime = opts.groundingRuntimeLoader ? false : groundingRelevant;
const loaded = await (
opts.readOnlyGroundingRuntimeLoader ?? loadReadOnlyGroundingRuntime
)(config, { loadRuntime: opts.groundingRuntimeLoader ? false : groundingRelevant });
)(config, { loadRuntime, ...(loadRuntime ? { allowSourceDrift: true } : {}) });
graphStatus = loaded.graphStatus;
groundingRuntime = loaded.runtime;
sourceDriftGrounding = !usesInjectedGroundingRuntime
&& groundingRuntime !== null
&& loaded.sourceDrift === true
&& graphStatus.status === "stale";

if (usesInjectedGroundingRuntime) {
// Preserve the historical injection seam while still inspecting graph
Expand All @@ -145,15 +157,16 @@ export async function runDriftCheckWithGraphStatus(
: null;
}

if (!usesInjectedGroundingRuntime && graphStatus.status !== "fresh" && groundingRuntime) {
if (!usesInjectedGroundingRuntime && graphStatus.status !== "fresh" && groundingRuntime
&& !sourceDriftGrounding) {
const staleRuntime = groundingRuntime;
groundingRuntime = null;
staleRuntime.close();
}

if (!usesInjectedGroundingRuntime && groundingRelevant && graphStatus.status !== "fresh") {
warnGraph(
graphFreshnessWarning(graphStatus, groundingRelevant, needsGroundingMigration),
graphFreshnessWarning(graphStatus, groundingRelevant, needsGroundingMigration, sourceDriftGrounding),
);
} else if (groundingRelevant && !groundingRuntime && !graphUpgradeNudgeShown) {
graphUpgradeNudgeShown = true;
Expand Down Expand Up @@ -226,7 +239,11 @@ export async function runDriftCheckWithGraphStatus(
}
}

if (usesInjectedGroundingRuntime || graphStatus?.status === "fresh") {
// A guard that saw the graph change mid-check marks the status degraded,
// so a source-drift batch is published only while it is still `stale`.
if (usesInjectedGroundingRuntime
|| graphStatus?.status === "fresh"
|| (sourceDriftGrounding && graphStatus?.status === "stale")) {
allIssues.push(...pendingGroundingIssues);
checkerIssueCounts.push(...pendingGroundingIssueCounts);
}
Expand Down Expand Up @@ -322,8 +339,13 @@ function graphFreshnessWarning(
status: GraphStatus,
groundingRelevant: boolean,
needsGroundingMigration: boolean,
sourceDriftGrounding = false,
): string {
const groundingNote = groundingRelevant ? "; grounding checks skipped" : "";
const groundingNote = !groundingRelevant
? ""
: sourceDriftGrounding
? "; groundings in changed source files were re-read or marked unverified"
: "; grounding checks skipped";
const command = graphRemediationCommand(status);
const primary = status.diagnostics.find((entry) =>
entry.code === "GRAPH_INDEX_READER_DATABASE_CHANGED"
Expand Down
13 changes: 10 additions & 3 deletions src/graph/grounding.ts
Original file line number Diff line number Diff line change
Expand Up @@ -25,11 +25,14 @@
// MOVED -> rebind identity; WARNING if the accepted body hash differs
// AMBIGUOUS -> WARNING (GROUNDING_AMBIGUOUS) + candidate id
// GONE -> ERROR (GROUNDING_GONE)
// Graph stale only by changed source (#228) — no reconciliation:
// file unchanged, or node re-derived exactly -> the Tier-1 rows above
// file deleted, node not located exactly -> WARNING (GROUNDING_UNVERIFIED)

import type { DriftIssue, ScaffoldFrontmatter, Grounding } from "../types.js";
import type { GraphEngine } from "./engine.js";
import type { Reconciler } from "./reconcile.js";
import { makeGroundingChecker } from "../drift/checkers/grounding.js";
import { makeGroundingChecker, type SourceDriftGrounding } from "../drift/checkers/grounding.js";

export type { Grounding };

Expand Down Expand Up @@ -120,11 +123,15 @@ export type GroundingChecker = (
* graph), the drift pipeline simply does not construct this checker (spec §7);
* the eleven filesystem/lexical checkers are unaffected.
*
* Phase-0 stub: returns a checker that throws. Track B provides the real body.
* `sourceDrift` is the one additive seam since Phase 0 (#228): a graph that is
* stale only because source files changed is still checked, with every node
* the snapshot cannot vouch for reported as GROUNDING_UNVERIFIED and no
* reconciliation attempted. Omit it for the frozen fresh-graph behaviour.
*/
export function createGroundingChecker(
graph: GraphEngine,
reconciler: Reconciler,
sourceDrift?: SourceDriftGrounding,
): GroundingChecker {
return makeGroundingChecker(graph, reconciler);
return makeGroundingChecker(graph, reconciler, sourceDrift);
}
Loading
Loading