Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions docs/code-graph-support.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,8 @@ extractor registry lives in
| **Supported** | Python | `.py` | [`sample.py`](../src/graph/__tests__/fixtures/sample.py), [`extractor-python.test.ts`](../src/graph/__tests__/extractor-python.test.ts), and the [`python-package`](../src/graph/__tests__/fixtures/python-package) integration fixture cover extraction and cross-file package resolution. |
| **Supported** | Rust | `.rs` | [`sample.rs`](../src/graph/__tests__/fixtures/sample.rs) and [`extractor-rust.test.ts`](../src/graph/__tests__/extractor-rust.test.ts) cover structs, traits, enums, modules, functions, methods, generics, imports, calls, implementations, construction, returns, and field types. |
| **Partial** | C# | `.cs` | [`sample.cs`](../src/graph/__tests__/fixtures/sample.cs) and [`extractor-csharp.test.ts`](../src/graph/__tests__/extractor-csharp.test.ts) cover namespaces (including nested/file-scoped), classes, interfaces, structs, enums, properties, overloaded indexers, field initializers, `const` fields, constructors/destructors, operators/conversions, static methods, parameters, attributes, `using` imports, calls with receivers, instantiation, and base-list extends/implements. [`engine-csharp.test.ts`](../src/graph/__tests__/engine-csharp.test.ts) verifies persistence and conservative call resolution. Ran clean (0 partial/failed) across 694 real-world `.cs` files in one large external repository. Marked partial, not supported: the `extends`/`implements` split on a class's base list is a first-listed-entry heuristic, not a semantic resolution (documented in `csharp.ts`), generics/type-parameter capture (`typeParameters`, matching Rust's `.rs` support) is not yet implemented, and call binding is limited to proven lexical scope. Calls through arbitrary objects or `base`, and qualified type references (including inheritance and construction), stay unresolved without semantic binding evidence. Recursive calls with multiple same-named overloads also remain unresolved; a unique lexical recursive call can bind to itself. Static constructors have distinct `static C` names so their identities survive reordering against instance constructors. |
| **Unsupported** | Go and other languages | All other extensions | These names may be reserved in [`src/graph/types.ts`](../src/graph/types.ts), but no grammar or extractor is registered for them. Unsupported files are skipped rather than failing a graph build. |
| **Supported** | Go | `.go` | [`sample.go`](../src/graph/__tests__/fixtures/sample.go) and [`extractor-go.test.ts`](../src/graph/__tests__/extractor-go.test.ts) cover structs, interfaces, type aliases, functions, methods, generics, imports, calls, and struct field types. |
| **Unsupported** | Other languages | All other extensions | These names may be reserved in [`src/graph/types.ts`](../src/graph/types.ts), but no grammar or extractor is registered for them. Unsupported files are skipped rather than failing a graph build. |

`src/graph/types.ts` contains a wider future-facing language vocabulary. A name
in that type union is not a support promise; the grammar and extractor
Expand Down Expand Up @@ -242,7 +243,7 @@ either resolve the requested declaration exactly or abstain.
until their language extractor and resolver work merges.
- **Support claims are fixture-bounded.** This page describes behavior exercised
in v0.7.0. It does not promise complete semantic analysis for every construct
in a supported language or support for unmerged Go, NestJS, or Next.js work.
in a supported language or support for unmerged NestJS or Next.js work.

For contributor interfaces, fixture requirements, and registration points, see
[Extending the code graph](extractors.md).
8 changes: 8 additions & 0 deletions docs/extractors.md
Original file line number Diff line number Diff line change
Expand Up @@ -127,6 +127,14 @@ When adding a language, document the grammar source and version.
- **Upstream grammar license:** MIT
- **Note:** `tree-sitter-wasms@0.1.12` also ships a `tree-sitter-c_sharp.wasm`, but it is built from a different grammar revision than the `node-types.json` published with `tree-sitter-c-sharp@0.23.5`. Several fields that `node-types.json` declares (`variable_declarator.name`, `using_directive.name`) return `undefined` via `childForFieldName` against that older build. The extractor is written and tested against 0.23.5's own field layout — use the grammar's own published wasm for this language, not `tree-sitter-wasms`'s copy.

### Go
- **Binary source:** `tree-sitter-go` package, version `0.25.0`
- **Vendored path:** `src/graph/wasm/tree-sitter-go.wasm`
- **SHA-256:** `9504573f352b20be7f2f1911754d710622aedc15afff16d5ed8fb5645681aee7` (matches `tree-sitter-go@0.25.0/tree-sitter-go.wasm`)
- **Binary package license:** MIT
- **Upstream grammar:** [tree-sitter/tree-sitter-go](https://github.com/tree-sitter/tree-sitter-go)
- **Upstream grammar license:** MIT

## Pull request proof

Before opening a pull request, run:
Expand Down
6 changes: 3 additions & 3 deletions src/graph/__tests__/cli-agent.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1375,7 +1375,7 @@ describe("runGraphQuery", () => {
"export function fetchOrders(userId: string): string[] {\n return [userId];\n}\n",
);
writeFileSync(join(mixedRoot, "OrderList.svelte"), "<script lang=\"ts\">\nlet userId = '';\n</script>\n");
writeFileSync(join(mixedRoot, "main.go"), "package main\n");
writeFileSync(join(mixedRoot, "main.java"), "class Main {}\n");
mixedEngine = createGraphEngine({ rootDir: mixedRoot });
await mixedEngine.build(mixedRoot);
db = openSqlite(join(mixedRoot, ".mex", "graph.db"));
Expand All @@ -1391,7 +1391,7 @@ describe("runGraphQuery", () => {
code: "TARGET_NOT_FOUND",
target: "refreshOrders",
filesIndexed: 1,
unindexedSources: { total: 2, byExtension: { ".go": 1, ".svelte": 1 }, truncated: false },
unindexedSources: { total: 2, byExtension: { ".java": 1, ".svelte": 1 }, truncated: false },
});
const coverageWarnings = (records: Record<string, unknown>[]) => ((records.at(-1)?.warnings ?? []) as string[])
.filter((warning) => warning.includes("coverage"));
Expand All @@ -1415,7 +1415,7 @@ describe("runGraphQuery", () => {
const changed = capture(() => runGraphQuery("where-defined", "refreshOrders", mixedRoot, mixedDeps, {}));
expect(changed.find((record) => record.code === "TARGET_NOT_FOUND")).toMatchObject({
filesIndexed: 1,
unindexedSources: { total: 2, byExtension: { ".go": 1, ".svelte": 1 }, truncated: false, observedAt: "last-build" },
unindexedSources: { total: 2, byExtension: { ".java": 1, ".svelte": 1 }, truncated: false, observedAt: "last-build" },
});
const changedScope = capture(() => runGraphScope("checkout cart pricing", mixedRoot, mixedDeps, {}));
expect(coverageWarnings(changedScope)).toEqual([expect.stringContaining("at the last graph build")]);
Expand Down
26 changes: 13 additions & 13 deletions src/graph/__tests__/coverage-histogram.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -33,16 +33,16 @@ function source(root: string, path: string, contents: string): void {
describe("unindexedExtensionHistogram", () => {
it("reports recognized source extensions no extractor handles", () => {
const root = temporaryRoot();
source(root, "main.go", "package main\n");
source(root, "internal/server/server.go", "package server\n");
source(root, "main.java", "class Main {}\n");
source(root, "internal/server/server.java", "package server;\n");
source(root, "src/App.svelte", "<script>let x = 1;</script>\n");
source(root, "src/api.ts", "export const api = true;\n");

const coverage = unindexedExtensionHistogram(root);

expect(coverage.total).toBe(3);
expect(coverage.entries).toEqual([
{ extension: ".go", files: 2 },
{ extension: ".java", files: 2 },
{ extension: ".svelte", files: 1 },
]);
expect(coverage.truncated).toBe(false);
Expand Down Expand Up @@ -92,29 +92,29 @@ describe("unindexedExtensionHistogram", () => {
const limits = { maxUnindexedFiles: 50, maxUnindexedEntries: 24 };
// Only counted files, so the processed-file cap maps exactly onto the total.
for (let i = 0; i < limits.maxUnindexedFiles + 10; i++) {
source(root, `generated/file-${i}.go`, "package main\n");
source(root, `generated/file-${i}.java`, "class File {}\n");
}

const coverage = unindexedExtensionHistogram(root, limits);

expect(coverage.truncated).toBe(true);
expect(coverage.total).toBe(limits.maxUnindexedFiles);
expect(coverage.entries[0]).toEqual({ extension: ".go", files: limits.maxUnindexedFiles });
expect(coverage.entries[0]).toEqual({ extension: ".java", files: limits.maxUnindexedFiles });
});

it("caps the reported entries at maxUnindexedEntries, most common first", () => {
const root = temporaryRoot();
const limits = { maxUnindexedFiles: 1000, maxUnindexedEntries: 3 };
for (const extension of [".go", ".rb", ".java", ".kt", ".scala"]) {
for (const extension of [".cpp", ".rb", ".java", ".kt", ".scala"]) {
source(root, `src/main${extension}`, "x\n");
}
source(root, "src/extra.go", "package extra\n");
source(root, "src/extra.cpp", "int extra;\n");

const coverage = unindexedExtensionHistogram(root, limits);

expect(coverage.total).toBe(6);
expect(coverage.entries).toEqual([
{ extension: ".go", files: 2 },
{ extension: ".cpp", files: 2 },
{ extension: ".java", files: 1 },
{ extension: ".kt", files: 1 },
]);
Expand All @@ -130,7 +130,7 @@ describe("build-time coverage cache", () => {
it("counts 600 candidates exactly in a 30k-file tree", () => {
const root = temporaryRoot();
for (let i = 0; i < 30_000; i++) source(root, `data/${i}.txt`, "");
for (let i = 0; i < 600; i++) source(root, `src/${i}.go`, "package main");
for (let i = 0; i < 600; i++) source(root, `src/${i}.java`, "class File {}");
const raw = captureGraphCoverage(root);
expect(readGraphCoverage(cachedDb(raw), root, true)).toMatchObject({ total: 600, truncated: false });
// Large fixture cleanup belongs to this test's explicit runtime budget.
Expand All @@ -140,7 +140,7 @@ describe("build-time coverage cache", () => {

it("reuses metadata without any directory discovery on reads", () => {
const root = temporaryRoot();
source(root, "src/main.go", "package main");
source(root, "src/main.java", "class Main {}");
const raw = captureGraphCoverage(root);
const spy = vi.spyOn(fs, "opendirSync").mockImplementation(() => { throw new Error("read-time walk"); });
try {
Expand Down Expand Up @@ -180,9 +180,9 @@ describe("build-time coverage cache", () => {

it("keeps build-time counts, unstamped, once the entry or directory budget is crossed", () => {
const root = temporaryRoot();
source(root, "a/one.go", "package a");
source(root, "a/one.java", "class One {}");
source(root, "b/two.vue", "<template />");
source(root, "c/three.go", "package c");
source(root, "c/three.java", "class Three {}");
const huge = vi.spyOn(fs, "opendirSync").mockImplementation(() => ({
readSync: () => ({ name: "entry" }), closeSync: () => {},
}) as unknown as fs.Dir);
Expand Down Expand Up @@ -216,7 +216,7 @@ describe("build-time coverage cache", () => {

it("verifies stamps with lstat alone, never realpath, on reads", () => {
const root = temporaryRoot();
source(root, "src/deep/main.go", "package main");
source(root, "src/deep/main.java", "class Main {}");
const raw = captureGraphCoverage(root);
const realpath = vi.spyOn(fs, "realpathSync").mockImplementation(() => { throw new Error("read-time realpath"); });
try {
Expand Down
147 changes: 147 additions & 0 deletions src/graph/__tests__/extractor-go.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,147 @@
import { readFileSync } from "node:fs";
import { dirname, join } from "node:path";
import { fileURLToPath } from "node:url";
import { beforeAll, describe, expect, it } from "vitest";
import { extractFile, loadGrammars } from "../extraction/index.js";
import type { FileExtraction } from "../extraction/index.js";

const FIXTURE = join(dirname(fileURLToPath(import.meta.url)), "fixtures", "sample.go");

describe("Go extractor", () => {
let result: FileExtraction;

beforeAll(async () => {
await loadGrammars(["go"]);
const source = readFileSync(FIXTURE, "utf-8");
result = extractFile("fixtures/sample.go", source, "go")!;
expect(result).not.toBeNull();
});

const node = (kind: string, name: string) =>
result.nodes.find((n) => n.kind === kind && n.name === name);
const hasEdge = (kind: string, targetName: string) =>
result.edges.some((e) => e.kind === kind && e.targetName === targetName);

it("emits a file node and stamps the language", () => {
expect(result.language).toBe("go");
expect(node("file", "sample.go")).toBeDefined();
});

it("extracts structs", () => {
const user = node("class", "User");
expect(user).toBeDefined();
expect(user!.isExported).toBe(true);
expect(user!.docstring).toContain("represents a user");

expect(node("property", "Name")).toBeDefined();
expect(node("property", "Age")).toBeDefined();

const order = node("class", "Order");
expect(order).toBeDefined();
expect(node("property", "ID")).toBeDefined();
expect(node("property", "Items")).toBeDefined();
});

it("extracts type aliases", () => {
expect(node("type_alias", "Role")).toBeDefined();
});

it("extracts functions and methods", () => {
const createUser = node("function", "CreateUser");
expect(createUser).toBeDefined();
expect(createUser!.isExported).toBe(true);

const greet = result.nodes.find(
(n) => n.kind === "method" && n.qualifiedName === "User::Greet",
);
expect(greet).toBeDefined();
expect(greet!.qualifiedName).toBe("User::Greet");
});

it("extracts interfaces", () => {
const greeter = node("interface", "Greeter");
expect(greeter).toBeDefined();
expect(greeter!.isExported).toBe(true);

const repo = node("interface", "Repo");
expect(repo).toBeDefined();
expect(repo!.typeParameters).toEqual(["T"]);
});

it("extracts constants and variables", () => {
expect(node("constant", "RoleAdmin")).toBeDefined();
expect(node("constant", "RoleMember")).toBeDefined();
expect(node("variable", "globalFlag")).toBeDefined();
// Every name of a multi-name declaration is emitted, not just the first.
expect(node("variable", "count")).toBeDefined();
expect(node("variable", "total")).toBeDefined();
});

it("emits import edges", () => {
expect(hasEdge("imports", "fmt")).toBe(true);
expect(hasEdge("imports", "strings")).toBe(true);
});

it("emits calls and implements references", () => {
expect(hasEdge("calls", "fmt.Sprintf")).toBe(true);
expect(hasEdge("calls", "processOrder")).toBe(true);
expect(hasEdge("calls", "consume")).toBe(true);
expect(hasEdge("instantiates", "Box")).toBe(true);
expect(hasEdge("instantiates", "Order")).toBe(true);
expect(hasEdge("instantiates", "User")).toBe(true);
});

it("nests methods under their class via contains edges", () => {
const userClass = node("class", "User")!;
const greet = result.nodes.find(
(n) => n.kind === "method" && n.qualifiedName === "User::Greet",
)!;
expect(
result.edges.some(
(e) => e.kind === "contains" && e.source === userClass.id && e.target === greet.id,
),
).toBe(true);
});

it("emits `type_of` edge for struct fields", () => {
const nameField = result.nodes.find(
(n) => n.kind === "property" && n.name === "Name",
)!;
expect(nameField).toBeDefined();
expect(
result.edges.some(
(e) => e.kind === "type_of" && e.source === nameField.id && e.targetName === "string",
),
).toBe(true);
});

it("handles generic type parameters", () => {
const box = node("class", "Box")!;
expect(box.typeParameters).toEqual(["T"]);

const makeBox = node("function", "makeBox")!;
expect(makeBox.typeParameters).toEqual(["T"]);

// A declaration can bind several names to one constraint: `[K, V any]`.
const convert = node("function", "Convert")!;
expect(convert).toBeDefined();
expect(convert.typeParameters).toEqual(["K", "V"]);
});

it("binds a method to a receiver type declared later in the file", () => {
const cache = node("class", "Cache")!;
expect(cache).toBeDefined();

const get = result.nodes.find(
(n) => n.kind === "method" && n.qualifiedName === "Cache::Get",
)!;
expect(get).toBeDefined();

// The method is contained by the receiver type, not the file.
const parents = result.edges.filter(
(e) => e.kind === "contains" && e.target === get.id,
);
expect(parents).toHaveLength(1);
expect(parents[0].source).toBe(cache.id);
});
});
80 changes: 80 additions & 0 deletions src/graph/__tests__/fixtures/sample.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
// Package main is a sample Go file for testing the code-graph extractor.

package main

import (
"fmt"
"strings"
)

// User represents a user in the system.
type User struct {
Name string
Age int
}

type Role string

const (
RoleAdmin Role = "admin"
RoleMember Role = "member"
)

var globalFlag = true
var count, total = 0, 1

// Greeter is an interface for greeting.
type Greeter interface {
Greet() string
}

func (u *User) Greet() string {
return fmt.Sprintf("Hello, my name is %s", u.Name)
}

func processOrder(order *Order) {
fmt.Println("Processing order:", order.ID)
}

type Order struct {
ID int
Items []string
}

func (o *Order) AddItem(item string) {
o.Items = append(o.Items, item)
}

type Box[T any] struct {
Value T
}

func makeBox[T any](val T) Box[T] {
return Box[T]{Value: val}
}

func consume(val any) {}

func CreateUser(name string) *User {
u := &User{Name: name, Age: 30}
processOrder(&Order{ID: 1})
u.Greet()
consume(makeBox(42))
return u
}

type Repo[T any] interface {
Get() T
}

func (c *Cache) Get(key string) string {
return c.items[key]
}

func Convert[K, V any](k K, v V) (K, V) {
return k, v
}

type Cache struct {
items map[string]string
}
Loading