Repository navigation
Add C/C++, C#, PHP, Ruby and Swift; extract Rust items inside cfg_*! wrappers; engine rules 10 - #9
Merged
NianJiuZst merged 13 commits intoNov 2, 2026
Conversation
Vendor the upstream walkers and the Swift grammar through a new kernel patch, enable the features in cg-extract, mark the languages supported and add stub facts extractors for them.
…t; extract Rust items inside cfg wrappers * facts extractors for C#, PHP and Ruby (scopes, typed bindings, imports, namespaces); C/C++ and Swift follow * module resolution: C/C++ #include by relative path and path suffix, C# namespaces (with parent namespaces), PHP namespaced names, Ruby require/require_relative, Swift modules per SwiftPM target * resolver: implicit-this members for the new families, call-name normalisation (PHP ->, parent::, self::; C# ?. base.; Ruby @ivar; C++ arity suffix), nested type paths, qualified callable lookup, partial classes / extensions / reopened classes / out-of-line members, per-family builtins and common methods * Rust: conditional item wrappers (cfg_rt! { .. }) are blanked in a position-preserving view before the kernel runs, so the items inside become nodes; node ids of everything else are unchanged * engine rules version 5
…, document results * facts extractors for C/C++ and Swift with tests; .h headers with C++ constructs are parsed as C++ (upstream looksLikeCpp rules) * a known receiver type without the member no longer falls back to a unique global name; Swift constructor calls of standard types are builtins even when the project extends the type * bench sync knows the new languages' comment, function, rename and import edits; incremental == full rebuild verified on six repos * fix file_node_only building a node row 8 bytes larger than the ABI * README (en/zh), CHANGELOG and docs/LANGUAGES-P5.md
Robustness: depth bounds and iterative helpers in the C/C++, C#, PHP, Ruby and Swift facts extractors; the cfg_*! pre-pass only blanks wrappers that really close and is kept only when it adds no syntax errors; a pre-pass timeout marks the file as timed out. Objective-C headers are recorded as file nodes instead of being parsed with the C grammar. Resolution: deterministic tie-breaking for cross-file members and qualified names (caller file, then included/imported files, then path and line); C++ `::a::f` and `::f()` resolve in the global namespace only, `::std::..` is builtin, and unqualified get/swap/move/.. in C++ files are standard functions; PHP parent::__construct() and namespaced names; Ruby single-segment require limited to load-path roots; Swift standard types from an explicit list (incl. Result, Error), shadowed by project types, nested ones only inside their enclosing declaration. The sample repositories were diffed reference-by-reference against the pre-review graph and every gained or retargeted edge was checked against source; docs/LANGUAGES-P5.md records the results.
Keep both sides: upstream's overload, framework and dynamic-dispatch resolution and the C/C++, C#, PHP, Ruby and Swift rules. Call paths added for the new languages now pass CallArgs through member_call and enclosing_member. Engine rules 8, above both 5 and upstream's 7, so indexes built by either rebuild.
cfg_*! wrappers; engine rules 5cfg_*! wrappers; engine rules 8
C/C++: port preParse, `using` declarations, constructors by arity, return and alias types, macro-opened namespaces and override edges. Ruby: bare calls as method calls, module members by name. Swift: argument-label overloads, chained calls, extension fields, nested-type and `Self` scoping, static-member chains.
A base written as fmt::detail::buffer<T>, with fmt opened by a macro, now resolves through the same scoped lookup as other C++ types, so inherited members of such bases are found. A call on a member of a member (pipe.write_end.close()) keeps its field chain instead of the bare method name the kernel gives it.
…ction
In a method of an object literal (_.mixin(Input.prototype, Emitter, {..})),
this has no enclosing class or prototype owner. The receiver is still the
object the method belongs to, not an arbitrary value, so a this.m() call
now reaches m when the file defines exactly one function of that name.
Calls on named receivers keep the existing no-guess rule.
P5-second-phase-languages.patch is rebuilt on top of main's P5-js-ts-graph.patch and now carries the later Swift and C++ walker changes; sync-kernel.sh reproduces crates/cg-kernel byte for byte. README and docs/LANGUAGES-P5.md give the upstream-oracle comparison for the six new-language repositories and against current main. rustfmt for three long lines in resolve.rs.
cfg_*! wrappers; engine rules 8cfg_*! wrappers; engine rules 10
Twelve repositories measured on one machine with the README method: one warmup and three timed rebuilds per side, two workers, a 2 GiB budget and 30-second gaps, upstream d2e0389 with its native kernel.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this does
Language coverage (issue item 2). Kernel patch P5 vendors the upstream C/C++, C#, PHP, Ruby and Swift walkers and the Swift grammar (
scripts/sync-kernel.shre-vendors; each language has a cargo feature). A.hheader with C++ constructs is parsed with the C++ grammar (upstreamlooksLikeCpprules); an Objective-C header is recorded as a file node instead of being forced through the C grammar. Each language gets a facts extractor with focused tests and resolution rules:#include "a/b.h"by includer-relative path, then path suffix;<...>by suffix only, else externalusing;using X = A.B.C/using staticbind declarationsuse A\B\Cto classes/interfaces/traits/functions/namespaces;$this->m()(emitted bare by the kernel) as an implicit-thismember;parent::,self::,static::,->,?->normalised;parent::__construct()reaches the parent classrequire/require_relativeby path suffix (single-segmentrequire 'x'only underlib/,spec/,test/,app/or the root, sorequire 'logger'stays the standard library);@ivar.m→self.@ivar.m;self.class.m→self.m; modules hold membersData(),Result { try .. },NSNumber(..)) are builtins even when the project has anextensionof them, unless a project struct/enum/protocol/typealias of that name shadows it (a nested one only inside its enclosing declaration); extensions read their type's fields;Type.staticMember.method()chains; argument-label overloads::a::fis an absolute qualified name and::f()a global-namespace function (only from the calling file or the files it includes);::std::..and unqualifiedget/swap/move/.. in C++ files are standard functions; a namespace opened by a macro (FMT_BEGIN_NAMESPACE) is recognised, sofmt::format(..)calls and bases such asfmt::detail::buffer<T>resolve; a call through a field chain (pipe.write_end.close()) keeps the chain and is typedDelegator.targetin Sinatra linked to a test class method)JS (one small, separate commit on top of #12, easy to drop):
this.m()in a function with no enclosing class and no prototype owner (an object-literal method in_.mixin(Input.prototype, Emitter, {..}), a getter passed todefineGetter) reachesmwhen the file defines exactly one function of that name. Calls on named receivers keep #12's no-guess rule.Rust call recall (issue item 1, Tokio). Items inside conditional wrapper macros (
cfg_rt! { pub struct X; impl X { .. } }, Tokio'scfg_*!family) are opaque token trees to tree-sitter, so the kernel extracted neither the declarations nor the calls inside them.cg-extract/src/macro_mask.rsnow blanks only the wrapper'sname! {and closing}in a position-preserving view (newlines kept, every other byte unchanged) before the kernel runs, so line/column positions and node ids of everything else are identical. Only wrappers that really close are blanked, and the masked text is used only when it does not add syntax errors.cfg_if!, macro definitions and other macros stay opaque; nothing is expanded.Fixes found along the way.
file_node_onlybuilt a node row 8 bytes larger than the kernel ABI row (debug-assertion panic on any kernel failure in debug/test builds).cgr bench syncnow knows the new languages' comment/function/rename/import edits.Engine rules version 10 (main is 9 after #12); indexes built by an older engine rebuild on the next sync. Format stays 2.
README evaluation (upstream oracle)
Upstream
colbymchenry/codegraph@d2e0389as the oracle, both sides indexing the same files, strict edge keys (source, target, kind, line, column), gate P ≥ 97% and R ≥ 95%.docs/)Where fmt and Alamofire fall short, much of the gap is call sites where the two sides pick different targets. Where the sampled cases were clear upstream mistakes, this PR does not copy them:
fmt::format/fmt::printcalls upstream links to thetext_styleoverloads incolor.h, and ~130 constructor calls where upstream takes the first declared constructor regardless of arity (bigint(42)→bigint(),uint128()→uint128(hi, lo),basic_string_view(s, n)→ the default constructor). The other ~340 are template overload picks (fill_n,format_decimal,write2digits, ...) where either side may be right. At the gate cgr-only can be at most ~615 edges, so even with only the ~340 clear errors counted as unavoidable, the remaining cgr-only edges would have to fall from ~884 to ~275 while shared edges grow by 463.jquery.min.js/typeahead.jquery.jsunderdocs/: minified code with many same-named nested functions, where the two sides pick different targets (~126 edges, not checked one by one) or upstream links calls this project leaves unresolved (~130).Against current main (
2c35e7a), same oracle:this.m()rule above: +6 / +13 shared edges, plus 2 + 2 edges upstream lacks that were checked correct (Expressthis.host→ thehostgetter; Zod{ callee: this.parse }→ZodMiniType::parse). Gin is identical; OkHttp and Django differ by under 0.05 points.cfg_*!items extracted here do not exist upstream, so their edges are not in the oracle.Measured performance (full index)
Same method as the README table: one warmup and three timed CLI rebuilds per side, two workers, a 2 GiB
cgrmemory budget, at least 30 seconds between runs, warm OS caches, the same files and bytes on both sides, upstreamd2e0389with its native kernel loaded. Measured 2026-11-02 on a second machine (AMD Ryzen 9 8940HX, 32 threads, 15 GiB, WSL2 Ubuntu 24.04, Node v24.18.0), so the six original repositories are re-measured and absolute times are not comparable with the README's existing table (upstream itself is about 1.7–1.9× faster there). Geometric-mean speedup 6.63× over all twelve (6.66× on the six new-language repositories); median peak RSS 55.2%–87.6% lower. Added to the README under "Second-phase languages".cgrfull indexcgrAgainst
main(2c35e7a) on the same machine: Express 158.6, Gin 153.8, Django 2,387.6, Zod 694.4 and OkHttp 860.0 ms, all within the three-sample spread of this branch. Tokio is 669.5 → 882.0 ms (+32%) because this branch also extracts the items insidecfg_*!wrapper macros. Sync and query latency were not re-measured.Validation
cargo test --workspace(incl. 6 × 100 random-edit incremental==full sequences); CI-scopecargo fmt --checkandcargo clippy --workspace --exclude cg-kernel --all-targets -- -D warningsscripts/smoke.sh;scripts/verify-repos.sh --edits 40on fmt, Alamofire and Express (incremental == full rebuild, determinism, checksums)scripts/sync-kernel.shreproduces the committedcrates/cg-kerneltree byte-for-byte from upstream9d3abb8+ P1/P2/P4/P5-js-ts-graph/P5-second-phase-languages (the last patch regenerated on top of Fix JavaScript and TypeScript graph resolution gaps #12's P5-js-ts-graph)x86_64-pc-windows-gnu(mingw-w64) cross-build succeeds including the vendored Kotlin and Swift C grammars; the binary was not executed (no Wine in the build environment) — the msvc target andinstall.ps1still need a real Windows runNot done / left for follow-up
auto x = Foo(1)constructor-vs-function by name shape). Details and caveats:docs/LANGUAGES-P5.md.