Fixes#1585. **Stacked on #1596** (the base branch is `fix/1588-rust-impl-type-qualification`; this PR's own diff is the second commit). Merge #1596 first, then retarget/merge this one.
## What was wrong
```rust
impl Outer {
pub fn run(&mut self) {
self.inner.run(); // inner: Inner
}
}
```
produced `Outer::run -> Outer::run` — recursion the source doesn't contain. The extractor collapsed every `self.<field>.<method>()` receiver to the bare method name (`run`), so the resolver only ever saw `run` and exact-matched the nearest same-named method — the calling method itself, or a method of an unrelated type. Nothing marked the edge as a guess, and no row stayed in `unresolved_refs`, so a consumer had no way to tell.
The same happened when the field's type isn't a project type at all (`its: std::vec::IntoIter<_>` → `self.its.next()`, `matcher: Regex` → `self.matcher.is_match()`): the bare `next` / `is_match` attached to whatever local method shared the name. ripgrep had 279 self-edges on `main`; the issue lists three sites, all of this shape.
(The issue's C++ control — "`Outer::run -> Inner::run` resolves correctly" — doesn't actually hold on `main`: `inner.h` is classified as C by the `.h` heuristic, so `Inner::run` never exists and the C++ repro self-edges too. That's #1592, fixed separately.)
## What this does
Rust struct fields are not graph nodes, so the field's type can only come from the struct's declaration text. This follows the Go 2-hop precedent exactly (`matchGoFieldChainCall`, #1276), including its exclusivity rule:
1. **Extraction (TS walker + native kernel, identical, parity-tested):** a call whose receiver is `self.<field>` keeps the owner-field shape — `self.inner.run()` is emitted as `self.inner.run`. Deeper chains (`self.a.b.m()`), call receivers (`self.f().m()`), parenthesized receivers and bare `self` keep the bare name, exactly as before.
2. **Resolution (`matchRustSelfFieldCall`):** owner type = the calling method's qualified-name prefix (`Outer::run` → `Outer`); the field's declared type is read from the owner struct's **own declaration lines** (comment-stripped, line by line — same discipline as the Go helper); the method is resolved **and validated** on that type by `resolveMethodOnType` (confidence 0.85, `instance-method`).
3. **Exclusive:** when the field is declared with an external type, a generic parameter (`T`), a container that doesn't auto-deref (`Option`/`Vec`/`Mutex`/…), or can't be found, the ref **stays unresolved** — it never falls through to the bare-name strategies. That is the safe behaviour the issue asks for, and it is what #1276 already chose for Go.
`rustFieldTypeName` looks through exactly the layers Rust's method-call auto-deref looks through: references (`&`, `&'a mut`) and the owning smart pointers `Box`/`Rc`/`Arc`. `Box<dyn Source>` yields the trait, whose method node the interface-impl synthesizer then fans out to every implementation. `Option<Inner>` is left alone — `self.inner.take()` is Option's method and must not become `Inner::take`.
Why it stacks on #1596: the owner is taken from the method's qualified name, which for a generic/lifetime impl was the trait's name before that fix.
## Measured on ripgrep (110 `.rs` files, #1596 build vs this branch)
| | #1596 | this PR |
|---|---|---|
| nodes | 4029 | 4029 |
| `calls` self-edges | 279 | **146** (none of the `self.<field>` shape remain — 116 bare-receiver, 30 other dotted) |
| `self.<field>.m()` calls resolved through a validated field type | — | **292** (`DecompressionMatcher::command -> GlobSet::matches`, `Parser::find_long -> FlagMap::find`, `Haystack::path -> DirEntry::path`, …) |
| `self.<field>.m()` calls left unresolved | — | **417** — every sampled one is a std/container method: `self.commands.push`, `self.child.wait`, `self.pre.is_some`, `self.colors.clone`, `self.path_terminator.unwrap_or` |
| `calls` edges total | 9150 | 8878 (the 272 removed are the former bare-name guesses for those 417) |
The issue's three sites: `walk.rs:824` now resolves to `IgnoreBuilder::add_custom_ignore_filename` (was a self-edge); `walk.rs:1195` (`self.its.next`, `IntoIter`) and `globset/lib.rs:983` (`self.matcher.is_match`, `Regex`) are parked as unresolved instead of guessed.
The issue's repro gives `Outer::run -> Inner::run` (`instance-method`, confidence 0.85) on both the kernel path and `CODEGRAPH_KERNEL=0`.
## Tests
- `__tests__/extraction.test.ts`: only the single-hop `self.<field>.<method>()` call keeps the prefix; deeper / call / parenthesized / bare-`self` receivers and a local receiver are unchanged.
- `__tests__/resolution.test.ts` (end-to-end, Cargo layout): the issue's repro → `Outer::run -> Inner::run`, no self-edge; an external field type (`std::vec::IntoIter`) with a local `next` decoy → no edge at all; `Box<Inner>` and `&'a mut Inner` resolve, `Option<Inner>` does not (even though `Inner` declares the method); a generic `T` field → no edge; genuine `self.run()` recursion keeps its self-edge; the #1588 repro's `UsesFile::go` / `UsesBuf::go` resolve to `FileSource::read` / `BufSource::read`, and a `Box<dyn Source>` field lands on `Source::read` with the synthesizer fanning out to both impls.
- `__tests__/fixtures/kernel-parity/torture.rs` grows the receiver shapes; all 15 kernel parity suites pass against the rebuilt kernel (147 tests).
- Full `npm test` on this branch: 189 files, 3187 passed, 9 skipped, 0 failed.
Re-index after upgrading.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
https://claude.ai/code/session_01LxZj6W6Y1SHXwvpT3uwJpK
Fixes#1588.
## What was wrong
The receiver of an `impl` block — the name that qualifies its methods, owns the `contains` edge, and sources the `implements` edge — was found positionally: the **last bare `type_identifier` child** of the `impl_item`. That works for `impl Source for FileSource`. But once the implementing type carries parameters it parses as a `generic_type`, and the only bare identifier left is the **trait's**:
```rust
impl Source for FileSource → FileSource::read ✓
impl<T> Source for BufSource<T> → Source::read ✗ (should be BufSource::read)
impl<'a> Iterator for Parents<'a> → Iterator::next ✗
impl Trait for &Foo → Trait::method ✗
```
Two consequences, both reproduced on `main`:
- `BufSource::read` did not exist in the graph, so `resolveMethodOnType("BufSource", "read")` and "who calls `BufSource::read`" had no answer, and every generic implementation of a trait collapsed onto the same trait-qualified name.
- Because the impl's method carried the trait's qualified name, the interface-impl synthesizer treated the impl **body** as a second trait declaration and emitted a dispatch edge from it (`Source::read -> FileSource::read`, registered at the generic impl's line — a body of `{ 0 }` containing no call at all).
The native kernel (`rustlang.rs`) mirrored the positional rule deliberately, bug-for-bug, to hold byte-parity with the TS walker — its header said "preserve, never fix via the grammar's trait:/type: fields". So the fix has to land on both sides at once.
## What this does
Both extractors now read the grammar's **named fields** instead of scanning children. One shared rule (`rustImplTypeName` in `languages/rust.ts`, `impl_type_name` in the kernel), applied to `impl_item.type`:
| implementing type | node | receiver |
|---|---|---|
| `Foo` | `type_identifier` | `Foo` |
| `Foo<T>` / `Foo<'a>` | `generic_type` → its `type` field | `Foo` |
| `m::Foo` | `scoped_type_identifier` → its `name` field | `Foo` (was: no receiver) |
| `&Foo` / `&'a mut Foo` | `reference_type` → its `type` field | `Foo` |
| `(A, B)`, `dyn Tr`, `*const T`, `u32`, fn types | anything else | none — extracted as plain functions, exactly as before |
The `implements` back-reference reads `impl_item.trait` (full text, so `fmt::Display` and `From<u32>` keep their spelling) and bails when the field is absent (inherent impl). Everything else — the no-scope impl quirk, the source-order `contains` owner scan, method extraction — is untouched; the `contains` edge simply lands on the implementing type now instead of the trait.
The kernel header comment, the parity test's description, and the two design docs that documented the quirk as "preserve" are updated to say what changed.
## Measured on ripgrep (110 `.rs` files, `main` build vs this branch)
| | main | this PR |
|---|---|---|
| nodes / methods | 4029 / 2202 | 4029 / 2202 |
| impl methods qualified by a **trait** name (node outside that trait's extent) | 61 | **0** |
| `Iterator::*` methods | 2 | 0 |
| duplicate method qualified names | 77 | 42 |
| synthesized `interface-impl` edges originating **outside** any trait declaration (the phantom fan-outs) | 38 | **0** |
| synthesized `interface-impl` edges originating at a real trait declaration | 33 | **52** |
| plain (non-heuristic) `calls` edges | 9098 | 9098 |
So the synthesizer lost every phantom edge and *gained* 19 legitimate fan-outs to implementations it could not previously see as implementations. `contains` edges went 5237 → 5224: the 13 removed were trait→impl-method edges produced by the mis-qualification.
The issue's repro now gives `BufSource::read` at line 12, `BufSource -> Source`, and both synthesized edges registered at the declaration (line 2) — identical on the kernel path and with `CODEGRAPH_KERNEL=0`. (The remaining `UsesFile::go -> BufSource::read` exact-match guess there is the separate `self.field.method()` receiver problem, #1585, which stacks on this.)
## Tests
- `__tests__/extraction.test.ts` (Rust Extraction): method qualified names for generic / lifetime / reference / scoped / generic-trait impls; the trait's qualified name names exactly one node; `implements` refs come from the implementing type for every shape; the `contains` edge lands on the type; tuple / `dyn` impls keep producing plain functions with no `implements` ref.
- `__tests__/resolution.test.ts` (end-to-end): `Source::read` names only the declaration; dispatch fans out to **both** `FileSource::read` and `BufSource::read`, every synthesized edge registered at line 2; neither impl body sprouts a synthesized call.
- `__tests__/fixtures/kernel-parity/torture.rs` grows all the new impl shapes; `kernel-rustlang-parity` (LF + CRLF) passes against the rebuilt kernel.
- `CODEGRAPH_KERNEL_EXPECT=1 npx vitest run __tests__/kernel-*.test.ts` — all 15 suites, 147 tests pass.
- Full `npm test`: 3180 passed, 9 skipped, 1 failed — `mcp-daemon.test.ts > daemon idle-times-out after the last client disconnects`, a 30 s timing test that passed on re-run in isolation (the machine was running four parallel suites and kernel builds at the time); unrelated to extraction.
Re-index after upgrading to pick up the corrected names.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
https://claude.ai/code/session_01LxZj6W6Y1SHXwvpT3uwJpK
The extraction half of #556 — indexing `.xsjs` / `.xsjslib` as JavaScript — already
landed on main via #654. This PR is now scoped to the remaining resolution gap:
the JS import-resolution list did not include the SAP HANA extensions, so an
extensionless `import { x } from './helpers'` in a `.xsjs` file resolved to
nothing and the cross-file call edge was dropped.
Add `.xsjs` / `.xsjslib` to the `javascript` entry in EXTENSION_RESOLUTION so
those imports resolve to their target file and `codegraph_callers` /
`codegraph_impact` see the edge. One resolution test covers the .xsjs -> .xsjslib
import; the now-redundant extraction/detection tests were dropped (covered by #654).
Four cases in extraction.test.ts: a Rust union carrying an
`impl Trait for` edge and owning the impl's method; a named C union
alongside a forward declaration that must NOT mint a node; a
`typedef union` taking the typedef name with no `<anonymous>` twin; a
C++ union with a member function.
Verified they fail without the fix on BOTH extraction paths — the wasm
walker via CODEGRAPH_KERNEL=0 and the kernel with a staged build.
torture.c / torture.rs gain the same shapes. The parity gate compares
the two walkers rather than a snapshot, so the fixtures do not detect
the bug on their own — they pin that the fix stays SYMMETRIC. The
regression tests above are what pin that it is present.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Census-driven cut of the top-ranked post-R7a lever. All passes TS-side,
C-only (preParseCSource), shared by both arms:
- parameterized-annotation whole-blank (__free/__printf/__counted_by/
__bpf_md_ptr…; extends through a stranded field `;`)
- type-keyword-arg scanner (kzalloc_obj(struct T), list_entry, multi-line
continuations behind nested-paren args; bounded hand scanner, head
exclusions + call-vs-declaration guard; blanks trailing stars)
- static/extern CAPS-macro declaration lines at any scope; the initialized
form is REWRITTEN to its expansion (name/tail keep exact offsets)
- va_arg qualified-type blank; GNU named-variadic #define dots-only blank
(post-restore); sandwiched notrace-family; C23 auto; multi-line
iterator-macro spans (hlist_for_each_entry_rcu + lockdep arg)
- word list += cacheline family (2- and 4-underscore spellings) + 10 more
census-confirmed annotations
Gates: five-repo parity sweeps 0 diffs (git deferral 16.1→12.2%, redis
25.3→24.1%, fmt/protobuf unchanged); linux full-tree both arms
2,049,153 nodes / 6,413,518 edges (+858/+6,585 vs R7a) with byte-identical
dumps (10,446,478 lines, sha256 6dd1185b); kernel-arm parse-loop 356→306s
at 2c; suite 2517 green under CODEGRAPH_KERNEL_EXPECT=1. Honesty note
recorded in the docs: error recovery was already salvaging most SYMBOLS on
deferred files — the graph win is relationships + phantom cleanup, and the
unreleased CHANGELOG entry was rewritten off the sweep-subset framing.
Also records §7a.5: post-R7a 8-core cg1212 re-run 16.4min (was 18.3min);
8c parse sits on the single-writer floor, so the <10min-on-8c gap re-ranks
to the per-ref resolution path.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Parity: 0 diffs on redis/git/fmt/protobuf/ALS sweeps; full-init dumps
byte-identical on all five + linux at kernel scale (10.4M dump lines,
same sha256 both arms). Linux 2c/6GB envelope: kernel-arm 19.1min vs
wasm-arm 22.9min (parse 356s vs 435s) on a much richer graph (the new
blanks recover error-swallowed code: git 2x nodes, linux kernel/+mm/ 3x).
Deferral guard corrected by measurement (C/C++ error incidence 9-42%;
--max-deferral flag); defer-reuse memo kills the 3x re-blank/re-parse
cost deferred files paid.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
SEC_ATTR UINT32 LostName(VOID) — an unknown attribute macro before a
typedef'd return type — misparses in tree-sitter's C grammar: the macro
becomes the type, the return type the declarator, and the PARAMETER
LIST is stored as the function name ("(VOID)"). The C++ grammar
recovers this shape via recoverMangledCppName, but in C the real name
never reaches the mangled string, so only a pre-parse blank can help.
Attribute macros are project-specific, so the blank keys on structure:
line-leading ALL-CAPS token followed by TWO identifiers then `(` — the
`MACRO Ret name(` definition shape. Plain typedef'd returns, ALL-CAPS
calls, #define lines, multi-word builtin returns, and mid-line uses are
all rejected by construction. Offset-preserving like the C++ blanks.
curl re-index: 5,531 C functions before and after, zero name changes;
7 nodes in memdebug.c improve start-line accuracy by 1 (the macro line
no longer counts as part of the definition).
Fixes#1211
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
An out-of-line member definition inside a namespace block takes its
qualifiedName from the declarator's receiver, which is spelled RELATIVE
to the enclosing namespace — so `namespace simulator {
ManifestStartup::Output ManifestStartup::Apply(...) {} }` indexed as
ManifestStartup::Apply while the class node carried
simulator::ManifestStartup. Fully-qualified call sites
(simulator::ManifestStartup::Apply(...)) never resolved; callers and
file impact came up empty (#1291).
The receiver-based qualifiedName now composes the active namespace
prefix, anchored at the first prefix segment the receiver re-spells
(so `namespace sim { void sim::M::f() {} }` doesn't double-prefix).
namespacePrefix is only ever non-empty for C++ — Go/Rust/Kotlin/Lua
receivers pass through unchanged.
leveldb re-index: node count byte-stable (3,044), calls edges +6,
namespace-qualified method names 947 -> 1,252.
Fixes#1291
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
template<typename T> T Box<T>::get() stored qualified_name Box<T>::get —
the <T> qualifier never matched the class node indexed as Box, so the
method didn't link to its class, while the inline form of the same
method produced Box::get. ICU-shaped multi-line template parameter
lists leaked whole <…> blocks (newlines included) into qualified_name,
exceeding NAME_MAX for downstream consumers.
extractCppReceiverType now applies stripCppTemplateArgs (the #1043
normalization for base-class refs) to the receiver qualifier.
fmt re-index: template-arg-in-qualifier names 25 -> 4 (remaining are a
FMT_BEGIN_EXPORT misparse artifact and gmock conversion-operator names,
both distinct pre-existing shapes), node count byte-stable at 7,536.
Fixes#1286
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* fix(cpp): resolve explicit operator calls (a.operator+(b)) to the operator method (#1247)
tree-sitter-cpp can't parse an operator_name in field position: the
call_expression carries `function: <receiver>` plus an ERROR child
wrapping the operator_name instead of a field_expression callee, so the
extractor emitted a calls ref named just the receiver (`a`) and the edge
never resolved — while the operator method itself indexed fine.
Two-part fix, scoped to the explicit call form (infix `a + b` / `a[i]`
need receiver type inference and are tracked in #1258):
- extraction: recover the operator_name from the ERROR child and emit
`<receiver>.operator+` (`->` receivers normalized, `this->` emits the
bare name), like any other member call
- resolution: matchMethodCall's dot pattern now admits an operator
method part (cpp-gated; symbol chars failed the \w match), so
receiver-type inference + resolveMethodOnType validate the target —
a same-named operator on an unrelated class can't capture the edge
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(cpp): harden explicit operator-call recovery against real-world shapes (#1247)
Validated on nlohmann/json (dozens of explicit operator[] / operator* /
operator< call sites). Two refinements the synthetic fixtures missed:
- normalize spaced call-site operator names (`it.operator * ()`,
`other.operator < (*this)`) to the compact form definitions index as
- drop the ref for a complex receiver (`obj()->operator+`, member chains
ending in a call) instead of emitting a bare operator name: exact-name
fallback GUESSED among unrelated same-named operators (linked a
std::map operator[] call to an in-repo operator[]) — silent miss,
never a wrong edge
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Carries #1133 forward onto current main (rebased for conflicts). A lean Unreal-Engine-style `.h` whose only C++ signal is `class ENGINE_API Foo : public Bar` (no public:/virtual/namespace/template) was misdetected as C and its class + inheritance edge silently dropped; looksLikeCpp now recognizes the export-macro-annotated class/struct shape, matching what blankCppExportMacros already recovers.
Fixes#1159.
Co-Authored-By: robertyluo <robertyluo@tencent.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The integration tests that shipped with #1158 use inputs too small to trigger
the tree-sitter class_specifier collapse, so they pass with OR without the fix
(verified: 0/3 reproduce on the pre-fix build) — only the offset-preserving unit
tests actually guard the behavior. This adds a real guard: a ~240-in-body-macro
reflected class (the density of the real CharacterMovementComponent.h) plus a
UENUM whose values carry mid-line UMETA. Asserting the decorated members and the
enum are extracted flips false->true across the three blank passes — verified the
same assertions FAIL on the pre-fix source (2a06d9a) and PASS on the fix.
The full class collapse is emergent from real engine-header content that can't be
shipped (Unreal source is EULA-licensed); this reproduces the recoverable-member
signal the collapse leaves, which regresses if any of blankCppAnnotationMacroCalls
/ blankCppApiPrefixMacros / blankCppInlineAnnotationMacros is reverted.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Heavily-reflected UE C++ classes (`UCharacterMovementComponent`, `AActor`, `UWorld`, …) were dropped from the index: in-body no-semicolon reflection macros (`UPROPERTY`/`UFUNCTION`/`GENERATED_BODY`/`UE_DEPRECATED_*`), member-level `*_API` prefixes, and mid-line `UMETA`/`UPARAM`/`UE_DEPRECATED` accumulate tree-sitter parse errors until the enclosing class_specifier collapses into an ERROR node. Three offset-preserving, C++-only pre-parse passes (`blankCppAnnotationMacroCalls`, `blankCppApiPrefixMacros`, `blankCppInlineAnnotationMacros`) blank the markup before parsing so the class survives.
Validated on the real Epic engine headers (CharacterMovementComponent.h / Actor.h / World.h): each main class + its base clause goes from DROPPED on main to recovered with the fix.
Closes#1160.
Thanks @luoyxy for the report and root-cause analysis.
Adds ArkTS (.ets, HarmonyOS/OpenHarmony) as a first-class language:
full TypeScript-grade extraction via the harmony-contrib tree-sitter
grammar (MIT, vendored byte-identical from the tree-sitter-arkts 0.2.0
npm tarball), plus the ArkUI constructs that make HarmonyOS apps
traceable:
- @Component/@ComponentV2 structs with decorators from both grammar
positions; members extract as class members with qualified names.
- build() component trees: child instantiation edges via
arkui_component_expression, no synthesizer needed.
- Attribute chains emitted dot-prefixed and resolved ONLY against
@Extend/@Styles/@AnimatableExtend/@Builder helpers (unique-or-drop) —
bare-name fallthrough produced 36,840 wrong edges (17% of calls) on
the OpenHarmony samples monorepo. All four grammar chain shapes
handled, including the detached-chain forms.
- .onClick(this.handler) method-reference bindings.
- ohpm workspace modules: bare imports follow oh-package.json5 file:
deps (ambiguous names dropped), honoring each module's main entry —
which also lets .ts consumers resolve .ets modules.
- ArkUI dynamic-dispatch bridges, all provenance:'heuristic' with
wiring-site metadata: assignment-gated state->build() re-render
(V1 @State family + V2 @Local/@Provider/@Consumer),
@ohos.events.emitter emit->subscriber pairing on static event keys
(numeric ids same-file, named constants same-module, fan-out capped),
and router.pushUrl literal urls -> the target page's @Entry struct.
- $r/$rawfile resource intrinsics treated as built-ins; arkts joins the
web language family, value-reference edges, re-export chase, and the
other TS-applicable gates.
Also ships a language-agnostic index-completeness guard: indexAll
stamps index_state (indexing -> complete/partial/failed), reconciles
discovered vs accounted files (a loaded run silently dropped 37 files),
and codegraph status surfaces truncated/partial indexes in human and
--json output.
Validated on HarmoneyOpenEye (82 files), CoolMallArkTS (528, modular
ohpm + ArkUI V2), and openharmony/applications_app_samples (11,693
files, 202,890 nodes stable across re-index, attribute false-positive
audit 36,840 -> 588 residual all-plausible). Supersedes PRs #656 and
#988 with credit — both informed this implementation.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Follow-ups noted in #1173:
- cloudposse/atmos remote-state: module.M.outputs.X emits a scoped
module.M:remote-output.X candidate; the resolver bridges it to the
target COMPONENT's own output when every gate holds — the module
source is the stack-config remote-state module, the component name is
static (a literal, or component = var.X whose variable declares a
literal default in the same directory), and exactly one directory in
the repo matches the component name and declares that output. Dynamic
(each.value) or ambiguous wiring stays unlinked. On
cloudposse/terraform-aws-components: 254 remote-state bridge edges,
every one re-derived from a matching source declaration (789/789
cross-directory output edges explained: 528 local-module + 254
remote-state + 7 checker-artifact false alarms under deprecated/);
coverage 66.4% -> 69.1%.
- provider aliases: provider "aws" { alias = "east" } is addressed as
provider.aws.east so aliased and default configurations stop
colliding; provider = aws.east on a resource/data block (and the
values of a module's providers map) reference the selected
configuration, resolved same-directory first then up the module tree
— the one construct Terraform genuinely inherits from parents. The
selection is no longer misread as a resource reference (aws.east).
- moved/import/removed blocks reference the resource addresses they
name (anchored to the file node — no phantom symbols), so a
refactor's paper trail joins the graph; check-assert conditions
contribute their references while check-scoped data blocks keep
indexing as before. Scoped module candidates are suppressed there:
module.a.aws_x.b names a resource inside a module instance, not an
output. +91 edges on cloud-foundation-fabric's moved-heavy stages.
Also fixes a latent test bug from #1173: cg.getNodeById is not public
API (cg.getNode is) — it only passed because the asserted edge list was
empty.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* feat(extraction): add Terraform and OpenTofu language support
Index .tf, .tfvars, and .tofu files via the tree-sitter-terraform dialect
of HCL (vendored from @tree-sitter-grammars/tree-sitter-hcl, Apache-2.0).
Symbols extracted:
- resource / data → class (qualified "type.name" / "data.type.name")
- module → module (qualified "module.name")
- variable → variable (qualified "var.name")
- output → variable (qualified "output.name")
- provider → namespace
- locals → constant per attribute (qualified "local.key")
References resolved cross-file:
- var.X, local.X, module.M[.out], data.T.N[.attr], <type>.<name>[.attr]
- built-ins skipped: each.*, count.*, self.*, path.*, terraform.workspace
The Terraform framework resolver disambiguates same-named candidates
across modules by preferring the one in the same directory as the
reference site, then by closest common-ancestor path, falling back to
the generic name matcher only when neither applies.
Validated on two Terraform monorepos (277 and 470 .tf files): indexing
runs in 1.3s and 2.4s respectively, query latency stays under 200ms,
and cross-module references resolve to the correct module 100% of the
time on inspected samples.
18 new extraction tests; full suite 1146/1148 green (2 pre-existing
flaky skips, 0 regressions).
* feat(terraform): bridge the module boundary and enforce directory scoping
Builds on #706. The module declaration was a dead end: module.M.out
resolved to the declaration and stopped, module inputs never reached the
child module's variables, and impact could not cross the boundary — on
real multi-module repos that breaks the core blast-radius question
("what breaks upstream if I change this module's variable/output").
- module blocks now wire across the boundary through :-scoped refs only
the Terraform resolver understands: module.M:var.<input> → the child's
variable node, module.M:output.<o> → the child's output node (emitted
alongside the module.M declaration ref), and module.M:file → the local
source directory's entry file (imports). Registry/git sources emit no
file ref and resolve nothing — an out-of-repo module stays a visible
boundary instead of a guess.
- .tfvars top-level assignments reference the variable they set, walking
up to the nearest ancestor directory (envs/prod.tfvars → root vars).
- Resolution now enforces Terraform's real scoping: same-directory only
(no cross-module fallback by common path prefix, no single-candidate
anywhere-in-tree binding), and terraform refs never fall through to
the generic name matcher — var.X can never legally bind outside its
module directory, so the fallback could only add wrong edges.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs(terraform): README language table + changelog entry + agent-eval corpus
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Javier Rodríguez Fernández <jfernandez@freepik.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
CUDA rides the C++ grammar via the Metal (#1121) dialect pattern:
blankCudaConstructs (offset-preserving) blanks execution-space specifiers
(__global__ family), __launch_bounds__(...), and <<<grid, block>>> launch
configs — which otherwise lex as shift operators and destroy the
host→kernel call edge entirely. Gated by .cu/.cuh extension OR by content
(looksLikeCudaSource), because much real CUDA lives in .h/.hpp headers:
cutlass launches most kernels from headers and flash-attention's launch
templates are .h. Safe by construction — no CUDA marker is valid C++
anywhere, and the launch blank is bounded + brace-balance-checked so a
stray <<< (committed merge-conflict markers) can never blank real code.
All real-world launch styles connect: plain, templated
(k<T, 256><<<...>>>), function-pointer (auto kernel = &fn<...>; with
branch reassignments each linked), dim3{...} brace-init configs, and
kernels defined through name-in-first-argument macros
(DEFINE_FLASH_FORWARD_KERNEL style — gtest TEST_F / PYBIND11_MODULE
shapes deliberately excluded by the two-lone-identifiers rule).
Two general C++ resolution wins the flow validation forced out:
- namespace blocks now prefix contained symbols' qualifiedNames
(prefix-only — no namespace nodes, avoiding #1093-style crowd-out), so
ns::fn(...) calls resolve; previously every namespace-qualified C++
call was a permanently dead edge. cutlass: +30,864 edges (~10%), node
count byte-identical.
- templated callees (fn<T, 256>(args)) strip template args at extraction
(mirroring #1043 for base classes), so they match their definitions.
Validated on llm.c (165 host→kernel launch edges, was 0),
flash-attention (run_flash_fwd → flash_fwd_kernel → compute_attn traces
in one codegraph_explore call), and NVIDIA CUTLASS; fmt as the plain-C++
control (unchanged). A/B n=2/arm: Read/Grep displacement decisive on all
three repos (flash-attention Reads 29,13 → 5,2).
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
escripts (.escript) index like any module — the ELP grammar has a
first-class shebang node, so no source transform is needed; main/1 and its
helpers get full function/call extraction.
OTP application resource files (<app>.app.src and compiled <app>.app) join
the graph as Erlang terms the grammar parses natively. They route by full
suffix (their last-dot extension, .src, is far too generic for the
extension map). The application tuple yields structure: {mod, {Mod, _}}
links the app to its callback module — the app's entry point — and
{applications, [...]} / {included_applications, [...]} connect umbrella
sibling apps, resolving through the OTP app-name == module-name convention;
kernel/stdlib and other out-of-repo apps stay unresolved.
App-file refs resolve only ever to MODULES: validation on emqx caught the
ssl OTP-app dependency resolving to a test helper FUNCTION named ssl (the
same defect class as the earlier -behaviour gate), so the matchReference
module-only gate now covers every ref an .app/.app.src file emits.
Validated on emqx: 2 app.src + 6 escripts indexed, entry-module and
umbrella-dependency edges all namespace-targeted post-gate, escript
functions extracted; a stray legacy/module.src stays unknown.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Calls hidden inside -define bodies were invisible: the extractor consumed
pp_define without walking the replacement, and macro use sites produced no
edges, so a call path routed through a macro (ejabberd's SQL upsert macros,
logging wrappers) was completely dark.
The macro's constant node now participates in the graph. The -define body's
calls are attributed to the MACRO — true exactly once, instead of a per-use
duplicate that would explode on logging macros — and each use site links
in: ?MACRO(...) with arguments emits a `calls` ref (inlined code joins the
call chain), a bare ?CONSTANT read emits `references` (answering "where is
this macro used" without polluting call paths). Compiler-predefined macros
(?MODULE, ?LINE, ?FUNCTION_NAME, ...) are excluded, macro-use arguments
keep walking so a call nested in ?assertEqual(ok, do_thing()) still
attributes to the enclosing function, and macro-to-macro chains connect.
Validated: node counts unchanged on cowboy/ejabberd/emqx; edges +26/+7.3K/
+42K with honest hub shapes (?T i18n, ?SLOG logging, ?QOS_1 protocol
constants); 40/40 sampled edges precise; +1.3s index cost on emqx's 2,273
files. The payoff chain on ejabberd: set_password_scram_t → ?SQL_UPSERT_T →
ejabberd_sql:sql_query_t — database writes through SQL macros now trace
end-to-end.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
gen_server:call/cast/send_request now connects to the TARGET module's
handle_call/handle_cast for every statically-named target, not just self:
a bare atom reaches the module of that name (OTP's {local, ?MODULE}
convention names a server after its module), and a macro defined as a bare
atom (-define(STORE, kv_store)) resolves the same way, alongside the
existing ?MODULE / -define(SERVER, ?MODULE) self paths. A registered name
that matches no module emits a qualified ref that never resolves — silent,
never guessed. Pid, var, and tuple targets ({global, Name}, {Name, Node})
stay unlinked.
Validated on emqx: 53 new edges, 53/53 precise (each source line is a real
registered-name gen_server request; each target module self-registers under
that name, macro-indirected registrations included). Nearly all are
test-suite → handler links — production code goes through API wrappers the
self path already covers — which is exactly the tests-exercising-this-
handler linkage blast-radius and test-gap reporting consume. ejabberd
yields zero (it always wraps): no false positives invented.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Vendored WhatsApp/tree-sitter-erlang 0.19 (the ELP grammar, ABI 14) with an
Erlang-shaped extractor: multi-clause/multi-arity functions merged into one
symbol, -spec signatures, records with fields, -type/-opaque aliases, -define
macros, -include/-include_lib file edges, and -export-driven visibility.
Modules wrap in a namespace so remote mod:fn(...) calls resolve through the
existing qualified-name matcher as mod::fn with zero resolver changes.
-behaviour declarations link to the behaviour module — gated to namespace
targets only (bare-name fallthrough linked -behaviour(supervisor) to an
unrelated macro constant on emqx). OTP indirection with static targets is
followed: spawn/apply/proc_lib/timer/rpc MFA-argument callees, and
gen_server:call/cast(?MODULE | ?SERVER) to the module's own
handle_call/handle_cast. Var-module dispatch and message sends stay
deliberately unlinked. codegraph_explore also normalizes Erlang-native query
spelling (mod:fn/3, init/2) so named symbols resolve as typed.
Benchmarked on cowboy (189 files), ejabberd (414), emqx (2,447): extraction
PASS on all three; with-codegraph arms reached 2/2/0 file Reads vs 10/5+/19
without, fastest on the largest repo.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Vendored patched govindbanura/tree-sitter-vbnet grammar (MIT, ~20-fix patch
+ new external scanner for XML literals and multi-line LINQ continuation;
provenance + rebuild instructions in docs/grammars/tree-sitter-vbnet.md),
vbnet extractor with VB-specific call/index disambiguation, Inherits/
Implements heritage, As New instantiation, events, Declare P/Invoke, and
MustOverride abstract members.
Parse health on five real repos: PolicyPlus 100%, CompactGUI 100%,
staxrip 95.2%, SCrawler 87.2%, PCL 87.5% (upstream grammar: 3-18%).
Retrieval A/B (sonnet): 26-43% faster with 0-5 file reads vs 7-20 without.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Programs, sections/paragraphs (reconstructed extents over the grammar's
flat header stream), PERFORM/THRU/GO TO/CALL call edges, COPY copybook
imports incl. standalone .cpy fragments, DATA DIVISION records/fields/
88-levels with write-site impact references, and CICS flows: EXEC
LINK/XCTL program targets (literal + same-file VALUE deref), EXEC SQL
INCLUDE, and pseudo-conversational RETURN/START TRANSID hops resolved
to the owning program via a CICS framework resolver. Fixed and free
source format (free format via a scanner wide-mode sentinel).
Grammar: vendored wasm built from a patched yutaro-sakamoto/
tree-sitter-cobol (EXEC blocks as an external-scanner token, copybook
fragment entry point, single-quote continuation, COPY REPLACING
pseudo-text, NOT=, CALL GIVING, ENTRY, FREE, bitwise ops, abbreviated
relations, COBOL-2002 usages, and more). Patch + provenance + upstream
PR draft in docs/grammars/. Parse health: AWS CardDemo 43/44 native
(upstream: 9/31), 44/44 through preParse; copybooks 28/29; CobolCraft
free-format 17/17 (upstream: 0); NIST COBOL85 unchanged at 373/382.
Copybook members resolve to files like C includes (basename index,
name-matcher short-circuit so compiler-supplied members stay honestly
unresolved): CardDemo imports 5 -> 285. Impact proof: ACCT-CURR-BAL
(CVACT01Y copybook) surfaces its 4 writer programs cross-file.
Also: run-all.sh now neutralizes the ambient prompt-hook in both A/B
arms (CODEGRAPH_NO_PROMPT_HOOK=1); COBOL corpus entries for agent-eval.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
.metal was absent from EXTENSION_MAP, so Metal Shading Language files were
silently skipped. MSL ≈ C++14, and the C++ grammar extracts its functions,
structs, type aliases, and call edges at parity with plain C++ — except MSL's
post-declarator [[attribute]] annotations, which misparse struct fields into
spurious extends refs from the struct to the field's own type (a wrong
inheritance edge whenever the repo typedefs float3/float4x4 itself, common in
shared ShaderTypes.h). blankMetalAttributes blanks them pre-parse,
offset-preserving, following the blankCppExportMacros pattern (#1061), gated
to .metal files only — in regular C++ the attribute position is legal syntax
the grammar parses natively. The preParse hook gains an optional filePath
param to support the gate.
Validated on llama.cpp's ggml-metal.metal (10.7k lines: 130 kernels vs 113
`kernel void` ground-truth lines, rope_yarn resolves its 4 kernel callers)
and SDL's shaders (PQtoLinear ← GetOutputColor), 0 bogus extends edges.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* fix(extraction): broaden the curated C++ inline-macro library list
Since #1102 the post-parse salvage already recovers the NAME for any macro, so
adding a library now buys full return-type recovery for it. Extend the curated
list across the major C++ ecosystem: Mozilla/SpiderMonkey, Protobuf, {fmt},
Hedley + nlohmann/json, GLM, Bullet (SIMD_FORCE_INLINE), Skia, OpenCV, EASTL,
Cocos2d-x, Chromium/WebKit (NEVER_INLINE), GLib, SQLite, and the unambiguous
Windows calling conventions (WINAPI / APIENTRY / STDMETHODCALLTYPE / WINAPIV —
which sit between the return type and the name, so blanking them recovers the
return type, e.g. `HRESULT WINAPI Foo()` -> Foo : HRESULT).
Every entry is an exact, curated token matched only in specifier position, so a
real all-caps return type is never touched. Anything still missed keeps its name
via the universal salvage. CARLA control unchanged (440->6 mangles, 0
regressions — none of these libs appear there, confirming no collateral). Eleven
representative full-recovery tests added.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs(changelog): note broadened C++ inline-macro library coverage (#1103)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(extraction): universal recovery of macro-mangled C/C++ function names
The curated inline-macro blank list (#1100/#1101) can't enumerate every
library's macro. Add a universal post-parse net so a function is findable by
name regardless of which macro decorates it, plus a batch of common libraries
to the curated list for full name+return-type recovery.
- recoverMangledCppName: after extraction, recover the real identifier from a
name still mangled by an un-blanked macro (`MACRO Ret name(…)` misparses to
"Ret name"). It's a new `recoverMangledName` extractor hook wired only onto
C/C++, applied to every name they produce. Safe by construction: it only
touches an already-mangled name (an internal space that isn't a legit
`operator …`/destructor), so a clean name is returned unchanged; guarded
against the `Ret (name)` parenthesized-name idiom and bare primitives. Scoped
to C/C++ so Kotlin/Scala backtick identifiers (which legitimately contain
spaces) are never touched.
- Curated list extended past UE/pugixml/Godot/Boost to Qt (Q_INVOKABLE, …),
Folly, Abseil, LLVM, V8, Eigen, and rapidjson.
Validated on CARLA (large UE project, 1131 C++/h files) vs the pre-fix baseline:
function-name mangles 440 -> 6, 431 fixed, and — critically — 0 regressions
(the salvage also recovers names that the pre-parse's own non-local error-recovery
shifts would otherwise re-mangle, erasing the 7 shifts seen in #1101). The 6
residual are all the moodycamel `Ret (name)` idiom, correctly left alone. On a
made-up macro with no list entry (`WEBKIT_EXPORT WTFString compute()`), the name
`compute` is still recovered. Full suite green; eleven regression/safety tests added.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs(changelog): note universal C++ macro-mangled name recovery (#1102)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(extraction): recognize common third-party C++ inline macros, not just UE
Extend blankCppInlineMacros beyond Unreal Engine's FORCEINLINE family to the
inline/linkage macros that vendored third-party libraries define and that
mangle function names the same way:
- pugixml: PUGI__FN / PUGI__FN_NO_INLINE (before the return type) and
PUGIXML_FUNCTION (linkage macro, between return type and name — the blank
mechanism handles both positions).
- Godot: _FORCE_INLINE_ / _ALWAYS_INLINE_.
- Boost: BOOST_FORCEINLINE / BOOST_NOINLINE.
- Generic cross-ecosystem hints: ALWAYS_INLINE / FORCE_INLINE / NOINLINE.
The list now drives a single generated alternation (longest-token-first), so
adding a codebase's macro is a one-line change. Still curated exact tokens in
specifier position only — a real all-caps return type like `HRESULT DoIt()` is
never touched (verified by controls).
Validated on CARLA (large UE project, 1131 C++/h files): function-name mangles
440 -> 16 (428 fixed). The 16 residual and 7 clean->mangled shifts are all in
third-party vendored files — chiefly pugixml.cpp, a 12k-line macro amalgamation
where error recovery is non-local, so blanking one of several *stacked* macros
(PUGI__FN + PUGI__UNSIGNED_OVERFLOW …) shifts an already-imperfect extraction.
Normal C++/UE code (ActionRoguelike, ALS) sees zero regressions — blanking a
macro there only helps. Chasing pugixml's internal attribute macros is left out
of scope. Seven regression tests added.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs(changelog): note third-party C++ inline macro recognition (#1101)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(extraction): recover C++ function names prefixed by an inline-specifier macro
An unknown inline-specifier macro before a function's return type
(`FORCEINLINE FString GetName(…)`) threw tree-sitter into error recovery: the
macro was read as the return type and — for a non-primitive return — the return
type was glued onto the name, so the function was indexed as
`"FString GetName"` instead of `GetName`, unfindable by name and with no caller
links. This is pervasive in Unreal Engine, where inline helpers are written
`FORCEINLINE <ret> <name>(…)` (e.g. ALS's `FORCEINLINE FString GetEnumerationToString`).
Add `blankCppInlineMacros`, a preParse that blanks the known UE inline macros
(`FORCEINLINE`, `FORCENOINLINE`, `FORCEINLINE_DEBUGGABLE`) with equal-length
spaces so byte offsets stay exact and the declaration parses as an ordinary
function — recovering both the real name AND the return type. This is the same
recover-don't-drop approach as blankCppExportMacros (#946/#1061), and the two
are composed into the cppExtractor preParse.
Matched tightly (exact known tokens, only in specifier position — followed by
the identifier that starts the return type/name), so ordinary identifiers, real
all-caps return types (`HRESULT DoIt()`), string literals, expression uses, and
longer words (`FORCEINLINE_COUNT`) are untouched — verified by controls. C++-only;
Kotlin/Scala re-index byte-for-byte identical. Five regression tests added.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs(changelog): note C++ inline-specifier-macro function name fix (#1100)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(extraction): correct C++ reference-return and conversion-operator method names
Two pre-existing C++ name-extraction bugs surfaced while validating the #1093
forward-declaration fix against real Unreal Engine repos (ActionRoguelike, ALS):
1. Inline methods/functions returning a reference were named after the whole
declarator. `const int& getRef() const {…}` parses with a reference_declarator
wrapping the function_declarator; extractName unwrapped pointer_declarator but
not reference_declarator, so the method was named "& getRef() const" instead
of "getRef" — polluting search and breaking caller linkage. Ubiquitous in UE
headers (`const FGameplayTagContainer& GetActiveTags() const`). Now the
reference wrapper is unwrapped alongside the pointer wrapper.
2. User-defined conversion operators were named with their full declarator —
`operator EALSMovementState() const` — instead of `operator EALSMovementState`,
so they didn't match the symbolic-overload style (`operator+`) and carried
`() const` noise. The operator_cast declarator is now named `operator <type>`.
Both are additive and C++-scoped (reference_declarator / operator_cast are C++
grammar nodes). Pointer, value, and out-of-line reference returns, and symbolic
operator overloads, are unchanged. Six regression tests added.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs(changelog): note C++ reference-return and conversion-operator name fixes (#1096)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A `class Foo;` forward declaration parses as a bodiless class_specifier.
extractStruct (#831) and extractEnum already skip their bodiless forms,
but extractClass did not — so every forward decl across dozens of headers
minted a phantom bodiless `class` node that competed with, and could be
picked as the blast-radius representative over, the single real definition.
Add an opt-in `skipBodilessClass` extractor flag (set only on cppExtractor)
and skip a bodiless class node when it's set, mirroring the struct/enum
skip. The flag keeps this C/C++-scoped: languages where a bodiless class is
a complete definition (Kotlin `class Empty`, Scala `case object`/`trait`)
leave it unset and are unaffected. The body is now resolved once at the top
of extractClass and reused for the member walk.
Regression tests cover the collapse to a single definition, elaborated-type
references creating no phantom, and Kotlin/Scala staying indexed.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A C++ class annotated with an export/visibility macro between `class`/`struct`
and the type name — `class MYMODULE_API UMyComponent : public UActorComponent`
(the standard Unreal-Engine `*_API` pattern), or the equivalent `*_EXPORT`/
`*_ABI` macros in Qt, Boost, LLVM, etc. — makes tree-sitter read `class MACRO`
as an elaborated type and the whole declaration as a function. #946 dropped the
resulting phantom function, but that also discarded the recoverable class name,
members, and base-class edge, so the class never entered the graph and
"find subclasses" / type-hierarchy / impact-through-inheritance returned
nothing for effectively every gameplay class in a UE project.
Add `blankCppExportMacros` as `cppExtractor.preParse`: it blanks the macro with
equal-length spaces before parsing (offset-preserving, like C#'s
`blankCsharpPreprocessorDirectives`/#237), so the declaration parses as a normal
class_specifier and existing extraction emits the node, members, and `extends`
edge. Generalized past UE `*_API` to any all-caps export macro, with two
false-positive guards: the trailing `[:{]` definition-guard (leaves elaborated
var decls like `struct FOO var;` alone) and requiring the macro to be followed
by the real name (leaves an all-caps class NAME such as `class FOO : public Base`
alone). C++-only, so C's heavier `struct TAG var;` never reaches it. The #946
drop stays as the fallback for any residual misparse the blanking doesn't catch.
Validated on google/leveldb (LEVELDB_EXPORT, 134 files): class/struct nodes
266→293, extends edges 292→359, phantom functions 588→513; every export-macro
real definition flips function→class and `EnvWrapper extends Env` goes
absent→present.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The gitlink discovery added in #1031/#1033 indexed a tracked 160000 gitlink even when the parent .gitignore excludes its directory, pulling a gitignored reference/benchmark corpus of git add'ed clones into the index (one report: ~138k files, 4.8 GiB, wedged "Resolving refs" watchdog).
Gate both gitlink-discovery sites on the same rule the untracked-embedded path already uses: skip a gitignored gitlink unless codegraph.json includeIgnored opts it in; index non-ignored gitlinks as before. Validated real-world on macOS, Linux, and Windows.
Closes#1065
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`instantiates` edges came only from heap `new Calculator(0)` (a
new_expression) and copy-init `Calculator c = Calculator(0)` (a
call_expression). Stack direct-init `Calculator calc(0)` and brace-init
`Widget w{1, 2}` parse as a `declaration` whose constructor arguments hang
directly off the declarator as an argument_list / initializer_list — there
is no call/new node — so the function-body walker saw no constructor
invocation and emitted no edge. A function that built objects with the
ordinary stack syntax looked like it didn't construct them, and the
dependency was missing from impact / callers.
In the body walker, a C++ `declaration` that is a stack/brace construction
now reuses extractInstantiation (a declaration's `type` field IS the
constructed class name, and extractInstantiation already strips template
args / namespace and emits the `instantiates` ref). Gated by
isCppStackConstruction, which requires BOTH a class-like type
(type_identifier / template_type / qualified_identifier — so `int x(0)`
and `auto z = …` are excluded) AND a declarator carrying args
(argument_list / initializer_list — so default `Calculator c;` and the
most-vexing-parse `Calculator c();` are excluded). The edge targets the
class node, not the same-named constructor method.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A C++ class deriving from a template — `class Derived : public Base<int>`,
a CRTP base `class App : public CRTPBase<App>`, a struct inheriting a
template, or a templated base mixed into a multi-base clause — recorded its
base as the full instantiation text (`Base<int>`). That never name-matched
the template, which is indexed as the bare node `Base`, so the `extends`
edge never resolved and the derived class looked like it inherited from
nothing — callers/impact analysis stopped at the boundary.
Strip the template arguments from the base-type reference name in the
`base_class_clause` handler via a new `stripCppTemplateArgs` helper: it
removes every balanced `<…>` group (any nesting/position), so `Base<int>`
→ `Base` and `ns::Tpl<int>` → `ns::Tpl`. The remaining qualified head is
exactly what the non-templated base case already produces, so resolution
treats templated and non-templated bases identically; a name with no
template args passes through unchanged.
Covers same-file and same-namespace bases (the dominant real-world
patterns). A base in a different namespace referenced with its qualifier
(`other_ns::Tpl<int>`) still doesn't resolve, but that's a pre-existing,
orthogonal namespace-resolution gap — the non-templated `other_ns::Plain`
fails identically — not a template issue.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A nested git repo tracked as a gitlink (mode 160000) — a clone `git add`ed
into the super-repo without a `.gitmodules` entry, or a submodule that
isn't active/initialized in this checkout — fell through both file-collection
passes: it's tracked, so the untracked `-o` listing skips it, but it's not
an active submodule, so `--recurse-submodules` won't expand it. Indexing the
top level therefore pulled in only the outer repo's own files and stopped at
the nested repo's boundary (one report: ~10 files at the root).
Switch the tracked scan to `ls-files -s` to expose file modes, collect the
unexpanded 160000 entries, and recurse into each that has a real working tree
on disk as its own embedded repo. Mirror the same discovery in
discoverEmbeddedRepoRoots so the watcher's scope stays equal to the indexer's.
Active submodules (#147) and untracked nested clones (#193) are unchanged;
gitlinks under default-ignored dirs (vendor/, node_modules/) stay excluded
(#407); an uninitialized submodule with no checkout on disk is left alone.
Adds four-shape coverage in extraction.test.ts.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Swift in-class properties are extracted by a dedicated branch in
TreeSitterExtractor.visitNode, not the generic nameField/variableTypes path
swift.ts declares. That branch had a `!isComputed` gate that dropped computed
properties entirely, so `codegraph query`/`codegraph_explore` returned "No
results found" for them — including a SwiftUI view's `var body: some View`,
the most important symbol in any SwiftUI app, and the heavily-read
`var isCloudProxy: Bool` from the report.
Stored properties were already fixed in #708 (v1.0.0); the reporter tested
v0.9.9 and confirmed "still present on main" by inspecting swift.ts only,
missing the dedicated branch — so only the computed-property half was real.
- Computed properties now index as `property` nodes; the getter is walked via
visitFunctionBody so its calls attribute to the property (a SwiftUI `body`'s
subview tree becomes the property's callees — the render flow is traceable
through it), not flattened onto the enclosing type.
- Protocol property requirements (`var x: T { get }`) — a third never-indexed
category — index as `property` too.
- Routing the getter through visitFunctionBody also stops getter-local
`let`/`var` declarations from being wrongly node-ified as struct fields
(the generic child-walk used to do this): Alamofire property 0→348, field
618→588, idempotent.
Stored/static behavior is unchanged.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A C++ class/struct annotated with an export/visibility macro —
`class MYLIB_EXPORT Foo : public Bar { … }` — makes tree-sitter read
`class MYLIB_EXPORT` as an elaborated type specifier and the whole declaration
as a `function_definition` named after the class, spanning the entire body. That
phantom `function` polluted callers/impact/blast-radius and skewed kind stats.
Detect the misparse structurally in cppExtractor.isMisparsedFunction — a
function_definition whose `type` field is a *bodyless* class/struct specifier
(the elaborated-type macro) and whose declarator is not a function_declarator —
and drop the bogus node, matching how macro-prefixed C prototypes are already
handled. The body is mangled by the same misparse and is unrecoverable. Precise
enough to leave genuine code alone: `struct P { int x; } makeP() {}` (real
inline-defined return type, has a field list) and `class Foo f() {}` (elaborated
return type on a real function, has a function_declarator) are untouched. The
leading macro alone triggers the misparse; a base clause is not required.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
R has no declaration syntax — everything is an expression — so the
extractor works through the visitNode hook: functions in every
assignment form (incl. nested, attributed to their enclosing scope),
top-level variables/constants, library()/require() imports and
source() file references (claimed, Lua-style), S4/RefClass/R6/ggproto
classes with their methods and extends edges, setGeneric/setMethod.
Grammar vendored from r-lib/tree-sitter-r v1.2.0 (ABI 14; npm package
is a security placeholder, tree-sitter-wasms has no R).
Benchmarked on AnomalyDetection (8/8 named defs), dplyr (1027 fns),
ggplot2 (150 ggproto classes / 597 methods / 128 extends edges —
adding ggproto mid-bench flipped the large-repo A/B from a regression
to 2.4x faster than the no-codegraph arm).
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
The shipped grammar parses every record form as record_declaration (no
record_struct_declaration node), so 'record struct' mis-kinded as class.
classifyClassNode now distinguishes the value-type form by its struct
keyword child, and extractStruct accepts bodiless positional records
(the no-body gate is for C/C++ forward declarations) instead of
crashing mid-file on them.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
.astro files were not indexed at all, leaving a typical Astro site mostly
invisible to search/impact/explore. New AstroExtractor (Svelte/Vue SFC
pattern): component node per file, TS frontmatter + <script> blocks
delegated to the TypeScript extractor, template {fn(...)} calls (incl. the
multiline `{posts.map((post) => (` opening line), PascalCase component-tag
references. New astroResolver: Astro global + astro:* virtual modules as
framework-provided, component resolution with the #764 ambiguity rule,
src/pages/ file-based routes ([param]→:param, [...rest]→*rest, _-prefixed
and *.config.* excluded). SFC languages now preload the TS/JS grammars
their extractors delegate to (a pure-SFC file set previously had none
loaded). Also fixes a pre-existing Svelte/Vue script-block off-by-one that
reported every script symbol one line low.
Validated per the playbook: stalux (the issue's repro) 54/54 .astro files
indexed, getIconNode found at its exact line, 14/14 routes, 93.0% fair
cross-file coverage; AstroPaper 27/27 components, 13/13 routes (underscore
dirs correctly excluded), explore connects page→Card→Datetime through the
jsx-render synthesizer; node/edge counts stable across re-syncs.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* fix(extraction): capture docstrings for export/const/decorator-wrapped symbols (#780)
getPrecedingDocstring walked previousNamedSibling from the EMITTED
declaration node, so it only found a leading comment when the comment was
a direct sibling of that node. For a declaration nested under a wrapper —
`export class X` / `export const f = () => {}` (export_statement /
lexical_declaration), a plain const arrow (variable_declarator), or a
decorated Python def/class (decorated_definition) — the comment is a
sibling of the WRAPPER, so the inner node had no preceding comment and
the docstring was stored as NULL.
Climb out through the wrapper node(s) before scanning for the comment.
Each wrapper holds exactly one declaration, so this can't mis-attribute a
comment to a sibling (verified: an uncommented method does NOT inherit its
class's comment). Also strip leading `#` from Python/Ruby/shell line
comments, which the cleanup chain missed (Python docstrings used to keep
their `#`).
Query/extraction-layer change to a parse helper; re-index to pick up
docstrings on already-indexed files. Verified on the reporter's JS/TS and
Python repros (8/8 now captured) plus over-walk controls; +3 tests.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(extraction): clean comment markers across all supported languages (#780)
Validating docstring capture across every README language surfaced that
the marker cleanup only knew C-style `//` and `/* */`, plus the `#` added
earlier this branch. Doc comments in other styles were captured but left
their markers in the stored text:
- Rust/Swift/Kotlin doc lines `///` and `//!` -> leading `/` / `!` leaked
- Lua/Luau `--` and `--[[ ]]` -> not stripped
- Pascal `{ }` and `(* *)` -> not stripped
Extract the cleanup into cleanCommentMarkers() and handle every style.
Paired block delimiters are stripped only when the comment OPENS with one,
so a line comment that happens to end with `}` / `*)` / `]]` is never
truncated; per-line markers stay anchored at line start.
Validated end-to-end (extract -> index -> codegraph_node output) across
all 19 tree-sitter code languages plus Svelte/Vue `<script>` blocks: every
one now stores and returns a clean docstring. +1 cross-language test.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A method called through a PHP fluent static factory — `ApiClient::for($c)->createOrder()`,
the canonical Laravel per-credential/per-tenant client idiom — produced no
`calls` edge: the receiver of `->createOrder` is the `Cls::for(...)` static
call, whose result type was never recovered, so the edge was dropped and
`codegraph_callers` returned nothing.
Same shape as the C++ singleton/factory fix (#645), reusing its return_type
column + the chained-call mechanism:
- Capture PHP return types (getReturnType): `: self` / `: static` / `$this`
stored as the `self` marker, a concrete `: Type` as its short name,
primitives/unions dropped.
- Encode the chained scoped-call receiver as `Cls::for().method` so the
resolver can split it (PHP-gated, in extractCall).
- New matchPhpCallChain: look up the factory's return type (`self` → the
factory's own class; concrete → that class), then resolve AND validate the
method on it — a wrong inference yields no edge, never a wrong one.
EXTRACTION_VERSION 4->5 (re-index to populate PHP return types + chained edges).
Validated on koel (1383 PHP files): node count identical (no explosion),
0 edges lost, +80 chained-call edges recovered; synthetic tests cover the
self-factory, concrete-return, namespace, decoy, and absent-method cases.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A function called only from an anonymous func_literal at package level — a
cobra `RunE: func(){…}` handler, a goroutine literal, a callback closure
stored in a `var` — had its call leak to the FILE node, because the Go
var-initializer walk ran with an empty scope. So `callers`/`impact` showed
the function with a file (or no meaningful) caller, unlike JS/TS where an
arrow-in-const becomes a named node whose calls attribute correctly.
Scope the Go top-level var/const initializer walk to the declared symbol, so
a call nested in any func_literal initializer (struct field, slice/map,
nested closure) attributes to the enclosing var. EXTRACTION_VERSION 3->4
(re-index to pick up the corrected attribution).
Validated on cli/cli (858 Go files): node/edge counts identical, file-level
dependents byte-identical (no regression), and 62 top-level-closure calls
correctly moved from file-attributed to var-attributed.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A .gitignore transparently encrypted in place by corporate DLP / endpoint
software (UTF-16 header + ciphertext), or one containing a pattern the
`ignore` library can't compile to a regex (`\[` -> "Unterminated character
class"), crashed the entire sync/index. The throw is LAZY — it surfaces at
match time (`ig.ignores()`), not `.add()` — so the existing add-time
try/catch never caught it, and the error never named the offending file.
Read .gitignore defensively: skip a file that isn't valid UTF-8 text whole
(NUL byte or fatal UTF-8 decode), drop only the individual uncompilable
patterns from a text one (probe-compile, then per-line fallback), and warn
with the file path. Indexing continues either way. The watcher inherits the
fix via buildDefaultIgnore.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
PHP's importTypes only captured namespace_use_declaration, so
include/require(_once) — the dependency mechanism in procedural and
script-style PHP — never produced edges. callers, impact, and trace
missed the entire file-include graph; only namespace `use` became a
dependency edge.
Capture the four include/require expression types and emit file→file
imports edges, reusing the path-based resolution that C/C++ #include
already goes through. Only static string-literal paths are resolved
(relative to the including file); dynamic forms (include $var,
require __DIR__ . '/x', interpolated strings) are skipped.
Include PATHS are distinguished from namespace `use` symbols by shape: a
path contains '/' or '.', which PHP identifiers and FQNs never do. A
path-shaped include that doesn't resolve to a known project file is left
unresolved and does NOT fall back to the symbol name-matcher, which would
otherwise mis-connect "inc/db.php" to an unrelated db.php elsewhere — a
wrong edge is worse than a missing one.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Colby McHenry <me@colbymchenry.com>
A C++ method call whose receiver is another call's result — `Foo::instance().bar()`,
`WidgetFactory::create().draw()`, `openSession()->run()`, or the same stored in an
`auto` local first — lost the receiver's type during extraction. The callee degraded
to a bare method name, so when two classes shared a method name the call silently
resolved to whichever was indexed first (or not at all), corrupting callers / impact /
trace with a plausible-but-wrong edge.
Three parts:
- Capture C++ return types (new nodes.return_type column, schema v5): the
function_definition's `type` field, normalized — smart-pointer pointee unwrapped,
void/primitives dropped.
- Preserve the inner-call receiver in extraction: a C/C++ field_expression whose
receiver is itself a call is encoded `inner().method` instead of dropping to the
bare name. Other languages keep the existing behavior.
- New resolution strategy (matchCppCallChain): infer the receiver's class from the
inner call's return type, then resolve AND validate the method on it. Handles
singletons/accessors, factories returning a different type, free-function
factories, make_unique/make_shared/new/direct construction, single-level member
chains, and namespace-qualified inner calls. A wrong inference yields no edge,
never a wrong one.
EXTRACTION_VERSION 2->3 (re-index to populate return types).
Validated on the issue repro + spdlog: node count stable (no explosion),
deterministic, and ~100 pre-existing wrong `.size()`-style edges removed.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Colby Mchenry