Commit Graph

10 Commits

Author SHA1 Message Date
Colby McHenry 1601e4ad12 feat(extraction): add Pascal/Delphi to value-reference edges (unit/class const → constant)
Pascal unit/class `const` sections already extracted as `constant`, so wiring was
add-to-`VALUE_REF_LANGS` + shadow prune + two Pascal-specific fixes:

**`constant`-only targets** — the Pascal extractor emits function parameters
(`const ATarget: TControl`, `var Dest: …`) and class fields as `variable` at
enclosing scope, which would collapse to noisy file-wide targets. Genuine shared
values are all `constant` (`declConst`), so targets are restricted to that kind.
(Unit `var` globals are the rare cost; the parameter/field noise dominates.)

**Sibling-body fix (same as Dart)** — Pascal attaches a proc body (`block`) as a
next sibling of the `declProc` header (the reader scope), both under `defProc`,
not as a child. Extended the existing Dart `function_body` sibling-pull to also
walk a `block` sibling so reads inside the body are found.

Shadow prune wired for `declConst`/`declVar` — a function-local `const X`
shadows a unit `const X`.

Validated S/M/L (horse / mORMot2 / castle-engine): node count identical on/off,
precision sample 100% TP (font/crypto/DB/FFI binding consts); headline is FFI
library-name constants read by hundreds of `external` declarations (impact
2→1880, 1→358). Caveats: low same-file density on app code (cross-unit reads;
horse gave 4 edges), the `constant`-only restriction, a rare tree-sitter-pascal
misparse of `const` params in complex Delphi signatures, and case-insensitivity
(exact-text scan misses a differently-cased reference — a miss, never an FP).
Coverage now 15 languages.
2026-06-16 09:31:29 -05:00
Colby McHenry 2ec74a2c36 feat(extraction): add Dart to value-reference edges (static const / static final constants)
Dart's grammar cleanly separates the constant idiom: a `static_final_declaration`
is exactly a top-level or class-`static` `const`/`final`, while instance fields
and `var` use `initialized_identifier` and method-locals use
`initialized_variable_definition`. A `visitNode` rule maps `static_final_declaration`
to `constant`, so there are no instance/local leaks to guard.

The reader-scan needed a Dart-specific fix: Dart attaches a method/function body
as a NEXT SIBLING of the signature node (which is stored as the reader scope),
not as a child, so the scan saw only the signature and produced zero edges until
it was taught to also walk a `function_body` next-sibling. The shadow prune counts
all three Dart declarator nodes so a method-local `const X` drops a file `const X`.

Validated S/M/L (http / flame-engine/flame / flutter/packages): node count
identical on/off; the headline is a single `static const String iconFont` read by
every Cupertino icon definition (impact 1 -> 1790). The common Dart codegen
suffixes (.g.dart/.freezed.dart/.pb.dart) are already skipped; header-only-marked
generators (JNIGEN _bindings.dart, pigeon .gen.dart) are not, so generated FFI/JNI
bindings are noisy while real source stays clean. Coverage now 15 languages.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 00:46:29 -05:00
Colby McHenry a3acc3626e feat(extraction): add Swift to value-reference edges (static let / enum-struct namespaced constants)
Top-level `let` and `static let` in a `struct`/`enum`/`class` (Swift's
shared-constant idiom — it namespaces constants in `enum`/`struct`) now extract
as `constant` and participate in impact analysis; instance stored `let`s stay
`field`, and computed properties (getter, no stored value) are skipped.

Reuses the Kotlin machinery: the name nests
`property_declaration → pattern → simple_identifier` (resolved via
firstSimpleIdentifier), and the reader-scan already matched `simple_identifier`.
Two Swift-specific touches: the value-ref target gate was widened to accept
`struct:`/`enum:` parents (every other language's targets sit at
file:/class:/module:), and computed properties are detected and skipped. Node
creation slots into the existing Swift property_declaration handler, so the
property-wrapper / type-annotation dependency extraction (@Published/@State) is
untouched.

Validated S/M/L (Alamofire / swift-argument-parser / swift-nio): node count
identical on/off, genuine static-let constants, computed properties skipped,
0-1 sibling-type collisions per repo. Coverage now 14 languages.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 00:29:04 -05:00
Colby McHenry 907098a703 feat(extraction): add C, Java, C#, PHP, Scala, and Kotlin to value-reference edges
Extends same-file constant-reader impact edges to six more languages, each
requiring a different technique:

**C** — file-scope `static const` / global nodes weren't extracted at all (name
nests in `init_declarator`, the generic fallback missed it). Added a C branch in
`extractVariable` with `cDeclaratorIdentifier` to walk the declarator chain.
Skips bare-`identifier` declarators to kill the macro-prefixed-prototype misparse
FP cluster (a `MACRO RetType fn(args);` mints a spurious type-named global).
Prune switch gains `init_declarator`.

**Java** — `static final` fields already existed but as `field` kind, which the
value-ref gate rejects. An `isConst` predicate (`static` + `final` modifiers)
re-kinds the constant subset to `constant` in `extractField`. Instance `final`
fields stay `field`.

**C#** — same `field`→`constant` approach: `const` modifier or `static readonly`
modifier pair → `constant`; instance `readonly` stays `field`.

**PHP** — constants already extracted as `constant`; the only gap was the
reader-scan. PHP represents a constant reference as a `name` node (not
`identifier`), so bare `X` and the const half of `self::X` / `Foo::X` were
invisible. Added `name` to the scan. No prune wiring needed: a `$var` local is a
`variable_name` — a different namespace — and can never shadow a bare constant.

**Scala** — top-level `val` was already `constant`; `object` and `class` vals
both came out as `field`. Fixed by walking to the enclosing AST definition in the
`val_definition` handler: `object_definition` → `constant`/`variable`; `class`/
`trait`/`enum`/`given` → `field`. Prune switch gains `val_definition`/
`var_definition`.

**Kotlin** — properties weren't extracted at all (`property_declaration →
variable_declaration → simple_identifier` is too deep for the generic path).
Added a `property_declaration` handler in `visitNode`: pulls the nested name,
walks to the enclosing scope for kind (`object`/`companion object`/top-level →
`constant`/`variable`; `class` → `field`; function body → local, skipped). Reader-
scan gains `simple_identifier`; prune switch gains `property_declaration`.

Validated S/M/L across all six languages (hiredis/curl/redis, gson/commons-lang/
guava, AutoMapper/Newtonsoft/efcore, guzzle/monolog/laravel, upickle/cats/pekko,
okio/coroutines/ktor): node count identical on/off, precision guards held, 0
shadow leaks, impact wins confirmed (`INDEX_NOT_FOUND` 4→165, `Curl_ssl` 3→57,
`_resourceManager` 22→1664, `STATE_IN_QUEUE` 1→32, `BLOCKING_SHIFT` 1→24).

C++ was attempted and reverted — tree-sitter-cpp parse fidelity on
template/macro-heavy code leaks class members to file scope as bogus constants;
did not reach the precision bar.
2026-06-16 00:06:43 -05:00
Colby McHenry b92576f660 feat(extraction): add Ruby to value-reference edges (class-scope constants)
Ruby keeps almost all constants inside a class/module (jekyll's lib: 0 top-level,
58 class-internal), so this needed the class-scope extension flagged since Go.
Three Ruby-specific changes:
- Extract Ruby constants as symbols at all: a `CONST = …` has a `constant`-typed
  LHS (not `identifier`), so they were never extracted — including class-internal
  ones (the variableTypes dispatch skipped class-internal assignments; it now
  makes an exception for `constant`-LHS assignments).
- The value-ref target gate accepts `class:`/`module:` parents, not just `file:`.
- The reader-scan matches `constant` nodes (Ruby represents both a constant's
  definition AND its references as `constant`, not `identifier`).

Effectively Ruby-only: Rust impl consts are already `file:`-parented (ripgrep
stayed at 144 edges) and TS/Python class members aren't constant/variable kind.

Class-scope precision uses a FILE-WIDE target map (not strict same-class) because
Ruby constant lookup is lexical+ancestor — a nested class legitimately reads an
enclosing class's constant (jekyll ERBRenderer→ThemeBuilder::SCAFFOLD_DIRECTORIES;
sinatra AcceptEntry→Request::HEADER_PARAM), and strict matching would drop those.
The only real FP is the same name in sibling classes in one file (~1.7% of
targets on rails; a NameError in valid Ruby, so rare). Net ~98–100% precision.

Validated small/medium/large on public Ruby OSS (sinatra, jekyll, rails): node
count identical on/off, ~98–100% precision, impact win holds (HEADER_PARAM 1→5).
Adds a Ruby test (top-level + class-internal const reads). Design matrix,
playbook (class-scope section now resolved), and CHANGELOG updated. Suite green
(1,558 passed).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-15 19:51:08 -05:00
Colby McHenry 6281a541e4 feat(extraction): add Rust to value-reference edges
Extend value-reference edges to Rust (module-level `const`/`static`). Rust's
declarators are `const_item`/`static_item` (the module consts) and
`let_declaration` (the local that shadows) — added to the per-grammar prune
switch. A synthetic probe caught the expected shadow FP (a `const TIMEOUT`
shadowed by a local `let TIMEOUT`), fixed by the addition.

No prune-RULE change needed: Rust's `#[cfg]`-conditional consts (`#[cfg(unix)]
const SEP = …; #[cfg(windows)] const SEP = …`) are kept correctly by the
Python-era file-scope-count rule (validated on tokio's io/interest.rs cfg-gated
flags). Consts written inside a config macro (`cfg_aio! { … }`) live in an
unparsed token tree, so the prune's syntax walk doesn't even see them.

Validated small/medium/large on public Rust OSS (ripgrep, tokio, rust-analyzer):
node count identical on/off (incl. 38.8k nodes on rust-analyzer), precision
samples all true positives, zero real shadow leaks, and the impact win reproduces
(rust-analyzer `INLINE_CAP` 2→183, tokio `PERMIT_SHIFT` 1→97).

Adds 2 Rust test cases (const/static read; the local-`let` shadow). Design
matrix, playbook, and CHANGELOG updated. Full suite green (1,557 passed).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-15 13:21:34 -05:00
Colby McHenry 7b0f204604 feat(extraction): add Python to value-reference edges (+ conditional-def-safe prune)
Extend value-reference edges to Python (module-level `NAME = …`; the `assignment`
declarator added to the per-grammar shadow-prune switch). A synthetic probe found
the expected shadow FP — a module `CONFIG` shadowed by a local `CONFIG = …` —
fixed by that addition.

Python also exposed a flaw in the prune RULE. Python conditionally defines module
constants (`try: X = a; except: X = b`), a very common idiom that binds a name
twice AT MODULE SCOPE. The old "declarators > 1 → drop" over-pruned these,
dropping a real const and all its readers. Refine the rule to compare declarator
count against the number of FILE-SCOPE NODES the name has: a conditional def makes
them equal (kept); a real local shadow makes declarators exceed file-scope nodes
(pruned). Strictly more correct for every language. Track file-scope node count
per name in captureValueRefScope; also suppress same-name value-ref edges (the two
halves of a conditional def would otherwise cross-reference via their own names).

Validated small/medium/large on public Python OSS (requests, sqlalchemy, django):
node count identical on/off, precision samples all true positives, guard holds,
conditional defs now kept (django recall +18 edges), zero same-name edges, and the
impact win reproduces (django `_trans` 1→138, sqlalchemy `COMPARE_FAILED` 1→26).

Adds 3 Python test cases (module read; the local-`=` shadow, verified to fail
without the prune extension; the conditional-def keep case). Design matrix,
playbook, and CHANGELOG updated. Full suite green (1,555 passed).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-15 13:01:56 -05:00
Colby McHenry c369d5d226 feat(extraction): add Go to value-reference edges (+ per-grammar shadow prune)
Extend value-reference edges to Go. Go's package-level const/var are file-scope,
so the target gate fits as-is. But Go's declarators are const_spec / var_spec /
short_var_declaration, not variable_declarator, so the shadow prune was a no-op
for Go — a package `const Timeout` shadowed by a local `Timeout := …` produced a
false positive (a synthetic probe caught it immediately). Generalise the prune's
declarator scan into a per-node-type switch (a file only contains its own
grammar's nodes, so listing all languages in one switch is safe).

Validated small/medium/large/very-large on public Go OSS (gin, hugo, prometheus,
kubernetes): node count identical on/off at every size (incl. 251k nodes on k8s),
precision samples all true positives, guard invariant clean, and the impact win
reproduces (k8s KubeletSubsystem 3→138, prometheus rdsLabelInstance 1→82).

Adds 2 Go test cases (package-const read; the shadow case, verified to fail
without the prune extension). Design matrix, playbook, and CHANGELOG updated.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-15 12:17:01 -05:00
Colby McHenry 4ca424785b test(extraction): cover the tsx/JSX value-ref read path
tsx is in VALUE_REF_LANGS and was exercised at scale by the excalidraw sweep
(tablerIconProps lives in icons.tsx), but those readers were normal expressions.
Add a unit test for the one tsx-specific path — a const read ONLY inside JSX
(`<Foo x={CONST}/>`), which depends on the reader-scan descending into the JSX
subtree — so it's locked against regression. Passes; no code change needed.
Note in the design doc that tsx needs no separate repo sweep.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-15 10:16:16 -05:00
Colby Mchenry 2f6316500d feat(extraction): enable same-file value-reference edges by default (TS/JS) (#895)
Value-reference edges (same-file `references` edges from a reader to the
file-scope const/var it reads) shipped behind CODEGRAPH_VALUE_REFS pending an
agent A/B. The A/B is in: on excalidraw the edges are correct and precise (node
count unchanged) and they transform the impact/blast-radius API — `impact` on a
const consumed by 103 readers goes from 1 affected symbol to the full radius.
That blast-radius API is what `codegraph impact` and CodeGraph Pro's verdict
engine consume, so the win is impact correctness; the agent path showed no
regression. Flip the default on; CODEGRAPH_VALUE_REFS=0 disables.

Also close the one precision gap the A/B surfaced: a bundled/Emscripten
`const Module` re-declared as an inner `var Module` / param produced false
positives (nested readers resolve to the inner binding). isGeneratedFile() is
path-only and can't catch content-minified bundles, so prune SHADOWED targets at
the syntax level — drop any value-ref target whose name is bound by more than one
`variable_declarator` in the file. On excalidraw this removes the 23 false
positives while preserving every real reader (impact unchanged at 170).

Adds regression coverage (there was none): same-file readers are edged, they
surface in the impact radius, shadowed consts are NOT edged, and
CODEGRAPH_VALUE_REFS=0 emits nothing.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-15 01:28:00 -05:00