detect() converts Office files and Google Workspace shortcuts into
markdown sidecars under a converted subdirectory of the output dir,
then ran the same scan ignore check on the sidecar's own path. The
documented gitignore advice puts the whole output dir inside a
gitignored tree, so that check rejected the tool's own output for
the same reason a user would gitignore it, and the source document
vanished from the corpus with nothing recorded about why. The check
exists to keep user files out of the scan, not to filter output this
same pass just produced from an already admitted source, so a
sidecar under the converted directory is now exempt from it. Fixes
#3504.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017qfdzgbA5KedGEjD1AayNh
(cherry picked from commit 900b376d39)
Review follow-up: a wide TS outDir keeping compiled files only under a
late-sorted module dir slipped past the 20-subdirectory cap. The level-two
probe now covers every subdirectory, bounded by total entries scanned.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JJbLfztSxm2tH5cJwBbe9q
(cherry picked from commit ff0a8f6d64)
Add a Robot Framework extractor using the official robot.api parser (pure-Python, optional
[robot] extra), emitting suite/test/keyword nodes plus keyword-call, resource-import, and
library edges. Keyword resolution is case/space/underscore-insensitive (RF's real semantics)
and handles BDD prefixes and ${CURDIR}/${EXECDIR} path resolution; degrades to an actionable
error when robotframework is absent.
Senior tweaks on the landed #3074:
- Lift the **-aware matcher out of the per-call @lru_cache closure (which referenced itself
and leaked a reference cycle every call) into a module-level function with an explicit
dict memo, plus a no-** fast path; port of the fix from the alternative #2845.
- Cap the process-global parsed-pattern cache so a long-lived watch across many repos cannot
grow it without bound.
- Add a differential test pinning _lexical_relative against pathlib relative_to, and a
gc test asserting the matcher leaves no cyclic garbage.
Ignore evaluation built a Path and called relative_to (with its CPython quadratic parents
cost) per pattern per file, and re-parsed each pattern's flags every call. Replace it with a
lexical string-space relative computation (no Path construction, no exception control flow),
parse each pattern once into a process-wide cache, and memoize per-anchor relatives, segment
splits and is_dir per evaluation. Ignore decisions are unchanged (differential-fuzz
verified); ~50x faster on a pattern-heavy monorepo.
Converted Office/Google-Workspace sidecars were written to `root/GRAPHIFY_OUT/converted`,
always anchored to the scanned source tree, so `graphify extract --out <dir>` against a
read-only or pinned checkout polluted the source tree. Route sidecar writes through the
cache root while keeping the content hash anchored to the scan root, so sidecar filenames
stay stable across checkouts.
save_manifest unconditionally stamped a fresh 'seen' timestamp for every file
on every run, so a no-op graphify update rewrote all manifest entries and left
graphify-out/ permanently dirty (a trailing graph commit on every push). Now
the per-file 'seen' is preserved for genuinely unchanged entries, and the disk
write is skipped entirely when the serialized manifest is byte-identical to
what is already on disk; a real change still refreshes and persists.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
tree-sitter-commonlisp-backed extractor for packages, classes, functions,
methods, generics, macros, variable definers, and same-file calls. Handles the
grammar's dedicated defun node (defun/defmacro/defmethod/defgeneric via
defun_header) and the generic list_lit + leading-symbol forms (defvar/defclass/
defpackage), recurses into wrapper macros (eval-when/progn) and reader
conditionals, and maps CL operator chars (= < > ? ! + *) to readable id
suffixes. Wired into detect/extract dispatch, the extractor registry, a
[commonlisp] optional extra, and the README; ships a fixture and a 23-test suite
behind an importorskip guard.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The #2799 fallback (utf-8 -> host codepage -> latin-1) turned a BOM'd UTF-16
ignore file (what PowerShell Set-Content / Notepad 'Unicode' write) into
NUL-laden mojibake via latin-1, so its rules matched nothing. Detect the
UTF-16 BOM and decode as utf-16 before the latin-1 fallback. Adds an
end-to-end UTF-16 exclusion test and a direct no-NUL-garbage test.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Ignore files were read with errors="ignore", so a mis-encoded byte in a rule
(e.g. a cp1252 accented directory name) was deleted, turning the pattern into
one that matches nothing — an exclusion that silently failed open. Both
rule-read sites now decode UTF-8-BOM first, then fall back to the host codepage
and latin-1 (never raising, never dropping bytes), with a one-time warning; the
rule survives intact instead of being truncated.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
#2766 ran `git ls-files` on every scan of a git repo. Gate it on .gitignore
actually contributing patterns beyond the explicit (.graphifyignore/--exclude)
set — with no .gitignore in play nothing can be gitignore-dropped, so the
tracked-file exemption is moot and an ordinary corpus pays no subprocess cost.
Adds a spy test asserting the probe is skipped without a .gitignore and runs
once with one. Also adds the CHANGELOG entry.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A file that is git-tracked but also matches a .gitignore pattern (a committed
file later added to .gitignore, or a force-added one) was silently dropped from
the corpus, even though git never un-tracks such a file — so the graph lost
real, shipping source. detect now runs `git ls-files` once per scan, keeps a
set of tracked-file identity keys plus their ancestor directories (so a
git-only-ignored parent dir is not pruned before the walk reaches the tracked
file below it), and exempts tracked paths from .gitignore. .graphifyignore and
--exclude stay authoritative, and a non-git corpus is unaffected.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
New graphify/extractors/ocaml.py handles both the implementation grammar
(language_ocaml, .ml) and the interface grammar (language_ocaml_interface,
.mli). Emits nodes for modules, top-level/module-level values and functions,
types and their variant constructors; edges for defines/contains, open ->
imports_from, and application -> calls. Qualified paths (Geo.area) resolve to
the final value name, not the module qualifier; local let ... in bindings do
not mint nodes or steal call attribution. Cross-file open/call targets are
sourceless stubs so the corpus rewire collapses them onto the unique real
definition (no #1402 sourced-stub leak).
Wired into detect.py (CODE_EXTENSIONS), extract.py (dispatch +
_EXTRA_FOR_EXTENSION), pyproject.toml ([ocaml] extra + all + dev dep), and
README. Adds tests/test_ocaml.py (behind importorskip).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The #2475 fix recomputed the match path against the explicit scan root but
skipped the _nfc() normalization rel_anchor gets, so an NFD-named path
could miss a parent ignore pattern under the #2544 NFC-matching regime.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
An ignore rule naming a directory with an accent silently does nothing on
macOS, and the files it was meant to exclude get scanned anyway.
macOS (APFS/HFS+) returns filenames in NFD — "ç" comes back as "c" + U+0327
COMBINING CEDILLA — while editors write ignore files in NFC, where the same
"ç" is the single codepoint U+00E7. The two render identically and compare
unequal, so `fnmatch` never matches and the pattern is a no-op.
Found in a real repo: a `.graphifyignore` containing `Orçamento/` failed to
exclude that directory, and 9 client contract PDFs were picked up for semantic
extraction — i.e. queued to be sent to an LLM — despite an explicit rule
against it. The failure is silent: there is no warning, and the only symptom
is a file count that does not match what you expect. A user who does not
count would never know. That is what makes this worth fixing rather than
documenting: the rule appears to work.
Both sides are now normalized to NFC before matching, at three boundaries:
the pattern (in `_parse_gitignore_line`, so it covers .graphifyignore,
.gitignore and $GIT_DIR/info/exclude alike) and the two path forms used in
`_is_ignored` (`target.name` and the anchor-relative path).
NFC is already the form Linux and Windows produce, so this is a no-op there
and only repairs the macOS mismatch.
Tests: two regression tests cover both directions (NFC pattern vs NFD path on
disk, and the reverse); both fail before this change and pass after. A third
asserts ASCII patterns are unaffected, so the normalization cannot regress
existing behavior.
Full suite: 3833 passed. The 13 failures in tests/test_terraform.py are
pre-existing on a clean upstream checkout (optional tree_sitter_hcl not
installed) and unrelated to this change.
#2543 (adopts PR #2546, thanks @michaelxer): a failed extraction is no
longer stamped in the incremental manifest as up-to-date, so graphify
update retries it instead of skipping it forever; a manifest already
poisoned by the old behavior is healed on the next run; genuinely
unchanged files are not re-processed. Extended to the watch save_manifest
paths too.
#2554 (adopts PR #2555, thanks @annieyii): the claude-cli backend now
inspects the stdout envelope for an is_error result (e.g. a rate limit
returned with exit code 0) and raises it on both the zero and non-zero
exit paths, instead of parsing it as an empty success and bisecting
against a live rate limit.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
#2495 (thanks @alisson-acioli): graphify update now evicts a file newly
added to .graphifyignore/--exclude even though it still exists on disk
(positive ignore-rule evidence), with .gitignore-driven eviction gated on
an explicit full update; unchanged in-corpus files and fail-closed #1795
cases are preserved.
#2504 (thanks @te7ina-honey): the Java type resolver runs before the
unique-label stub rewire and parks imported-but-external types on their
FQN, and cross-file import resolution checks the package, so a local
class and a same-named external annotation (e.g. Spring @Component) no
longer collapse into one node. In-corpus annotation resolution unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
"coverage" was an unconditional _SKIP_DIRS entry and _is_noise_dir matches
directory names at any depth, so a repo where coverage is a legitimate
package name lost the whole package from the graph — no warning, no
skipped_sensitive entry, nothing in the report. The failure is quiet in
the worst way: the package's dependents survive, so queries keep
returning plausible neighbours while the package itself has no nodes.
The entry's comment cites Vitest/Istanbul/nyc HTML reports (#870), but
the pruning is language-agnostic, so it also removes Python/Go/Rust
packages that happen to be called coverage.
detect.py has been fixed twice for this exact shape — #1666 gated a bare
snapshots/ on real .snap evidence, #2058 gated env/.env/*_env on real
virtualenv markers — so this applies the established pattern rather than
a new mechanism. _has_coverage_artifacts() mirrors _has_venv_markers():
same OSError guard, same "cannot verify, keep a possibly-real code dir"
contract when no parent is available. Evidence is a file a coverage tool
actually writes (lcov.info, coverage-final.json, clover.xml,
coverage.xml, cobertura-coverage.xml, jacoco.xml, .coverage, index.html)
or an lcov-report/ / html-report/ subtree, covering lcov, nyc/Istanbul,
coverage.py, JaCoCo and Cobertura.
lcov-report stays unconditional: it has no false-positive class, so
gating it would add filesystem probes for no benefit — the same split
#1666 made between the unambiguous __snapshots__ and the bare snapshots.
Stage 2's .env regex treated .env.example / .sample / .template / .dist
like live secret files and dropped them from the graph. Carve out those
suffixes for .env / .envrc basenames only — real .env.local etc. stay blocked.
Fixes#2184
Stamped keys can already be NFC while Path(root).resolve() is NFD, so
os.path.relpath treated in-root files as ../ and kept them absolute.
Normalize both operands inside _to_relative_for_storage (and the join in
_to_absolute_from_storage).
macOS yields NFD paths from os.walk/getcwd while skill path literals are
often NFC. Raw string compare treated every file as deleted+new and forced
a full re-extract. Canonicalize at the manifest boundary (same idea as #1226).
Fixes#2221
.gitignore/.graphifyignore/info-exclude read with encoding=utf-8 kept a
leading BOM (U+FEFF) on the first line, so the first pattern (e.g. *.log
or .fable-wt/) silently matched nothing and a BOM'd full-line comment
became a bogus pattern. git strips a single leading BOM; switching the
two ignore read sites to utf-8-sig matches git exactly (strips at most
one, file-start only).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The .graphifyinclude loader and its two matcher helpers had no consumers:
commit df40e4d (#873, index dot dirs) removed the blanket dot-prefix
exclusion and with it the only call sites, leaving detect() parsing the
file on every run and then discarding the result. A .graphifyinclude was
silently a no-op.
Delete _load_graphifyinclude, _is_included, _could_contain_included_path
and the orphaned assignment; add .graphifyinclude to _SKIP_FILES so a
leftover file no longer lands in unclassified; and print a one-time
stderr note when one is present at the scan root, pointing to ! negation
patterns in .graphifyignore. Bump to 0.9.25.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The heuristic over-matched and silently dropped legitimate files:
- prose `.md`/`.rst` whose topic slug ends in a keyword (privacy-tokens.md,
token-economics.md) — only code was exempt, not prose;
- the unbounded Stage-2 `service.account` substring (regex `.` wildcard) matched
real source (google/oauth2/service_account.py) and prose slugs.
It also MISSED real secrets (.npmrc, .pypirc, secring, .git-credentials, and
case variants on case-insensitive filesystems), which were being indexed.
Fix: move service_account/aws_credentials to the boundary-checked keyword path
(so real source is spared, downloaded key files still drop), add a prose-note
carve-out (multi-word slugs indexed, bare `secrets.md`/`token.md` still dropped),
tighten id_rsa with a left boundary, add the missed secret dotfiles + secring,
lowercase the dir/segment comparisons, and count multi-dot slugs as multi-word.
Net effect is stricter on real secrets and stops the false-positive data loss.
Traceability: `graphify extract` now names the files skipped as sensitive (not
just a count), so a wrongly-flagged file is visible.
#2058: `_is_noise_dir` treated any directory named `env`/`.env`/`*_env` as a
Python virtualenv and pruned it during the walk — before `.graphifyignore`
negation, with zero trace in any returned bucket. Real source dirs with those
names (common in UVM/ASIC verification trees) were silently lost. The venv
heuristic for those names is now gated on actual markers (`pyvenv.cfg`,
`bin`/`Scripts/activate`, `lib/python*`, `conda-meta/`); `venv`/`.venv`/`*_venv`
stay name-only. Pruned-as-noise dirs are recorded in a new `pruned_noise_dirs`
bucket for traceability, and extract.py's walk call sites pass the parent so
genuine venvs are still marker-checked and pruned.
#2059: Office and Google-Workspace sidecars were named with a hash of the
resolved ABSOLUTE source path, so the same tracked file in two clones/worktrees
produced two differently-named byte-identical sidecars — unbounded duplicates
when graphify-out/ is committed, each ingested as a distinct source doc. The
hash is now over the scan-root-relative (NFC-normalized) path, stable across
checkouts while still disambiguating same-stem files; out-of-root sources fall
back to the old absolute form. Also fixes the same bug in google_workspace's
`_sidecar_path` (which additionally never had the #1226 NFC fix).
The #1943 carve-out rescues genuine source under secrets/ / credentials/, but
.tfvars sits in CODE_EXTENSIONS while being Terraform's canonical values store
(routinely real secrets), so it would now be indexed. Add .tfvars to
_SECRET_PRONE_DATA_EXTS so the shared graphable-source predicate drops it in
both Stage 1 and Stage 3; .tf/.hcl stay graphable as genuine infra source.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
graph.json (the clustered `to_json` write and the `--no-cluster`/merge raw
dumps) and manifest.json were written with a direct `open()`/`write_text`, so a
crash, kill, or disk-full mid-write left a truncated, unparseable file that the
next load or `detect_incremental` then failed on.
Add `write_text_atomic`/`write_json_atomic` in graphify.paths (temp file in the
same directory + `os.replace`; JSON is streamed into the temp, not materialized
as one string) and route the graph.json writers (export.to_json,
cli._prune_graph_json_sources, the merge driver) plus detect.save_manifest
through them. The helper preserves the destination's mode (an atomic replace
never tightens 0644 to mkstemp's 0600), writes through a symlinked destination
(shared-output setups), and falls back to copy-then-delete on a Windows
os.replace lock — matching graphify.cache's existing atomic writer. On failure
the previous file is left intact and the temp removed. Not a power-loss
durability guarantee (no fsync, consistent with the rest of the codebase).
- #1925: a missing manifest.json no longer degrades `extract --code-only`
into a full scan that discards the committed semantic layer. An existing
graph.json is a sufficient incremental baseline (detect_incremental treats
an absent manifest as "all new / none deleted"), so out-of-scope doc/paper/
image nodes are preserved while genuinely deleted sources still evict.
- #1920: _stamped_manifest_files now counts hyperedge output, so a doc whose
only chunk output is a hyperedge is stamped instead of re-extracted forever.
- #1923: new namespace/use-aware PHP resolver (mirrors the Java resolver, runs
before the unique-name rewire) so App\Models\Page and an imported
Filament\Pages\Page stay distinct — no more false inherits/imports edge.
- #1922: detect() records ignored files/dirs in a new `ignored` diagnostic
field (the nested-ignore scoping bug itself shipped in 0.9.16 / #1873).
Regression tests added for each; full suite 3325 passed, 3 skipped.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
.cjs was half-registered: the language maps in build.py and
extract.py already routed it to the JS grammar, but it was missing
from CODE_EXTENSIONS, the extractor _DISPATCH, _LANG_FAMILY, and the
JS resolution/cache suffix sets — so .cjs sources (Electron
main/preload scripts, CommonJS escape hatches in "type": "module"
packages) were silently skipped during a build: classified as
non-code and never handed to the JS extractor.
Add .cjs alongside .mjs in the six lists that gate/route JS files
(detect.py, extract.py _DISPATCH, analyze.py _LANG_FAMILY,
extractors/models.py cache-bypass, extractors/resolution.py resolve
exts, cli.py _HOOK_SOURCE_EXTS), with regression locks mirroring the
.mts/.cts fix (#1607).
Real-world impact: an Electron app whose ~2000-line main.cjs backend
was invisible went from 201 to 294 nodes and 256 to 441 edges on
rebuild.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Two coupled excluded-vs-deleted fixes:
#1909 — incremental extract's prune set was derived from the manifest
alone (manifest - corpus), so a file that became excluded without ever
being manifest-listed (every pre-#1897 graph) kept its stale nodes in
graph.json forever. The prune set is now also derived from the existing
graph's own node source_files reconciled against the post-exclude detect
corpus (_stale_graph_sources), restricted to in-root paths; out-of-root
--include/symlinked entries and remote (://) sources are never pruned.
Relative source_files are anchored against both the scan root and the
--out root (the #555/#1899 relativized form). The --no-cluster
incremental early exit never runs build_merge, so an exclusion-only
change now prunes the raw graph.json in place instead.
#1908 — save_manifest retained any prior row whose file still existed
on disk, so an excluded-but-alive file survived as a permanent phantom
that detect_incremental reported as deleted on every run. Full-scan
callers (extract's saves, watch._rebuild_code's saves) now pass the RAW
detect corpus via a new scan_corpus parameter and in-root rows outside
it are dropped; the corpus is deliberately not the #933 stamp-filtered
files dict, so failed-chunk/omitted-doc rows and --code-only doc rows
survive. Subset saves (changed_paths hooks, #917) keep the seeding
default. detect_incremental now splits manifest rows that left the scan
into deleted_files (gone from disk) and excluded_files (alive but out of
scan), mirroring the watch-side #1795 distinction, and the extract
summaries report the two separately.
Ordering matters: extract's cleanup of newly-excluded nodes previously
worked only through the #1908 conflation, so the graph-source prune
lands together with the manifest split to avoid regressing #1909.
- pg_introspect: the psycopg-missing ImportError suggested pip install
'graphify[postgres]', but the PyPI package is graphifyy (double-y)
- detect: add .skill (Markdown with YAML frontmatter agent files) to
DOC_EXTENSIONS so they are no longer dropped as unclassified
- extract: route .skill through extract_markdown for the structural
quick-scan
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Non-anchored patterns from a nested .gitignore/.graphifyignore were
matched against root-relative paths first, so a nested ignore file's
patterns leaked outside its directory. In the wild, .hypothesis/.gitignore
(a bare "*" auto-written by the hypothesis library) ignored the entire
repository and detect() returned 0 files.
Per gitignore semantics, patterns from A/.gitignore apply only to paths
under A. Match every pattern against the path relative to its own anchor
(the anchor dir itself exempt — an ignore file governs its directory's
contents, not the directory), and skip patterns whose anchor does not
contain the target.
Regression introduced with nested-ignore support in 8a5287a (#1206).
tests/test_detect.py: 143 passed, including the existing nested-ignore
and nested-negation tests plus two new regressions.
Fixes#1873
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
detect() only read .gitignore/.graphifyignore in the scan root and its
ancestor directories (up to the nearest VCS root), loaded once before the
walk began. A .gitignore sitting in a descendant directory — e.g.
vendor/sub/.gitignore — was never read, so files/dirs it excluded leaked
into the graph. Real git (and every other gitignore-aware tool) honors
.gitignore at every directory level, not just the ancestor chain.
Extracts the per-directory read+parse logic into a shared
_load_dir_own_ignore() helper (used by both the existing ancestor-chain
loader and the new call site) and invokes it live inside detect()'s
os.walk loop for every directory visited, before that directory's
children are pruned — so a nested ignore file governs its own subtree
with the same closer-file-wins precedence git uses.
Adds three regression tests: nested file exclude, nested directory prune
(the walk never descends into it), and nested negation overriding a
broader root-level rule.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
#1810 — detection read only .gitignore/.graphifyignore, never
.git/info/exclude, which is where git records local-only excludes and
where `git worktree add` writes nested worktree paths. graphify walked
into those worktree copies and the graph exploded (one 5-worktree repo:
9.4k nodes/10MB -> 210k nodes/311MB, ~77% duplicate). detect now loads
info/exclude at lowest precedence (below every per-dir .gitignore, per
git, so a nearer `!` still wins) and resolves the linked-worktree /
submodule case where `.git` is a file to the shared common git dir.
#1809 — two git-hook gaps: (a) post-checkout never honored
GRAPHIFY_SKIP_HOOK, so the var stopped commit rebuilds but not
branch-switch ones; now checked in both. (b) with core.hooksPath shared
across worktrees, a commit in any linked worktree fired post-commit,
which wrote a rogue delta-only graph.json into it and raced deploy/CI
`git clean` against the detached rebuild. Both hooks now short-circuit
in a linked worktree (git-dir != git-common-dir), comparing ABSOLUTE
paths so the primary checkout (where --git-common-dir is the relative
".git") is never false-positived and skipped.
Adds regression tests: info/exclude honored + negation precedence;
both hooks honor the skip env and carry the worktree guard; and an
end-to-end guard check against a real `git worktree`.
Reported by @cdahl86-cyber (#1810, #1809); the worktree guard was
co-developed with @Claude-Madera's PR #1806.
Co-Authored-By: cdahl86-cyber <cdahl86-cyber@users.noreply.github.com>
Co-Authored-By: Claude-Madera <Claude-Madera@users.noreply.github.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
#1807 — piping graphify into a reader that stops early (head,
Select-Object -First N, sed q) disconnected stdout mid-write, raising an
unhandled BrokenPipeError (OSError(EINVAL) on Windows) and exiting 255,
so CI wrappers and agent harnesses read a successful query as a failure.
The console entry point now wraps the CLI body: a closed-pipe reader is
treated as success — stdout is redirected to devnull so shutdown flush
can't raise again, and the process exits 0. Adds a subprocess regression
test.
#1804 — .nox/ (nox virtualenvs, tox's successor, same .nox/ tree shape)
was missing from _SKIP_DIRS while .tox was present, so nox site-packages
got fully indexed (one repo came out 91% venv noise). Added next to .tox
with a regression test.
Reported by @varuntej07 (#1807) and @igorregoir-lgtm (#1804).
Co-Authored-By: varuntej07 <varuntej07@users.noreply.github.com>
Co-Authored-By: igorregoir-lgtm <igorregoir-lgtm@users.noreply.github.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
.rake files are plain Ruby (Rake's task DSL is ordinary method calls), but the
extension was gated out everywhere, so rake tasks were classified as
unsupported, skipped, and their calls invisible. Add `.rake` to all seven `.rb`
gates the reporter mapped:
- detect.CODE_EXTENSIONS (classification)
- extract._DISPATCH (extractor dispatch)
- extract._LANG_FAMILY_BY_EXT-adjacent language-name map (.rake -> ruby)
- the ruby_member_calls LanguageResolver suffix set
- both `.rb`-suffix filters in ruby_resolution.py (raw-call gather + class-def index)
- analyze language-stats map
- build repo-tag map
The extractor already parsed the content; this is purely extension routing.
Regression test: a `.rake` task's `Widget.tally` resolves cross-file to the
`.rb` definition.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Case 1 — `extract <corpus> --out <dir>`: the graph went to <dir> (cache_root is
already passed to the AST extractor), but detect()'s word-count/stat-index cache
uses the scan root, so a stray graphify-out/cache/ was created inside the corpus
(and left behind even when the run aborted at the no-LLM-key gate). Thread an
optional cache_root through detect() -> cached_word_count() -> _ensure_stat_index()
and pass out_root from the extract CLI, so the stat index lives under --out. Entry
keys are absolute paths, so relocating the index file is safe.
Case 2 — `cluster-only --graph <elsewhere>/graphify-out/graph.json`: outputs
(GRAPH_REPORT.md, re-clustered graph.json, labels, analysis, html) were written to
the CWD's graphify-out/, ignoring where --graph lives. They now write beside the
input graph when it sits in a graphify-out/ dir (another project/tenant's output),
while still falling back to the CWD for an arbitrary archived backup/graph.json —
the restore-into-place workflow #934 pins.
Regression tests for both cases; #934 still passes.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ayushcodes10