- fix SQL extractor using bare path.stem as node ID prefix — collides across same-named files in different dirs; use _file_stem() (directory-qualified) instead
- fix Python import resolver keying stem_to_entities by bare stem; add bare_to_qualified secondary index for absolute imports so cross-file edges survive duplicate filenames
- add stat-based mtime fastpath to file_hash: skip full SHA256 when size+mtime_ns unchanged, flush index atomically via atexit (same trade-off as make)
- add cache-check, merge-chunks, merge-semantic CLI subcommands so the skill pipeline can use library functions instead of inline Python
- fix absolute source_file paths from semantic subagents not being relativized before graph storage (#932): add root param to build_from_json/build/build_merge, pass scan target at both call sites
- fix failed semantic chunks permanently freezing their files in the manifest (#933): filter _manifest_files to only include doc/paper/image files that appear in sem_result nodes/edges before calling save_manifest
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
detect: add scan_root to return dict so skill can strip absolute prefix
when computing relative subdirectory breakdown; remove stale --no-semantic
flag reference from large-corpus warning (flag does not exist)
skill: clarify fast path checks CWD graphify-out/graph.json (project root);
remove hardcoded --backend gemini from multi-subfolder example — users
should pass whichever backend key they have; expand large-corpus gate
instruction to use scan_root for relative paths, filter graphify-out/
converted sidecars, and handle flat repos with no subdirectories
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Fast path: if graphify-out/graph.json exists and user is asking a question
(not an explicit rebuild), skip detect entirely and run graphify query —
prevents the skill from refusing large already-built corpora (#930)
- Raise FILE_COUNT_UPPER 200 → 500 so typical 200-500 file codebases no longer
hit the large-corpus size gate on fresh extraction (#930)
- Subdirectory breakdown now strips the scan-root prefix so agent shows
relative names (core/, service/) not absolute paths rooted at / (#930)
- Document multi-subfolder CLI pattern: graphify extract ./sub/ places
graphify-out/ inside each subfolder; skill clobbers single root graphify-out
when run on subfolders separately (#930)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Security: _is_sensitive now flags underscore-prefixed names (api_token.txt, oauth_token.json) by replacing \b with lookarounds; adds _SENSITIVE_DIRS check on parent path components (parts[:-1]) so .ssh/, secrets/, .aws/ directories are always skipped; aligns both patterns to (?![a-zA-Z]) for consistent underscore-after-keyword behavior (#920)
Fix: --wiki Relationships section always empty because _cross_community_links read community from node attrs (always None) instead of the communities dict; _god_node_article had the same bug and never linked to the owning community; fixed by building a node->community map in to_wiki() and threading it through (#925)
Fix: --watch now respects .graphifyignore; patterns loaded once at startup, handler checks _is_ignored before extension filter so node_modules/, .venv/, build/ churn no longer triggers rebuilds (#928)
Fix: C++ struct inheritance edges via base_class_clause; initialize base="" per iteration to prevent stale carryover (#915)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The OpenAI-compatible API can return HTTP 200 with an empty `choices`
list or with `choices[0].message = None` (e.g. content-filtered
responses on Gemini, overwhelmed Ollama instances). Without a guard,
both sites raise an unhandled IndexError or AttributeError.
`_call_openai_compat` already documents this hazard ("Ollama can return
HTTP 200 with empty/null content") and has `_response_is_hollow` logic
downstream, but `_response_is_hollow` is unreachable when the choices
list itself is empty. The new guard closes that gap.
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
- hooks.py: drop --path-format=absolute (added git 2.31), validate no newlines in path, anchor relative paths on repo root (#907)
- detect.py: seed save_manifest from existing manifest before loop so incremental callers don't erase untouched file entries (#917)
- cluster.py: drop round(..., 2) from cohesion_score so split threshold 0.05 fires correctly; add resolution param to _partition and cluster; add exclude_hubs_percentile to cluster with majority-vote reattachment (#919)
- report.py: format cohesion with :.2f for display
- __main__.py: wire --resolution and --exclude-hubs into extract and cluster-only commands (#919)
- C++ inheritance already written to disk by analysis agent (#915)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(analyze): exclude npm dep-block keys from god-node selection
Extends _JSON_NOISE_LABELS in graphify/analyze.py with the six npm
package.json dependency-block keys (dependencies, devDependencies,
peerDependencies, optionalDependencies, bundledDependencies,
bundleDependencies — lowercased to match the existing .strip().lower()
comparison in _is_json_key_node).
On JS/TS corpora with non-trivial dependency counts, the dep-block key
node accumulates contains+imports edges to every package entry and was
surfacing as the top god-node in the report. The fix is a one-line
extension of the existing frozenset; no new helpers or code paths.
Includes a parametrized regression test in tests/test_analyze.py
covering all five npm keys: dependencies, devDependencies,
peerDependencies, optionalDependencies, bundledDependencies.
Validated live against rsl-siege-manager @ 6085fd66 — zero npm dep-block
keys in the top-10 god-nodes after the fix. Pre-existing failures
(Windows symlink + SQL tree-sitter): 17, unchanged.
Closes#2
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* style: drop fork-local issue refs from analyze comments
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(extract): force UTF-8 encoding on subprocess + loud failure on chunk errors
On Windows cp1252, subprocess.run(..., text=True) without encoding= raises
UnicodeEncodeError for chars like → ✅ ≥ in chunk content. Both
_call_claude_cli (llm.py:426) and the _call_llm claude-cli branch (llm.py:959)
lacked encoding=.
- Add encoding="utf-8" to both subprocess.run sites.
- Track failed_chunks in extract_corpus_parallel merged result dict.
- Print [graphify] WARNING: N/M semantic chunk(s) failed summary to stderr at
end of run when any chunk failed, so silent partial failures are visible.
- Add tests/test_charmap_encoding.py: 10 regression tests covering subprocess
encoding kwarg, loud-failure summary, and substitution validation.
Closes#3
* style: drop fork-local issue refs from llm/test comments
- Revert .h -> extract_c (C++ grammar rejects C++ keywords used as identifiers
in Linux-kernel-style headers; .hpp/.hxx/.hh already route to extract_cpp)
- Fix field_declaration block: use children_by_field_name("declarator") instead
of iterating all children with wrong type guard; replace ensure_node (undefined)
with add_node
- Fix _import_c include resolution: use _make_id(str(resolved)) to match the
file_nid scheme _extract_generic uses, not _make_id(_file_stem(resolved))
- Fix exact_merges counter in dedup Pass 1 to count only within-file merges
actually performed, not the raw unpartitioned group sizes
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Route .h files through extract_cpp (was extract_c), fixing missing method nodes in C++ headers
- Extend _get_cpp_func_name to handle field_identifier, destructor_name, operator_name
- Add CPP-specific field_declaration branch in _extract_generic to emit class method/field nodes
- Partition dedup Pass 1 by source_file: only union same-label nodes within the same file;
cross-file matches fall through to Pass 2 fuzzy, preventing generic-label god nodes
- Add _resolve_c_include_path: resolve quoted #include paths to real files on disk so
target node IDs match what _extract_generic creates, fixing dangling include edges
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Common terms like 'error'/'exception' were stealing BFS seed slots from
rare identifiers like 'FooBarService', burning the token budget on noise.
- _compute_idf: weights query terms by inverse document frequency, cached
on G.graph so cost is paid once per graph load not per query
- _score_nodes: multiplies each tier bonus by IDF weight
- _pick_seeds: replaces fixed top-3 with gap-ratio selection — stops adding
seeds when score drops below 20% of the top match
- _subgraph_to_text: truncation hint now tells Claude to narrow with
context_filter or use get_node instead of just saying 'truncated'
Fixes#897
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Clean merge — fixes real token-bloat regression where GRAPH_REPORT.md was loaded on every question. All 10 install surfaces now route to graphify query first. Also fixes hidden bug where re-running graphify install after upgrade didn't refresh stale hook text.
The current install writes "ALWAYS read graphify-out/GRAPH_REPORT.md
before reading any source files, running grep/glob searches, or
answering codebase questions" into CLAUDE.md and equivalents, plus a
PreToolUse hook with the same instruction. On real corpora that report
is 47-91K characters, so Claude Code sessions pay roughly 12-25K tokens
of context up front for every search-able question. Three users on #580
reported this making token usage worse than no install at all.
Reproduced on a 1500-file Go monorepo: a "where is X defined" question
takes 5 tool calls and 34k agent tokens with stock graphify, 4 calls
and 30k tokens with no install, and 1 call and 30k tokens after this
patch. Stock graphify's Read of GRAPH_REPORT.md hit Claude Code's 25k
token cap and failed entirely, then recovered via a partial read plus
graphify explain.
Demote GRAPH_REPORT.md to a fallback for broad architecture review and
route first action to the existing scoped commands: graphify query,
path, explain. The 2k-budget BFS subgraph already exists in serve.py;
the install just wasn't pointing at it.
Updated across all ten install surfaces: _SETTINGS_HOOK,
_CLAUDE_MD_SECTION, _AGENTS_MD_SECTION, _GEMINI_MD_SECTION,
_GEMINI_HOOK, _VSCODE_INSTRUCTIONS_SECTION, _ANTIGRAVITY_RULES,
_KIRO_STEERING, _CURSOR_RULE, _OPENCODE_PLUGIN_JS. Plus the matching
sentence in README.md, which also fixes an inaccuracy about Codex
hooks (Codex's installed hook is intentionally a no-op because Codex
rejects additionalContext, so the guidance there comes from AGENTS.md,
not the hook).
Five installers (claude, agents, vscode, gemini, kiro, cursor) were
also writing their section only when no marker was present, so users
who installed pre-fix kept the old "ALWAYS read" text after upgrading.
Added _replace_or_append_section helper that updates in place when the
graphify marker is found. claude_install also no longer returns before
re-running _install_claude_hook, so stale settings.json hook payloads
get refreshed on upgrade.
Tests:
- tests/test_install_strings.py (3): every install constant still
mentions `graphify query` and matches no banned report-first regex.
- tests/test_install_upgrade.py (7): seeds each platform's instruction
file with pre-fix text, runs install, asserts the on-disk file
reflects the new policy.
- test_claude_md.py idempotency tests still pass.
Fixes#580.
When `root` has at least one direct symlinked child, default to following
symlinks instead of silently dropping their contents. This makes "fake
working dir" patterns (a folder full of symlinks pointing at scattered
source dirs) work transparently — the caller no longer has to know to
pass `follow_symlinks=True`.
Concrete motivating shape:
~/projects/research-corpus/
├── papers -> /external/drive/papers
├── notes -> /external/drive/notes
└── code -> /external/drive/code
Before: `--update` invoked `detect_incremental(root)` without passing
`follow_symlinks=True`, so the scan saw the small set of files actually
inside the root and missed everything reachable only through a symlink.
Result: every legitimate new file was missed, and the manifest paths
were marked as deleted.
After: when `follow_symlinks` is left at its default `None`, `detect()`
runs `_auto_follow_symlinks(root)` (one cheap `iterdir()` + `is_symlink()`
loop) and follows symlinks if any direct child is one. Behaviour is
unchanged for ordinary scans (no direct symlinks → False, as before).
Override is always possible by passing an explicit `follow_symlinks=True`
or `follow_symlinks=False`; existing tests confirming the explicit
behaviour continue to pass unchanged.
Backwards compatibility:
- Type annotation: `bool` -> `bool | None`. Callers passing `True`/`False`
continue to work identically. Callers passing nothing get the new
auto-detect.
- No new dependencies.
- Cheap: one `iterdir()` call once per detect() invocation.
Tests:
- 3 new tests in tests/test_detect.py:
- test_detect_auto_detects_direct_symlink_child
- test_detect_default_does_not_follow_when_no_symlinks
- test_detect_explicit_false_overrides_auto_detect
- Full tests/test_detect.py + tests/test_incremental.py: 49/49 pass.
In Python+TypeScript monorepos, the call and import resolvers match by
label across language boundaries (AuthError -> Member), producing false
positives that dominate 'Surprising Connections' due to cross-dir (+2)
and cross-community (+1) bonuses.
Expand the existing calls guard to also cover uses edges, and zero the
cross-dir and cross-community bonuses for these pairs — not just conf_bonus.
Leaves semantically_similar_to, EXTRACTED, and AMBIGUOUS edges unaffected.
5 new tests: calls suppressed, uses suppressed, semantically_similar_to
preserved, same-language INFERRED preserved, cross-language EXTRACTED preserved.
Alembic/Flask-Migrate revisions, Django migrations, and protobuf/OpenAPI
generated files produce hundreds of degree-1 rationale nodes labeled as
'possible documentation gaps'. Their module docstrings are revision
annotations or boilerplate, not architectural rationale.
- Add _is_autogenerated_python() in extract.py detecting Alembic, Django
migrations, and generic DO-NOT-EDIT markers; skip module docstring only
- Function/class docstrings inside those files still extracted as normal
- report.py: exclude file_type=rationale nodes from isolated-node gaps
section — rationale nodes are degree-1 by construction; flagging them
as missing edges was always wrong
- 5 new tests covering Alembic, Django, protobuf, false-positive guard,
and function-docstring passthrough
Move all intermediate data files and temp step scripts from the
project root into graphify-out/ to match the bash skill behaviour.
Also fix the > redirect on detect.json which wrote UTF-16LE in
PowerShell 5.1 - now uses Out-File -Encoding utf8 consistently.
Files moved: .graphify_detect.json, .graphify_ast.json,
.graphify_extract.json, .graphify_semantic.json,
.graphify_analysis.json, .graphify_labels.json,
.graphify_incremental.json, .graphify_old.json,
all .graphify_step_*.py temp scripts.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
antigravity install on Windows now installs skill-windows.md instead
of skill.md by redirecting to a new antigravity-windows platform config
at install time via sys.platform check.
Python detection in Find-GraphifyPython now uses uv tool dir (respects
UV_TOOL_DIR) and pipx environment --value PIPX_LOCAL_VENVS (respects
PIPX_HOME) instead of guessing from the shim location. The graphify.exe
shim and python.exe live in different directories for both uv and pipx
on Windows so the previous Get-Command approach was wrong.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
graphify.exe and python.exe are co-located in the same Scripts\
directory for both uv tool and pipx installs. Use Get-Command graphify
to find that directory and pick up the correct Python instead of
falling through to bare python (which is a different interpreter
that doesn't have graphify).
Also replace all bare `python .graphify_*` invocations throughout
the skill with `& (Get-Content graphify-out\.graphify_python)` so
every step uses the saved interpreter, and fix the bash $(cat ...)
syntax to PowerShell & (Get-Content ...).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
#873: Remove blanket dot-prefix exclusion from detect.py and
extract.py collect_files(). Add framework caches (.next, .nuxt,
.turbo, .angular, .idea, .cache, .parcel-cache, .svelte-kit,
.terraform, .serverless, .graphify) to _SKIP_DIRS so they stay
blocked. Meaningful dot dirs (.github, .claude, etc.) are now
indexed.
#874: Add _maybe_reload() with mtime+size stat key and threading.Lock
to serve.py. call_tool and read_resource call _maybe_reload() on
every request; the graph reloads automatically when graph.json changes
without restarting the MCP server.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add .sh, .bash, .json to CODE_EXTENSIONS in detect.py so files reach extractor
- Fix test_detect_incremental manifest path collision with new .json extension
- Update test_watch to reflect .json/.sh are now watched extensions
- B-1: only emit source imports for paths that exist on disk
- J-1: replace stat()+read() with bounded read to eliminate TOCTOU
- J-3: move pair_count cap inside loop so it is honoured exactly
- J-4: namespace $ref/extends refs with "ref_" prefix to prevent ID collision
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
In a linked worktree .git is a file not a directory, so root/.git/hooks
fails with NotADirectoryError. Fall back to git rev-parse --path-format=absolute
--git-path hooks which resolves the correct hooks dir from any worktree.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- skill-windows.md: add graphify-out/ prefix to 5 paths in Step B3 that
were being read/written without prefix (lines 244,245,373,387,388,407,413)
causing semantic cache to always appear empty on Windows (#862)
- detect.py: gate converted office/Google Workspace sidecar paths through
_is_ignored() before appending to files list so .graphifyignore rules
apply to graphify-out/converted/ sidecars (#861)
- detect.py: split manifest entries from {mtime,hash} to
{mtime,ast_hash,semantic_hash}; save_manifest() accepts kind="ast"|
"semantic"|"both"; detect_incremental() accepts kind= and checks the
right hash field so graphify update (kind=ast) no longer prevents
graphify extract from re-running semantic extraction (#857)
- watch.py: all three save_manifest() calls use kind="ast" (AST-only rebuild)
- __main__.py: both extract save_manifest() calls use kind="both"
- backwards compatible: legacy {mtime,hash} entries treated as ast_hash only
775 tests passing
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Several host agents (Claude Code in particular) misread the current
phrasing and prompt the user for `ANTHROPIC_API_KEY` when no Gemini
key is set, instead of falling through to subagent dispatch. The
Gemini tip stays as-is; this just adds an explicit one-line callout
that no other provider keys are consulted, with `ANTHROPIC_API_KEY`
called out by name since that is the specific misread we hit in the
wild.
No behavior change — pure documentation.
The rebuild lock file accumulated concatenated PIDs across post-commit
rebuilds without a separator, and was never removed when the rebuild
finished. Two practical consequences for users:
1. Downstream tooling that polls for `.rebuild.lock` to disappear before
doing post-rebuild work (publish scripts copying graph.html to a web
root, etc.) blocked forever / until its own timeout.
2. The accumulated digit string could not be parsed by humans or tooling
to find the owning PID.
The `_rebuild_lock` context manager now:
- Opens the lock file with `a+` so a non-acquiring caller does not
truncate the existing holder's PID.
- After flock acquisition, truncates and writes a single `<pid>\n` line
so external readers can `kill -0 $(cat .rebuild.lock)` to check
liveness.
- Unlinks the lock file in the finally block (only when *we* held the
lock), restoring the "signal-by-absence" convention users rely on.
Four regression tests added under `tests/test_watch.py` covering the
PID-with-newline payload, post-release unlink, no-accumulation across
sequential acquisitions, and the non-blocking-caller-does-not-clobber
invariant.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Astro files have a `---...---` TypeScript frontmatter block at the top
containing nearly all imports, followed by an HTML-with-expressions
template body, and optionally `<script>` blocks for client-side JS.
Feeding the whole file to the JS tree-sitter parser produces a top-level
ERROR node because the template isn't valid JS, so `import_statement`
nodes are never reached.
Mirrors `extract_svelte` (#713) — same regex-rescue strategy:
- Dynamic `import('...')` pass over the full source.
- Static `import X from '...'` pass over the frontmatter block plus any
`<script>` blocks.
Both static and dynamic imports honor tsconfig.json path aliases and
resolve relative paths through `_resolve_js_module_path` so node ids
match the targets that `_extract_generic` produces for sibling files.
Wired in:
- `detect.py:27` — `.astro` added to `CODE_EXTENSIONS`.
- `extract.py:5492` — `".astro": extract_astro` in the dispatch table.
Tests in `tests/test_astro_extraction.py` cover: extension registration,
frontmatter static imports, dynamic import in frontmatter, client-side
`<script>` block imports, frontmatter-less pages (must not crash), and
tsconfig.json path-alias resolution.
- skill.md + skill-windows.md: add encoding="utf-8" to all read_text()/write_text()
calls and ensure_ascii=False to json.dumps — bare calls defaulted to system
codepage on Chinese-locale Windows, mojibaking non-ASCII content (#832)
- skill.md + skill-windows.md: prefer uv tool install --upgrade graphifyy over
pip in the Step 1 install fallback — pip installs to the wrong env when
graphify was installed via uv tool (#831)
- serve.py + __main__.py: replace flat substring scoring in _score_nodes with
three-tier precedence (exact 1000 / prefix 100 / substring 1); _find_node
returns results ordered exact→prefix→substring; both path CLI and MCP now
emit a clear error when src and tgt resolve to the same node (#828)
- cache.py: normalize path key via .as_posix().lower() in file_hash so Windows
junction/case variants hash identically; mirror abs-path normalization from
save_semantic_cache into check_semantic_cache so relative source_file paths
resolve the same way on both sides (#826)
- __main__.py: add /graphify skill trigger line to _AGENTS_MD_SECTION — affects
all 7 AGENTS.md platforms (OpenCode, Codex, Aider, Trae, Hermes, Claw, Droid)
so typing /graphify actually invokes the skill tool (#827)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Safi