31 Commits

Author SHA1 Message Date
safishamsi 6a56763051 fix(build): form-insensitive prune + scan-root marker (#2012)
build_merge now prunes a deleted file's nodes, edges, and hyperedges regardless
of whether their stored source_file is absolute or relative. When the caller
passed no root (the --update runbook), a node that kept an absolute path slipped
past the relative prune set and the deleted file's graph survived silently.
Matching is now form-insensitive (raw, normalized-relative, then an absolute-
identity fallback); a re-extracted file is still never pruned (#1796 preserved).

`graphify extract` also writes the .graphify_root marker after every graph write
so a later build_merge relativizes deleted-file paths correctly even under a
custom --out (its grandparent-of-graph.json fallback pointed at the wrong dir).

Regression tests in tests/test_build_merge_hyperedges_and_prune.py.
2026-07-20 13:29:54 +01:00
oleksii-tumanov c57d5711ca fix(extract): honor persisted excludes on re-extract 2026-07-20 11:31:36 +01:00
oleksii-tumanov b35898aafc fix(extract): initialize detection for pathless Postgres 2026-07-20 11:31:36 +01:00
shazeb 08166306ba fix(cache): anchor semantic cache writes to cache_root so --out round-trips (#1990, #1991)
With `graphify extract --out <dir>`, the semantic cache write and read
sides disagreed on both location and key anchoring, breaking the cache
round-trip in two ways:

- Checkpoints (#1990): `_checkpoint_chunk` called `save_semantic_cache`
  with only `root=target`, so per-chunk recovery checkpoints were written
  under `<corpus>/graphify-out/` while the reader consulted
  `<out>/graphify-out/` — creating an unwanted graphify-out/ inside the
  analyzed source tree and making every interrupted run re-extract (and
  re-bill) completed chunks.

- Final save (#1991): cli.py passed `root=out_root`, so corpus-relative
  `source_file` paths resolved against the --out directory, failed
  `p.is_file()`, and every result group was silently skipped — the cache
  the reader would consult was never populated at all, with no warning.

Fix, following the split the AST cache already uses (#1774):

- `save_semantic_cache` and `check_semantic_cache` gain a `cache_root`
  parameter mirroring `load_cached`/`save_cached`: `root` stays the
  source-key anchor (content-hash keys, source_file resolution and
  relativization), `cache_root` selects where cache files live. Omitting
  it keeps `root` for both, so existing callers are unchanged.
- `extract_corpus_parallel` plumbs `cache_root` into `_checkpoint_chunk`.
- cli.py extract passes `root=target, cache_root=out_root` at the cache
  read, the checkpoint path, and the final save, and re-anchors the
  prune sweep's live hashes to `target` (keys anchored to out_root would
  mismatch every entry and sweep the fresh cache as orphaned).
- `save_semantic_cache` now warns loudly when every result group is
  dropped because its source_file does not resolve to a real file — the
  silent-0-writes failure mode #1991 asked to surface.

Regression tests cover: checkpoint written under cache_root (not the
corpus, no corpus graphify-out/ created), recovery read finds the
checkpoint via the same root/cache_root split, the final-save call shape
writes entries where the reader looks, the all-groups-dropped warning,
and backward compatibility when cache_root is omitted.

Fixes #1990
Fixes #1991
2026-07-18 19:09:53 +01:00
shazeb 0224bcaea4 fix(install): fire the search nudge on Claude Code's Grep tool, not just Bash (#1986)
ee1df22 narrowed the Claude Code search-guard matcher from "Glob|Grep" to
"Bash" on the premise that dedicated search tools were removed and searches
go through Bash. Current Claude Code routes content search through its
first-class Grep tool (its Bash tool description actively steers away from
shell grep), so the graphify-first nudge never fired on the agent's primary
exploration path and the graph was silently bypassed.

Three-part fix, per the issue's analysis:

- Matcher: "Bash" -> "Bash|Grep" in _claude_pretooluse_hooks. Glob already
  fires the read nudge via "Read|Glob", so Grep was the only orphaned tool.
- Guard body: the hook-guard search branch only inspected tool_input.command,
  which a Grep call doesn't carry (it has pattern/path/glob). A Grep-shaped
  input (pattern present, no command) is now treated as a search — it IS one
  by definition — and nudges whenever a fresh graph exists. The Bash
  token-matching path is unchanged, and a command-carrying input never
  triggers the Grep shape, so non-search Bash calls stay silent.
- Idempotency: "Bash|Grep" added to the four install/uninstall dedup filters
  (claude + codebuddy), so upgrading replaces the stale "Bash" hook in place
  instead of appending a duplicate — verified against a pre-fix settings.json.

Tests: new regression tests feed Grep-shaped tool_input through
hook-guard search and assert the nudge (with graph), silence (without),
valid PreToolUse JSON, and no blocking; plus a guard that a non-search Bash
command with a stray pattern key does not nudge. Existing matcher assertions
updated across test_search_hook/test_install/test_claude_md/test_codebuddy/
test_hook_strict. Hook+install suites: 397 passed. Full suite: 3224 passed;
the 13 failures are pre-existing on clean v8 in this environment.

Fixes #1986
2026-07-18 19:07:16 +01:00
safishamsi caa6f9edb9 fix(extract): persist --no-gitignore instead of clobbering it (follow-up to #1979)
The PR always wrote gitignore=not no_gitignore into .graphify_build.json, so a
flag-less `graphify extract` after `--no-gitignore` reset it to True and the
git-ignored code silently disappeared again — the exact #1971 complaint. Write
False only when the flag is set (None = leave as-is, mirroring #1886 excludes),
and honor the persisted value for the run when the flag is absent.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-18 12:09:08 +01:00
mzt006 f17e2c5638 fix: add --no-gitignore extraction opt-out 2026-07-18 12:05:00 +01:00
safishamsi 689dd6ccfd feat(hook): opt-in strict PreToolUse guard + stop crying wolf (#1840)
Agents routinely ignore the advisory "run graphify query first" nudge and read
raw files anyway. `graphify install --project --strict` (or `graphify claude
install --strict`) now installs a hook that BLOCKS the first raw source read of a
session via permissionDecision:"deny" with a redirect to graphify query, then
downgrades to the soft nudge — it fires at most once per session (atomic
per-session marker) so it can never strand the agent, and a recent
query/explain/path refreshes a stamp that suppresses it. Claude Code only;
Bash-grep and Glob stay nudge-only; Gemini/Codex/OpenCode are unchanged.
GRAPHIFY_HOOK_STRICT=1/0 toggles at runtime without a reinstall; default installs
are byte-identical (soft nudge).

Also fixes #1840 for the default soft nudge: the guard no longer fires for reads
of out-of-project files, and softens to a non-mandatory nudge when the graph is
stale for the target file. Gating is ~3 stat calls (no corpus walk) and fails
open. Begins the 0.9.19 cycle.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 17:10:41 +01:00
SinghAman21 be80ee82d5 fix(extract): anchor source_file on the scan root, not the --out dir (#1941)
`graphify extract <root> --out <dir>` reduced every node's `source_file` to a
bare filename, so a graph.json could no longer be resolved back to files on
disk by joining source_file onto the scan root.

--out passes the output dir as cache_root to relocate the cache, but that value
also anchored relativization. Every scanned file then failed relative_to(root),
fell through to the #1899 out-of-root fallback, tripped its `updepth > 3`
walk-up guard -- written for a stray ProjectReference, not a whole corpus -- and
collapsed to a basename. On Windows an --out on another drive hit the
cross-drive branch and basenamed unconditionally, which is what the reporter
saw: 0 of ~120k source_files kept a separator. The directory survived only in
the node id slug, lossily (`.`, `/`, `\`, `-`, spaces all map to `_`), and no
other field carried it -- origin_file is stripped (#1516) and the export has no
file table -- so 0% of nodes resolved.

extract() now takes an explicit `root` anchor for source_file/ids/symbol
resolution, which the CLI pins to the scan root independent of where the cache
lives. This completes the cache/anchor decoupling #1774 started and matches
build(root=target), which already anchored on the scan root -- extract was the
lone component keying off --out.

cache_root keeps its fallback-anchor role, so callers that pass the scan root as
cache_root (watch, the no---out CLI path, tests) are unchanged, as are cache
location and out-of-root portability (#1899).
2026-07-17 11:35:11 +01:00
Alpha Nury 824cac7086 fix(detect): clear stale semantic_hash for dispatched-but-omitted files (#1948)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 11:30:53 +01:00
oleksii-tumanov 5a480a83a7 fix(merge-chunks): fail when no chunk validates 2026-07-17 10:54:07 +01:00
safishamsi 479f1af455 fix(cache): mark files partial on empty-parse truncation (follow-up to #1950)
The _partial item-marker approach couldn't fire on the most common truncation
shape: a mid-JSON cut parses to zero items, so a sliced document whose second
slice truncated empty was still stamped complete (the PR's own headline case).

- The adaptive-retry give-up sites now record the chunk's own source files in a
  result-level _partial_files list, independent of parsed items; it propagates
  through _merge_two / the recursion merges / _merge_into so it reaches both the
  per-chunk checkpoint and the run-level manifest stamp.
- _partial_source_files unions _partial_files with the item markers.
- save_semantic_cache seeds an empty group for a named partial file with no
  items so its entry is stamped partial, and carries a partial prev entry's flag
  forward so a later clean slice merging over it can't re-promote it to complete.
- the CLI final save now passes partial_source_files (computed before the save)
  so an empty-parse file isn't written back as a complete entry.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 00:02:15 +01:00
tpateeq 90dcb6a2ef fix(cache): don't promote truncated LLM chunks to the semantic cache as complete
A chunk whose LLM response is truncated (`finish_reason="length"`) and can't be
recovered by splitting, or that hits the adaptive-retry depth cap, returns a
partial node set. Today that set is checkpointed and written to the content-hash
semantic cache + manifest-stamped as complete, so the incomplete nodes are
served forever until the file content changes or `--force`.

Truncated give-up results are now tagged with an internal `_partial` marker.
`save_semantic_cache` stamps the affected file's entry `partial: True` (detected
from the marker or an explicit `partial_source_files` arg), and `load_cached`
treats a partial entry as a cache MISS, so the file re-dispatches and retries.
The file is also left unstamped in the manifest (like a failed chunk, #933) so
detect_incremental re-queues it on the next incremental run — not only on a full
/ `--force` / content-change run. A file sliced across chunks accumulates via a
partial-aware `merge_existing` peek (`load_cached(allow_partial=True)`) so a
truncated slice is never dropped or silently promoted to complete. Self-heals: a
later complete extraction overwrites the same key with a non-partial entry. The
marker is stripped after the final save so it never leaks into graph.json.
2026-07-16 23:48:07 +01:00
safishamsi 16fe8f3020 fix(io): route the remaining graph/manifest writers through the atomic helper (follow-up to #1952)
The atomic-write PR left several writers on the old truncate-then-write path.
Route them through write_json_atomic so a crash mid-write can't corrupt them:
- the --no-cluster raw graph.json dump (a core graph.json writer)
- merge-graphs / merge-chunks / merge-semantic output
- .graphify_analysis.json and .graphify_labels.json sidecars
- global_graph.py's global-graph.json and global-manifest.json

write_json_atomic gains an ensure_ascii flag so the raw-UTF-8 writers
(labels, merge outputs) keep byte-for-byte output. Adds tests for the
Windows PermissionError copy fallback and ensure_ascii=False.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 23:48:07 +01:00
tpateeq f38e98012d fix(io): write graph.json and manifest.json atomically
graph.json (the clustered `to_json` write and the `--no-cluster`/merge raw
dumps) and manifest.json were written with a direct `open()`/`write_text`, so a
crash, kill, or disk-full mid-write left a truncated, unparseable file that the
next load or `detect_incremental` then failed on.

Add `write_text_atomic`/`write_json_atomic` in graphify.paths (temp file in the
same directory + `os.replace`; JSON is streamed into the temp, not materialized
as one string) and route the graph.json writers (export.to_json,
cli._prune_graph_json_sources, the merge driver) plus detect.save_manifest
through them. The helper preserves the destination's mode (an atomic replace
never tightens 0644 to mkstemp's 0600), writes through a symlinked destination
(shared-output setups), and falls back to copy-then-delete on a Windows
os.replace lock — matching graphify.cache's existing atomic writer. On failure
the previous file is left intact and the temp removed. Not a power-loss
durability guarantee (no fsync, consistent with the rest of the codebase).
2026-07-16 23:42:47 +01:00
safishamsi 7d53a02704 fix(extract): fail closed on malformed existing graph + honor walk_errors (follow-up to #1951)
Two gaps the review found in the incomplete-build shrink guard:
- existing_graph_node_count() returned None ("proceed") on a present-but-
  unparseable graph.json, so the --no-cluster path could clobber a complete
  graph whose file was corrupt/mid-write. It now returns a MALFORMED_GRAPH
  sentinel and the caller fails closed, matching to_json's #479 handling.
- A walk that couldn't fully enumerate the corpus (permission-denied subtree,
  I/O error) is now treated as an incomplete extraction: detect()/
  detect_incremental() already record walk_errors; the extract path consumes
  them so a walk-truncated graph can't force-overwrite a complete one.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 23:42:47 +01:00
tpateeq 0f936850ba fix(extract): extend the incomplete-build shrink guard to the --no-cluster path
The clustered write is guarded by to_json's #479 shrink check, but the
`--no-cluster` raw-dump path writes graph.json directly and had no guard, so an
incomplete `--no-cluster` build could still overwrite a larger complete graph
with a partial one — the residual gap noted in the original change.

Add `existing_graph_node_count` in graphify.export (mirrors to_json's guard and
respects the graph-size cap) and, on the raw path, refuse the write with exit 1
before the manifest when the build was incomplete and the new graph has fewer
nodes than the existing one — unless --allow-partial is passed. Both write paths
now enforce the same guarantee.
2026-07-16 23:38:09 +01:00
tpateeq fcc01dbcdf fix(extract): don't force-write a partial graph over a complete one
A full `graphify extract` writes the final graph with `to_json(..., force=True)`,
which bypasses the #479 shrink guard. That is correct for a clean build that
legitimately shrinks (dedup collapse, deleted code), but when this run's
extraction was incomplete — an AST pass crashed, or some semantic chunks failed —
forcing the write lets a partial graph silently overwrite a good complete one.

The build now tracks incompleteness (AST-pass failure, semantic-pass crash, or
succeeded < total chunks) and falls back to the shrink guard (force=False) on an
incomplete run, so a smaller partial graph is refused rather than written. It
exits non-zero before the manifest is written, so the manifest is never stamped
for a graph we declined to write and the next run re-attempts. `--allow-partial`
restores force=True to override intentionally.

Note: the `--no-cluster` raw-dump path writes graph.json directly and has no
shrink guard; this change covers the normal clustered build path only.
2026-07-16 23:38:09 +01:00
safishamsi 3305ef1059 fix(merge-chunks): coerce non-numeric chunk token counts (follow-up to #1953)
An untrusted chunk with a non-numeric input_tokens/output_tokens would abort
the whole merge with a TypeError after other chunks had already merged. Coerce
to 0 so a bad token field can't defeat the per-chunk validation guard.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 23:38:09 +01:00
tpateeq ee74d80b45 fix(merge-chunks): validate untrusted subagent chunk JSON before merging
`graphify merge-chunks` concatenates agent-written `.graphify_chunk_*.json`
files with only a JSON-decode guard, so an oversized payload or a crafted
node/edge id (e.g. `../../etc/passwd`) flowed straight into the merged graph.

Route each chunk through `load_validated_semantic_fragment`, which stats the
file size BEFORE reading it (a multi-GB chunk can't blow up memory), parses the
JSON, and validates the byte/count caps + the node/edge id charset that blocks
path traversal (#825). An invalid chunk is skipped with a warning (filter
semantics; never abort). Left OUT of build_from_json/load_graph_json on purpose:
those must keep loading valid pre-existing graphs.

Also relax two over-strict checks in the shared validator that would otherwise
silently drop whole legitimate chunks (a relaxation — never a new rejection):
- file_type is no longer gated: build coerces any value via _FILE_TYPE_SYNONYMS
  (unknown -> "concept", #840), so synonyms like "markdown"/"tool" the loader
  maps must not fail validation.
- the id charset now allows Unicode word chars (build's normalize_id preserves
  CJK/Cyrillic/accented-Latin ids); the explicit path-separator/".." check still
  blocks directory escape.
Corrects the stale module docstring (the validator serves the devin skill path
and merge-chunks, not skill-opencode/codex).
2026-07-16 23:36:49 +01:00
SinghAman21 0d018b4c6d fix(cache): key semantic cache on the extraction prompt (#1939)
The semantic cache keyed entries on sha256(file content + path) alone, with
no component for the extraction prompt that produced them. After an upgrade
that changed the prompt, every unchanged file was a cache hit and replayed
the older prompt's extraction: the run exited 0, cost.json looked cheap, and
the graph silently carried two prompt generations side by side. The reporter
saw 506 of 512 docs replay an older vintage on a rebuild they expected to be
cold; deleting the whole cache was the only workaround.

output, and invalidating them on every release would re-bill extraction for
unchanged files. Fingerprinting the prompt itself keeps both properties:
entries survive releases that don't touch the prompt, and invalidate only
when it actually changed.

Semantic entries now live under cache/semantic/p{fingerprint}/, mirroring the
AST cache's v{version}/ layout. Both extraction paths pass their prompt: the
Python/CLI path from llm.py's _EXTRACTION_SYSTEM (shared by every backend),
and the skill path via a new prompt_file argument in Step B0/B3 naming the
references/extraction-spec.md the subagents were handed. The fingerprint
normalizes line endings so a CRLF checkout isn't mistaken for a new prompt.

Pre-existing entries predate fingerprinting and have unknowable vintage, so
they are still served rather than re-billing a whole corpus on upgrade — but
check_semantic_cache now warns with the count, turning "no signal at all"
into a visible one. merge_existing refuses to fuse such an entry into a
current-vintage write, which would mix two prompts inside one entry and then
attest the result to a prompt that produced half of it.

Old-fingerprint entries are pruned by liveness only, never swept wholesale
the way stale AST versions are: two hosts with different prompts (verbose vs
compact extraction-spec) can share one graphify-out/, and a wholesale sweep
would have each run delete the other's entries and re-bill on every
alternation. prune/clear/cached_files glob recursively so fingerprinted
entries can't become unprunable orphans (the #1527 failure mode).

The two monolith skills (aider, devin) inline their prompt instead of
shipping a spec sidecar and stay on the unfingerprinted path for now.
2026-07-16 16:38:12 +01:00
safishamsi cb96bdaa0c fix: preserve semantic layer, stamp hyperedges, PHP namespaces, ignore diagnostic (#1925 #1920 #1923 #1922)
- #1925: a missing manifest.json no longer degrades `extract --code-only`
  into a full scan that discards the committed semantic layer. An existing
  graph.json is a sufficient incremental baseline (detect_incremental treats
  an absent manifest as "all new / none deleted"), so out-of-scope doc/paper/
  image nodes are preserved while genuinely deleted sources still evict.
- #1920: _stamped_manifest_files now counts hyperedge output, so a doc whose
  only chunk output is a hyperedge is stamped instead of re-extracted forever.
- #1923: new namespace/use-aware PHP resolver (mirrors the Java resolver, runs
  before the unique-name rewire) so App\Models\Page and an imported
  Filament\Pages\Page stay distinct — no more false inherits/imports edge.
- #1922: detect() records ignored files/dirs in a new `ignored` diagnostic
  field (the nested-ignore scoping bug itself shipped in 0.9.16 / #1873).

Regression tests added for each; full suite 3325 passed, 3 skipped.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 19:51:53 +01:00
safishamsi b7ddee3c28 fix(extract): make --mode deep effective over a warm cache; add --force (#1894)
`graphify extract --mode deep` over a warm tree was a silent no-op, for
three stacked reasons:

1. The semantic cache ignored mode: deep runs were served standard-mode
   entries (and vice versa). check_semantic_cache/save_semantic_cache now
   take `mode` (default None, byte-identical when omitted so older
   installed callers keep working) and map it to a namespaced kind —
   cache/semantic/ for None, cache/semantic-{mode}/ otherwise. The
   per-chunk checkpoint in llm.extract_corpus_parallel and the extract /
   cache-check consumers thread the run's mode through (cache-check grows
   --mode/--deep). cached_files, clear_cache, and prune_semantic_cache
   sweep BOTH namespaces; prune uses the same live-hash set for both
   (liveness is content-based, mode-independent) so semantic-deep/ can't
   regrow the #1527 unbounded-orphan problem and inherits the
   files_by_type-derived exclusion gating for free.

2. extract had no --force — the flag was silently swallowed by the
   parser's unknown-arg fallthrough. It is now real (plus GRAPHIFY_FORCE
   env parity with `update`): force disables the incremental gate so
   detection is a full scan and skips the semantic cache READ so every
   semantic file re-dispatches, while the post-run save and manifest
   stamping still happen.

3. The incremental gate dispatched zero files on a warm unchanged tree
   before the cache was ever consulted, so namespacing alone couldn't fix
   the repro. In deep+incremental runs the semantic pass now widens to the
   full live doc/paper/image set from detect_incremental's files_by_type
   (already exclusion-filtered, #1908/#1909) and lets the mode-namespaced
   cache decide hits/misses, with a loud count line so the first deep
   run's full re-dispatch is visible.

Skill-side threading of mode is deliberately deferred to PR-2; mode
defaults keep generated skills byte-compatible.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 14:14:13 +01:00
Kookwater 2851f31a07 fix: treat .cjs (explicit CommonJS) as a code extension
.cjs was half-registered: the language maps in build.py and
extract.py already routed it to the JS grammar, but it was missing
from CODE_EXTENSIONS, the extractor _DISPATCH, _LANG_FAMILY, and the
JS resolution/cache suffix sets — so .cjs sources (Electron
main/preload scripts, CommonJS escape hatches in "type": "module"
packages) were silently skipped during a build: classified as
non-code and never handed to the JS extractor.

Add .cjs alongside .mjs in the six lists that gate/route JS files
(detect.py, extract.py _DISPATCH, analyze.py _LANG_FAMILY,
extractors/models.py cache-bypass, extractors/resolution.py resolve
exts, cli.py _HOOK_SOURCE_EXTS), with regression locks mirroring the
.mts/.cts fix (#1607).

Real-world impact: an Electron app whose ~2000-line main.cjs backend
was invisible went from 201 to 294 nodes and 256 to 441 edges on
rebuild.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-15 13:13:30 +01:00
safishamsi 75cf56bfbc fix(detect,cli): excluded files are pruned from graph and manifest, not misreported as deleted (#1908 #1909)
Two coupled excluded-vs-deleted fixes:

#1909 — incremental extract's prune set was derived from the manifest
alone (manifest - corpus), so a file that became excluded without ever
being manifest-listed (every pre-#1897 graph) kept its stale nodes in
graph.json forever. The prune set is now also derived from the existing
graph's own node source_files reconciled against the post-exclude detect
corpus (_stale_graph_sources), restricted to in-root paths; out-of-root
--include/symlinked entries and remote (://) sources are never pruned.
Relative source_files are anchored against both the scan root and the
--out root (the #555/#1899 relativized form). The --no-cluster
incremental early exit never runs build_merge, so an exclusion-only
change now prunes the raw graph.json in place instead.

#1908 — save_manifest retained any prior row whose file still existed
on disk, so an excluded-but-alive file survived as a permanent phantom
that detect_incremental reported as deleted on every run. Full-scan
callers (extract's saves, watch._rebuild_code's saves) now pass the RAW
detect corpus via a new scan_corpus parameter and in-root rows outside
it are dropped; the corpus is deliberately not the #933 stamp-filtered
files dict, so failed-chunk/omitted-doc rows and --code-only doc rows
survive. Subset saves (changed_paths hooks, #917) keep the seeding
default. detect_incremental now splits manifest rows that left the scan
into deleted_files (gone from disk) and excluded_files (alive but out of
scan), mirroring the watch-side #1795 distinction, and the extract
summaries report the two separately.

Ordering matters: extract's cleanup of newly-excluded nodes previously
worked only through the #1908 conflation, so the graph-source prune
lands together with the manifest split to avoid regressing #1909.
2026-07-15 10:58:57 +01:00
safishamsi b1e313cc5f fix(cli): stamp freshly-extracted semantic docs in the manifest (#1897)
The #933 manifest filter built its extracted-set from node/edge
source_file values, which are root-relative on a fresh extraction, and
compared them against files_by_type entries, which are absolute (from
detect()). The raw string membership test therefore never matched, so
every freshly-extracted semantic doc was dropped from the manifest and
re-queued as changed on the next run; only code files and cache-replayed
docs got stamped. The filter now lives in _stamped_manifest_files(),
which resolves BOTH sides against the scan root before the membership
test — the same Path/is_absolute/resolve normalization the #1890
reconciliation uses in graphify.llm. Genuinely omitted zero-node docs
still have no source_file entry and stay unstamped, preserving the
intentional #933 re-queue behavior.

Regression tests: a CLI extract with a mocked corpus extractor returning
root-relative source_files lands the doc in manifest.json with a
non-empty semantic_hash while a zero-node doc stays out; a unit test
covers relative (fresh) and absolute (cache-hit) source_file shapes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 00:41:37 +01:00
safishamsi 2795a18bda fix(exclude): persist --exclude so rebuilds re-apply it (#1886)
--exclude was honored only by the initial extract scan; update/watch/hook
rebuilds re-ran detect() without it and silently re-indexed excluded
paths. extract now writes the patterns to a graphify-out/.graphify_build
.json sidecar, and _rebuild_code reads and re-applies them on every
rebuild (layered after the ignore files, so they still win). Pre-existing
graphs without the sidecar behave as before. Adds a regression test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 14:03:20 +01:00
tpateeq 579ba1da3c fix(cache): scope semantic cache writes to extracted files (#1757) 2026-07-13 00:35:32 +01:00
CJNA b688a26f71 fix(path): prefer full-token label matches when resolving path endpoints (#1785)
`graphify path` committed each endpoint to _score_nodes()[0]. The full-query
bonus tier only fires when the query equals/prefixes a label, so a query that is
a token subset of the intended label ("Reject-everything judge" vs "Degenerate
Reject-Everything Judge") got no bonus and a node prefix-matching one rare token
("Rejection Summary") could out-score it on IDF alone — anchoring the path on an
unrelated, often disconnected node and returning a false "No path found".

_pick_scored_endpoint() scans the score-ordered list and takes the first
candidate whose label contains EVERY query token, falling back to scored[0] when
none does — so when the head already full-matches (the common case) resolution
is unchanged. Wired into both the `path` CLI and the MCP _tool_shortest_path.
The close-runner-up ambiguity warning now fires only when the picked endpoint is
the raw score head (a full-token override was chosen on coverage, not score, so
the head's margin is irrelevant).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 11:56:38 +01:00
bbqboogiedwonsen c5db9ffb06 fix(cli): keep outputs/cache with --out / --graph, not the corpus or CWD (#1747)
Case 1 — `extract <corpus> --out <dir>`: the graph went to <dir> (cache_root is
already passed to the AST extractor), but detect()'s word-count/stat-index cache
uses the scan root, so a stray graphify-out/cache/ was created inside the corpus
(and left behind even when the run aborted at the no-LLM-key gate). Thread an
optional cache_root through detect() -> cached_word_count() -> _ensure_stat_index()
and pass out_root from the extract CLI, so the stat index lives under --out. Entry
keys are absolute paths, so relocating the index file is safe.

Case 2 — `cluster-only --graph <elsewhere>/graphify-out/graph.json`: outputs
(GRAPH_REPORT.md, re-clustered graph.json, labels, analysis, html) were written to
the CWD's graphify-out/, ignoring where --graph lives. They now write beside the
input graph when it sits in a graphify-out/ dir (another project/tenant's output),
while still falling back to the CWD for an arbitrary archived backup/graph.json —
the restore-into-place workflow #934 pins.

Regression tests for both cases; #934 still passes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-10 02:06:26 +01:00
tpateeq 247b2ae603 refactor(main): move the command dispatch into graphify/cli.py
main() was a 2,760-line function that was almost entirely a 2,538-line
if/elif chain dispatching every non-install subcommand (query, path, explain,
diagnose, affected, reflect, save-result, extract, update, cluster-only, label,
build, global, merge-graphs, merge-driver, watch, tree, export, benchmark,
clone, prs, provider, hook*, check-update, and the `graphify <path>` redirect).

Move that chain, plus its 5 chain-only helpers (_StageTimer, _clone_repo,
_default_graph_path, _enforce_graph_size_cap_or_exit, _run_hook_guard) and 4
chain-only constants (_SEARCH_NUDGE, _READ_NUDGE, _HOOK_SOURCE_EXTS,
_GEMINI_NUDGE_TEXT), into graphify/cli.py as dispatch_command(cmd). main() now
does its setup (encoding, stale-skill check, version/help) then
`if dispatch_install_cli(cmd): return` else `dispatch_command(cmd)`.

The chain ends in its own unknown-command exit, so it's called as a statement —
no return-value plumbing. The one cli->__main__ coupling, the path-redirect's
recursive main() call, is handled by a lazy import (_reenter_main), so import
direction stays __main__ -> cli with no cycle. __main__ re-exports every moved
symbol, so importers/tests are unchanged.

__main__.py 3,388 -> 662 LOC (5,368 at branch start, -88%). Verified end-to-end:
version/help/unknown-command, the `graphify <path>` redirect (rebuilds a graph),
and query/path/explain. Full suite unchanged: 3036 passed, 29 skipped; skillgen OK.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-09 02:13:04 +05:30