Commit Graph

48 Commits

Author SHA1 Message Date
Safi 7467c1b6a4 feat/fix: land PRs #1118 #1110 #1159 #1107 #1103 (graph quality + new features)
#1118 — prune stale AST nodes on full re-extraction (#1116)
Stamps every AST-extracted node with _origin="ast" in extract(). On a
full rebuild _rebuild_code drops any AST-marked node absent from the
fresh output even when its source file survives, fixing stale symbols.
Backward-compat: marker-less nodes from pre-1118 graphs survive one
cycle then self-heal.

#1110 — stop reading images and PDFs as garbage in headless extract
Images route through per-backend vision payloads (base64/data-URI/bytes
for claude/openai/bedrock); non-vision backends get _strip_pixels for
graceful degradation. PDFs reuse pypdf. 5MB cap, 20-image chunk limit.

#1159 — Salesforce Apex extractor (.cls, .trigger)
Regex-based extractor: classes, interfaces, enums, methods, triggers,
SOQL/DML edges. No new dependency. Dispatched as .cls and .trigger.

#1107 — Azure OpenAI Service backend (--backend azure)
Uses AzureOpenAI SDK client (from existing openai package). Auto-detects
when AZURE_OPENAI_API_KEY + AZURE_OPENAI_ENDPOINT both set. Uses
max_completion_tokens (not deprecated max_tokens).

#1103 — live PostgreSQL introspection (--postgres DSN)
graphify extract --postgres "postgresql://..." introspects tables, views,
routines, and FK relations via information_schema (SERIALIZABLE READ ONLY).
Credentials sanitized on error. New graphify[postgres] extra (psycopg3).

Union-resolved llm.py conflict: Azure functions + bedrock images= param.
Fixed test_image_vision.py mock to accept timeout= kwarg (our #1112).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-07 01:20:38 +01:00
Dennis Cornwell 25df580061 fix: relativize manifest, .graphify_root, and cache source_file fields (#777)
Fixes #777. Relativize manifest keys, .graphify_root, and cache source_file fields on persist; re-anchor on load. In-memory callers still see absolute paths. Symlink round-trip fixed in follow-up commit 8f09326.
2026-06-03 21:03:06 +01:00
Safi f0badd9899 fix Windows claude-cli WinError 2 and post-commit hook silent drop on rapid commits
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-29 11:41:42 +01:00
Safi d1d5751fa4 fix(watch): evict stale nodes in full re-extraction path when changed_paths is None (#1007)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-27 11:21:55 +01:00
Safi 3f8efaebc2 fix graphify update ghost nodes: as_posix() and relativize existing graph before eviction
Three issues in _rebuild_code (watch.py):
1. _relativize_source_files was called on result after eviction list was built,
   so existing nodes with absolute source_file were never normalized before comparison
2. deleted_paths and evict_sources used str() (backslashes on Windows) while
   graph.json stores forward-slash paths via _norm_source_file
3. _relativize_source_files itself used str() instead of as_posix()

Also fix extract.py source_file relativization to use as_posix(). Closes #1007.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-26 20:05:31 +01:00
mohamadkattan12 6fba4e4594 fix(watch): bypass shrink-guard when caller declared explicit deletions (#1000)
The post-commit hook passes `git diff --name-only HEAD~1 HEAD` as
changed_paths to `_rebuild_code`. That list includes deletions, and
`_rebuild_code` correctly identifies them (lines 352-367 in watch.py)
and evicts the stale nodes from the preserved set. The rebuilt graph
is intentionally smaller.

`_check_shrink` then refuses to overwrite the existing graph.json
because it only sees the node-count delta, not the cause. The guard
fires with "Refusing to overwrite — you may be missing chunk files
from a previous session. Pass --force to override."

Result: every commit that deletes a tracked file silently leaves stale
nodes in graph.json. The user must either pass --force (which also
disables the guard for legitimate failure modes) or manually re-run
`graphify update . --force` after delete-heavy commits.

Fix: thread a `had_explicit_deletions` flag from `_rebuild_code` into
`_check_shrink`. When the caller has declared the deletions, the
smaller graph is the expected outcome and the guard is skipped. The
guard remains intact for SILENT shrinkage — its actual purpose —
from failed semantic chunks or corrupted runs.

The fix is opt-in by design: callers that don't pass `changed_paths`
(e.g. the post-checkout full rebuild path) keep the old conservative
behavior. Only paths that explicitly track deletions get the bypass.

Tests added (tests/test_watch.py):
- `test_check_shrink_blocks_silent_shrink` — pre-existing behavior intact
- `test_check_shrink_allows_force_override` — pre-existing behavior intact
- `test_check_shrink_allows_explicit_deletions` — new: deletion bypass
- `test_check_shrink_allows_no_existing_data` — first-run case
- `test_check_shrink_allows_growth` — sanity
- `test_check_shrink_unlinks_tmp_on_refuse` — cleanup on refusal
- `test_check_shrink_keeps_tmp_when_deletions_declared` — no spurious unlink
- `test_rebuild_code_prunes_deleted_file_nodes` — end-to-end probe of the
  exact scenario the post-commit hook triggers (git init, build, delete
  one file, re-run with the deleted path in changed_paths, verify the
  graph shrinks and the surviving file's nodes are preserved)

All 8 new tests pass; full test_watch.py + test_build.py + test_export.py
(74 tests) pass with no regressions.
2026-05-24 20:35:19 +01:00
hypnwtyk b6127aa5a7 feat(multigraph): add runtime compatibility probe (#956)
* feat(bash): harden extractor — literal filtering, entrypoint nodes, AST-ancestry-aware command detection

Builds on tree-sitter-bash extractor from #866. Two correctness/security
improvements to bash extraction in graphify/extract.py:

1. Reject command/process substitutions at extraction time. Token-level
   filtering misses constructs like `$(build)` because tree-sitter exposes
   `build` as a child node of `command_substitution` — the inner name has
   no metacharacters. Added `is_inside_expansion(node)` that walks
   `node.parent` until it finds `command_substitution` or
   `process_substitution`. Used as a gate in both `walk` and `walk_calls`.
   Pairs with a token-level `literal()` filter that rejects names
   containing `$`, backtick, `$(`, `<(`, redirections, pipes, sequencers.

2. Entrypoint node. Every .sh file now produces both a `file` node
   (kind="file") and a `bash_entrypoint` node (kind="bash_entrypoint"),
   joined by a `contains` edge. A separate top-level `walk_calls(root,
   entry_nid, ...)` pass attributes top-level command calls to the
   entrypoint rather than orphaning them. Matches the entrypoint pattern
   other-language extractors use. Node metadata gains language+kind.

Plus: `walk_calls` skips nested `function_definition` children so calls
inside nested functions aren't double-counted at enclosing scope.

Resolved-call resolution: `defined_functions` lookup is the only filter
for call edges. User-defined functions named like external commands
(install, find, git, ...) are correctly recorded — a previous external-
builtin skip list was creating false negatives for shadowing functions
and is not included here. Skip list belongs with raw/unresolved call
recording (not in this PR).

Devtools (bundled): pyproject.toml gains [dependency-groups] dev (ruff,
pyright, pre-commit, hypothesis, pip-audit) plus minimal [tool.ruff],
[tool.ruff.lint], [tool.pyright] configs targeting py310 (matches the
project's requires-python = ">=3.10").

Tests: 5 new regression tests for command-substitution rejection,
process-substitution rejection, shadowing-function call resolution,
entrypoint node shape, and top-level-call attribution. 826/826 pass
(was 821); 15/15 bash-relevant tests pass (was 10).

* feat(detect): parse macOS/BSD and GNU env(1) shebang option forms

Upstream's _shebang_file_type parses shebangs via line[2:].split() and only
handles `#!/usr/bin/env <interp>`. Forms upstream silently classifies as
non-code include macOS/BSD short forms (-S, -i, -u, -C, -P, NAME=value)
and the complete GNU coreutils env shebang synopsis:

    #!/usr/bin/env -[v]S[option]... [name=value]... command [args]...

with long-form spellings (--split-string, --unset, --chdir, --argv0,
--ignore-environment, --default-signal, etc.), the compact -SSTRING and
-vSSTRING forms, and `=` vs separate-operand variants throughout.

Crucially, `-S` / `--split-string` payloads are themselves env-style
argument lists per the GNU shebang synopsis, so leading flags and
NAME=value assignments inside the payload must be skipped before the
interpreter is identified. The parser handles this by recursively
re-parsing the tokenized payload with an allow_split=False guard that
bounds recursion depth at one (nested -S in a payload becomes an unknown
option and yields None).

Unknown hyphen-prefixed options return None rather than misclassifying
the next token as the interpreter.

_shebang_file_type becomes a 4-line wrapper. Read buffer raised 128 -> 256
to accommodate longer env -S strings.

Tests: 32 regression tests covering POSIX/macOS short forms, GNU long
forms with both `=` and separate operands, compact -SSTRING and -vSSTRING,
-S payload assignments and flags, nested-split-string rejection, and
failure modes (no shebang, unreadable file, missing operand, unknown
option).

* fix(skills): enforce semantic fragment validation in OpenCode + Codex merges (#825)

Closes #825. Adds graphify.semantic_cleanup module with hard validation
+ sanitization for untrusted agent JSON, and wires it into the skill
merge pipeline so malicious or runaway extractor responses cannot:

- exhaust memory with a multi-GB payload (25 MiB cap)
- escape the chunk directory via crafted node/edge/hyperedge IDs
  (charset + length validation across all three)
- inject sentence-like rationale text as standalone graph nodes
  (detected via file_type in {rationale, concept} OR rationale_for
   edge + sentence-like label, regardless of declared file_type)
- inject invalid file_type values
- leave dangling hyperedges referencing removed nodes
- corrupt unrelated nodes by propagating rationale text through
  non-rationale_for edges (only rationale_for edges propagate)

Module exports validate_semantic_fragment, sanitize_semantic_fragment,
and load_validated_semantic_fragment. Wired into skill-opencode.md and
skill-codex.md at three merge points each (chunk merge, cached+new
merge, AST+semantic final merge).

Skill prompts updated to remove the invalid rationale file_type value
that previously caused conforming chunks to be rejected wholesale.
Valid set is now {code, document, paper, image}.

Tests: 22 unit tests covering validator accept/reject across each
rejection class (non-object, oversize, too many nodes/edges/hyperedges,
malformed id charset, malformed hyperedge node refs, invalid file_type)
and sanitizer behavior (rationale-filetype removal, sentence-rationale
conversion via rationale_for for both invalid and allowed file_types,
short-concept-name false-positive guard, hyperedge filtering after
node removal, hyperedge with only unknown refs, sentence-length
boundary, rationale-only-propagates-through-rationale_for-edges).

880/880 tests pass.

* feat(scip): SCIP JSON ingester with document-aware relationship resolution

Adds graphify.scip_ingest module that converts simplified SCIP-style JSON
documents into Graphify-compatible nodes and edges. Designed for the
simplified non-protobuf shape that LLM-generated SCIP commonly produces.

Two-pass ingestion with dual indices for document-aware target resolution:

  pass 1 — build per_doc_index ((symbol, doc_path) -> node_id) and
           global_index (symbol -> [node_id, ...]) across every valid
           symbol in every valid document. Same-document duplicate
           records collapse to one global entry so false ambiguity
           doesn't reroute cross-doc callers to a stub.
  pass 2 — emit nodes for indexed symbols, then walk relationships.
           Resolution order:
             1. same-doc match (per_doc_index)
             2. unique cross-doc match (global_index[symbol] len == 1)
             3. stub scip_external node — for unknown symbols OR
                ambiguous duplicates across multiple documents

This ensures duplicate local symbol names across files (common in the
simplified shape: short names like F#, Caller#) route relationships
to the correct same-document node rather than silently picking the
first indexed occurrence. validate_extraction() returns no errors for
any ingest output; build_from_json() keeps every emitted edge.

Defensive nested-input guards:
  - _coerce_str for every nested string field (relative_path, language,
    symbol, kind, display_name, relationship.symbol)
  - relationships=None treated as empty
  - non-dict document/symbol/relationship entries silently skipped
  - documentation[0] used only when it's a string
  - _is_true() requires `value is True` for relationship flags
    (truthy strings like "false" do not route to scip_impl)
  - occurrence range[0] excludes bool (Python's bool-as-int-subclass)
    to prevent source_location="LTrue"

Module is stdlib-only (hashlib, re, typing.Any). Not wired to the CLI
in this phase — importable as `from graphify.scip_ingest import
ingest_scip_json`.

Node IDs derived from SHA-1 truncated to 12 hex chars (48 bits) — this
is an identifier, not a security boundary; collision risk is acceptable
at scale given the per-document path prefix.

Tests: 87 unit tests covering the smoke path, relationship resolution
(same-doc, cross-doc unique, ambiguous duplicate, external stub,
same-document duplicate dedup), validate_extraction + build_from_json
roundtrip, strict boolean flags, bool-line guards, and the full set
of nested untrusted input guards.

1044/1044 tests pass.

* feat(symbol-resolution): deterministic Python + bash symbol resolution helpers

Adds graphify.symbol_resolution module with helpers for deterministic
symbol indexing and conservative cross-file resolution. Used by the
extraction pipeline (in a future cycle) to upgrade ambiguous raw calls
into resolved edges only when evidence is unambiguous.

Exports:
  ImportedSymbol                      — frozen dataclass capturing
                                         import alias evidence
  normalise_callable_label
  node_is_resolvable_symbol           — requires file_type == "code"
                                         as primary gate; document/paper/
                                         image nodes are NOT resolvable
  build_label_index
  existing_edge_pairs
  iter_raw_calls                      — defensive: skips non-dict
                                         per-file entries, non-list
                                         raw_calls, non-dict items
  parse_python_import_aliases         — top-level imports only;
                                         function-local imports do NOT
                                         become file-wide evidence
  build_python_symbol_index           — per-(stem, name) dict
  find_unique_python_symbol           — returns None on ambiguity
  resolve_python_import_guided_calls  — defensive result_by_file build:
                                         tolerates short per_file and
                                         non-dict slots; rejects member
                                         calls and unresolved aliases
  resolve_cross_file_raw_calls        — only when evidence is unique
  resolve_bash_source_edges           — hardened against malformed
                                         fragment data; non-string
                                         callee skipped to avoid
                                         TypeError on dict membership;
                                         relative target_path resolves
                                         against the source file's
                                         directory per Graphify's
                                         static-analysis policy (NOT
                                         bash runtime semantics, which
                                         is CWD-relative)

Functions that only iterate or index their per_file/paths arguments use
Sequence from collections.abc for proper covariance. Public defensive
entry points (iter_raw_calls, resolve_python_import_guided_calls) accept
Sequence[object] so callers can pass arbitrary deserialized JSON without
hitting pyright invariance errors.

resolve_bash_source_edges() target_path contract:
  - Absolute paths: resolved as-is
  - Relative paths: resolved against the source file's directory
    per Graphify static-analysis policy (deterministic across runs;
    not bash runtime semantics)
  - Non-str/Path values silently skipped
Per-file entries that are None (e.g. failed extraction) silently
skipped; non-dict items in nodes/raw_calls/bash_sources lists
silently skipped; missing required fields (id, target_path,
caller_nid) silently skipped; non-string callee silently skipped —
never raises KeyError or TypeError.

Module is stdlib-only (ast, re, dataclasses, pathlib, typing,
collections.abc). Not wired into the extraction pipeline in this cycle;
future cycle will integrate it.

Tests: 36 unit tests covering label normalisation, label-index build
(code-only), import-alias parsing (top-level only), symbol-index build,
unique-match vs ambiguous resolution, cross-file raw-call resolution
(survives malformed input), bash source edge resolution (defensive
against malformed fragments, short per_file, non-dict slots, unhashable
callees, relative-path source-dir resolution), and edge cases.

* feat(security): cap graph.json loaders at 512 MiB before parsing

exhaustion on adversarial or pathological inputs.

- graphify.security: add _MAX_GRAPH_FILE_BYTES + check_graph_file_size_cap
- graphify.serve._load_graph: call cap after existence check
- graphify.__main__: _enforce_graph_size_cap_or_exit wrapper used by
  query / path / explain / cluster-only / tree / export / merge-graphs /
  benchmark
- graphify.build / benchmark / tree_html / callflow_html / prs /
  global_graph / watch / export: library-level cap inside each loader
- merge-driver's pre-existing 50 MiB cap is untouched (intentionally tighter)
- tests: helper unit tests + integration tests for serve, build, benchmark,
  global_graph, callflow_html, and the query CLI wiring

* feat(security): sanitize_metadata at graph export boundaries

Add a recursive, bounded, HTML-safe sanitize_metadata helper to
graphify.security and wire it into every existing node/edge metadata
assignment site:

- scip_ingest.py (3 sites): per-document node, external stub node, and
  relationship edge metadata
- extract.py (1 site): bash extractor's add_node metadata
- symbol_resolution.py (1 site): Python import-guided call edge metadata

Helper policy:
- Strip control chars, html.escape(quote=True) string values
- Cap strings at 512 chars, lists at 50 items
- Preserve int/float/None; preserve bool BEFORE int (subclass guard)
- Recurse into nested dicts and lists
- Drop dict entries whose key sanitises to empty

Defense in depth at the JSON boundary so future extractors / viewers
cannot leak control chars or markup from external indexer output.

* feat(security): pin vis-network CDN with SRI hash

Pin the vis-network <script> tag in to_html() to a versioned URL
(vis-network@9.1.6) with a sha384 Subresource Integrity hash and
crossorigin="anonymous". Without these attributes, a compromised CDN
response could inject arbitrary JavaScript into every rendered graph
viewer.

Hash verified live against
https://unpkg.com/vis-network@9.1.6/standalone/umd/vis-network.min.js:

  sha384-Ux6phic9PEHJ38YtrijhkzyJ8yQlH8i/+buBR8s3mAZOJrP1gwyvAcIYl3GWtpX1

Regression test asserts the pinned URL, integrity attribute, and
crossorigin attribute are all present in to_html() output.

Follow-up: tree_html.py (D3) and callflow_html.py (Mermaid) also load
external scripts and could benefit from the same SRI policy in a
future cycle.

* fix(review): address real Copilot review findings in base stack

Resolves 7 issues found in upstream code review of PRs #893 and #954:

1. extract.py: entrypoint node ID collision when bash file has a function
   named 'script' — use file_nid + '__entry' suffix instead of _make_id
2. extract.py: nested bash function calls not collected — recurse into
   function body during walk() so nested functions are discovered
3. extract.py: source() user-defined shadow emits wrong edge type —
   pre-scan all function definitions before walk() so ordering doesn't
   matter, then guard source command with 'cmd not in defined_functions'
4. extract.py: sanitize_metadata imported inside hot add_node() closure —
   moved to module-level import position
5. symbol_resolution.py: _bash_make_id() diverged from extract._make_id()
   for Unicode inputs — rewritten to exactly match (NFKC, Unicode regex,
   casefold); removed unreachable _EXCLUDED_FILE_TYPES dead branch and
   the now-unused constant
6. semantic_cleanup.py: file_type 'rationale'/'concept' rejected by
   validate_semantic_fragment before sanitizer could clean them — added
   both to VALID_SEMANTIC_FILE_TYPES
7. scip_ingest.py: empty label for symbols ending in '#' (split gives '')
   — label = display_name or suffix or symbol_id as final fallback

All 7 issues covered by new failing-first regression tests (red → green).
Full pytest suite: 1239 passed, 4 pre-existing env-specific failures.

* fix(review): address PR #956 Copilot findings in watch.py and symbol_resolution.py

- watch.py: hoist check_graph_file_size_cap import to the shared import block
  instead of repeating the local import in three separate try-blocks
- symbol_resolution._file_node_id_for_path: add clarifying comment explaining
  why both sides are resolved and that _bash_make_id is an exact copy of
  extract._make_id (addressing reviewer concern about ID mismatch)

* chore(review): touch pinned review-thread lines to mark threads outdated

Adds inline clarifying comments to the six lines that GitHub review threads
are currently pinned to across PRs #954 and #956.  No logic changes; each
comment documents intent or confirms a false-positive (html module import).

* feat(diagnostics): report multigraph edge-collapse risk

Add graphify.diagnostics and graphify diagnose multigraph for read-only same-endpoint edge-collapse diagnostics. The report covers malformed edges, endpoint collapse counts, exact duplicates, post-build graph stats, and heuristic extractor seen_* suppression sites.

Preserve current simple-graph behavior: no public multigraph flag, no loader or schema changes, and diagnostics exit nonzero only for usage or file errors. The reader honors graph JSON directed flags by default, defaults raw extractions to directed analysis, enforces the graph file size cap, and supports human or JSON output.

* feat(multigraph): add runtime compatibility probe

New module graphify.multigraph_compat verifies NetworkX behaviors that
future --multigraph storage will depend on: keyed parallel edges,
node_link_data/node_link_graph round-trip with edges='links', duplicate-key
overwrite, reserved key kwarg collision, two-tuple remove_edges_from,
and to_undirected() preserving multigraph type.

Behavior probe, not version check. Both NX 3.4.2 (Py 3.10 lane) and
NX 3.6.1+ (Py 3.11+ lane) pass. Result cached for the process lifetime.

No call sites added — this PR adds the API surface only. Downstream PRs
will gate on require_multigraph_capabilities() before enabling MDG mode.

Refs: Wave 1 MultiDiGraph implementation order.

* test: filter known third-party analyze warnings

---------

Co-authored-by: vampyre <vampyre@local.net>
2026-05-22 13:22:51 +01:00
Safi 6939494b3e add backup_if_protected to snapshot graph before overwrite when semantic/curated (#834)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-18 21:57:09 +01:00
Safi 47e65658c7 bump version to 0.8.12 — security and wiki fixes
Security: _is_sensitive now flags underscore-prefixed names (api_token.txt, oauth_token.json) by replacing \b with lookarounds; adds _SENSITIVE_DIRS check on parent path components (parts[:-1]) so .ssh/, secrets/, .aws/ directories are always skipped; aligns both patterns to (?![a-zA-Z]) for consistent underscore-after-keyword behavior (#920)

Fix: --wiki Relationships section always empty because _cross_community_links read community from node attrs (always None) instead of the communities dict; _god_node_article had the same bug and never linked to the owning community; fixed by building a node->community map in to_wiki() and threading it through (#925)

Fix: --watch now respects .graphifyignore; patterns loaded once at startup, handler checks _is_ignored before extension filter so node_modules/, .venv/, build/ churn no longer triggers rebuilds (#928)

Fix: C++ struct inheritance edges via base_class_clause; initialize base="" per iteration to prevent stale carryover (#915)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-18 15:43:45 +01:00
Safi 972fdc1b87 fix #862 #861 #857: skill-windows.md path prefix, sidecar graphifyignore bypass, manifest ast/semantic split
- skill-windows.md: add graphify-out/ prefix to 5 paths in Step B3 that
  were being read/written without prefix (lines 244,245,373,387,388,407,413)
  causing semantic cache to always appear empty on Windows (#862)
- detect.py: gate converted office/Google Workspace sidecar paths through
  _is_ignored() before appending to files list so .graphifyignore rules
  apply to graphify-out/converted/ sidecars (#861)
- detect.py: split manifest entries from {mtime,hash} to
  {mtime,ast_hash,semantic_hash}; save_manifest() accepts kind="ast"|
  "semantic"|"both"; detect_incremental() accepts kind= and checks the
  right hash field so graphify update (kind=ast) no longer prevents
  graphify extract from re-running semantic extraction (#857)
- watch.py: all three save_manifest() calls use kind="ast" (AST-only rebuild)
- __main__.py: both extract save_manifest() calls use kind="both"
- backwards compatible: legacy {mtime,hash} entries treated as ast_hash only

775 tests passing

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-14 23:31:14 +01:00
voidborne-d 2c975ee9fb fix(watch): unlink .rebuild.lock on release and rewrite single PID line (#858)
The rebuild lock file accumulated concatenated PIDs across post-commit
rebuilds without a separator, and was never removed when the rebuild
finished. Two practical consequences for users:

1. Downstream tooling that polls for `.rebuild.lock` to disappear before
   doing post-rebuild work (publish scripts copying graph.html to a web
   root, etc.) blocked forever / until its own timeout.
2. The accumulated digit string could not be parsed by humans or tooling
   to find the owning PID.

The `_rebuild_lock` context manager now:

- Opens the lock file with `a+` so a non-acquiring caller does not
  truncate the existing holder's PID.
- After flock acquisition, truncates and writes a single `<pid>\n` line
  so external readers can `kill -0 $(cat .rebuild.lock)` to check
  liveness.
- Unlinks the lock file in the finally block (only when *we* held the
  lock), restoring the "signal-by-absence" convention users rely on.

Four regression tests added under `tests/test_watch.py` covering the
PID-with-newline payload, post-release unlink, no-accumulation across
sequential acquisitions, and the non-blocking-caller-does-not-clobber
invariant.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-14 08:05:55 +08:00
Safi f7c9a9ad82 fix watch.py labels churn, edges/links schema, shrink-check duplication, and skill.md ID edge cases 2026-05-14 00:29:54 +01:00
Ahmad Fathallah 28fc71eddc skip reclustering when topology is unchanged
Add a pre-cluster topology comparison fast path in update rebuilds so unchanged graphs short-circuit before clustering and report generation, preventing residual run-to-run community-count drift.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-12 03:36:26 +03:00
Ahmad Fathallah ef0e6ee681 harden community serialization fallbacks
Use safe JSON serialization fallbacks for deterministic sort keys in clustering and graph canonicalization, and skip invalid community IDs with a stderr warning instead of raising during update rebuilds.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-12 02:37:38 +03:00
Ahmad Fathallah 8e4c803f31 reduce graph update churn and stabilize community IDs
Make `graphify update` idempotent by skipping output rewrites when graph/report content is unchanged, add `update --no-cluster`, and preserve community IDs across runs via overlap-based remapping with deterministic partition inputs.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-12 02:23:51 +03:00
Safi 5ead190891 wire callflow HTML into watch/hook rebuild: regenerates automatically if file exists 2026-05-09 23:29:43 +01:00
Safi dc50979a68 fix hook OOM, lift worker cap, fix antigravity .agents path and frontmatter
#791: per-repo fcntl flock in _rebuild_code prevents concurrent hook rebuilds
from exhausting memory; changed_paths wired through so only modified files are
re-extracted; stale nodes evicted on deletion; SIGALRM watchdog with
GRAPHIFY_REBUILD_TIMEOUT; Darwin-aware RLIMIT_DATA memory cap

#792: remove hard 8-worker cap (GRAPHIFY_MAX_WORKERS env var); add
--max-workers, --token-budget, --max-concurrency, --api-timeout CLI flags
to graphify extract; fix ollama API key gate for loopback URLs; explicit
timeout on OpenAI client (GRAPHIFY_API_TIMEOUT, default 600s); per-chunk
progress prints during extraction

#453 + #785: rename .agent -> .agents throughout antigravity install/uninstall;
add trigger:always_on YAML frontmatter to _ANTIGRAVITY_RULES so Antigravity
recognises the rules file

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-09 16:57:05 +01:00
shym 1026695da7 feat: add .qmd file extension support
Adds support for Quarto markdown (.qmd) files by:
- Adding '.qmd' to document file extensions in detection
- Updating export logic to handle .qmd in filename sanitization
- Adding .qmd extractor dispatch using the existing markdown extractor
- Updating watch comments to include .qmd files
2026-05-07 11:13:14 +09:00
imfarhanm 09b33b795b fix: include .md/.mdx document files in graphify update rebuild path 2026-05-04 20:01:20 +05:30
Safi b3c99ec0ed Fix hook rebuild losing community labels; fix gemini platform install (#705, #706) 2026-05-04 18:20:16 +01:00
Safi 8e81720bc8 Fix #686 #652: GRAPHIFY_OUT env var for worktrees, Antigravity install auto-updates rules/workflow
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-03 17:36:17 +01:00
Safi ca48092413 Add --force flag and GRAPHIFY_FORCE env var to graphify update
Bypasses the node-count safety check in to_json for refactors that
legitimately shrink the graph (renames, package deletions). Also
honored by post-commit and post-checkout hooks via GRAPHIFY_FORCE=1.

Implements the approach from #639 (targeted at v5) adapted for v6.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-02 13:52:09 +01:00
Safi c8969edd3c Fix semantic node preservation on incremental rebuild and detach git hooks
watch.py: filter preserved nodes by ID membership in new AST output instead
of file_type — INFERRED/AMBIGUOUS nodes on code files also carry file_type=code
and were being wrongly dropped, triggering the to_json safety check refusal.

hooks.py: detach post-commit and post-checkout rebuilds with nohup + disown
so git commit returns immediately instead of blocking for the full rebuild
duration. Rebuild log written to ~/.cache/graphify-rebuild.log.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-02 08:53:17 +01:00
Safi 3fdae8f334 fix #623 #621 #605 #638 #589 #586 #593: kimi thinking, manifest, inline comments, query boost, cache race, markdownify, content hash
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-01 22:15:24 +01:00
Safi f9c344b546 Remember scan root so graphify update works without a path arg
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 09:43:51 +01:00
Safi 4563b043f2 Fix 6 bugs: ID collisions, path portability, alias resolution, HTML controls, desync guard, rationale prompt
- #550: _file_stem() includes parent dir to prevent node ID collisions for same-named files
- #555: extract() relativizes source_file paths before returning for cross-machine portability
- #562: to_json() returns bool; _rebuild_code() writes report/html only if json succeeded
- #563: skill prompts store rationale as node attribute, not separate node; enforce calls direction
- #566: Show All / Hide All buttons added to HTML community panel
- #575: _import_js() resolves tsconfig.json compilerOptions.paths aliases before external fallback

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-27 21:44:22 +01:00
Safi 64f38acf3e implement #488 #482 #472 #490: legacy schema canonicalization, Java inheritance, aggregated HTML viz, check-update subcommand 2026-04-22 23:28:09 +01:00
Safi dbde78ca03 Fix four bugs from community PRs
- cache: skip directory source_file in save_cached to prevent IsADirectoryError (#444)
- report: skip structural-only communities with no real nodes (#443)
- hooks: allow @ in python path allowlist for Homebrew paths (#474)
- watch: keep source_file paths project-relative after rebuild (#434)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-21 22:26:17 +01:00
Safi 4738e88aaf fix #453 antigravity install uses .agents not .agent, fix #436 _is_sensitive false positives on directory names, fix #433 update command writes absolute paths in artifacts
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-21 21:33:33 +01:00
Safi 9c2275ad82 fix #432 to_html crash on large graphs, fix #431 Go import node ID collision 2026-04-18 10:54:39 +01:00
Safi c24cde9d9e v0.4.22: fix #429 AST cache root, fix #428 add .mdx to DOC_EXTENSIONS 2026-04-18 09:47:06 +01:00
Safi a507c97822 v0.4.20: fix #414 JS imports_from path normalisation, fix #418 graph.html missing from CLI 2026-04-17 15:47:56 +01:00
Safi 8a474eb1b3 v0.4.19: fix #390 #298 #410 #401 #385, team workflow docs, Windows/pipx tips
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-17 12:46:42 +01:00
Safi b2523fcdec v0.4.16: fix watch NameError, .mjs dispatch, exclude llm.py from wheel
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-16 07:01:04 +01:00
Safi 429e46a665 v0.4.15: VS Code Copilot Chat, OpenCode/Gemini Windows fixes, .mjs/.ejs, macOS watch, god_nodes degree rename
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-15 23:08:43 +01:00
Safi 4210de270f Fix watch edge key, claw path, Blade support, WSL MCP docs (0.4.7)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-12 20:49:24 +01:00
Safi 5ab8eb34c5 fix #261, #249, #266 and bump to 0.4.4
- watch.py: preserve INFERRED/AMBIGUOUS edges (code<->doc) across rebuilds (#261)
- __main__.py: fix Codex hook - use additionalContext instead of permissionDecision:allow (#249)
- detect.py: skip common lockfiles (package-lock.json, yarn.lock, Cargo.lock etc.) (#266)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-12 18:37:54 +01:00
Safi d1808bab8c fix bugs #256, #253, #244, #226, #254 and bump to 0.4.3
- extract.py: resolve relative JS/TS imports to full-path IDs (fixes 0 import edges on TS codebases) (#256)
- extract.py: resolve relative Python imports to full-path IDs (#256)
- watch.py: merge fresh AST with existing semantic nodes instead of overwriting (#253)
- hooks.py: add python fallback after python3 for Windows; exit 0 if neither found (#244)
- analyze.py: guard stale _src/_tgt hints with node membership check (#226)
- detect.py + extract.py: add .vue and .svelte to CODE_EXTENSIONS and _DISPATCH (#254)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-12 12:59:15 +01:00
Safi 8c17230586 cherry-pick PRs #212, #220, #204, #221 into 0.4.2
- build/validate: accept NetworkX <=3.1 "links" key alongside "edges" (#212)
- __main__: skip version check during install/uninstall, deduplicate paths (#220)
- all file IO: explicit encoding="utf-8" to prevent crashes on Windows CJK locales (#204)
- hooks: add newline="\n" on write to prevent CRLF shebang breakage on Windows (#204)
- export: strip trailing .md from safe_name so "CLAUDE.md" doesn't become "CLAUDE.md.md" (#221)
- report: add Community Hubs navigation block so Obsidian vault stays connected (#221)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-11 21:07:39 +01:00
Safi df77d5f8ce Add Cursor support, fix _rebuild_code KeyError and node_link_data crash (#137, #148, #149)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-09 17:19:40 +01:00
Safi c3817d6144 Apply PRs #82 #93 #102 #109: extension drift, click detection, skill coverage, .graphify_python persistence
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-09 08:22:01 +01:00
Safi 91cbc74bb1 release 0.3.18: fix watch .graphifyignore, codex hook, trae link, Korean README, save-result CLI 2026-04-09 08:09:36 +01:00
Safi a78d25d4fc Add ObjC support, C# inheritance, CLI query, symlink support, bug fixes (v0.3.7-0.3.9)
- Add Objective-C extractor (.m/.mm) with @interface, @implementation, @protocol, imports, calls
- Add C# inheritance extraction via base_list nodes (inherits edges)
- Add --obsidian-dir flag to skill.md and skill-windows.md
- Add graphify query CLI command with --dfs, --budget, --graph flags
- Add follow_symlinks parameter to detect() and collect_files() with cycle detection
- Fix semantic cache relative path resolution (was saving only 4/17 files)
- Fix validate.py missing rationale file_type causing 75 warnings per run
- Add tree-sitter-objc dependency
- Update README with .m/.mm extensions, --obsidian-dir usage, Codex $ skill trigger
- 367 tests passing

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-07 20:20:49 +01:00
Safi c529e2217f Fix 8 bugs: SSRF in tweet fetch, Cypher injection, XSS in community labels, graph_diff edge ordering, cache key collision, hook uninstall correctness, watch mode set, rebuild word count 2026-04-06 22:23:55 +01:00
Safi e2fd4f944e watch: auto-rebuild graph on code changes without LLM, notify on doc/image changes 2026-04-06 16:06:31 +01:00
Safi 5db8f7ce39 docs: update surprising connections description, test count
style: replace all em dashes with hyphens

fix: explain hidden .graphify/ folder in skill output and README

fix: rename .graphify/ to graphify-out/ so output is visible by default
2026-04-06 16:06:31 +01:00
Safi 81a43f028f feat: 13-language AST support and token benchmark
Java, C, C++, Ruby, C#, Kotlin, Scala, PHP via tree-sitter (13 total)
benchmark.py measures BFS subgraph tokens vs corpus tokens
5 skill bug fixes (cohesion crash, dead step, missing MCP tool)
2026-04-04 18:56:38 +01:00
Safi ce47198be1 feat: Claude Code skill, Obsidian vault, install, tests
skill.md with full pipeline steps, Obsidian as default output (canvas, tags,
dataview, graph colors), two-command install, 71 tests, .gitignore, deps
2026-04-04 18:53:43 +01:00