mirror of
https://github.com/Ar9av/obsidian-wiki.git
synced 2026-09-14 20:36:34 +08:00
fix/loopback-default
9 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Arnav
|
c4b71c5f83 |
Merge pull request #171 from mike840609/fix/code-understanding-config-and-doctor
fix(cli): complete code-understanding config resolution and doctor hardening |
||
|
Zorion Arrillaga
|
f085d14f04 |
feat(lint): report pages that share a slugged stem
`graph_analysis` identifies a page by its slugged filename stem, so two pages whose stems slug alike are one node — and one entry in the graphrag index. On a vault holding concepts/vector-search.md and entities/vector-search.md, build_index keeps only the entities page; the concepts page is absent from the index that memory_search and graph-query read. duplicate_titles does not cover this: it keys on the title: value, and stems can collide while titles differ. Add duplicate_stems, built over graph_analysis.iter_pages with graph_analysis._page_slug so the report keys as the merge does. It reads pages rather than links, so a collision nothing references is still reported. Severity warn, beside duplicate_titles. Borrowing the graph's page selection means a collision inside _bootstrap/ is now visible to this check, where lint's own SKIP_DIRS excludes that directory. The existing bootstrap exclusion test still passes; the tradeoff is called out in the pull request rather than settled here. |
||
|
Arnav
|
8b5859d0f8 |
Vault state management: manifest write safety, lifecycle transitions, and equilibrium mode (#195)
* fix(cache): lock and atomically write .manifest.json update_source was a plain read-modify-write with no mutual exclusion, so any two overlapping writers lost an entry silently: the second reader's snapshot predated the first writer's write. This fires in three real configurations — batch-plan fan-out dispatching parallel ingest subagents, the Dockerized server writing a vault a local skill is also using, and a scheduled daily-update overlapping an interactive session. Reproduced with 12 concurrent update_source processes against one vault: 6 of 12 entries recorded before, 12 of 12 after. Writes now take an advisory lockfile and replace the manifest atomically via a temp file plus os.replace. O_CREAT|O_EXCL rather than fcntl because this ships on Windows. Hashing stays outside the lock — it can take seconds on a large source tree and nothing in the manifest depends on it. The contention test spawns real subprocesses; a single-process test cannot observe this race. * feat(trust): flag illegal lifecycle transitions in lint The lifecycle field is a state machine, but nothing enforced its edges. Ledger entries now record the lifecycle at review time, giving lint a baseline to compare the current frontmatter against. Only edges that are illegal for every possible sequence of unobserved intermediate states are flagged: falling back to draft (only ingest sets draft) and exiting archived (terminal). draft -> verified is deliberately not flagged — ledger snapshots are sparse, so a legitimate intermediate reviewed may simply never have been recorded, and flagging that pair would fire on valid history. Additive: entries predating the lifecycle field carry no baseline and are skipped, so existing vaults behave identically until their next trust-record. Owner-extended lifecycle states never match the illegal pairs, so the check degrades to a no-op rather than firing on states it does not understand. Warns by default, fails under --strict-trust, matching how the surrounding trust checks were already staged. * feat(wiki-status): add vault equilibrium mode The maintenance skills each optimize one shared vault for a different objective, so any of them can undo another's work and no single skill reporting clean means maintenance is done. Equilibrium mode runs every maintenance audit in report-only form and reports whether any skill still has a pending change. The vault is converged only when all of them propose nothing. It also detects oscillation via a snapshot in _insights.md, alongside the existing GRAPH_SNAPSHOT: a skill whose pending count keeps returning after being driven to zero is being reversed by another skill, and the vault can never settle. Reported, never auto-fixed — resolving it is a decision about whose objective wins. Pending moves are attributed back to their manifest source, so a source that repeatedly produces pages needing repair surfaces as a suggestion to review its source_quality. Report-only; the owner still decides through the existing trust flow. Skill-only, no Python: lint already emits JSON counts and the other three audits are LLM passes. * docs: document manifest write safety, lifecycle checks, and equilibrium Covers the advisory lock and atomic manifest write (including the parallel-run contract), the illegal_lifecycle_transitions check and why draft -> verified is excluded, and equilibrium mode in the architecture feature list. |
||
|
Mike Tsai
|
6831c5f6e9 |
docs(configuration): document code-understanding settings resolution order
CODE_UNDERSTANDING_BACKEND / CODE_UNDERSTANDING_CODEGRAPH_BIN resolve like OBSIDIAN_VAULT_PATH: env (empty = unset) > nearest project .env > global config > default. Also note the --backend flag precedence in the CLI reference. |
||
|
Arnav
|
5cab5c16c4 |
feat(graph): make the vault graph queryable , and stop routing paths through index.md (#175)
* chore(sync): ignore the regenerable graph cache
New vaults get `.graph-cache.json` in their .gitignore — it is derived data
that churns on every graph change and should never enter history.
An existing .gitignore is the user's file and is left byte-identical; the
command prints a tip instead of editing it.
* feat(graph): bridges, cohesion, paths, neighbourhoods and a betweenness cache
Brings the graph analyser up to the algorithm family graphify runs on code
graphs, in pure Python (no networkx/numpy — the package stays dependency-free):
- `bridges` — exact Brandes betweenness centrality. The pages that sit on the
most shortest paths; removing one fragments the vault.
- `cohesion` per community — intra-cluster edge density; < 0.15 with 5+ pages
marks a fragmented cluster.
- `--path A B` — shortest wikilink chain between two pages.
- `--around PAGE --depth N --direction in|out|both` — N-hop neighbourhood;
`--direction in` is the blast radius if the page were renamed or removed.
- `--diff-against` / `--snapshot` — compare against a previous `_insights.md`
snapshot (added/removed pages and links, newly connected, lost incoming).
- `suggested_questions` — derived from bridges, sink hubs, isolates and
low-cohesion clusters.
Two correctness fixes fall out of this:
- Vault bookkeeping files (`index`, `log`, `hot`, `_insights` at the vault
root, plus `_meta/` and `_readouts/`) are no longer graph nodes. They link
to nearly every page, so they dominated every degree and centrality ranking.
A nested `concepts/index.md` is still an ordinary page.
- Pages are now iterated in sorted order rather than filesystem order, so
results are reproducible across machines. That exposed a latent flaw in the
label-propagation fallback: a sequential sweep cascades one label along
chains and collapsed a barbell graph into a single community. Nodes are now
visited in a seeded shuffle — deterministic, and clusters survive.
Betweenness is O(V*E) and the only expensive step in the module (~43s on a
5,000-page vault, versus 0.13s to parse it and 0.03s for communities), so it
is memoised in `.graph-cache.json`. The cache key is a hash of the graph
topology rather than file mtimes, which makes a stale hit impossible by
construction: retargeting any link changes the key, while editing prose
without touching links keeps the cached result valid. The file is only
written when the computation actually exceeded 0.5s, so small and medium
vaults never accumulate one; it is bounded to 3 keys, written atomically,
and ignored if corrupt.
Sampled betweenness (k pivots) was measured as an alternative and rejected:
25-50x faster but only 60-90% top-10 overlap and wrong ordering even on a
38-page vault, which is not acceptable for a ranked list a human reads.
* fix(query): stop routing paths through index.md; answer structural questions
`graph-query` is the front door `wiki-query` calls first, and it built its
own graph in graphrag.py with its own page-selection rules, its own slug
helper and its own BFS. Those rules did not exclude the vault bookkeeping
files, and `index.md` links to every page — so almost any two pages looked
about two hops apart and the returned "connection" was meaningless:
owasp-agentic-quotes -> prismor-overview
before: owasp-agentic-quotes -> index -> arnav -> prismor-overview
after: owasp-agentic-quotes -> agentic-governance-maturity
-> prismor -> arnav -> prismor-overview
On a 38-page vault, 52 of 630 page pairs (8%) routed through a bookkeeping
file, and 51 of those were shorter than the real semantic path, so BFS
preferred them. The share grows with vault size.
graphrag now uses the shared page selection from graph_analysis and delegates
to its `shortest_path`, so there is one graph and one BFS in the codebase.
On top of that, `graph-query` now answers questions about the shape of the
vault, phrased in plain English, and returns them fully computed in a new
`graph` field (null for ordinary queries, so the response stays backwards
compatible):
impact "what breaks if I delete X" / "what depends on X"
bridges "which pages bridge my clusters" / "what would fragment my vault"
hubs "what's central" / "top hubs" / "my main topics"
clusters "what clusters do I have" / "how is my wiki organised"
surprising "surprising connections" / "unexpected links"
These are computed lazily: only `bridges` needs betweenness, so every other
intent stays as cheap as an ordinary lookup even on a 5,000-page vault
(<= 0.71s measured). A structural question that names an unresolvable page
falls back to ordinary retrieval rather than guessing.
* docs: document the structural intents, query modes and graph cache
- docs/cli.md: the new graph-analyse flags, the graph-query structural
questions, and a section on the cache (what it stores, why the key is the
graph topology, and when the file is written).
- wiki-query: route structural questions to graph-query and report the
`graph` payload directly instead of re-deriving it from page reads.
- wiki-status: insights mode now reads bridges, cohesion, suggested questions
and the graph delta straight from the analyser rather than instructing the
model to compute them by hand.
* docs(readme): add the structural-query benchmark vs a plain agent
Same model, same vault, same plain-English questions — the only difference is
whether obsidian-wiki is installed: 81s -> 19s, 44% -> 83% correct, at
unchanged API cost.
Includes the per-question breakdown and, in a collapsed block, the method and
its limits: ground truth verified against networkx rather than this project's
own code, n=2 per cell on a single 38-page vault, and the one run in the
"with" column that failed outright. The accuracy gap is attributed to its
actual cause — the plain agent routing through index.md, which links to every
page.
Both READMEs updated together per the translation-parity rule.
|
||
|
Arnav
|
77d74e0261 |
feat(config): XDG-style config location, with legacy fallback (#172) (#173)
* feat(config): resolve global config from an XDG-style location (#172) The global config lived at ~/.obsidian-wiki, which isn't XDG-compliant. It now resolves to $XDG_CONFIG_HOME/obsidian-wiki (default ~/.config/obsidian-wiki). Existing installs are unaffected: if ~/.obsidian-wiki is present and the XDG path is not, the legacy directory is used as-is. No migration is required, and nothing moves on upgrade. Once the XDG path exists it wins, so `mv ~/.obsidian-wiki ~/.config/obsidian-wiki` is all a migration takes. Resolution is implemented identically in Python (cli.py), bash (setup.sh, scripts/daily-update.sh, scripts/wiki-notify.sh), and the Config Resolution Protocol that every skill defers to, so vault-scoped state and named `config.<name>` profiles stay in one place. * fix(config): stop setup from wiping user-added config keys `write_config` rewrote the global config with only the three setup-managed keys, so every `obsidian-wiki setup` — including the one a pip upgrade prompts for — silently destroyed anything else in the file: OBSIDIAN_LINK_FORMAT, QMD_WIKI_COLLECTION, WIKI_SKIP_PROJECTS, sync settings, and comments. Setup now owns only OBSIDIAN_VAULT_PATH, OBSIDIAN_WIKI_REPO and OBSIDIAN_WIKI_VERSION, updating those in place and carrying every other line over untouched. Duplicate definitions of a managed key collapse to one. Predates the XDG change (same behavior on main) but is fixed here because this branch already touches config writing, and the XDG work makes re-running setup more likely. Also pins PYTHONPATH in both new test files so they exercise this checkout rather than whatever obsidian_wiki happens to be installed. |
||
|
Mike Tsai
|
fd30720223 |
feat: improve project ingest with optional code-graph-assisted understanding (#167) (#169)
* test: add code-understanding fixtures (fake codegraph CLI, repo builder)
* feat: add code_understanding module with builtin + codegraph providers
Provider abstraction behind code_understand(): builtin uses the regex
ast_extractor plus rg cross-file references (dependency-free fallback);
codegraph shells out to the codegraph CLI with defensive JSON parsing.
Focus-map output is ranked symbols with file:line citations, capped by
max_symbols, and never contains source bodies.
* feat(cli): wire code-understand command and doctor code-understanding checks
New 'obsidian-wiki code-understand --project' subcommand emits the focus
map JSON; doctor --project gains 5 code-understanding capability checks
(builtin, rg, codegraph, codegraph-index, codegraph-fresh). Missing
optional CodeGraph reports warn (exit 0); explicit CODE_UNDERSTANDING_
BACKEND=codegraph with a broken backend fails (exit 1).
* feat(skills): add optional code-understanding focus-map step to wiki-update
Step 3b runs 'obsidian-wiki code-understand --since <last_commit_synced>'
between deciding what to distill and distilling. GUARD-soft-fails like
wiki-ingest Step 1c; mandates file:line citations for architectural
claims and a pruning rule that removes relationships no longer
supported by the codebase. Registers CODE_UNDERSTANDING_BACKEND and
CODE_UNDERSTANDING_CODEGRAPH_BIN in the llm-wiki env reference.
* docs: document CODE_UNDERSTANDING_BACKEND and code-understand command
* docs: add CodeGraph install steps; teach wiki-update to offer install
configuration.md gains a Setup subsection under Code understanding
(npm install -g @colbymchenry/codegraph + verify + CODE_UNDERSTANDING_
CODEGRAPH_BIN fallback). wiki-update Step 3b gains an optional offer:
when codegraph is missing and the user wants the enhanced backend, the
agent may install it (with consent) and re-run the step.
* docs(readme): note code-graph-aware /wiki-update (EN + zh-TW parity)
* docs(readme): clarify zh-TW /wiki-update codegraph note
* fix(codegraph): cap impact queries at 25 seeds on full-project scans
Full scans (no --changed/--since) seed from every tracked file, so an
unbounded impact loop fired one subprocess per symbol — hundreds of
codegraph CLI calls on real repos (observed 120s+ timeout on a 4.3k-node
project). Cap impact at 25 seeds, matching the existing callers/callees
cap (seeds[:5]), keeping full scans bounded while --changed/--since
delta scans stay exhaustive for their small seed sets.
Regression test: test_codegraph_caps_impact_queries_on_full_scan pins the
25-call ceiling (RED: 30 calls -> GREEN: <= 25).
* feat(skills): add standalone code-understand skill
Exposes the code-graph-assisted focus map directly to any coding
agent session, not just as wiki-update Step 3b — so it can be invoked
on demand ("help me understand this codebase", "what breaks if I
change X") independent of a wiki sync.
---------
Co-authored-by: ar9av <ar9avg@gmail.com>
|
||
|
dj-wishket
|
5ef66b6bec | feat(trust): enforce owner authority and correction provenance (#162) | ||
|
ar9av
|
4bf87a8d19 |
feat(sessions): add session brain; restructure README behind docs/
Two strands of work. Session brain — a topic graph over raw agent session history, so you can find the session where something happened instead of remembering its UUID. Reads every Claude transcript plus the pruned sessions that survive only as prompts in history.jsonl, clusters them with local TF-IDF and community detection (no embeddings, no API calls, no new dependencies), and writes a sidecar at ~/.claude/session-brain/ — the vault is never touched. Ships five modules, the sessions-* CLI commands, an interactive graph.html, and the /session-brain and /wiki-sessions skills. This is retrieval, deliberately distinct from wiki-history-ingest, which distils sessions into permanent vault pages. Docs — README.md was 598 lines and put trust-check flags and a 17-row agent matrix in front of anyone evaluating the project. Cut it to a 129- line landing page (problem statement, 60-second install, four blocks of runnable commands, graph screenshot, why-not-a-notes-folder) and move the reference material into docs/: installation, agents, skills, cli, configuration, architecture, session-brain, contributing. README_TW.md mirrors the new structure; docs/ stays English-only for now. The restructure also closed real drift: - Ten skills existed but appeared in no reference — session-brain, session-search, wiki-dedup, wiki-digest, wiki-import, wiki-context-pack, wiki-stage-commit, wiki-switch, vault-skill-factory, and obsidian-layout-adjustment. - SETUP.md listed 12 of 39 skills and still claimed "no scripts, no dependencies" despite the pip package. It is now a signpost into docs/. - Eight config variables were undocumented anywhere, including WIKI_STAGED_WRITES, which wiki-setup prompts for and three skills read. Added to .env.example and docs/configuration.md. - sessions-name and a number of CLI flags were missing from the docs. Two details worth noting. README links are absolute GitHub URLs because pyproject sets readme = "README.md" as the PyPI long_description, and PyPI resolves relative hrefs against pypi.org rather than the repo — docs/ links would have 404'd on the package page. And three doc-drift guards asserted their facts against README.md; since those facts moved rather than disappeared, the assertions were repointed at the page that now owns each one instead of padding content back into the README. Also gitignore /vault/ (a local knowledge vault, not a fixture) and the packed browser extension. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |