`_staging/` promotion existed only as agent instructions in
.skills/wiki-stage-commit/SKILL.md — `grep -rn "_staging" obsidian_wiki/`
returned nothing. That was fine while an agent was the only actor, but it
meant a UI or a CLI could not promote a staged page without writing a second
copy of the rules, which is exactly what #208 was trying to avoid.
obsidian_wiki/staging.py now owns the parts with one correct answer:
list_staged(vault) inventory, classified new / update / patch, with both
sides' content revisions pinned
promote(...) atomic os.replace to the live path
discard(...) move back to _raw/rejected-…, never clobbering an
earlier rejection of the same page
Promotion is a rename, so the page that lands is byte-for-byte the one that
was reviewed: arbitrary frontmatter, prose and wikilinks survive untouched.
Judgment stays with the skill. `.patch.md` files are listed but refused by
promote() — merging a human-readable diff into a page whose surrounding text
may have moved is not mechanical.
The revision check is the reason this exists. An agent can write to _staging/
or to the live page while a human is mid-review, so callers pass the hashes
they reviewed and get StagingConflict instead of a silent overwrite.
pin_live_absent covers the case a bare None cannot express: "there was no
live page when I looked."
Also:
- `obsidian-wiki staging list|promote|discard`, exit 9 on conflict so a
caller can tell "re-read and ask again" from "bad input"
- server._resolve and the /v1/staging listing now use the shared module
instead of their own copies
- SKILL.md calls the commands rather than restating the steps
Closes#212.
main() bound 0.0.0.0 unconditionally. That was defensible while the server
was an API for remote agents; now that /ui serves a browser console over the
whole vault, a bare `python -m obsidian_wiki.server` on a box with a public
address exposes it to the internet.
Default to 127.0.0.1 and read WIKI_HOST to override. A container has to
listen on every interface, so the Dockerfile now sets WIKI_HOST=0.0.0.0
explicitly — `docker run -p 8080:8080` behaves exactly as before.
Closes the loopback-by-default point raised in #208.
`graph_analysis` identifies a page by its slugged filename stem, so two pages
whose stems slug alike are one node — and one entry in the graphrag index.
On a vault holding concepts/vector-search.md and entities/vector-search.md,
build_index keeps only the entities page; the concepts page is absent from
the index that memory_search and graph-query read.
duplicate_titles does not cover this: it keys on the title: value, and stems
can collide while titles differ.
Add duplicate_stems, built over graph_analysis.iter_pages with
graph_analysis._page_slug so the report keys as the merge does. It reads
pages rather than links, so a collision nothing references is still reported.
Severity warn, beside duplicate_titles.
Borrowing the graph's page selection means a collision inside _bootstrap/ is
now visible to this check, where lint's own SKIP_DIRS excludes that
directory. The existing bootstrap exclusion test still passes; the tradeoff
is called out in the pull request rather than settled here.
* fix(cache): lock and atomically write .manifest.json
update_source was a plain read-modify-write with no mutual exclusion, so
any two overlapping writers lost an entry silently: the second reader's
snapshot predated the first writer's write. This fires in three real
configurations — batch-plan fan-out dispatching parallel ingest subagents,
the Dockerized server writing a vault a local skill is also using, and a
scheduled daily-update overlapping an interactive session.
Reproduced with 12 concurrent update_source processes against one vault:
6 of 12 entries recorded before, 12 of 12 after.
Writes now take an advisory lockfile and replace the manifest atomically
via a temp file plus os.replace. O_CREAT|O_EXCL rather than fcntl because
this ships on Windows. Hashing stays outside the lock — it can take
seconds on a large source tree and nothing in the manifest depends on it.
The contention test spawns real subprocesses; a single-process test
cannot observe this race.
* feat(trust): flag illegal lifecycle transitions in lint
The lifecycle field is a state machine, but nothing enforced its edges.
Ledger entries now record the lifecycle at review time, giving lint a
baseline to compare the current frontmatter against.
Only edges that are illegal for every possible sequence of unobserved
intermediate states are flagged: falling back to draft (only ingest sets
draft) and exiting archived (terminal). draft -> verified is deliberately
not flagged — ledger snapshots are sparse, so a legitimate intermediate
reviewed may simply never have been recorded, and flagging that pair
would fire on valid history.
Additive: entries predating the lifecycle field carry no baseline and are
skipped, so existing vaults behave identically until their next
trust-record. Owner-extended lifecycle states never match the illegal
pairs, so the check degrades to a no-op rather than firing on states it
does not understand. Warns by default, fails under --strict-trust,
matching how the surrounding trust checks were already staged.
* feat(wiki-status): add vault equilibrium mode
The maintenance skills each optimize one shared vault for a different
objective, so any of them can undo another's work and no single skill
reporting clean means maintenance is done.
Equilibrium mode runs every maintenance audit in report-only form and
reports whether any skill still has a pending change. The vault is
converged only when all of them propose nothing.
It also detects oscillation via a snapshot in _insights.md, alongside the
existing GRAPH_SNAPSHOT: a skill whose pending count keeps returning
after being driven to zero is being reversed by another skill, and the
vault can never settle. Reported, never auto-fixed — resolving it is a
decision about whose objective wins.
Pending moves are attributed back to their manifest source, so a source
that repeatedly produces pages needing repair surfaces as a suggestion to
review its source_quality. Report-only; the owner still decides through
the existing trust flow.
Skill-only, no Python: lint already emits JSON counts and the other three
audits are LLM passes.
* docs: document manifest write safety, lifecycle checks, and equilibrium
Covers the advisory lock and atomic manifest write (including the
parallel-run contract), the illegal_lifecycle_transitions check and why
draft -> verified is excluded, and equilibrium mode in the architecture
feature list.
CODE_UNDERSTANDING_BACKEND / CODE_UNDERSTANDING_CODEGRAPH_BIN resolve like
OBSIDIAN_VAULT_PATH: env (empty = unset) > nearest project .env > global
config > default. Also note the --backend flag precedence in the CLI
reference.
Everything else here is local: skills on your machine, a vault on your disk.
This puts a vault behind a URL so an agent elsewhere can search and write it.
One container, one vault, one API key. The vault stays plain markdown on a
mounted volume, so Obsidian still opens it. The container does no LLM work --
search and packing delegate to the existing graphrag/context_pack modules, and
the caller's agent does the thinking.
- obsidian_wiki/server.py: four vault functions, five REST routes and four MCP
tools as transport over them. Auth is one bearer key; the process refuses to
start without WIKI_API_KEY unless WIKI_ALLOW_ANONYMOUS=1. Every path is
resolved inside the vault or refused.
- Dockerfile, docker-compose.yml, .dockerignore, and a ghcr push on v* tags.
- New [server] extra; the core wheel stays dependency-free.
- docs/deployment.md, plus the WIKI_* vars in configuration.md and .env.example.
Requires mcp>=2.0: 2.0 renamed FastMCP to MCPServer, and session_manager only
exists once streamable_http_app() has been called -- a mounted sub-app's own
lifespan is ignored, so it is wired into the FastAPI lifespan by hand.
Static, idempotent postgres.sql output (wiki_pages/wiki_edges tables,
ON CONFLICT upserts) loadable via psql — no driver dependency needed.
Documents the format in README, README_TW, and docs/.
* chore(sync): ignore the regenerable graph cache
New vaults get `.graph-cache.json` in their .gitignore — it is derived data
that churns on every graph change and should never enter history.
An existing .gitignore is the user's file and is left byte-identical; the
command prints a tip instead of editing it.
* feat(graph): bridges, cohesion, paths, neighbourhoods and a betweenness cache
Brings the graph analyser up to the algorithm family graphify runs on code
graphs, in pure Python (no networkx/numpy — the package stays dependency-free):
- `bridges` — exact Brandes betweenness centrality. The pages that sit on the
most shortest paths; removing one fragments the vault.
- `cohesion` per community — intra-cluster edge density; < 0.15 with 5+ pages
marks a fragmented cluster.
- `--path A B` — shortest wikilink chain between two pages.
- `--around PAGE --depth N --direction in|out|both` — N-hop neighbourhood;
`--direction in` is the blast radius if the page were renamed or removed.
- `--diff-against` / `--snapshot` — compare against a previous `_insights.md`
snapshot (added/removed pages and links, newly connected, lost incoming).
- `suggested_questions` — derived from bridges, sink hubs, isolates and
low-cohesion clusters.
Two correctness fixes fall out of this:
- Vault bookkeeping files (`index`, `log`, `hot`, `_insights` at the vault
root, plus `_meta/` and `_readouts/`) are no longer graph nodes. They link
to nearly every page, so they dominated every degree and centrality ranking.
A nested `concepts/index.md` is still an ordinary page.
- Pages are now iterated in sorted order rather than filesystem order, so
results are reproducible across machines. That exposed a latent flaw in the
label-propagation fallback: a sequential sweep cascades one label along
chains and collapsed a barbell graph into a single community. Nodes are now
visited in a seeded shuffle — deterministic, and clusters survive.
Betweenness is O(V*E) and the only expensive step in the module (~43s on a
5,000-page vault, versus 0.13s to parse it and 0.03s for communities), so it
is memoised in `.graph-cache.json`. The cache key is a hash of the graph
topology rather than file mtimes, which makes a stale hit impossible by
construction: retargeting any link changes the key, while editing prose
without touching links keeps the cached result valid. The file is only
written when the computation actually exceeded 0.5s, so small and medium
vaults never accumulate one; it is bounded to 3 keys, written atomically,
and ignored if corrupt.
Sampled betweenness (k pivots) was measured as an alternative and rejected:
25-50x faster but only 60-90% top-10 overlap and wrong ordering even on a
38-page vault, which is not acceptable for a ranked list a human reads.
* fix(query): stop routing paths through index.md; answer structural questions
`graph-query` is the front door `wiki-query` calls first, and it built its
own graph in graphrag.py with its own page-selection rules, its own slug
helper and its own BFS. Those rules did not exclude the vault bookkeeping
files, and `index.md` links to every page — so almost any two pages looked
about two hops apart and the returned "connection" was meaningless:
owasp-agentic-quotes -> prismor-overview
before: owasp-agentic-quotes -> index -> arnav -> prismor-overview
after: owasp-agentic-quotes -> agentic-governance-maturity
-> prismor -> arnav -> prismor-overview
On a 38-page vault, 52 of 630 page pairs (8%) routed through a bookkeeping
file, and 51 of those were shorter than the real semantic path, so BFS
preferred them. The share grows with vault size.
graphrag now uses the shared page selection from graph_analysis and delegates
to its `shortest_path`, so there is one graph and one BFS in the codebase.
On top of that, `graph-query` now answers questions about the shape of the
vault, phrased in plain English, and returns them fully computed in a new
`graph` field (null for ordinary queries, so the response stays backwards
compatible):
impact "what breaks if I delete X" / "what depends on X"
bridges "which pages bridge my clusters" / "what would fragment my vault"
hubs "what's central" / "top hubs" / "my main topics"
clusters "what clusters do I have" / "how is my wiki organised"
surprising "surprising connections" / "unexpected links"
These are computed lazily: only `bridges` needs betweenness, so every other
intent stays as cheap as an ordinary lookup even on a 5,000-page vault
(<= 0.71s measured). A structural question that names an unresolvable page
falls back to ordinary retrieval rather than guessing.
* docs: document the structural intents, query modes and graph cache
- docs/cli.md: the new graph-analyse flags, the graph-query structural
questions, and a section on the cache (what it stores, why the key is the
graph topology, and when the file is written).
- wiki-query: route structural questions to graph-query and report the
`graph` payload directly instead of re-deriving it from page reads.
- wiki-status: insights mode now reads bridges, cohesion, suggested questions
and the graph delta straight from the analyser rather than instructing the
model to compute them by hand.
* docs(readme): add the structural-query benchmark vs a plain agent
Same model, same vault, same plain-English questions — the only difference is
whether obsidian-wiki is installed: 81s -> 19s, 44% -> 83% correct, at
unchanged API cost.
Includes the per-question breakdown and, in a collapsed block, the method and
its limits: ground truth verified against networkx rather than this project's
own code, n=2 per cell on a single 38-page vault, and the one run in the
"with" column that failed outright. The accuracy gap is attributed to its
actual cause — the plain agent routing through index.md, which links to every
page.
Both READMEs updated together per the translation-parity rule.
* feat(config): resolve global config from an XDG-style location (#172)
The global config lived at ~/.obsidian-wiki, which isn't XDG-compliant.
It now resolves to $XDG_CONFIG_HOME/obsidian-wiki (default
~/.config/obsidian-wiki).
Existing installs are unaffected: if ~/.obsidian-wiki is present and the
XDG path is not, the legacy directory is used as-is. No migration is
required, and nothing moves on upgrade. Once the XDG path exists it wins,
so `mv ~/.obsidian-wiki ~/.config/obsidian-wiki` is all a migration takes.
Resolution is implemented identically in Python (cli.py), bash (setup.sh,
scripts/daily-update.sh, scripts/wiki-notify.sh), and the Config
Resolution Protocol that every skill defers to, so vault-scoped state and
named `config.<name>` profiles stay in one place.
* fix(config): stop setup from wiping user-added config keys
`write_config` rewrote the global config with only the three setup-managed
keys, so every `obsidian-wiki setup` — including the one a pip upgrade
prompts for — silently destroyed anything else in the file:
OBSIDIAN_LINK_FORMAT, QMD_WIKI_COLLECTION, WIKI_SKIP_PROJECTS, sync
settings, and comments.
Setup now owns only OBSIDIAN_VAULT_PATH, OBSIDIAN_WIKI_REPO and
OBSIDIAN_WIKI_VERSION, updating those in place and carrying every other
line over untouched. Duplicate definitions of a managed key collapse to
one.
Predates the XDG change (same behavior on main) but is fixed here because
this branch already touches config writing, and the XDG work makes
re-running setup more likely.
Also pins PYTHONPATH in both new test files so they exercise this
checkout rather than whatever obsidian_wiki happens to be installed.
A host that answers `vaults` with an error is a staged copy older than the
popup, which looked exactly like "you only have one vault" — the picker
silently fell back to its single default option. A missing host stays silent,
since pressing Fill already explains that one.
Also document that a second vault needs a ~/.obsidian-wiki/config.<name>
profile before it can appear in the list.
The fill flow always used whatever ~/.obsidian-wiki/config resolved to, so
answering a work questionnaire from the work vault meant re-pointing the
wiki-switch symlink for everything else too. This is the extension's
equivalent of the @name override.
The host gains a `vaults` request that lists the active-config default plus
every ~/.obsidian-wiki/config.<name> profile with its resolved path and an
existence flag, and `fill` accepts a vault name. Named resolution never falls
back to the default: silently answering from the wrong vault is exactly what
picking a vault is meant to prevent.
The popup remembers the choice per browser profile, disables profiles whose
path is gone, and resets to the default (loudly) if the remembered one
disappears. If the host is not installed the picker stays on its single
default option, so fill behaves as before.
* test: add code-understanding fixtures (fake codegraph CLI, repo builder)
* feat: add code_understanding module with builtin + codegraph providers
Provider abstraction behind code_understand(): builtin uses the regex
ast_extractor plus rg cross-file references (dependency-free fallback);
codegraph shells out to the codegraph CLI with defensive JSON parsing.
Focus-map output is ranked symbols with file:line citations, capped by
max_symbols, and never contains source bodies.
* feat(cli): wire code-understand command and doctor code-understanding checks
New 'obsidian-wiki code-understand --project' subcommand emits the focus
map JSON; doctor --project gains 5 code-understanding capability checks
(builtin, rg, codegraph, codegraph-index, codegraph-fresh). Missing
optional CodeGraph reports warn (exit 0); explicit CODE_UNDERSTANDING_
BACKEND=codegraph with a broken backend fails (exit 1).
* feat(skills): add optional code-understanding focus-map step to wiki-update
Step 3b runs 'obsidian-wiki code-understand --since <last_commit_synced>'
between deciding what to distill and distilling. GUARD-soft-fails like
wiki-ingest Step 1c; mandates file:line citations for architectural
claims and a pruning rule that removes relationships no longer
supported by the codebase. Registers CODE_UNDERSTANDING_BACKEND and
CODE_UNDERSTANDING_CODEGRAPH_BIN in the llm-wiki env reference.
* docs: document CODE_UNDERSTANDING_BACKEND and code-understand command
* docs: add CodeGraph install steps; teach wiki-update to offer install
configuration.md gains a Setup subsection under Code understanding
(npm install -g @colbymchenry/codegraph + verify + CODE_UNDERSTANDING_
CODEGRAPH_BIN fallback). wiki-update Step 3b gains an optional offer:
when codegraph is missing and the user wants the enhanced backend, the
agent may install it (with consent) and re-run the step.
* docs(readme): note code-graph-aware /wiki-update (EN + zh-TW parity)
* docs(readme): clarify zh-TW /wiki-update codegraph note
* fix(codegraph): cap impact queries at 25 seeds on full-project scans
Full scans (no --changed/--since) seed from every tracked file, so an
unbounded impact loop fired one subprocess per symbol — hundreds of
codegraph CLI calls on real repos (observed 120s+ timeout on a 4.3k-node
project). Cap impact at 25 seeds, matching the existing callers/callees
cap (seeds[:5]), keeping full scans bounded while --changed/--since
delta scans stay exhaustive for their small seed sets.
Regression test: test_codegraph_caps_impact_queries_on_full_scan pins the
25-call ceiling (RED: 30 calls -> GREEN: <= 25).
* feat(skills): add standalone code-understand skill
Exposes the code-graph-assisted focus map directly to any coding
agent session, not just as wiki-update Step 3b — so it can be invoked
on demand ("help me understand this codebase", "what breaks if I
change X") independent of a wiki sync.
---------
Co-authored-by: ar9av <ar9avg@gmail.com>
Loading an unpacked extension needs a directory on disk, so pip users had no
way to reach the feature at all. The extension now ships under
_data/extension/ and `obsidian-wiki info` prints the path to point Chrome at.
install.sh now handles both origins: a checkout stages a copy of the package
(so the host survives the checkout later moving into a TCC-protected folder),
while a wheel install uses the already-importable package and stages nothing.
The private signing key moved out of the repo to ~/.obsidian-wiki/. This is
load-bearing, not tidiness: hatchling's force-include copies straight from the
working tree and honours neither .gitignore nor the target's exclude list, so
every file in a force-included directory ships. Verified empirically — the key
appeared in both wheel and sdist until it was moved out, and exclude patterns
did not stop it. pyproject now says so at the include site.
The brain-capture extension only went one way: browser -> vault. This adds
the return path — read the form on the current page, answer what the vault
supports, and leave the rest blank — and folds both directions into a single
two-tab extension renamed from "Brain Vault Capture" to "Brain".
Retrieval never calls a model. `obsidian-wiki context-pack --json` ranks the
vault deterministically and the model does one narrow job: map retrieved facts
onto fields. One `claude -p` / `codex exec` invocation per fill, billed to the
subscription the user already has. Nothing is ever submitted.
A native messaging host bridges the two, staged into ~/.obsidian-wiki/ rather
than run from the checkout. Chrome's children do not inherit your shell, and
all three consequences bite:
- macOS TCC: ~/Documents is mode 700 with an ACL, and a native host inherits
Chrome's TCC context, so a checkout there cannot be exec'd at all. Chrome
reports this as "Native host has exited", identical to a crash.
- PATH: Chrome passes a minimal PATH, hiding claude/codex in ~/.local/bin.
- USER/LOGNAME: Claude Code reads OAuth credentials from the macOS login
Keychain and needs these to find them, else "Not logged in" despite a valid
session.
The host bookends every run in its log so those failures stay distinguishable,
re-validates every returned fill against the real form schema (an injected
value cannot reach a field the page never advertised), and forwards the pack's
instruction_policy so vault excerpts stay data rather than instruction.
Form-schema extraction resolves labels via aria-labelledby before aria-label:
Google Forms puts aria-label="Your answer" on every paragraph field while the
real question sits behind aria-labelledby, so checking aria-label first
silently mislabelled exactly the textarea fields. Values are written through
the native setter with input/change dispatched, so React-controlled forms see
a genuine edit.
Docs: docs/browser-extension.md, linked from both READMEs and the docs index.
Also backfills the untranslated tagline from #159 into README_TW.md.
Two strands of work.
Session brain — a topic graph over raw agent session history, so you can
find the session where something happened instead of remembering its
UUID. Reads every Claude transcript plus the pruned sessions that survive
only as prompts in history.jsonl, clusters them with local TF-IDF and
community detection (no embeddings, no API calls, no new dependencies),
and writes a sidecar at ~/.claude/session-brain/ — the vault is never
touched. Ships five modules, the sessions-* CLI commands, an interactive
graph.html, and the /session-brain and /wiki-sessions skills. This is
retrieval, deliberately distinct from wiki-history-ingest, which distils
sessions into permanent vault pages.
Docs — README.md was 598 lines and put trust-check flags and a 17-row
agent matrix in front of anyone evaluating the project. Cut it to a 129-
line landing page (problem statement, 60-second install, four blocks of
runnable commands, graph screenshot, why-not-a-notes-folder) and move the
reference material into docs/: installation, agents, skills, cli,
configuration, architecture, session-brain, contributing. README_TW.md
mirrors the new structure; docs/ stays English-only for now.
The restructure also closed real drift:
- Ten skills existed but appeared in no reference — session-brain,
session-search, wiki-dedup, wiki-digest, wiki-import, wiki-context-pack,
wiki-stage-commit, wiki-switch, vault-skill-factory, and
obsidian-layout-adjustment.
- SETUP.md listed 12 of 39 skills and still claimed "no scripts, no
dependencies" despite the pip package. It is now a signpost into docs/.
- Eight config variables were undocumented anywhere, including
WIKI_STAGED_WRITES, which wiki-setup prompts for and three skills read.
Added to .env.example and docs/configuration.md.
- sessions-name and a number of CLI flags were missing from the docs.
Two details worth noting. README links are absolute GitHub URLs because
pyproject sets readme = "README.md" as the PyPI long_description, and PyPI
resolves relative hrefs against pypi.org rather than the repo — docs/
links would have 404'd on the package page. And three doc-drift guards
asserted their facts against README.md; since those facts moved rather
than disappeared, the assertions were repointed at the page that now owns
each one instead of padding content back into the README.
Also gitignore /vault/ (a local knowledge vault, not a fixture) and the
packed browser extension.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
ar9av