Commit Graph

6 Commits

Author SHA1 Message Date
Arnav 8b5859d0f8 Vault state management: manifest write safety, lifecycle transitions, and equilibrium mode (#195)
* fix(cache): lock and atomically write .manifest.json

update_source was a plain read-modify-write with no mutual exclusion, so
any two overlapping writers lost an entry silently: the second reader's
snapshot predated the first writer's write. This fires in three real
configurations — batch-plan fan-out dispatching parallel ingest subagents,
the Dockerized server writing a vault a local skill is also using, and a
scheduled daily-update overlapping an interactive session.

Reproduced with 12 concurrent update_source processes against one vault:
6 of 12 entries recorded before, 12 of 12 after.

Writes now take an advisory lockfile and replace the manifest atomically
via a temp file plus os.replace. O_CREAT|O_EXCL rather than fcntl because
this ships on Windows. Hashing stays outside the lock — it can take
seconds on a large source tree and nothing in the manifest depends on it.

The contention test spawns real subprocesses; a single-process test
cannot observe this race.

* feat(trust): flag illegal lifecycle transitions in lint

The lifecycle field is a state machine, but nothing enforced its edges.
Ledger entries now record the lifecycle at review time, giving lint a
baseline to compare the current frontmatter against.

Only edges that are illegal for every possible sequence of unobserved
intermediate states are flagged: falling back to draft (only ingest sets
draft) and exiting archived (terminal). draft -> verified is deliberately
not flagged — ledger snapshots are sparse, so a legitimate intermediate
reviewed may simply never have been recorded, and flagging that pair
would fire on valid history.

Additive: entries predating the lifecycle field carry no baseline and are
skipped, so existing vaults behave identically until their next
trust-record. Owner-extended lifecycle states never match the illegal
pairs, so the check degrades to a no-op rather than firing on states it
does not understand. Warns by default, fails under --strict-trust,
matching how the surrounding trust checks were already staged.

* feat(wiki-status): add vault equilibrium mode

The maintenance skills each optimize one shared vault for a different
objective, so any of them can undo another's work and no single skill
reporting clean means maintenance is done.

Equilibrium mode runs every maintenance audit in report-only form and
reports whether any skill still has a pending change. The vault is
converged only when all of them propose nothing.

It also detects oscillation via a snapshot in _insights.md, alongside the
existing GRAPH_SNAPSHOT: a skill whose pending count keeps returning
after being driven to zero is being reversed by another skill, and the
vault can never settle. Reported, never auto-fixed — resolving it is a
decision about whose objective wins.

Pending moves are attributed back to their manifest source, so a source
that repeatedly produces pages needing repair surfaces as a suggestion to
review its source_quality. Report-only; the owner still decides through
the existing trust flow.

Skill-only, no Python: lint already emits JSON counts and the other three
audits are LLM passes.

* docs: document manifest write safety, lifecycle checks, and equilibrium

Covers the advisory lock and atomic manifest write (including the
parallel-run contract), the illegal_lifecycle_transitions check and why
draft -> verified is excluded, and equilibrium mode in the architecture
feature list.
2026-08-25 17:19:03 -07:00
ar9av bba4b3a74c feat(wiki-export): add Postgres SQL export alongside Neo4j Cypher
Static, idempotent postgres.sql output (wiki_pages/wiki_edges tables,
ON CONFLICT upserts) loadable via psql — no driver dependency needed.
Documents the format in README, README_TW, and docs/.
2026-08-19 01:21:40 -07:00
Mike Tsai ef7bedd558 Agent/document project ingest flow (#182)
* docs: explain project ingest lifecycle

* docs: add code understanding verification command
2026-08-17 22:22:02 -07:00
Arnav 77d74e0261 feat(config): XDG-style config location, with legacy fallback (#172) (#173)
* feat(config): resolve global config from an XDG-style location (#172)

The global config lived at ~/.obsidian-wiki, which isn't XDG-compliant.
It now resolves to $XDG_CONFIG_HOME/obsidian-wiki (default
~/.config/obsidian-wiki).

Existing installs are unaffected: if ~/.obsidian-wiki is present and the
XDG path is not, the legacy directory is used as-is. No migration is
required, and nothing moves on upgrade. Once the XDG path exists it wins,
so `mv ~/.obsidian-wiki ~/.config/obsidian-wiki` is all a migration takes.

Resolution is implemented identically in Python (cli.py), bash (setup.sh,
scripts/daily-update.sh, scripts/wiki-notify.sh), and the Config
Resolution Protocol that every skill defers to, so vault-scoped state and
named `config.<name>` profiles stay in one place.

* fix(config): stop setup from wiping user-added config keys

`write_config` rewrote the global config with only the three setup-managed
keys, so every `obsidian-wiki setup` — including the one a pip upgrade
prompts for — silently destroyed anything else in the file:
OBSIDIAN_LINK_FORMAT, QMD_WIKI_COLLECTION, WIKI_SKIP_PROJECTS, sync
settings, and comments.

Setup now owns only OBSIDIAN_VAULT_PATH, OBSIDIAN_WIKI_REPO and
OBSIDIAN_WIKI_VERSION, updating those in place and carrying every other
line over untouched. Duplicate definitions of a managed key collapse to
one.

Predates the XDG change (same behavior on main) but is fixed here because
this branch already touches config writing, and the XDG work makes
re-running setup more likely.

Also pins PYTHONPATH in both new test files so they exercise this
checkout rather than whatever obsidian_wiki happens to be installed.
2026-08-16 20:40:09 -07:00
ar9av d3e545b012 feat(extensions): fill web forms from the vault; rename to Brain
The brain-capture extension only went one way: browser -> vault. This adds
the return path — read the form on the current page, answer what the vault
supports, and leave the rest blank — and folds both directions into a single
two-tab extension renamed from "Brain Vault Capture" to "Brain".

Retrieval never calls a model. `obsidian-wiki context-pack --json` ranks the
vault deterministically and the model does one narrow job: map retrieved facts
onto fields. One `claude -p` / `codex exec` invocation per fill, billed to the
subscription the user already has. Nothing is ever submitted.

A native messaging host bridges the two, staged into ~/.obsidian-wiki/ rather
than run from the checkout. Chrome's children do not inherit your shell, and
all three consequences bite:

- macOS TCC: ~/Documents is mode 700 with an ACL, and a native host inherits
  Chrome's TCC context, so a checkout there cannot be exec'd at all. Chrome
  reports this as "Native host has exited", identical to a crash.
- PATH: Chrome passes a minimal PATH, hiding claude/codex in ~/.local/bin.
- USER/LOGNAME: Claude Code reads OAuth credentials from the macOS login
  Keychain and needs these to find them, else "Not logged in" despite a valid
  session.

The host bookends every run in its log so those failures stay distinguishable,
re-validates every returned fill against the real form schema (an injected
value cannot reach a field the page never advertised), and forwards the pack's
instruction_policy so vault excerpts stay data rather than instruction.

Form-schema extraction resolves labels via aria-labelledby before aria-label:
Google Forms puts aria-label="Your answer" on every paragraph field while the
real question sits behind aria-labelledby, so checking aria-label first
silently mislabelled exactly the textarea fields. Values are written through
the native setter with input/change dispatched, so React-controlled forms see
a genuine edit.

Docs: docs/browser-extension.md, linked from both READMEs and the docs index.
Also backfills the untranslated tagline from #159 into README_TW.md.
2026-08-14 23:43:19 -07:00
ar9av 4bf87a8d19 feat(sessions): add session brain; restructure README behind docs/
Two strands of work.

Session brain — a topic graph over raw agent session history, so you can
find the session where something happened instead of remembering its
UUID. Reads every Claude transcript plus the pruned sessions that survive
only as prompts in history.jsonl, clusters them with local TF-IDF and
community detection (no embeddings, no API calls, no new dependencies),
and writes a sidecar at ~/.claude/session-brain/ — the vault is never
touched. Ships five modules, the sessions-* CLI commands, an interactive
graph.html, and the /session-brain and /wiki-sessions skills. This is
retrieval, deliberately distinct from wiki-history-ingest, which distils
sessions into permanent vault pages.

Docs — README.md was 598 lines and put trust-check flags and a 17-row
agent matrix in front of anyone evaluating the project. Cut it to a 129-
line landing page (problem statement, 60-second install, four blocks of
runnable commands, graph screenshot, why-not-a-notes-folder) and move the
reference material into docs/: installation, agents, skills, cli,
configuration, architecture, session-brain, contributing. README_TW.md
mirrors the new structure; docs/ stays English-only for now.

The restructure also closed real drift:

- Ten skills existed but appeared in no reference — session-brain,
  session-search, wiki-dedup, wiki-digest, wiki-import, wiki-context-pack,
  wiki-stage-commit, wiki-switch, vault-skill-factory, and
  obsidian-layout-adjustment.
- SETUP.md listed 12 of 39 skills and still claimed "no scripts, no
  dependencies" despite the pip package. It is now a signpost into docs/.
- Eight config variables were undocumented anywhere, including
  WIKI_STAGED_WRITES, which wiki-setup prompts for and three skills read.
  Added to .env.example and docs/configuration.md.
- sessions-name and a number of CLI flags were missing from the docs.

Two details worth noting. README links are absolute GitHub URLs because
pyproject sets readme = "README.md" as the PyPI long_description, and PyPI
resolves relative hrefs against pypi.org rather than the repo — docs/
links would have 404'd on the package page. And three doc-drift guards
asserted their facts against README.md; since those facts moved rather
than disappeared, the assertions were repointed at the page that now owns
each one instead of padding content back into the README.

Also gitignore /vault/ (a local knowledge vault, not a fixture) and the
packed browser extension.

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-30 12:07:48 -07:00