Everything else here is local: skills on your machine, a vault on your disk.
This puts a vault behind a URL so an agent elsewhere can search and write it.
One container, one vault, one API key. The vault stays plain markdown on a
mounted volume, so Obsidian still opens it. The container does no LLM work --
search and packing delegate to the existing graphrag/context_pack modules, and
the caller's agent does the thinking.
- obsidian_wiki/server.py: four vault functions, five REST routes and four MCP
tools as transport over them. Auth is one bearer key; the process refuses to
start without WIKI_API_KEY unless WIKI_ALLOW_ANONYMOUS=1. Every path is
resolved inside the vault or refused.
- Dockerfile, docker-compose.yml, .dockerignore, and a ghcr push on v* tags.
- New [server] extra; the core wheel stays dependency-free.
- docs/deployment.md, plus the WIKI_* vars in configuration.md and .env.example.
Requires mcp>=2.0: 2.0 renamed FastMCP to MCPServer, and session_manager only
exists once streamable_http_app() has been called -- a mounted sub-app's own
lifespan is ignored, so it is wired into the FastAPI lifespan by hand.
The OBSIDIAN_RAW_DIR comment still described the pre-#113 behaviour. wiki-ingest
moves a promoted draft into _raw/_archived/ instead of deleting it
(.skills/wiki-ingest/SKILL.md), wiki-status excludes that subdirectory from
pending work, and llm-wiki states the same. .env.example was the last place
still promising deletion, which is the outcome #113 was filed about.
* test: add code-understanding fixtures (fake codegraph CLI, repo builder)
* feat: add code_understanding module with builtin + codegraph providers
Provider abstraction behind code_understand(): builtin uses the regex
ast_extractor plus rg cross-file references (dependency-free fallback);
codegraph shells out to the codegraph CLI with defensive JSON parsing.
Focus-map output is ranked symbols with file:line citations, capped by
max_symbols, and never contains source bodies.
* feat(cli): wire code-understand command and doctor code-understanding checks
New 'obsidian-wiki code-understand --project' subcommand emits the focus
map JSON; doctor --project gains 5 code-understanding capability checks
(builtin, rg, codegraph, codegraph-index, codegraph-fresh). Missing
optional CodeGraph reports warn (exit 0); explicit CODE_UNDERSTANDING_
BACKEND=codegraph with a broken backend fails (exit 1).
* feat(skills): add optional code-understanding focus-map step to wiki-update
Step 3b runs 'obsidian-wiki code-understand --since <last_commit_synced>'
between deciding what to distill and distilling. GUARD-soft-fails like
wiki-ingest Step 1c; mandates file:line citations for architectural
claims and a pruning rule that removes relationships no longer
supported by the codebase. Registers CODE_UNDERSTANDING_BACKEND and
CODE_UNDERSTANDING_CODEGRAPH_BIN in the llm-wiki env reference.
* docs: document CODE_UNDERSTANDING_BACKEND and code-understand command
* docs: add CodeGraph install steps; teach wiki-update to offer install
configuration.md gains a Setup subsection under Code understanding
(npm install -g @colbymchenry/codegraph + verify + CODE_UNDERSTANDING_
CODEGRAPH_BIN fallback). wiki-update Step 3b gains an optional offer:
when codegraph is missing and the user wants the enhanced backend, the
agent may install it (with consent) and re-run the step.
* docs(readme): note code-graph-aware /wiki-update (EN + zh-TW parity)
* docs(readme): clarify zh-TW /wiki-update codegraph note
* fix(codegraph): cap impact queries at 25 seeds on full-project scans
Full scans (no --changed/--since) seed from every tracked file, so an
unbounded impact loop fired one subprocess per symbol — hundreds of
codegraph CLI calls on real repos (observed 120s+ timeout on a 4.3k-node
project). Cap impact at 25 seeds, matching the existing callers/callees
cap (seeds[:5]), keeping full scans bounded while --changed/--since
delta scans stay exhaustive for their small seed sets.
Regression test: test_codegraph_caps_impact_queries_on_full_scan pins the
25-call ceiling (RED: 30 calls -> GREEN: <= 25).
* feat(skills): add standalone code-understand skill
Exposes the code-graph-assisted focus map directly to any coding
agent session, not just as wiki-update Step 3b — so it can be invoked
on demand ("help me understand this codebase", "what breaks if I
change X") independent of a wiki sync.
---------
Co-authored-by: ar9av <ar9avg@gmail.com>
Two strands of work.
Session brain — a topic graph over raw agent session history, so you can
find the session where something happened instead of remembering its
UUID. Reads every Claude transcript plus the pruned sessions that survive
only as prompts in history.jsonl, clusters them with local TF-IDF and
community detection (no embeddings, no API calls, no new dependencies),
and writes a sidecar at ~/.claude/session-brain/ — the vault is never
touched. Ships five modules, the sessions-* CLI commands, an interactive
graph.html, and the /session-brain and /wiki-sessions skills. This is
retrieval, deliberately distinct from wiki-history-ingest, which distils
sessions into permanent vault pages.
Docs — README.md was 598 lines and put trust-check flags and a 17-row
agent matrix in front of anyone evaluating the project. Cut it to a 129-
line landing page (problem statement, 60-second install, four blocks of
runnable commands, graph screenshot, why-not-a-notes-folder) and move the
reference material into docs/: installation, agents, skills, cli,
configuration, architecture, session-brain, contributing. README_TW.md
mirrors the new structure; docs/ stays English-only for now.
The restructure also closed real drift:
- Ten skills existed but appeared in no reference — session-brain,
session-search, wiki-dedup, wiki-digest, wiki-import, wiki-context-pack,
wiki-stage-commit, wiki-switch, vault-skill-factory, and
obsidian-layout-adjustment.
- SETUP.md listed 12 of 39 skills and still claimed "no scripts, no
dependencies" despite the pip package. It is now a signpost into docs/.
- Eight config variables were undocumented anywhere, including
WIKI_STAGED_WRITES, which wiki-setup prompts for and three skills read.
Added to .env.example and docs/configuration.md.
- sessions-name and a number of CLI flags were missing from the docs.
Two details worth noting. README links are absolute GitHub URLs because
pyproject sets readme = "README.md" as the PyPI long_description, and PyPI
resolves relative hrefs against pypi.org rather than the repo — docs/
links would have 404'd on the package page. And three doc-drift guards
asserted their facts against README.md; since those facts moved rather
than disappeared, the assertions were repointed at the page that now owns
each one instead of padding content back into the README.
Also gitignore /vault/ (a local knowledge vault, not a fixture) and the
packed browser extension.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
QMD_WIKI_COLLECTION pointed at OBSIDIAN_VAULT_PATH, which contains
_raw/, so a literal reading of the setup docs merges the wiki and
papers collections wiki-query depends on keeping disjoint. Document
the ignore-based exclusion in .env.example (and fix the nonexistent
`qmd index` command name), have wiki-setup verify/create that
exclusion when QMD_WIKI_COLLECTION is configured, and have wiki-query
defensively drop any _raw/ path surfaced from the wiki collection so
a misconfigured collection degrades instead of citing stale drafts
as compiled knowledge.
Fixes#157
setup.sh had an interactive git-sync flow (git init, .gitignore, remote,
a generated sync.sh, optional cron) that `obsidian-wiki setup` — the entry
point pip/uv installs actually run — never had, so those installs silently
skipped it and .env.example documented a var nothing ever read.
Move the git plumbing into obsidian_wiki/sync.py (configure_sync/run_sync/
get_remote) and expose it as `obsidian-wiki sync-setup <url>` and
`obsidian-wiki sync`. `obsidian-wiki setup` now offers the same interactive
prompt setup.sh always had (or accepts --remote non-interactively), and
setup.sh's own prompt delegates into the same module via the CLI instead of
re-implementing it. Sync status is read live from the vault's own `git
remote` (via `obsidian-wiki info`) rather than a config var that could drift.
wiki-ingest previously walked repo folders with a hardcoded SKIP_DIRS
list, so custom-ignored paths (anything not literally node_modules/
.venv/dist/...) leaked into the ingest plan. batch-plan now detects a
`.git` folder and enumerates files via `git ls-files`, so a repo's own
.gitignore decides what's skipped. Docs updated to spell out the
"clone locally, add to OBSIDIAN_SOURCES_DIR" workflow for public/private
repos on any host.
Implements the proposal from issue #96. Turns a cluster of mature, curated
Obsidian wiki pages into a portable Agent Skill (SKILL.md + references/),
the inverse of wiki-capture.
- .skills/vault-skill-factory/SKILL.md: new skill with hard guardrails
(never writes to .skills/, never auto-installs), QMD-based cluster
discovery, maturity filtering, and provenance manifest.
- .env.example: SKILL_FACTORY_OUTPUT_DIR and SKILL_FACTORY_MATURITY keys.
- AGENTS.md: routing row for /vault-skill-factory and related phrasings.
Closes#96
* feat(wiki-ingest): optional PageIndex long-PDF preprocessing
Implements the proposal from issue #97. When PAGEINDEX_REPO is set and a
PDF has >= PAGEINDEX_MIN_PAGES pages, wiki-ingest builds a structure-aware
TOC tree (section titles + summaries + page ranges) before reading, so only
relevant sections are fetched — yielding page-cited provenance at a fraction
of context cost. Falls back to direct page reads if unset or on error.
- wiki-ingest/SKILL.md: gated step-1 branch before the Academic papers section
- wiki-ingest/references/pageindex.md: full recipe (run command, JSON shape,
step-by-step reasoning approach, caching notes)
- .env.example: PAGEINDEX_REPO, PAGEINDEX_MODEL, PAGEINDEX_MIN_PAGES,
PAGEINDEX_WORKSPACE keys
Fully additive and backward-compatible — unset PAGEINDEX_REPO = no change.
Closes#97
* feat(ingest): add extract-jsonl.py pre-processor for 50-200x JSONL compression
Raw Claude Code JSONL files are 80-90% noise (tool_use, thinking, progress,
file-history-snapshot entries). This adds a stdlib pre-extraction script that
strips all noise and writes compact signal-only JSON, enabling the ingest skill
to process 5-10x more conversations per run within the same token budget.
Measured on real sessions:
- 559 KB JSONL → 8.6 KB extracted (65x)
- 12 MB JSONL → 64 KB extracted (188x)
scripts/extract-jsonl.py:
- Scans ~/.claude/projects/*/*.jsonl, extracts user + assistant text turns only
- Writes ~/.claude/extracted/<project>/<session-id>.json (compact JSON)
- --since DATE for incremental runs (only files modified after the date)
- --skip A,B to exclude projects (also reads WIKI_SKIP_PROJECTS env var)
- --dry-run / --verbose for inspection
- Own extract-manifest at ~/.claude/extracted/.extract-manifest.json tracks
which source files have been extracted, preventing redundant re-extraction
- Pure stdlib, no dependencies; follows the same conventions as manifest.py
.skills/claude-history-ingest/SKILL.md:
- New "Pre-extraction" section with usage examples and the extracted format
- Step 3 updated: check ~/.claude/extracted/<proj>/<uuid>.json first,
fall back to raw JSONL if not present
- Sampling heuristic fixed: projects with memory files now also ingest new
conversations not yet in the manifest (previously all conversations were
silently skipped once any memory file existed)
Source keys could be stored as both ~-relative and absolute paths, so the
same file got tracked twice and re-ingested in append mode (5 such collisions
found in a real 137-source vault). Also no supported way to exclude projects
from history ingest, and manifest updates were ad-hoc inline Python.
- scripts/manifest.py: stdlib helper. `normalize` rewrites source keys to
canonical absolute paths and merges collisions (keeps newest ingested_at,
unions pages lists); `delta` lists new/modified sources honoring
WIKI_SKIP_PROJECTS + --skip.
- llm-wiki/SKILL.md: canonical-source-key rule + provenance recording note;
WIKI_SKIP_PROJECTS in env table.
- wiki-status/SKILL.md: schema example now uses absolute paths consistently
(was mixing ~ and absolute); canonical-key callout.
- claude-history-ingest/SKILL.md: Project Scoping step, canonical-path compare
note (with helper usage), conversation sampling heuristic, desktop-session
pre-check, hot.md frontmatter-timestamp reminder.
- .env.example: WIKI_SKIP_PROJECTS.
Closes#86, #87, #88
Adds an optional GitHub sync step to setup.sh that mirrors the common
`brainsync` alias pattern but reads vault path from ~/.obsidian-wiki/config.
- Prompts during setup: repo URL, shell alias, optional hourly cron
- git-inits vault if needed, writes .gitignore for Obsidian workspace files
- Writes ~/.obsidian-wiki/sync.sh (idempotent, skips if nothing staged)
- Adds wiki-sync alias to .zshrc/.bashrc on confirmation
- Summary line shows wiki-sync if sync was configured
- README: new "Syncing your vault to GitHub" section with manual setup notes
- .env.example: documents VAULT_GITHUB_REMOTE config key
Thanks for the well-structured PR — mirrors the existing `claude-history-ingest` / `codex-history-ingest` pattern cleanly and the Pi session-format details (tree JSONL with id/parentId, message roles, compaction/branch_summary) line up with pi.dev.
Reviewed:
- New `pi-history-ingest` skill, routing across `wiki-history-ingest`, `wiki-agent` (`/wiki-pi`), and `memory-bridge`
- `setup.sh`, `.env.example`, `AGENTS.md`, `README.md`, `SETUP.md`, and all four IDE rule files updated consistently
- 38 committed symlinks under `.pi/skills/` + 5 cross-agent symlinks
One small follow-up: the SKILL.md cites `references/pi-data-format.md` but the file isn't shipped (the claude/codex skills both ship that companion reference). I'll add it in a follow-up so the family stays consistent.
Adds a new `OBSIDIAN_LINK_FORMAT` config variable (default: `wikilink`)
that lets users opt into standard relative Markdown links instead of
Obsidian `[[wikilinks]]` for all newly generated or updated pages.
Closes#26
- .env.example: document QMD_WIKI_COLLECTION, QMD_PAPERS_COLLECTION, and
OBSIDIAN_RAW_DIR with install instructions and fallback notes
- wiki-ingest: add Raw Mode for promoting _raw/ drafts to wiki pages;
add no-QMD fallback note pointing to grep path and .env.example
- wiki-query: add no-QMD fallback note in Step 2b directing to grep path
- wiki-setup: create _raw/ on vault init; ask about QMD during setup
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Agent-readable skills for building and maintaining an AI-powered Obsidian
knowledge base using Karpathy's LLM Wiki pattern. No scripts, no API keys —
the agent is the LLM.
10 skills: llm-wiki, obsidian-setup, obsidian-ingest, data-ingest,
claude-history-ingest, wiki-status, wiki-rebuild, obsidian-query,
obsidian-lint, skill-creator.
Features: delta tracking via manifest, project-based organization,
archive/rebuild/restore, any text source ingestion.
Arnav