Closes a gap from issue #113: _raw/ deletion after promotion conflicted
with the "immutable raw layer" framing in llm-wiki, since some drafts
have no other copy. wiki-ingest now moves promoted files into
_raw/_archived/ instead of rm'ing them; wiki-status, wiki-setup, and
llm-wiki docs updated to match.
Adds a ## Strongest Objection section between Tensions and Open Questions,
requiring each synthesis page to name the best case against its own cross-cutting
insight plus a testable search query. Adds matching quality checklist item.
Closes#105
A naive last_updated (no tz offset) passes fromisoformat cleanly but throws
TypeError at mtime > last_updated since mtime is always UTC-aware. Guard added
right after parse: if tzinfo is None, replace with UTC. No change for already-aware timestamps.
Closes#101
Add lossless OKF v0.1 round-trip to wiki-export and wiki-import. OKF
(Google's knowledge-catalog) is a vendor-neutral spec for knowledge as
markdown + YAML frontmatter — structurally near-identical to this vault.
- wiki-export: new "OKF bundle" output mode (wiki-export/okf/), the
canonical frontmatter mapping (category/summary/sources/updated ⇄
type/description/resource/timestamp), [[wikilink]]→file-relative
markdown link transform, per-folder index.md generation, okf_version.
- wiki-import: source-type detection (graph.json vs OKF dir) + Step 4-OKF
that reconstructs full pages (not stubs) via reverse mapping.
- AGENTS.md: routing triggers + OKF-conformance architecture note.
Native frontmatter keys ride along as OKF extension frontmatter, making
the round-trip lossless — unlike the stub-only graph.json round-trip.
Forward-references (wikilinks to not-yet-written pages) are preserved as
links per OKF §5.3 rather than dropped to plain text.
Verified on st3ve (46-page vault): 46/46 exported, 0 OKF §9 conformance
issues, 25/25 frontmatter lossless, 46/46 body/link-graph lossless.
Implements the proposal from issue #96. Turns a cluster of mature, curated
Obsidian wiki pages into a portable Agent Skill (SKILL.md + references/),
the inverse of wiki-capture.
- .skills/vault-skill-factory/SKILL.md: new skill with hard guardrails
(never writes to .skills/, never auto-installs), QMD-based cluster
discovery, maturity filtering, and provenance manifest.
- .env.example: SKILL_FACTORY_OUTPUT_DIR and SKILL_FACTORY_MATURITY keys.
- AGENTS.md: routing row for /vault-skill-factory and related phrasings.
Closes#96
Implements the proposal from issue #97. When PAGEINDEX_REPO is set and a
PDF has >= PAGEINDEX_MIN_PAGES pages, wiki-ingest builds a structure-aware
TOC tree (section titles + summaries + page ranges) before reading, so only
relevant sections are fetched — yielding page-cited provenance at a fraction
of context cost. Falls back to direct page reads if unset or on error.
- wiki-ingest/SKILL.md: gated step-1 branch before the Academic papers section
- wiki-ingest/references/pageindex.md: full recipe (run command, JSON shape,
step-by-step reasoning approach, caching notes)
- .env.example: PAGEINDEX_REPO, PAGEINDEX_MODEL, PAGEINDEX_MIN_PAGES,
PAGEINDEX_WORKSPACE keys
Fully additive and backward-compatible — unset PAGEINDEX_REPO = no change.
Closes#97
* feat(wiki-ingest): optional PageIndex long-PDF preprocessing
Implements the proposal from issue #97. When PAGEINDEX_REPO is set and a
PDF has >= PAGEINDEX_MIN_PAGES pages, wiki-ingest builds a structure-aware
TOC tree (section titles + summaries + page ranges) before reading, so only
relevant sections are fetched — yielding page-cited provenance at a fraction
of context cost. Falls back to direct page reads if unset or on error.
- wiki-ingest/SKILL.md: gated step-1 branch before the Academic papers section
- wiki-ingest/references/pageindex.md: full recipe (run command, JSON shape,
step-by-step reasoning approach, caching notes)
- .env.example: PAGEINDEX_REPO, PAGEINDEX_MODEL, PAGEINDEX_MIN_PAGES,
PAGEINDEX_WORKSPACE keys
Fully additive and backward-compatible — unset PAGEINDEX_REPO = no change.
Closes#97
* feat(ingest): add extract-jsonl.py pre-processor for 50-200x JSONL compression
Raw Claude Code JSONL files are 80-90% noise (tool_use, thinking, progress,
file-history-snapshot entries). This adds a stdlib pre-extraction script that
strips all noise and writes compact signal-only JSON, enabling the ingest skill
to process 5-10x more conversations per run within the same token budget.
Measured on real sessions:
- 559 KB JSONL → 8.6 KB extracted (65x)
- 12 MB JSONL → 64 KB extracted (188x)
scripts/extract-jsonl.py:
- Scans ~/.claude/projects/*/*.jsonl, extracts user + assistant text turns only
- Writes ~/.claude/extracted/<project>/<session-id>.json (compact JSON)
- --since DATE for incremental runs (only files modified after the date)
- --skip A,B to exclude projects (also reads WIKI_SKIP_PROJECTS env var)
- --dry-run / --verbose for inspection
- Own extract-manifest at ~/.claude/extracted/.extract-manifest.json tracks
which source files have been extracted, preventing redundant re-extraction
- Pure stdlib, no dependencies; follows the same conventions as manifest.py
.skills/claude-history-ingest/SKILL.md:
- New "Pre-extraction" section with usage examples and the extracted format
- Step 3 updated: check ~/.claude/extracted/<proj>/<uuid>.json first,
fall back to raw JSONL if not present
- Sampling heuristic fixed: projects with memory files now also ingest new
conversations not yet in the manifest (previously all conversations were
silently skipped once any memory file existed)
Academic PDFs (arXiv/conference) carry their substance in figures,
equations, and results tables, but the generic page template flattens
everything to a terse Key Ideas bullet list and the PDF reader drops
the text layer's figures. ML/AI/LLM/VLM paper pages end up text-only
and hard to use.
Add an opt-in path, scoped to academic papers landing in references/,
that leaves the terse default untouched for every other source:
- llm-wiki: new Paper Deep-Dive Template (TL;DR callout, Problem,
Method/Architecture with an embedded real figure + a Mermaid
schematic, Key Equations in LaTeX, Results table, Limitations).
- wiki-ingest: an Academic papers sub-step that re-reads figure/equation
pages with vision, embeds the paper's real architecture figure when it
can be extracted (PyMuPDF: get_image_info -> extract_image, or render
the bbox for vector figures) and always draws a Mermaid schematic;
Step 5 branches to the new template.
- ingest-prompts: a Paper Extraction Frame reading checklist.
Mermaid stays the zero-dependency default; real-figure extraction is an
opt-in enhancement. No behavior change for non-paper sources.
Implements Option B from #91 — creates a tmpfile keyed to session_id on
first trigger so subsequent Stop events in the same session are no-ops.
Avoids the repeated nudge loop when cumulative transcript counts stay
above the threshold after a capture has already been requested.
Closes#91
Source keys could be stored as both ~-relative and absolute paths, so the
same file got tracked twice and re-ingested in append mode (5 such collisions
found in a real 137-source vault). Also no supported way to exclude projects
from history ingest, and manifest updates were ad-hoc inline Python.
- scripts/manifest.py: stdlib helper. `normalize` rewrites source keys to
canonical absolute paths and merges collisions (keeps newest ingested_at,
unions pages lists); `delta` lists new/modified sources honoring
WIKI_SKIP_PROJECTS + --skip.
- llm-wiki/SKILL.md: canonical-source-key rule + provenance recording note;
WIKI_SKIP_PROJECTS in env table.
- wiki-status/SKILL.md: schema example now uses absolute paths consistently
(was mixing ~ and absolute); canonical-key callout.
- claude-history-ingest/SKILL.md: Project Scoping step, canonical-path compare
note (with helper usage), conversation sampling heuristic, desktop-session
pre-check, hot.md frontmatter-timestamp reminder.
- .env.example: WIKI_SKIP_PROJECTS.
Closes#86, #87, #88
Continues the skill-surface consolidation (#84). Removes three skills
whose capability fully overlaps a survivor, so users and the router stop
disambiguating near-duplicates. No functionality lost.
- obsidian-wiki-ingest: a self-described "wrapper for wiki-ingest" that
never did anything wiki-ingest doesn't. Deleted outright.
- wiki-quick-chat-capture -> wiki-capture --quick: quick mode folded in
as a documented fast path (gate, _raw/ staging, <60s, no manifest
writes). RAW-FORMAT.md moved to wiki-capture/references/. The
session-end Stop hook now asks for `/wiki-capture --quick`.
- ingest-url -> wiki-ingest: URL ingestion folded in. The full URL
mechanics (project detection, defuddle, dedup, slug, project/misc
frontmatter, affinity scoring, stub handling, INGEST_URL log) moved
verbatim to wiki-ingest/references/url-sources.md, pointed to from a
new "Web URL sources" section.
Repointed every reference: AGENTS.md (+CLAUDE.md symlink), README,
wiki-setup, wiki-query, llm-wiki, RAW-FORMAT.md.
data-ingest was effectively wiki-ingest plus a format-detection
preamble — its own process step said "Follow the wiki-ingest skill's
process for creating/updating pages". The split forced users and the
skill router to disambiguate two overlapping ingest skills (a .md
ChatGPT export fit either one).
Fold it in to keep the skill surface small:
- wiki-ingest now handles structured docs AND raw/unstructured text
(chat exports, Slack/Discord JSON, CSV/JSON, transcripts, logs).
- Added an "Unstructured & conversational sources" section to Step 1
with format detection, chat-export shapes, and "distill substance,
not dialogue" guidance carried over from data-ingest.
- Expanded the description to absorb data-ingest's triggers.
- Removed .skills/data-ingest/ and repointed every reference
(CLAUDE.md/AGENTS.md, README, SETUP, llm-wiki, wiki-status,
wiki-rebuild, wiki-quick-chat-capture, and the mirror rule files
for cursor/windsurf/agent/kiro/copilot).
* feat(capture): auto-capture findings at session end via Stop hook
Ports hivemind's session-end capture pattern into the wiki framework:
- `.claude/hooks/wiki-stop-capture.sh` — fires on Claude Code Stop event,
reads the session transcript JSONL, and if ≥1 file edit or ≥4 shell calls
are detected, feeds back a message prompting /wiki-quick-chat-capture.
The stop_hook_active guard prevents re-entry on the follow-up turn.
- `.claude/settings.json` — wires the hook into the project's Claude settings
so it activates automatically for any Claude Code session in this repo.
- `wiki-quick-chat-capture/SKILL.md` — adds a KEEP/SKIP gate as Step 0.
Makes the skill safe to call automatically: purely conversational sessions
with no concrete findings exit immediately. Manual invocations err toward KEEP.
- `wiki-setup/SKILL.md` — adds an optional step to install the hook globally
into ~/.claude/settings.json so it works across all projects, not just this repo.
* fix(stop-hook): route rewake message to stderr
Claude Code Stop hooks deliver content back to Claude via stderr, not stdout.
The echo was going to stdout so the hook silently exited 2 with no visible effect.
Confirmed: exit=2 with stderr content triggers the /wiki-quick-chat-capture prompt.
wiki-query had no read-only guard — its only sanctioned write was the
Step 6 log.md append, but nothing forbade editing content pages. A query
whose message read like a finding/action request ("save this", "ban X")
could drift into editing wiki pages instead of just answering.
- Add a prominent READ-ONLY guard up top: never create/edit content pages,
index.md, hot.md, _insights.md, or .manifest.json. Findings/action
requests are proposed and routed to the right write skill
(wiki-quick-chat-capture / wiki-capture / wiki-update). Reinforced at Step 6.
- Surface the project's source path for project-scoped queries: resolve
.manifest.json -> projects.<name>.source_cwd (fallback to page source_path)
and include a "Source code:" line naming real files, with an offer to
switch into implementation mode as an explicit next step.
Adds contents:write permission and a gh release create step to the
publish job so that each v* tag push produces a GitHub Release with
the built wheel and sdist attached, alongside the existing PyPI publish.
Adds an optional GitHub sync step to setup.sh that mirrors the common
`brainsync` alias pattern but reads vault path from ~/.obsidian-wiki/config.
- Prompts during setup: repo URL, shell alias, optional hourly cron
- git-inits vault if needed, writes .gitignore for Obsidian workspace files
- Writes ~/.obsidian-wiki/sync.sh (idempotent, skips if nothing staged)
- Adds wiki-sync alias to .zshrc/.bashrc on confirmation
- Summary line shows wiki-sync if sync was configured
- README: new "Syncing your vault to GitHub" section with manual setup notes
- .env.example: documents VAULT_GITHUB_REMOTE config key
Add Step 4b to wiki-query: bounded BFS over the relationships: typed-edge
adjacency to answer multi-hop "how is X connected to Y" / "trace the chain"
/ "what does X depend on transitively" path queries.
- New "Path / multi-hop query" class in the query classifier
- Frontmatter-only adjacency build (single relationships: grep, no body reads)
- Reverse edges traversable so "connected to" is symmetric
- Renders the full chain with edge types; reports no-path-within-N-hops as a graph gap
- Updates trigger description and the llm-wiki relationships: consumer note
Stamps OBSIDIAN_WIKI_VERSION in ~/.obsidian-wiki/config each time
`obsidian-wiki setup` runs. On any subsequent CLI invocation (list,
info, etc.), _check_stale() compares the stamped version against the
running version and warns if they differ, or if ~/.claude/skills/ is
missing bundled skills.
`obsidian-wiki info` now also shows a per-agent skill install count
(N/38) so users can see at a glance whether setup has been run.
Fixes#76.
The combined-entry template was repeating capture_source for both
agent: and source: fields, and referenced undefined tokens (<project>,
<lifecycle_changed>). Use the actual sources: field value from the
_raw/ frontmatter instead, which already encodes project and date.
When promoting _raw/ files in Raw Mode, synthesize the sources: entry
from capture_source + sources frontmatter fields. The _raw/ path is a
staging artifact and must never appear as provenance in promoted pages.
wiki-export: add Project Filter section — '/wiki-export <name>' now filters
nodes by path prefix (projects/<name>/) or tag match, with combined support
alongside the existing visibility filter. Closes#73.
wiki-import: change default conflict mode from skip to merge — existing pages
are updated (frontmatter, wikilinks) while new pages are created as stubs.
skip and overwrite remain available as explicit opt-ins. Closes#74.
Pairs with wiki-export: takes a graph.json snapshot and rebuilds page stubs
in the target vault with correct frontmatter, wikilinks, and typed relationships.
Supports skip/overwrite/merge conflict modes and updates all vault metadata files.
* feat(pypi): package obsidian-wiki as a pip-installable distribution (#71)
Add a PyPI package so users can install without cloning the repo.
- pyproject.toml (hatchling, no runtime deps); force-include .skills/ and
bootstrap files into the wheel under obsidian_wiki/_data/ — repo root stays
the single source of truth. Force-include .skills/ into the sdist too, or
hatchling's VCS walker drops it (the committed agent symlink mirrors shadow
the real .skills/<name> files).
- obsidian_wiki CLI: Python port of setup.sh. `obsidian-wiki setup` writes
~/.obsidian-wiki/config and symlinks all skills into every supported agent
(--copy / --project / --project-only); plus `list` and `info`.
- .github/workflows/publish.yml: build + twine check + skill-bundle assertion,
publish to PyPI on v* tags via Trusted Publishing (OIDC).
- README + SETUP: pip install instructions; gitignore build artifacts.
* build(pypi): derive version from git tag via hatch-vcs; skip-existing on publish
Align the package version with the repo's CalVer release tags (v2026.05.2 ->
2026.5.2) so tagging a release is the version bump and future release tags
auto-publish the matching version.
* feat(skill): add wiki-quick-chat-capture for zero-friction mid-session finding capture
Implements the skill proposed in issue #69. Extracts reusable technical findings
(bugs, gotchas, API quirks) from the current conversation and stages them as
structured _raw/ files in under 60 seconds — no subagents, no QMD, no manifest
writes. Full promotion to wiki pages happens later via /wiki-ingest or /data-ingest.
* refactor(wiki-quick-chat-capture): add agentskills.io frontmatter, RAW-FORMAT reference, OBSIDIAN_RAW_DIR support
- Add compatibility, metadata, and allowed-tools frontmatter fields for cross-agent portability
- Extract body format spec and provenance/confidence calibration table to references/RAW-FORMAT.md
(keeps SKILL.md under 500 lines; details loaded on demand)
- Support OBSIDIAN_RAW_DIR config key so _raw/ can live outside the vault root
- Trim inline format docs from SKILL.md now that they live in the reference file
Incorporates feedback from @BenRoe in PR #70 comment.
When a Hermes user runs under a named profile ($HERMES_HOME points to
~/.hermes/profiles/<name>/), setup.sh was only installing skills to
~/.hermes/skills/ (the default profile), making them invisible to the
active profile session.
Implements Option C from issue #66:
- Always installs to ~/.hermes/skills/ (default profile baseline)
- If $HERMES_HOME is set and differs from the default, also installs
to $HERMES_HOME/skills/ (active profile)
- Scans ~/.hermes/profiles/ and installs to every named profile,
skipping any already handled via $HERMES_HOME to avoid duplicates
Closes#66
Before running `git log <last_commit>..HEAD`, check ancestry with
`git merge-base --is-ancestor`. If the stored SHA is no longer
reachable (branch rebased or force-pushed), warn the user and fall back
to a full scan rather than silently under/over-ingesting.
Tested on st3ve: verified exit 0 for valid ancestor, exit 1 after
simulated rebase and amend.
Closes#64 (partial — addresses the rebase fragility raised in discussion)
Thanks for the well-structured PR — mirrors the existing `claude-history-ingest` / `codex-history-ingest` pattern cleanly and the Pi session-format details (tree JSONL with id/parentId, message roles, compaction/branch_summary) line up with pi.dev.
Reviewed:
- New `pi-history-ingest` skill, routing across `wiki-history-ingest`, `wiki-agent` (`/wiki-pi`), and `memory-bridge`
- `setup.sh`, `.env.example`, `AGENTS.md`, `README.md`, `SETUP.md`, and all four IDE rule files updated consistently
- 38 committed symlinks under `.pi/skills/` + 5 cross-agent symlinks
One small follow-up: the SKILL.md cites `references/pi-data-format.md` but the file isn't shipped (the claude/codex skills both ship that companion reference). I'll add it in a follow-up so the family stays consistent.
Adds a standardised 'QMD Refresh After Vault Writes' section to all 25
write-capable skills. QMD is treated as a best-effort search index:
refresh is skipped when QMD_WIKI_COLLECTION is unset or the CLI is
unavailable, and failures are reported without rolling back vault writes.
Conflict in wiki-lint/SKILL.md resolved by keeping both the --consolidate
mode (from main) and the QMD refresh section (from PR #55).
Co-authored-by: georgelichen <georgelichen@users.noreply.github.com>
When stdin is closed (e.g. CI with < /dev/null), the interactive
read -p prompt returns exit code 1 (EOF). With set -e this silently
killed the script after creating .env. Adding || true lets the read
fail gracefully so setup continues with an empty vault path.
Closes#49. Adds WIKI_STAGED_WRITES config flag that routes all LLM-written
pages to _staging/<category>/ for human review before they land in the live
wiki. New pages go as full page files; updates go as .patch.md files with
structured additions/deletions.
New skill: wiki-stage-commit — interactive promotion flow with per-file
accept/reject, conflict detection (target modified since staging), --all /
--reject-all / --list flags, and consolidation report.
wiki-ingest: checks WIKI_STAGED_WRITES before Step 5 and writes staged files
accordingly; index.md and log.md always updated immediately.
wiki-status: shows "Staged writes pending" in Overview when flag is enabled;
adds staged writes as priority-0 item in "What to Do Next" (above _raw/).
wiki-setup: asks about WIKI_STAGED_WRITES in Step 1; creates _staging/
directory alongside _raw/; verifies _staging/ in Step 6.
CLAUDE.md routing updated for wiki-stage-commit.
Closes#48. Adds a --consolidate flag to wiki-lint that switches from
report-only to act-and-report mode. Six consolidation actions: fix broken
wikilinks (fuzzy match), add missing cross-refs for orphans, correct lifecycle
states (draft→reviewed when age>30d and confidence>0.7; stale callout for
verified pages), demote tier to peripheral for stale zero-link pages, normalize
tag aliases from taxonomy, add contradiction callouts. Always dry-runs first
with structured action list and asks for confirmation before any writes. Writes
a consolidation report to synthesis/consolidation-YYYY-MM-DD.md. All changes
logged to log.md. CLAUDE.md routing updated.
Closes#46. Adds a Step 3b to wiki-status that computes per-tier token
estimates using a 4 chars/token heuristic, an index-only estimate, and a
typical query estimate. Emits a warning when the full-wiki estimate exceeds
WIKI_TOKEN_WARN_THRESHOLD (default 100K). Includes a Token Footprint table
in every standard status output. wiki-setup now prompts for the threshold.
Arnav