mirror of
https://github.com/daymade/claude-code-skills.git
synced 2026-09-14 16:15:21 +08:00
main
7 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 daymade
|
80115b243c |
fix(read-claude-code-history): prune injected Codex records, dedupe boilerplate chunks, replace the fixed embed sleep (#527)
Two staged changes to daymade-claude-code/read-claude-code-history's hybrid index (scripts/history_index.py), plus a round of independent review and fixes on top. P0: Codex's injected preamble blocks (<goal_context>, <subagent_notification>, <skill>, plus the existing preamble tags) are now skipped at extraction time and swept from already-stored records during update_index (anchored with substr(fts_text,1,n)=prefix, not LIKE, so `_` can't wildcard-match a real record). These blocks measured 27.6M of a 60.7M-token backlog. Schema v3 adds chunks.text_hash (content sha1), backfilled in batches of 5,000 with per-batch commits. A new boilerplate policy keeps one usable copy of any chunk duplicated 3+ times (68% of the chunk backlog, 94% cross-session) and reverses the demotion if the copy count later drops back below the threshold. P1: host-pressure pause is now bounded (EMBED_PAUSE_CEILING_SECONDS=600, stop_reason=host_pressure, exits 0) instead of pausing forever while holding the writer lock; token/ETA progress uses a windowed rate; embed_stop_reason covers running/complete/max_seconds/host_pressure/memory_boundary/ warmup_failed and is written before work starts; vectors_drop_deferred is renamed non_embeddable_chunks; the cross-process writer lock retries for up to 900s instead of failing an entire nightly run on a one-second overlap. Round-1 review (three fresh-context lenses: data-safety, nightly-compat, embed-loop; data-safety found 0 issues) found and this change fixes 3 more minors: index --json leaked a progress line to stdout while building a fresh/--rebuild database (now stderr); embed's deferred vector deletes (orphaned and boilerplate-demoted) were not counted in either return path (now orphan_vectors_dropped/demoted_vectors_dropped, verified against real sqlite-vec rowcount); the reference doc filed host_pressure alongside the stop reasons that raise before writing their marker, when it actually exits 0 (doc now routes alerting through embed_stop_reason and a non-decreasing remaining, not $?). All three were reproduced as failing before the fix and independently re-confirmed on this exact head. Tests: 221 in this suite (194 after P0 alone, 218 after P1 and its own review-fix round). 17/17 registered CI suites, check_marketplace.py (57 plugins), check_shell_syntax.sh (51 scripts) and check_version_progression.py all pass. daymade-claude-code bumped 3.25.0 -> 3.26.0. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
daymade
|
4f7d96d734 |
fix(read-claude-code-history): survive real Codex/Kimi stores, and make the Kimi route executable (#525)
* fix(read-claude-code-history): make Codex/Kimi indexing survive real stores Fixtures passed; the real stores did not. Indexing 8,924 Codex rollouts crashed twice, and the real Kimi store sits nowhere near the documented path. All findings below have regression tests. Codex: - A rollout with no session_meta record at all (truncated or interrupted runs) made the meta lookup return None, which went into the id reader and aborted the whole sweep. The id now falls back to the filename UUID that codex_session_id already knows how to read. - Resuming a session writes a second rollout that keeps the original session_meta.id and appends a fork id to its filename. Two refs with one id violated the sessions primary key, and because the resumed half restarts its ordinals at 1, a bare codex:<ordinal> record key would have silently discarded it. Same-id rollouts now attach as segments of one session with per-file record keys. - Per-file read errors become warnings instead of aborting a sweep over thousands of files. Kimi: - The store is not at ~/.kimi-code when the Kimi desktop app is installed: the app bundles kimi-code inside its Electron runtime and keeps sessions under kimi-desktop/daimon-share/daimon/runtime/ kimi-code/home. Found by following a transcript record's own sourcePath. The layout matches the documented wire contract exactly, so only --kimi-home was needed. - Newer builds drop cwd (and id) from state.json and keep the working directory only in session_index.jsonl. Without reading that map every Kimi session collapsed into one "kimi" project label instead of joining the Claude and Codex sessions for the same repository, which is the entire point of one shared index. - Kimi runs internal agents in the same sessions/ tree as real conversations, separated only by a directory prefix: ctitle- title generation, dvlt- vault-memory maintenance, sklsum- skill summarization. On the real store that was 51 of 83 wires. Left in, a title-generation run outranks the human's own words for the very query that quotes them. Excluded by denylist rather than a conv- allowlist, so a future user-facing prefix shows up as noise to fix rather than as history that silently went missing, and the skip is reported. Verified on the real stores: Codex 8,922 sessions / 398,214 records in 194s, Kimi 32 conversations, zero existing sessions removed, all three providers reachable through recall. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(local-conversation-history): make the Kimi route actually executable The router is Kimi's only entry point, and on this machine it produced a confident empty answer for a store holding 32 real conversations. Three defects, then eight more found by having a fresh agent execute the files rather than read them. Found by executing the skills' own instructions: - A provider whose home was never found reports the same zero as one with no history. Kimi's documented default (~/.kimi-code) does not exist when the desktop client is installed; the client bundles the CLI inside its Electron runtime and keeps sessions there. - The inventory listed internal agent runs as conversations. Title generation, vault maintenance and skill summarization live in the same sessions/ tree and outnumbered real conversations 51 to 32, with titles like "Generate a concise title for the conversation below...". This contradicted the Kimi provider's own documented contract about keeping shared system-prompt boilerplate out. They are now excluded as automated sessions, counted in the reported excluded line, and --include-automated restores the old listing exactly. The prefix denylist moved to the shared core so inventory and index apply one list. - The router had no recall row, so a cross-provider content question routed to an exhaustive scan even where an index covering those providers existed: 234s versus 0.69s on this machine. Found by an independent fresh-context agent, all reproduced before acting: - The home-discovery procedure was unexecutable as written. The bundled CLI sits four levels below the client's user-data dir through segment names the file never gave; the directory it named is the CLI install, not the store (its home/ child is); and the transcript field it cited had no stated location, so the reviewer looked in a file that has no paths at all. - history_index.py status already prints the Kimi home absolutely, in a command the skill already cited. It is now locate step 1. - The executor's task table presented --source kimi and --kimi as parallel Kimi surfaces. --kimi means "also search": the reviewer's run swept 5,565 Claude sessions and put the Kimi verdict on the last of 39,735 lines. - The "readers print a diagnostic line" promise held for the inventory only; the search path prints none in either case. - Worst case: a correct --kimi-home plus the table's bare inventory command returns 0 conversations with no diagnostic at all, because the inventory cwd-scopes by default. This is the zero most likely to be reported as "you have no Kimi conversations". - Search has no automated-session filter while inventory does, and no bundled command reads a located Kimi session. Both are now documented gaps carrying their manual remedies rather than silent surprises. 186 + 168 registered tests green; three skill regression audits passed; review record committed in the maintainer's private knowledge repo. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
daymade
|
51e0b2f22b |
feat(read-claude-code-history): index Codex and Kimi in hybrid recall (#524)
* feat(read-claude-code-history): index Codex and Kimi in hybrid recall The recall index held Claude only: all 14 bound sources were provider claude (the profiles named kimi/glm/step are Claude API profiles, not other CLIs), and history_index.py referenced neither other store. So a cross-provider keyword sweep had to fall back to a full JSONL scan for every provider, including the Claude share the index already covered. Measured on a real query: 234s scan vs 0.69s recall. - sources: add discover_history_sources() and per-provider discovery. Providers are opt-in (--codex/--kimi, with --codex-home/--kimi-home) and never auto-widened; an absent requested home is a reported gap, not a silent zero. Non-Claude display labels carry their provider, since a Claude profile named "kimi" would otherwise collide with the Kimi CLI store. - schema v1 -> v2 adds sessions.provider via an in-place migration. Rebuilding would have discarded 288k valid embeddings for one metadata column; measured 0.051s with all 200,591 records, 5,527 sessions and 288,539 chunks preserved, and it is idempotent. - scope changes now separate widening from narrowing. Adding a provider to an existing index is accepted and announced; narrowing stays refused, because the reconciliation loop prunes out-of-scope sessions and that is exactly when it would destroy covered history. - recall carries provenance: results print their provider, coverage names the providers the index does NOT hold, and --provider for an uncovered store fails loudly instead of returning zero rows that read as absence. - Codex's injected <user_instructions>/<environment_context> preambles and Kimi's config.update boilerplate stay out of the index; a keyword appearing only in a shared preamble would match every session. Kimi subagent wires are keyed per stream so they add turns rather than de-duplicating one another. - fix a silent miss found in testing: --project compared a resolved path against a raw stored cwd, so a macOS /tmp project (resolving to /private/tmp) indexed zero sessions while reporting success. SKILL.md routes a broad sweep with no Session ID, date or project to recall first, then a scan scoped by what it returns. The reference documents Codex as a deliberate opt-in: ~9k rollouts and 40GB on this machine, ~9% of records indexable prose. 9 new tests; 178 + 168 registered tests green. Migration and both provider paths verified end-to-end against real Codex rollouts and a Kimi fixture. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(read-claude-code-history): use a synthetic Codex session id The Codex fixtures reused a UUID copied from a real local rollout. It grants no access and is not itself identifying, but this repository is public and fixtures should not carry identifiers lifted from real sessions. Replace it with an obviously synthetic UUIDv7-shaped value. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
daymade
|
77f83b116a |
fix(read-claude-code-history): search exact sessions globally (#432)
* fix(read-claude-code-history): search exact sessions globally * test(read-claude-code-history): pin selected session workspace |
||
|
daymade
|
44785fa918 |
fix(history): make Claude search candidate-first without losing fidelity (#412)
* fix(history): prefilter candidates before metadata parsing Bounded keyword searches now discover native file candidates first while preserving archive unions, untimed-record accounting, Unicode recall, and exact no-prefilter output. Co-Authored-By: Codex <noreply@openai.com> * fix(history): preserve mixed JSON Unicode matches Match every raw/Unicode-escaped mixture with a linear-size regex, including uppercase hex and decoded ASCII case semantics. Keep grep fail-safe by falling back to full parsing, update help, and add regressions for the independently reproduced false negative. Co-Authored-By: Codex <noreply@openai.com> * fix(history): retain escaped full-fold candidates When any query character is ASCII, keep files containing Unicode escapes so mixed CJK+ASCII searches cannot miss decoded full-fold equivalents such as Kelvin-sign k. Pure CJK queries keep the exact fast path. Co-Authored-By: Codex <noreply@openai.com> * fix(history): join renamed copies by session identity Peek only the first internal sessionId in each physical JSONL before candidate filtering, then retain every active/archive filename for a matching identity. This restores provenance, untimed-record counts, and exact session totals without full-corpus transcript parsing. Co-Authored-By: Codex <noreply@openai.com> * fix(history): bound identity peeks and fold markers Cap session identity discovery at 32 records or 256 KiB, promote unresolved files only after the native scan, and keep their full parsing candidate-scoped. Also derive markers for casefolds containing ASCII substrings such as dotted-I so date-bounded candidate discovery remains complete. Co-Authored-By: Codex <noreply@openai.com> * perf(history): keep pure CJK on exact candidate path Only enable ASCII fold and broad Unicode-escape markers when the query contains an ASCII character. Pure CJK now uses the exact raw/escaped regex and ignores unrelated Kelvin, dotted-I, and ligature text. Co-Authored-By: Codex <noreply@openai.com> * fix(history): select candidates by physical path Prevent same-named files with different internal sessionIds from entering the full parse set, while keeping renamed copies of one identity joined. Scope the release note to Claude candidate-first discovery and the shared Unicode prefilter actually used by both readers. Co-Authored-By: Codex <noreply@openai.com> * fix(history): make identity gaps conservative Apply --exclude-session only after resolving the internal session identity, so renamed aliases cannot disappear early. Treat unreadable or dangling JSONL prefixes as unknown and fall through to the existing tolerant full-scan behavior instead of raising. Co-Authored-By: Codex <noreply@openai.com> * fix(history): resolve exclusion after full identity Treat a provisional prefix ID that appears excluded as unknown, promote it through the native candidate path, and apply exclusion only after the full scanner resolves the final internal session identity. Co-Authored-By: Codex <noreply@openai.com> * fix(history): expand candidates from final identities Full-scan each matched or uncertain candidate once, cache its summary for merge, and rebuild copy unions and totals from the final internal sessionId. Prefix-to-final identity changes now preserve renamed provenance, untimed counts, exclusions, and single-pass candidate parsing. Co-Authored-By: Codex <noreply@openai.com> * fix(history): derive final identity from EOF Read at most 32 complete records and 64 KiB backwards from EOF, so any discovered sessionId is exactly the full scanner's final identity. Unknown suffixes remain conservative candidates; final-ID totals, exclusions, copy unions, and cached single-pass summaries stay exact. Co-Authored-By: Codex <noreply@openai.com> --------- Co-authored-by: Codex <noreply@openai.com> |
||
|
daymade
|
6f21bb0489 |
fix(doc-to-markdown): PDF 后处理三件套 + heavy 退化大声警告（v1.12.0）+ docs(skill-creator): needle proves line survival (v1.32.1) (#409)
* |
||
|
daymade
|
3f7aaa858f |
feat(history): split read and continue skills by provider (#357)
* feat(history): split read and continue skills by provider * fix(history): fail closed on incomplete session evidence * fix(history): retain every textual session state * fix(history): require record-level Claude identity |