Commit Graph

3 Commits

Author SHA1 Message Date
Auree Aubert 57254032cf Record which skills a session actually loaded (#78)
* Record which skills a session actually loaded

Routing is the thing this bundle turns on, and it has been argued rather
than measured. evals/routing.jsonl now scores 100% on the real-model
tier — a ceiling effect, not a victory: every prompt in it was written by
someone looking at the descriptions they were grading, and several were
added to pin behavior that had just been implemented. A suite that always
passes has stopped discovering anything. Prompts nobody wrote to pass
have to come from real sessions.

- hooks/skill-loads.sh reads a Claude Code transcript and emits a marker
  naming the skills invoked and the recipes/playbooks/guides opened:
  `[cargo-skills: cargo-gtm | docs: build-tam,waterfall]`. The session
  hooks append it to the row's summary, so it is queryable today over
  workspace_management.sessions with no schema change.
- Wired into both session-checkpoint.sh and session-end.sh. Recomputed
  from the whole transcript each time rather than accumulated, so a
  session that dies abruptly still carries it. Emits nothing at all for
  sessions that never touched Cargo.
- evals/README.md documents the three questions the data answers (which
  skills never load, which sessions loaded two before settling, which
  load with no doc read after) and how each turns into an eval case.
- Self-test wired into skills-lint, because a regression here corrupts
  telemetry silently instead of failing loudly.

Two bugs the fixtures passed and real transcripts caught:

- **Prose mentions counted as reads.** The doc regex matched a bare path
  anywhere in the transcript, and a skill's own catalog table names every
  recipe and playbook — so one cargo-gtm load reported all 43 playbooks
  as opened. Now anchored to a `"file_path":` tool input. Fixture 7 is
  that exact case.
- **Dangling prefix** when docs were read without a Skill call, giving
  `[cargo-skills: | docs: …]`. Both halves are now independently optional.

Also: the doc list caps at 12, and says `+N more` rather than looking
complete.

PRIVACY: skill and document names only, all of them already public in
this repository. Never a prompt, file contents, an argument, a record, or
anything a user typed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* Fix Bugbot issues: preserve marker in sync finalize, retain skill load order

1. End hook drops skill marker (high severity): The sync finalize now
   includes the skill marker in the placeholder summary, so it persists
   even when refine() fails, times out, or is skipped. The marker is
   fast (no LLM call, just grep/sed).

2. Skill order lost to sorting (medium severity): Replaced 'sort -u'
   with 'awk !seen[$0]++' to preserve load order. The first skill in
   a multi-skill session is the misroute candidate, so order matters
   for harvest question 2.

Added test case 10 to verify load order preservation.

Co-authored-by: Auree Aubert <aureeaubert@users.noreply.github.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Auree Aubert <aureeaubert@users.noreply.github.com>
2026-08-12 14:33:36 -07:00
Auree Aubert 9b5a78ba46 Stop the SessionEnd hook from spawning an endless summarizer chain (#79)
hooks/session-end.sh refines the session title by piping the transcript
to `claude -p`. That subprocess is a full Claude Code session, so when it
exits it fires SessionEnd, which runs the hook again, which spawns
another summarizer. Nothing broke the chain except a child that happened
to have no transcript.

Reported from the field on v1.17.1: 264 headless sessions against 8 real
ones in a day (~16.5M tokens), each spawn burning ~66K tokens — 46.5K
cache write plus 20K cache read — to produce a 150-token title. The
session log shows the signature: distinct session ids seconds apart, all
refined with an identical title, because each child summarizes the
previous child's transcript.

The fix is a CARGO_SESSION_SUMMARIZER marker exported across the
`claude -p` call. It reaches the child's own hooks through the
environment, and all three lifecycle hooks now exit on it — so a
summarizer child registers no session row, refreshes no CLI, writes no
checkpoint, and spawns no second summarizer.

session-end.sh also keeps a rolling-hour spawn budget (20 by default,
CARGO_SUMMARIZER_MAX) as a mechanism-independent backstop: if the marker
is ever lost to a hook runner that re-execs with a scrubbed environment,
a runaway is capped instead of unbounded. Exhausting the budget costs a
refined title, nothing else — the row is already finalized with the
placeholder.

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 13:47:58 -07:00
Auree Aubert a418d76ee7 Fan the plugin out to Codex and Cursor; bundle session-lifecycle hooks (#50)
* Fan the plugin out to Codex and Cursor; bundle session-lifecycle hooks

- One plugin source, three targets: .agents/plugins/marketplace.json
  (Codex) + .cursor-plugin/marketplace.json, with .codex-plugin/ and
  .cursor-plugin/ plugin manifests version-locked to the router
  (linter-enforced across all three, negative-tested). The approval hook
  is wired per target - PreToolUse (Claude Code), PermissionRequest via
  hooks/codex-hooks.json (Codex), beforeShellExecution via
  hooks/cursor-hooks.json (Cursor) - reusing approve-cli.sh's existing
  per-agent verdict dispatch.
- Plugin-bundled lifecycle hooks (Claude Code): session-start (pinned
  CLI refresh from the plugin's own cli-version + placeholder session
  row; deliberately no skills add - the plugin owns the skills),
  session-checkpoint (Stop; jq-derived title, 45s throttle), and
  session-end (sync finalize + detached claude -p refinement). Each
  defers when the installer's counterpart exists at ~/.claude/hooks/,
  so installer + plugin never double-register a session.
- .gitignore: un-ignore .agents/plugins/ (vendored skills stay ignored).
- README: per-target install steps; router 1.11.0, plugins 1.11.0.

Verified: lint 0/0; scripts behavior-tested with stubbed npm/cargo-ai
(pin read from plugin root, defer guards, checkpoint derivation +
throttle, --finished finalize); live plugin install on Claude Code
2.1.205 shows all four hook events and both extra manifests in the
1.11.0 cache.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Make the plugin the single source of truth; agent-portable docs

- session-start.sh now also refreshes the plugin itself (marketplace
  update + plugin update, detached, effective next session) - the plugin
  channel's replacement for the installer's skills-add refresh, with
  zero duplication by construction.
- Docs made agent-portable: the pinned-install command in cargo/SKILL.md
  and prerequisites.md reads cli-version from the skill directory the
  agent loaded instead of hardcoding ~/.claude/skills/ (which silently
  degraded Codex/Cursor to @latest); manual-path session titles are
  agent-neutral; job 1 says to skip skills add when skills came from a
  plugin; the permission-prompts note covers all three agents' events.
- README Claude Code section describes the refactored installer flow
  (installs the plugin; skills add only as fallback; legacy scaffolding
  migrated away). cargo 1.12.0, plugins 1.12.0.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Single-source the lifecycle scripts: dual-mode plugin/standalone

The three hooks/session-*.sh scripts are now the only copies of the
session lifecycle - the installer's fallback channel downloads them
instead of embedding heredocs. Mode auto-detects from location
(~/.claude/hooks/ = standalone; anywhere else = plugin) with an explicit
plugin|standalone arg override:

- standalone: skills add refresh, pin from ~/.claude/skills/cargo/,
  no plugin self-update, never defers.
- plugin: pin from plugin root, detached plugin self-update, no skills
  add, defers when a standalone copy exists (exactly one lifecycle runs).

Log lines carry the mode. Behavior-tested both modes of all three
scripts with stubbed npm/npx/cargo-ai/claude (mode detection, defer,
no-self-defer, per-mode pin paths, per-mode refresh, mode-aware logs).
cargo 1.13.0, plugins 1.13.0.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(lifecycle): registration-aware defer, wider PATH, checkpoint hygiene

Review feedback on the lifecycle scripts:

- Defer guard now requires the standalone copy to be REGISTERED in
  ~/.claude/settings.json, not merely present - an unregistered leftover
  file no longer suppresses the plugin copy (which would silently skip
  CLI pinning and session logging). Falls back to file existence when jq
  is unavailable.
- session-start's PATH widening for the detached plugin self-update now
  matches session-end's full list (Volta, asdf shims, nvm version dirs,
  /opt/node*) so `claude` resolves under version managers and the
  update isn't silently skipped.
- Checkpoint prompt snippets strip control characters. The reported
  command-substitution injection does not exist - shell assignments and
  quoted expansions never evaluate a value's content, and the upsert
  passes --title/--summary as single argv words (verified empirically:
  $(touch ...) and backticks pass through as literal data) - but control
  chars could mangle logs/rows, so they are removed and the inertness is
  documented in a comment for future reviewers.

Verified: unregistered leftover -> plugin runs; registered standalone ->
plugin defers; clean env -> runs; injection payload inert with ESC
sequences stripped from the stored title.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(lifecycle): run when standalone registration is unverifiable

Review follow-up: the defer guard's no-jq / no-settings.json fallback
returned "owned" (defer), which silently skipped CLI pinning and session
logging whenever registration couldn't be checked - exactly the failure
the registration-awareness was meant to prevent. Flipped: when jq or
settings.json is unavailable, default to NOT owned so the plugin copy
runs. Safe direction - the session upsert is idempotent, so a duplicate
with an active standalone is harmless, whereas a wrong defer is a silent
skip. Unit-tested all five branches (registered->defer; not-registered,
no-settings, no-jq, no-file->run).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 18:33:21 -07:00