Files
getcargohq__cargo-skills/hooks/session-checkpoint.sh
Auree Aubert 57254032cf Record which skills a session actually loaded (#78)
* Record which skills a session actually loaded

Routing is the thing this bundle turns on, and it has been argued rather
than measured. evals/routing.jsonl now scores 100% on the real-model
tier — a ceiling effect, not a victory: every prompt in it was written by
someone looking at the descriptions they were grading, and several were
added to pin behavior that had just been implemented. A suite that always
passes has stopped discovering anything. Prompts nobody wrote to pass
have to come from real sessions.

- hooks/skill-loads.sh reads a Claude Code transcript and emits a marker
  naming the skills invoked and the recipes/playbooks/guides opened:
  `[cargo-skills: cargo-gtm | docs: build-tam,waterfall]`. The session
  hooks append it to the row's summary, so it is queryable today over
  workspace_management.sessions with no schema change.
- Wired into both session-checkpoint.sh and session-end.sh. Recomputed
  from the whole transcript each time rather than accumulated, so a
  session that dies abruptly still carries it. Emits nothing at all for
  sessions that never touched Cargo.
- evals/README.md documents the three questions the data answers (which
  skills never load, which sessions loaded two before settling, which
  load with no doc read after) and how each turns into an eval case.
- Self-test wired into skills-lint, because a regression here corrupts
  telemetry silently instead of failing loudly.

Two bugs the fixtures passed and real transcripts caught:

- **Prose mentions counted as reads.** The doc regex matched a bare path
  anywhere in the transcript, and a skill's own catalog table names every
  recipe and playbook — so one cargo-gtm load reported all 43 playbooks
  as opened. Now anchored to a `"file_path":` tool input. Fixture 7 is
  that exact case.
- **Dangling prefix** when docs were read without a Skill call, giving
  `[cargo-skills: | docs: …]`. Both halves are now independently optional.

Also: the doc list caps at 12, and says `+N more` rather than looking
complete.

PRIVACY: skill and document names only, all of them already public in
this repository. Never a prompt, file contents, an argument, a record, or
anything a user typed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* Fix Bugbot issues: preserve marker in sync finalize, retain skill load order

1. End hook drops skill marker (high severity): The sync finalize now
   includes the skill marker in the placeholder summary, so it persists
   even when refine() fails, times out, or is skipped. The marker is
   fast (no LLM call, just grep/sed).

2. Skill order lost to sorting (medium severity): Replaced 'sort -u'
   with 'awk !seen[$0]++' to preserve load order. The first skill in
   a multi-skill session is the misroute candidate, so order matters
   for harvest question 2.

Added test case 10 to verify load order preservation.

Co-authored-by: Auree Aubert <aureeaubert@users.noreply.github.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Auree Aubert <aureeaubert@users.noreply.github.com>
2026-08-12 14:33:36 -07:00

122 lines
5.3 KiB
Bash
Executable File

#!/usr/bin/env bash
# Cargo session checkpoint hook (Stop event): keep the session row fresh
# mid-session.
#
# SINGLE SOURCE OF TRUTH for both delivery channels — runs as the PLUGIN copy
# (default) or as the STANDALONE copy the installer's fallback channel downloads
# to ~/.claude/hooks/. Behavior is identical in both; only deference differs:
# the plugin copy exits when a standalone copy exists, so the row is never
# double-checkpointed. Mode auto-detects from the script's location and can be
# forced with a first argument: plugin | standalone.
#
# Derives a lightweight title/summary from the transcript WITHOUT an LLM call
# (latest user prompt + timestamp) and upserts the row WITHOUT --finished, so a
# session that never reaches SessionEnd (crash, timeout, reclaimed container)
# still shows recent context instead of being stuck on "Session in progress."
# Throttled to at most one update per CARGO_CHECKPOINT_INTERVAL seconds
# (default 45) so it never adds a network call to every turn.
set -u
# Exit inside the SessionEnd summarizer child — it would checkpoint a phantom
# session row on every turn of a process that exists only to write a title.
# See the recursion note in hooks/session-end.sh.
if [ "${CARGO_SESSION_SUMMARIZER:-}" = "1" ]; then
exit 0
fi
SCRIPT_DIR="$(CDPATH= cd -- "$(dirname -- "$0")" && pwd)"
MODE="${1:-}"
if [ -z "$MODE" ]; then
if [ "$SCRIPT_DIR" = "$HOME/.claude/hooks" ]; then MODE="standalone"; else MODE="plugin"; fi
fi
# A standalone copy owns the lifecycle only when it is REGISTERED in
# ~/.claude/settings.json — a leftover file nothing invokes must not suppress
# the plugin copy. When registration cannot be verified (no jq, or no
# settings.json), default to NOT owned: the plugin copy runs. That is the safe
# direction — the session upsert is idempotent, so a duplicate with an active
# standalone is harmless, whereas a wrong defer silently skips CLI pinning and
# session logging with no signal.
standalone_owns() {
s="$HOME/.claude/hooks/$1"
[ -x "$s" ] || return 1
command -v jq >/dev/null 2>&1 || return 1
[ -f "$HOME/.claude/settings.json" ] || return 1
jq -e --arg cmd "$s" \
'[.hooks[]?[]? | .hooks[]? | select(.command | contains($cmd))] | length > 0' \
"$HOME/.claude/settings.json" >/dev/null 2>&1
}
if [ "$MODE" = "plugin" ] && standalone_owns "session-checkpoint.sh"; then
exit 0
fi
LOG="${CARGO_SESSION_LOG:-$HOME/.claude/cargo-session.log}"
mkdir -p "$(dirname "$LOG")" 2>/dev/null || true
log() { printf '[%s] checkpoint(%s): %s\n' "$(date -u +%Y-%m-%dT%H:%M:%SZ)" "$MODE" "$*" >>"$LOG" 2>/dev/null || true; }
INPUT="$(cat 2>/dev/null || true)"
SESSION_ID="$(printf '%s' "$INPUT" | jq -r '.session_id // "unknown"' 2>/dev/null || echo "unknown")"
TRANSCRIPT_PATH="$(printf '%s' "$INPUT" | jq -r '.transcript_path // ""' 2>/dev/null || echo "")"
command -v cargo-ai >/dev/null 2>&1 || exit 0
# Throttle: skip if this session was checkpointed within the last INTERVAL secs.
INTERVAL="${CARGO_CHECKPOINT_INTERVAL:-45}"
STAMP="${TMPDIR:-/tmp}/cargo-checkpoint-${SESSION_ID}.ts"
NOW="$(date +%s)"
if [ -f "$STAMP" ]; then
LAST="$(cat "$STAMP" 2>/dev/null || echo 0)"
case "$LAST" in (*[!0-9]*|"") LAST=0 ;; esac
if [ $((NOW - LAST)) -lt "$INTERVAL" ]; then
exit 0
fi
fi
TITLE="Claude Code session ${SESSION_ID}"
SUMMARY="Session in progress."
if [ -n "$TRANSCRIPT_PATH" ] && [ -f "$TRANSCRIPT_PATH" ]; then
# tail by lines (not bytes) so we never feed jq a half-written JSON line.
TAIL="$(tail -n 600 "$TRANSCRIPT_PATH" 2>/dev/null || true)"
if [ -n "$TAIL" ]; then
# Latest real user prompt (string or text blocks; tool_result entries skipped).
LAST_USER="$(printf '%s\n' "$TAIL" | jq -rR '
fromjson? | select(.type=="user") | .message.content
| if type=="string" then .
elif type=="array" then (map(select(.type=="text").text) | join(" "))
else empty end
| select(. != "")' 2>/dev/null | tail -n 1)"
if [ -n "$LAST_USER" ]; then
# Prompt-derived text is DATA end-to-end: shell assignments and quoted
# expansions never evaluate $(…)/backticks inside a variable's VALUE,
# and the upsert below passes --title/--summary as single quoted argv
# words — so there is no execution path. Control characters are stripped
# anyway so a pathological prompt can't mangle logs or the session row.
SNIP="$(printf '%s' "$LAST_USER" | tr '\n\t' ' ' | tr -d '\000-\037\177' | cut -c1-80)"
TITLE="$SNIP"
SUMMARY="In progress. Latest request: \"${SNIP}\". Updated $(date -u +%Y-%m-%dT%H:%M:%SZ)."
fi
fi
fi
# Which skills this session has loaded so far — see hooks/skill-loads.sh.
# Recomputed each checkpoint rather than accumulated, so the marker always
# reflects the whole transcript and a session that ends abruptly still carries
# it. Empty for sessions that never touched Cargo.
MARKER=""
if [ -x "$SCRIPT_DIR/skill-loads.sh" ]; then
MARKER="$(bash "$SCRIPT_DIR/skill-loads.sh" "$TRANSCRIPT_PATH" 2>/dev/null || true)"
fi
[ -n "$MARKER" ] && SUMMARY="$SUMMARY $MARKER"
if cargo-ai workspaceManagement session upsert \
--session-id "$SESSION_ID" \
--title "$TITLE" \
--summary "$SUMMARY" >>"$LOG" 2>&1; then
printf '%s' "$NOW" > "$STAMP" 2>/dev/null || true
log "checkpointed $SESSION_ID"
else
log "checkpoint upsert failed for $SESSION_ID"
fi
exit 0