mirror of
https://github.com/vectorize-io/hindsight.git
synced 2026-09-14 19:31:49 +08:00
feat/coding-agents-usage-stats
3 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Nicolò Boschi
|
682f77a221 |
fix(cursor): register hooks on init, ship rules_file.py, flush on sessionEnd (#4041)
* fix(cursor): make init register sessionStart hooks session_start imports lib.rules_file but init never copied it, and Cursor never saw the hooks because project .cursor/hooks.json was missing. Fixes #3864 * fix(cursor): flush sessions on sessionEnd, and harden init/uninstall Follow-up on top of lei_lei's #4023, which fixed the two reasons the plugin never ran at all: `scripts/lib/rules_file.py` was missing from `_PLUGIN_FILES` (so `session_start.py` ImportError'd on every invocation), and `init` never wrote a project `.cursor/hooks.json` (Cursor only loads hooks from the workspace or user file, not from `.cursor-plugin/`). Verified end to end against a real Cursor install and a local hindsight-embed: hooks register, `sessionStart` fires, and the agent answered from a seeded memory that existed nowhere in the project files. Four gaps surfaced while testing. Retain never ran for short sessions. `stop` is the only retain trigger, and every firing is subject to the turn window — at the default retainEveryNTurns of 10, a seven-turn conversation fires `stop` seven times, is gated seven times, and is never stored. `retain.py` is now registered on `sessionEnd` too and treats it as a final flush that bypasses the window. The two events overlap at the end of a conversation, so retains are made idempotent: a per-session watermark of messages already retained (`retained.json`) makes a run with nothing new a no-op. In chunked mode the flush stores only the tail past the watermark, keeping chunks non-overlapping. `python3` was hardcoded into the generated hook commands. Windows has no `python3` on PATH and the plugin explicitly supports it (`state.py` has a win32 branch), so every hook was a silent no-op there. `init` reported success over an install that copied nothing. When the bundled payload is incomplete it warned per file and still printed "Plugin files copied." and "Done!" — the same silent-no-op shape as #3864. It now aborts with a non-zero exit and no half-installed state. `uninstall` left the generated session rules file behind. It is written with `alwaysApply: true`, so a dead session's memories kept being injected into every agent turn after removal; its `.gitignore` line is stripped too. Co-authored-by: lei_lei <96427312+leilei3167@users.noreply.github.com> Claude-Session: https://claude.ai/code/session_01F3i5UVdZRK16AZ9oFQqsg4 --------- Co-authored-by: lei_lei <96427312+leilei3167@users.noreply.github.com> |
||
|
Nicolò Boschi
|
d7d137fa1e |
fix(daemon): drop the idle timeout that could kill an in-flight request (#3930)
* fix(daemon): drop the idle timeout that could kill an in-flight request The daemon's idle checker measured idleness as "time since the last request *started*" (IdleTimeoutMiddleware stamped last_activity on entry, never on completion), so a retain, reflect or consolidation that outlived the configured timeout was SIGTERM'd mid-flight — the client saw a connection error and the work was lost. Raising the timeout only lowered the odds; 0 (the default everywhere but the legacy cursor hook package) was the only safe value. Rather than teach the middleware to count in-flight requests, remove the feature: a shared local daemon that quietly exits under a long operation is not worth the resource reclamation it buys. The daemon now runs until it is stopped. - Delete IdleTimeoutMiddleware, the idle-checker thread and DEFAULT_IDLE_TIMEOUT. - Keep `--idle-timeout` parseable — hindsight-embed and every coding-agent integration still pass it — but ignore it, printing a note when it is non-zero. The integration packages therefore need no change and no release. - Drop it from the current docs, the SDK/integration READMEs and the generated docs skill. The coding-agents README keeps the row marked deprecated because docs-freshness.test.ts requires every readable RawConfig field to be documented; versioned docs, blog posts and changelogs are left as history. Closes #3903 Claude-Session: https://claude.ai/code/session_012gXUp1i7YrWmLJVUYki53g * review: drop the now-dead logger and refresh the comments the removal invalidated - daemon.py: the module logger only served the deleted idle checker. - coding-agents daemon.ts / runtime.ts / runtime.test.ts and openclaw: comments, a debug line and the plugin-schema description still described an auto-exit that can no longer happen. The config fields stay (inert) as agreed. Claude-Session: https://claude.ai/code/session_012gXUp1i7YrWmLJVUYki53g |
||
|
DK09876
|
91d767cdcb |
feat(cursor): add Hindsight memory plugin for Cursor (#866)
* feat(cursor): add Hindsight memory plugin for Cursor
Adds a complete Cursor integration using the plugin architecture
(hooks, skills, rules). Automatically recalls relevant memories
before each prompt and retains conversation transcripts on task
completion. Modeled after the claude-code integration with
Cursor-specific adaptations.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs(cursor): add integration docs, blog post, and sidebar entry
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs(cursor): clarify plugin vs MCP modes, add hook diagnostics
- Add plugin-vs-MCP comparison table near top of integration doc
- Add "Verifying Plugin Hooks" section with state file commands
- Add troubleshooting note: visible tool calls = MCP, not plugin
- Write last_retain.json state file in retain.py for diagnostics
- Add mode: plugin and query_length to recall state file
- Fix test_settings_file_loaded to isolate from user config
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(cursor): install path, always-write diagnostics, Cloud snippets
- Add mkdir -p before cp -r in all install examples (first-run fix)
- Add "fully quit and reopen Cursor" note to all setup flows
- Recall/retain hooks now write status on every invocation
(success, empty, skipped, error) not just on success
- Fix docs to show ~/.hindsight/cursor-state/ default path
- Add concrete Hindsight Cloud config snippet to Quick Start
- Add Cloud option to blog post setup section
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(cursor): add session field to dynamic bank IDs, add changelog
- Support "session" in dynamicBankGranularity for per-conversation banks
- Add changelog page for cursor integration
- Add test for session-based dynamic bank ID
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(cursor): sync integration README with cookbook/blog setup guidance
- Add mkdir -p for plugin install path
- Add "fully quit and reopen Cursor" instruction
- Show Cloud as Option A, local as Option B, daemon as Option C
- Match the setup flow documented in the cookbook and blog
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat(cursor): add pip/uvx installer, fix review findings
- Add hindsight_cursor package with CLI `init` and `uninstall` commands
- Add pyproject.toml for PyPI publishing via existing release pipeline
- Update README install path: `pip install hindsight-cursor && hindsight-cursor init`
- Fix rule/skill files to describe plugin behavior instead of MCP tools
- Add diagnostics on get_api_url failure paths in both hooks
- Remove missing assets/avatar.png reference from plugin manifest
- Add Cloud token retrieval guidance (Settings > API Keys)
- Add test_cli.py with 8 tests for init/uninstall commands
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(cursor): daemon timeout, config defaults, full config docs
- Set daemonIdleTimeout default to 300s (was 0/infinite with no cleanup hook)
- Fix retainEveryNTurns fallback from 1 to 10 in retain.py
- Fix DEFAULTS: hindsightApiUrl="" and bankId="cursor" to match settings.json
- Document all config settings in README (was missing ~15 entries)
- Fix pytest version discrepancy in pyproject.toml
- Fix plugin.json author to "Vectorize" for consistency
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs(cursor): streamline setup with init flags, add Docker instructions
- Restructure Quick Start around Cloud vs Local as two clear paths
- Use hindsight-cursor init --api-url/--api-token for one-command setup
- Add Docker run command for users without a local Hindsight server
- Remove separate "configure" step that contradicted init behavior
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor(cursor): replace beforeSubmitPrompt with sessionStart + MCP
beforeSubmitPrompt does not support additionalContext in Cursor's hook
system — the old recall.py was silently ignored. This rewrites the
architecture to use Cursor's native mechanisms:
- sessionStart hook for ambient project-level recall (supports additionalContext)
- MCP integration for on-demand recall/retain/reflect tools mid-session
- stop hook for auto-retain (unchanged, works correctly)
Also fixes Python floor (3.9 -> 3.10, pytest 9 requires it) and
updates docs/blog to match the new architecture.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(cursor): workaround broken sessionStart additionalContext
Cursor's sessionStart hook accepts additionalContext output but silently
drops it before the agent's composer handle is ready — a race condition
acknowledged by Cursor staff in 2026-04, still present in 3.6.31
(verified 2026-06-02 with a marker-emitting test hook). Without a
workaround the plugin's "auto-recall memories at session start" feature
silently does nothing in every install.
Per Cursor staff guidance (Dean Rie, thread 158452), the documented
escape hatch is to write a workspace .cursor/rules/<file>.mdc with
alwaysApply: true — the rules engine injects those reliably. Plugin-
local rules dirs (~/.cursor/plugins/local/...) are NOT reliable per
thread 159101.
Implementation:
- scripts/lib/rules_file.py (new): owns the workaround. Three helpers:
* rotate_session_rules() — deletes any prior rules file at the top
of each sessionStart so an empty recall doesn't leave stale
memories from a previous session.
* write_session_rules() — writes the .mdc with alwaysApply: true,
an HTML comment that explains what the file is and links to the
Cursor bug, and the recalled memories inside a
<hindsight_memories> block (same wrapper the broken native path
used, so the static rules guidance is unchanged).
* ensure_gitignored() — idempotently appends the file path to
<workspace>/.gitignore when the workspace is a git repo. No-ops
otherwise. Matches both /-anchored and bare relative forms so we
don't double-add against an existing entry.
- scripts/session_start.py: rotates at the top, writes the fallback
file after recall succeeds, gates both behind config flags
(useRulesFileFallback, appendToGitignore, both default True). Still
emits additionalContext to stdout below — when Cursor fixes the
upstream bug, dropping the workspace write is the only code change
needed; the same plugin works on the native path with no protocol
rev.
- scripts/lib/config.py: two new config keys + HINDSIGHT_USE_RULES_
FILE_FALLBACK / HINDSIGHT_APPEND_TO_GITIGNORE env overrides.
- rules/hindsight-memory.mdc: tells the agent where recalled memories
now appear (the new .cursor/rules/hindsight-session.mdc file) and
notes that the file is plugin-generated and safe to delete.
- tests/test_rules_file.py: 18 tests pinning the on-disk shape:
frontmatter, alwaysApply, bug link, rotation, idempotent gitignore
with both anchor forms, falsy workspace handling, write-error
degradation.
Why this design (vs. alternatives):
- Just shipping MCP-only and documenting the limitation would repeat
the OpenAI Agents notebook-10 Pattern-1 failure mode: the agent has
to choose to call recall, and small models reliably skip it. Auto-
inject doesn't depend on tool-call choice.
- Reverting to beforeSubmitPrompt would mean a recall per turn instead
of per session, and Cursor staff have signalled additional_context
on that hook is unimplemented (forum 150707).
- The workspace file is the price of Cursor's bug being open with no
ETA. Mitigations: auto-rotate, auto-gitignore, in-file explanatory
comment, config opt-outs.
Verification:
- Full suite: 74 passed (56 prior + 18 new).
- Smoke end-to-end against a fresh git repo: rules file written with
correct frontmatter, .gitignore appended cleanly with both an
explanatory comment and the path entry, no duplicate-add on re-run.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test(cursor): adopt requires_real_llm bucketing + live E2E + lockfile + docs
Aligns cursor with the standing test-bucketing convention from PR #1469
("Split test suite into deterministic mock and real LLM buckets") that the
other eight Python integrations already follow.
Changes:
- pyproject.toml: register the `requires_real_llm` marker so the live
E2E suite is selectable as a discrete bucket (and excluded from the
deterministic CI path via `pytest -m "not requires_real_llm"`). Add
hindsight-client as a dev dep — the E2E driver needs it to seed and
verify banks; the runtime plugin scripts still use stdlib only.
- tests/test_e2e.py (new): four-test gated suite that drives the actual
hook scripts the way Cursor does — JSON on stdin, env vars for config
— against a live Hindsight server. Covers:
1. session_start writes the rules-file workaround with recalled
content, appends `.gitignore`, and emits the forward-compat
`additionalContext` to stdout.
2. empty-bank case: hook succeeds without writing a rules file.
3. opt-out: `useRulesFileFallback=false` produces no `.cursor/` or
`.gitignore` mutations even when recall surfaces content.
4. retain end-to-end: drives `retain.py` with a JSONL transcript
(the on-disk shape Cursor actually emits, not an inline messages
array), then verifies the bank holds the fact via direct recall.
Two non-obvious fixtures the suite needs:
- `HOME` / `CURSOR_PLUGIN_DATA` redirected to tmp so the test doesn't
touch the developer's real `~/.hindsight/cursor.json` or state.
- `HINDSIGHT_BANK_MISSION` overridden to a focused mission that aligns
with the seeded fixtures — the production default mission is broad
boilerplate, fine for real users but too diffuse to reliably
surface targeted test content within a deadline.
- `HINDSIGHT_RETAIN_EVERY_N_TURNS=1` because retain.py batches every
N turns (10 by default) and a single-shot test only has one turn.
- uv.lock: committing per the convention every other Python
integration follows. 258 KB, 29 packages resolved, `uv lock --check`
clean.
- README.md: new "How session memory reaches the agent" section
documenting why the plugin writes `<workspace>/.cursor/rules/
hindsight-session.mdc` (Cursor's native `additionalContext` channel
is broken, forum thread 158452, still open in 3.6.31). Captures the
empirically-verified behaviour: Cursor blocks prompt submission
until sessionStart returns, so every new agent's first prompt has
memories, the rules file is regenerated each session, and the file
is auto-gitignored. Two new config knobs (`useRulesFileFallback`,
`appendToGitignore`) added to the Session Recall table.
Verification:
- Deterministic bucket: 74 pass / 4 deselected (the new gated E2E).
- Live bucket (HINDSIGHT_API_URL=http://127.0.0.1:8888): 4 pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(cursor): default to hosted backend + give each retain a distinct document_id
V2 audit (2026-06-02) caught two real bugs in cursor that were missed by
the V1 pass:
1) Goal-5 (Default to Cloud) FAIL — settings.json shipped
hindsightApiUrl='' and the daemon path treated empty as "fall back to
local daemon at 127.0.0.1:9077". Users following the docs ("just enable
the plugin") never reached the hosted backend without explicitly
passing --api-url. Every other integration's empty-config path lands on
https://api.hindsight.vectorize.io.
2) The retain path used document_id=session_id in full-session mode,
which silently upserts the same Hindsight document on every retain.
The audit's 5-turn distinct-fact driver exposed this as "5-turn cloud
→ 1 topic surfaced" — earlier turns got overwritten because each
retain rewrote the single per-session document with whatever
transcript snapshot was current.
Both are addressed below; the live test suite still passes against the
local server and the new deterministic tests pin the cloud-default
resolution + the unique-document-id derivation.
Changes:
- scripts/lib/config.py — add ``DEFAULT_HINDSIGHT_API_URL`` constant
(``https://api.hindsight.vectorize.io``). Add ``useLocalDaemon`` flag
(default ``False``) so self-hosters can opt back into the auto-managed
daemon path. New env override ``HINDSIGHT_USE_LOCAL_DAEMON``.
- scripts/lib/daemon.py — rewrite ``get_api_url`` resolution:
1. Explicit ``hindsightApiUrl`` wins.
2. A locally-running server on the configured port is used (preserves
the "developer already started a daemon" path).
3. ``useLocalDaemon=True`` AND ``allow_daemon_start=True`` (retain
path) triggers the auto-managed daemon. Recall path never starts a
daemon on its own.
4. Otherwise → ``DEFAULT_HINDSIGHT_API_URL``. A failed daemon-start
under (3) also falls back here rather than hard-erroring, so the
plugin keeps working when ``hindsight-embed`` isn't on PATH.
- scripts/retain.py — every retain now derives
``document_id = f"{session_id}-{int(time.time() * 1000)}"`` regardless
of retainMode. The chunked-vs-full-session distinction at the doc-id
layer was always a misfeature; full-session mode now means "the
transcript ingested per retain may span the whole session", not "every
retain writes the same document".
- tests/test_daemon.py (new) — pin the four-tier resolution + env
override + the source-shape of retain.py's document_id derivation.
Verification:
- Deterministic bucket: 81 pass / 4 deselected (74 prior + 7 new).
- Live bucket: 4 pass / 0 fail against 127.0.0.1:8888.
- Manual smoke for empty-config → returns ``DEFAULT_HINDSIGHT_API_URL``.
- Live server still resolves to ``http://127.0.0.1:8888`` when healthy.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(cursor): parse Cursor 3.x role-nested transcript format
retain.py's read_transcript only recognized two transcript shapes:
- Flat: {role, content}
- Type-nested: {type: "user"|"assistant", message: {role, content}}
Cursor 3.6.31 writes a third shape to its stop-hook transcript:
{"role":"user","message":{"content":[
{"type":"text","text":"..."},
{"type":"tool_use","name":"...","input":{...}}
]}}
Top-level has `role` (not `type`), and `content` lives under `message`
as a list of typed blocks (not at the top level as a string). The old
parser's two branches both missed every line: `entry.get("type")` was
None and `"content" in entry` was False. read_transcript silently
returned [] for every Cursor 3 transcript, and retain.py bailed with
status=skipped reason=empty_transcript on every stop hook.
Visible symptom: auto-retain silently stops working under Cursor 3
even though the stop hook fires correctly and transcript_path points
at a real, populated file (verified by reading
~/Library/Application Support/Cursor/logs/.../cursor.hooks.*.log —
the input JSON includes a valid transcript_path that the parser then
ignores). End users see recall continue to work (sessionStart writes
the rules-file workaround) but new turns never get retained.
Fix:
- Add _normalize_blocks_to_text to flatten typed-block lists to a
single string, inlining a compact [tool_use:<name>] marker so
downstream Answer:/Thought: handling still sees coherent structure.
- Recognize the role-nested Cursor 3 shape explicitly.
- Keep flat and type-nested handling intact.
Verified end-to-end against a real Cursor 3.6.31 transcript captured
from ~/.cursor/projects/.../agent-transcripts/<conv>/<conv>.jsonl:
read_transcript now returns the 15 messages it should (1 user + 14
assistant turns) instead of 0.
Regression tests (3 added):
- test_read_transcript_parses_flat_format pins the flat shape.
- test_read_transcript_parses_type_nested_format pins the type-nested
shape.
- test_read_transcript_parses_cursor3_role_nested_with_block_content
is the regression: fails on the pre-fix parser (returns []), passes
now. Also asserts the [tool_use:Shell] marker survives.
14/14 tests in test_hooks.py pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore(docs): drop missing image refs in cursor blog post
The 2026-04-03 cursor-persistent-memory blog references
/img/blog/cursor-persistent-memory.png in both frontmatter and
inline markdown, but the image was never added to the repo. build-docs
fails MDX compilation with "Markdown image with URL
/img/blog/cursor-persistent-memory.png couldn't be resolved to an
existing local image file".
Strip the two references so the post renders. The prose stands on its
own without an illustration; an image can be added in a follow-up PR
if/when one is produced.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore(cursor): sync openapi.json with main
Branch carried an older snapshot of hindsight-docs/static/openapi.json
that pre-dated main's addition of the OperationProgress schema.
check-openapi-compatibility flagged the missing 'progress' field on
GET /v1/default/banks/{bank_id}/operations/{operation_id} as a
backwards-incompatible removal.
Re-checkout main's openapi.json onto the branch.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore(cursor): drop cursor-persistent-memory blog post
The blog post was added as marketing for the Cursor integration but
the accompanying illustration was never produced. Earlier commit
|