mirror of
https://github.com/nexu-io/open-design.git
synced 2026-09-20 06:15:06 +08:00
lefarcen
792480fe21
* test(daemon): red specs for unbounded persisted run events
Incident 2026-09-14: a cursor-agent conversation (~169 turns) stored every
unrecognised stdout line verbatim as a `raw` event. One `editToolCall`
completion repeats the edited file twice (~1.2 MB) and the `user` line echoes
the whole composed prompt, so each turn stored ~1.4 MB that nothing renders.
The renderer and daemon ran out of memory, and `GET .../messages` failed with
`RangeError: Invalid string length`.
These specs pin the invariants the fix must establish, against 0.22.x-shaped
rows (seeded straight into the daemon's tables) and the real parser +
persistence path:
- I1: every persisted run event fits a 64 KiB budget; meaningful events
(status, done_key, text, usage) are stored unchanged; nothing is dropped;
live `raw` lines are bounded too.
- I2: GET messages returns a bounded transcript; the PUT sibling-Run guard and
the project run-status projection keep their verdicts without any database
read handing a run's event log to JS.
- I3: a background, idempotent, one-row-per-transaction heal bounds existing
rows, skips active runs, and is safe to interrupt.
* fix(daemon): bound persisted run events and heal oversized rows
A persisted run event never carries an unbounded payload (I1):
`runtimes/run-event-payload-budget.ts` bounds every stored event to 64 KiB of
JSON. An oversized event keeps its kind, identity fields and count; its
payload is shortened deterministically to a 16 KiB head+tail with an inline
`[open-design: ...]` marker (bytes cut, original size, digest) and a
`truncated: { originalBytes }` field. cursor-agent `tool_call` lines drop their
full-file copies (`beforeFullFileContent`, `afterFullFileContent`, read
`content`) first so the retained JSON stays valid and meaningful; Write/Edit
rows keep their `+N -M` via `od_diff_stat`; text/thinking are never cut.
Applied at the json-event-stream source (live SSE + run buffer), at the
SSE->persisted conversion, and at every events_json/batch write.
Loading a conversation never materializes unbounded data (I2): GET messages
streams rows and bounds legacy rows on read; the PUT sibling-Run guard reads
each sibling's first done_key in SQLite (`listSiblingRunDoneKeys`); the
project run-status projection picks each project's latest run in SQLite and
reads only the events `eventsEndedWithUnfinishedWork` can see.
Existing oversized rows are healed once (I3):
`storage/message-event-payload-heal.ts` runs after the daemon is listening,
one row per transaction, streaming each row's events via json_each, skipping
queued/running runs, and records completion in `daemon_maintenance_passes`.
Contracts gain the optional `AgentEventPayloadTruncation` field on raw,
tool_use and tool_result events (and the SSE raw payload).
* fix(daemon): keep listMessages working on all()-only DB adapters
Streaming rows with `iterate()` is what keeps a pre-heal conversation read
from holding every row's legacy event log at once, but some integration
boundaries hand listMessages a lightweight adapter that models only
`prepare().all()` (as `hasMessageEventBatchStorage` already tolerates). Fall
back to `all()` there. Also prepare the run-status completeness statement once
per listing instead of once per project.
* test(daemon): red spec for unbounded live raw lines from every parser
claude-stream, copilot-stream and qoder-stream hand unrecognised stdout lines
to the SSE fan-out verbatim; only json-event-stream bounded them at the
source. Pin that every parser's live raw event fits the 64 KiB per-event
budget on both the newline and flush() paths, and that a line that fits is
passed through unchanged.
* fix(daemon): bound raw lines at the source in every stream parser
Apply boundedRawAgentEvent to the raw emitters of claude-stream,
copilot-stream and qoder-stream, as json-event-stream already does, so the
live SSE stream and the in-memory run buffer carry the same bounded line the
transcript stores. Live tool_use/tool_result stay unbounded on purpose:
Langfuse tool spans and artifact detection read them from the live run.
* fix(daemon): deliver agent prompts intact and reap leftover agents after a daemon crash
Plain-text-stdin runtimes (cursor-agent, codex, opencode, copilot, qwen,
amp, ...) received the composed prompt through an async pipe write. When
the daemon died right after spawn, only the kernel pipe buffer (64 KiB)
reached an agent that reads stdin late, and the detached agent (running
with --force) kept going on the truncated prompt, whose last visible
request was the conversation's oldest one.
- Deliver the prompt as a file-backed stdin: a 0600 file under the run
directory, passed as the child's stdin fd at spawn and deleted right
after. If it cannot be written, the run fails before any agent exists.
Claude's stream-json pipe is unchanged.
- Record each spawned agent (pid, process group, spawn window, daemon
identity) in its run directory. On the next daemon start, terminate
leftovers whose recording daemon is gone and whose identity still
matches, and sweep stale prompt files. A failed record write fails the
spawn instead of leaving an agent nothing can reap.
- Add process-identity helpers to @open-design/platform.
By design a leftover agent keeps running until the next daemon start;
the interrupted run reads back failed / DAEMON_RESTARTED.
Red first: with main's behaviour a late-reading agent received 65,536
bytes and its process group survived a restart. The agent-process specs
and e2e/tests/dialog/leftover-agent-reap.test.ts go red there and are
green on this change.