Fundamental fix: previous execInMicroVm used a poll-based protocol where
host wrote to a shared directory and polled for results. But Firecracker
uses virtio-block devices (not shared directories), so host and guest
can't simultaneously access the same filesystem without page cache issues.
New model (Lambda-style):
1. Host creates workspace ext4 image (64MB)
2. Host loop-mounts it, writes command.sh + request.json, unmounts
3. Host boots Firecracker VM with workspace as /dev/vdb
4. Guest-runner (one-shot) mounts /dev/vdb, executes command, writes
result.json, powers off the VM
5. Host waits for VM process to exit
6. Host loop-mounts workspace, reads result.json, unmounts
7. Clean up
Changes:
- guest-runner.sh: converted from polling daemon to one-shot executor
that halts the VM after writing results
- microvm.ts execInMicroVm: complete rewrite for sequential model with
createWorkspaceImage, mountExt4, unmountExt4 helpers
- dag-orchestrator.ts executeMicroVm: simplified — no longer manages
boot/destroy lifecycle, delegates to one-shot execInMicroVm
All existing tests pass. Rootfs rebuilt and deployed to PVC.
Implement Gate B proof for sandbox runtime: execute a deterministic tiny coding task that creates one small change in sandbox scope, runs verification, and exports a clean patch artifact before any real acceptance workload runs.
Created packages/agent-execution/__tests__/gate-b-smoke.test.ts with 5 passing tests that prove:
- Sandbox executor can materialize a repo at a specific SHA using materializeRepo()
- Code changes can be made in isolation (add comment to schema.ts)
- Git operations work (add, commit, format-patch via generatePatchArtifact())
- Verification commands execute and results are captured (bunx tsc --noEmit)
- Patch artifacts are generated with full commit metadata
- Touched-file reporting comes from sandbox-local checkout via getTouchedFiles()
- Host checkout stays clean (zero dirt verified in beforeEach/afterEach)
- Patch is reviewable and promotable (git format-patch format)
- Verification success/failure is reported truthfully in artifacts.verification
- Tests are rerunnable deterministically (not tribal knowledge)
Updated documentation:
- packages/restate/README.md: Mark Gate B as proven, document what's proven and known gaps
- docs/inngest-functions.md: Update sandbox mode gate status to reflect Gate B proven
- skills/system-bus/SKILL.md: Update execution mode documentation with Gate B status
All tests pass. Host checkout remains clean. Gate B acceptance criteria met.
Precondition: Gate A passing (verified before work).
Verification: bunx tsc --noEmit, pnpm biome check (passing), bun test packages/agent-execution (105 tests passing).
Files touched: packages/agent-execution/__tests__/gate-b-smoke.test.ts, docs/inngest-functions.md, packages/restate/README.md, skills/system-bus/SKILL.md.
Known gaps: No k8s Job launcher (that's Gate C), no multi-story orchestration, no cancellation/timeout.
Implement Gate A proof for the sandbox runtime PRD:
- Create packages/agent-execution/__tests__/gate-a-smoke.test.ts with 4 passing tests
- Prove truthful state transitions: running → completed
- Prove artifact generation (read file, write temp artifact)
- Prove zero host dirt (operator checkout stays clean)
- Prove failure state handling with honest error reporting
- Prove JSON serialization round-trip
- Add simple local sandbox executor (not k8s Job launcher - that's Gate B)
- Tests are rerunnable (not tribal knowledge)
- Document Gate A contract in packages/restate/README.md with proven/gaps sections
- Update docs/inngest-functions.md to reference Gate A status
- Update skills/system-bus/SKILL.md to reference Gate A proof
Gate A proves the contract validity and state machine. Known gaps:
- Local executor only (no k8s)
- No real git operations (deterministic SHA)
- No network isolation
- No resource limits
- No cancellation support
Next gates: B (k8s Job launcher), C (multi-story orchestration), D (cancellation/timeout)
Implement sandbox runtime PRD Story 5: ensure duplicate requestIds do not spawn duplicate work, terminal snapshots always land with logs attached, and cancellation kills the sandbox job honestly.
Changes:
1. Add stdout/stderr log surfacing in ExecutionArtifacts type and schema validators
2. Enhance serve.ts with isTerminalState() helper and terminal-aware deduplication
3. Add requestId-level deduplication at agent-dispatch function entry
4. Track active processes in activeProcesses map for cancellation support
5. Implement onFailure handler that kills subprocess and writes cancelled snapshot
6. Capture and attach stdout/stderr (10KB truncated) to all terminal results
7. Add terminal-results.test.ts with validation tests for all execution states
8. Update dag-orchestrator.ts with cancellation contract note
9. Document terminal state guarantees, cancellation, and log surfacing in:
- docs/inngest-functions.md
- packages/restate/README.md
- skills/system-bus/SKILL.md
Verification: bunx tsc --noEmit, bun test packages/agent-execution all pass
Result: No execution can pretend to be running when it finished. Duplicate dispatches return existing terminal results. Cancellation terminates the subprocess and writes honest state. Logs always attached for debugging.
Add execution-mode flag to route deterministic Restate story execution between host (shared checkout) and sandbox (isolated k8s Jobs) execution paths, preserving the current stable host behavior as default while laying the foundation for sandbox pilot.
Changes:
- Add ExecutionMode type ('host' | 'sandbox') to @joelclaw/agent-execution with schema validators
- Add PRD_EXECUTION_MODE environment variable to trigger-prd.ts (default: 'host')
- Pass executionMode in agent-dispatch payload from trigger-prd
- Route agent-dispatch to sandbox stub when executionMode='sandbox' (returns error until k8s Job launcher is implemented)
- Capture executionMode in InboxResult for observability
- Update Restate README with execution mode documentation and operator contract
- Update inngest-functions.md to document executionMode parameter
- Update system-bus skill with execution mode routing rules
Verification:
✅ bunx tsc --noEmit
✅ pnpm biome check (touched files only)
✅ bun test packages/restate packages/agent-execution (84 pass)
Operator-facing contract:
- PRD_EXECUTION_MODE=host (default): stable shared-checkout path
- PRD_EXECUTION_MODE=sandbox: stub error until k8s Job launcher ships
- Result polling (/internal/agent-result/:requestId) works for both modes
- Stable requestId/workflowId/storyId/agent identity preserved end-to-end
Story: ADR-0217 Sandbox Runtime PRD Story 4
Implement sandbox runtime PRD Story 3: clean repo materialization and patch-artifact export so sandbox runs mutate only their own checkout and return auditable output instead of touching the host worktree.
New capabilities:
- materializeRepo(): Clone or checkout repo at exact SHA in sandbox-local workspace. Fresh clone if target doesn't exist, fetch+checkout otherwise. SHA verification with automatic unshallow. Isolated from host worktree.
- generatePatchArtifact(): Export auditable patch from baseSha..headSha with touched-file inventory, verification summary, and log references. Uses git format-patch for commits, git diff for uncommitted changes.
- getTouchedFiles(): Capture modified/untracked files via git status --porcelain.
- verifyRepoState(): Validate repo is at expected SHA.
- writeArtifactBundle()/readArtifactBundle(): Serialize ExecutionArtifacts to/from JSON.
Promotion boundary: Phase 1 output is patch bundle + metadata. Runtime does NOT merge to main or push to remote. Operator reviews patch + verification, then applies to host repo or discards.
Tests: Full coverage for repo materialization, artifact export, touched-file inventory, and bundle serialization. All 84 tests pass.
Docs: Updated deploy.md, architecture.md, and system-architecture skill with new contract details and Phase 1 promotion boundary explanation.
Files:
- packages/agent-execution/src/repo.ts (new)
- packages/agent-execution/src/artifacts.ts (new)
- packages/agent-execution/src/index.ts (exports)
- packages/agent-execution/__tests__/repo.test.ts (new)
- packages/agent-execution/__tests__/artifacts.test.ts (new)
- docs/deploy.md (updated)
- docs/architecture.md (updated)
- skills/system-architecture/SKILL.md (updated)
Verification: bunx tsc --noEmit ✓, pnpm biome check ✓, bun test packages/agent-execution ✓ (84/84 pass)
Joel Hooks