mirror of
https://github.com/callstack/agent-device.git
synced 2026-09-14 20:06:34 +08:00
main
11 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Michał Pierzchała
|
0da105e3c3 |
docs: simplify agent context and resolve conflicting guidance (#2287)
* docs: clarify agent task scope and validation * docs: remove redundant and conflicting agent guidance |
||
|
Michał Pierzchała
|
9941330dcc |
docs(agents): PR diff budget, move PRs, gates commit, validation lifecycle (#2247)
* docs(agents): PR diff budget, move PRs, gates commit, validation lifecycle * docs(agents): merge-base diff budget, subprocess lane facts, gate-stage validation * docs(agents): keep volatile test topology at its declaration |
||
|
Michał Pierzchała
|
dbc4f2f955 |
chore(test): start the subprocess-stub kill-criterion experiment (#1823) (#2007)
Deletes the serialized `subprocess-stub` Vitest project and drops SUBPROCESS_STUB_TESTS from unit-core's exclude, so its two real spawners (client-metro.test.ts, harness.test.ts — corpus-replay.test.ts already left for fuzz-worker in #1994) run un-serialized in the default forks pool per #1823's own kill criterion. Revert if a timeout-shaped failure shows up before 20 consecutive CI runs pass clean. The files stay excluded from the mutation lane (SERIALIZED_TESTS): that exclusion is about mutant-rerun cost, independent of Vitest project structure. Updated the comments/docs/scripts that described the old project by name so none of them assert a project that no longer exists. Claude-Session: https://claude.ai/code/session_015YPgKE1xmjdqh7T1q987DA Co-authored-by: Claude <noreply@anthropic.com> |
||
|
Michał Pierzchała
|
104fe75248 |
fix(ci): run the fuzz corpus replay outside the coverage lane (#1994)
The Coverage job intermittently ends with no failing test and one file's
results missing:
Test Files 1070 passed (1071)
Errors 1 error
Error: [vitest-pool]: Worker forks emitted error.
Caused by: Error: Worker exited unexpectedly
This is shape (B) of #1824 — the half #1854 did not fix. Scanning every
failed Coverage job across the 120 CI runs since #1854 merged finds the
signature five times, and the vanished file is
scripts/fuzz/corpus-replay.test.ts all five (six for six with #1866's
occurrence): 23% of Coverage failures in that window, ~4% of all CI runs.
The ~40s gap before the error is coverage report generation, not test
time — the pool surfaces its AggregateError only once every task settles.
Control, from a green attempt of the same run: the file passes in 3152ms
at 09:37:35.9 and the summary prints at 09:38:12.5. So the file is not
slow in CI, nothing else is in flight when it dies, and neither a missed
per-case budget nor STARTUP_BUDGET_MS is implicated. Partial test counts
(3/11 and 9/11 reported) place the death mid-file, inside runCases.
So the corpus replay gets its own serialized project that the coverage
run skips, and a second uninstrumented Vitest invocation in
`test:coverage:ci` runs it, keeping the tests on every PR. Measured
against two full runs, this costs zero coverage: the cases execute in
worker threads, a separate isolate the fork's inspector never
instruments, so the lines reported are identical with and without it.
Membership is by demonstrated failure, not by a property of the code:
`session-replay-runtime-maestro.test.ts` also constructs a
node:worker_threads Worker and stays in unit-core, instrumented and
green, so "nests a Worker" is explicitly not the criterion.
The second leg goes through `test:fuzz-worker`, which blanks
AGENT_DEVICE_COVERAGE_SHARD and AGENT_DEVICE_COVERAGE_MERGE. ci.yml sets
those as job-level env over a single `gate: unit-ci` step, so both legs
would otherwise inherit them and the shard would die: Vitest refuses
`--shard=1/2` over this one-file project, and the blob reporter
overwrites the instrumented shard's report on its way out. Verified on
the merged tree — shard 1/2 (549 files), shard 2/2 (548), and the merge
job (1097 files, 90.38% lines) all pass, and the leg still fails without
the blanking.
Refs #1824
|
||
|
Michał Pierzchała
|
d713988c5a |
docs(agents): simplify testing and pull-request guidance wording (#1997)
* refactor(lint): replace the facade import scan with a lint rule The surviving half of `contracts-entry-closure.test.ts` walked ~490 candidate files and parsed each one to assert that nothing value-imports the two wide contracts facades. `eslint/no-restricted-imports` already states exactly that, and `allowTypeImports` already draws the one distinction that made the walker seem necessary: `import type` is erased, so it stays legal. Verified rather than assumed, because the override semantics are not additive: a same-rule override REPLACES the parent, so a top-level rule would have been silently dropped for `src/**`, and the existing `"off"` entry for `exec.ts` and the test tree would have exempted the files that carried most of the cost #1959 removed. So the paths are added per zone, and the blanket `"off"` becomes a facade-only config that keeps the `node:child_process` exemption it existed for. Planted red in all three zones — `src/core/capabilities.ts`, a `src/__tests__` file, and `packages/capture-kit/src` — each flagged, while a type-only import in the same probe file was not. A first probe read as a pass because the sed that built it produced a type-only import; the zone was re-probed with a real value import rather than trusting the green. Misconfiguration fails loudly, which is why this is safe to rely on: a typo'd rule name makes oxlint exit 1 with "Rule not found in plugin", not pass silently (the failure mode #1976 records for the `rg` assertions). What a linter cannot replace, and stays: the eager-closure budgets. Those are a transitive-weight property — a module already imported grows an import, and the cost arrives without any single file's import list changing. Per-file rules cannot see that, and `no-restricted-imports` can only ban specifiers named in advance, which is precisely what #1950/#1956/#1959 could not have named. * docs(agents): simplify testing and pull-request guidance wording testing.md sat 15 bytes under the 10k per-doc check:agent-guidance cap. Rewrite both docs in shorter, plainer sentences without dropping any fact, threshold, or identifier (backtick-identifier sets verified unchanged against the previous revision). Also fix testing.md's gate catalog sentence being separated from its code block and the missing blank line before pull-requests.md's Reviewing section. |
||
|
Michał Pierzchała
|
40e4b0dd3e |
docs(agents): restore and enforce progressive disclosure (#1888)
* docs(agents): restore and enforce progressive disclosure * test(maestro): pin typed selector fallback signal * docs(agents): address progressive disclosure review * docs(agents): restore orphaned traps and close guidance-gate bypasses - AGENTS.md: skills carry a minimal start/routing card; command semantics stay in versioned CLI help (the skills contract enumerates two skills by hand, so prose retains ownership for the rest) - testing.md: restore the two local-only XCTest snags CI never hits (unsigned-bundle policy refusal signature + first-run automation permission) - scripts/gate/routing.ts: record GitHub's 300-changed-file path-filter limit at the paths-ignore assertion it bounds - agent-guidance-contract.test.ts: recurse docs/agents so nested guidance cannot evade the byte budgets while the gate stays green |
||
|
Michał Pierzchała
|
ef2094c9d7 |
docs: keep size review in CI and local feedback fast (#1842)
* feat(size): measure a base ref in one command (pnpm size --base <ref>) The Size workflow already compares base and PR builds; locally that needed a manual checkout, install, build, --json, and --compare dance, so budgets were negotiated late. --base <ref> does the workflow's recipe in a detached worktree under .tmp/size-base/<sha> (kept for reuse, other bases pruned) and compares against it: first run ~1-2 min, later runs against the same base ~3s. Documents the local caveat: npm tarball/unpacked rows compare a fresh base against a working tree that may carry locally built helper artifacts. * feat(tooling): pnpm pr:evidence — one paste-ready, SHA-stamped evidence block for PR bodies Composes what the repo already measures instead of hand-transcribing it after every rebase: exact merge-base and head, changed-file areas, the affected selector's plan (local vs GitHub-authoritative, fail-open summarized), the layering guard verdict, depgraph counts with a real delta against the base (a throwaway git worktree, no install — the script analyzes its cwd while its imports resolve from this checkout), and, behind flags, the changed-line coverage table and pnpm size --base. It claims nothing about CI: the last line links the head's checks. ~20s default tier. The pure model (grouping, report parsing, rendering) has node:test coverage registered as the pr-evidence-model gate, run in the Affected-check Selector job next to the selector it reads. * fix(tooling): pr:evidence measures pristine head/base worktrees from an os.tmpdir scratch; size --base gets a per-SHA lock, completion stamp, and non-destructive eviction Review (three P1s): - pr:evidence created its scratch under an untracked .tmp/ that a fresh checkout lacks (ENOENT). Scratch now lives under os.tmpdir(), which exists by construction; a real entrypoint regression runs the whole pipeline with --base HEAD (no origin/main needed) and asserts JSON shape plus cleanup of both worktrees and the scratch. - Untracked or uncommitted production files could move the layering/depgraph numbers the block labels as HEAD's. Head is now measured from a pristine worktree of the head commit exactly like base, and the affected plan takes the head SHA (the literal HEAD folds the working tree in). The dirty flag now counts untracked files and says they are not in the block. - size --base force-pruned other cached bases without locking and trusted a dist/src that could be half-built. Per-SHA .lock (pid, O_EXCL) held from before the worktree exists until the base report is read; a live lock on the same base fails fast, a stale one is replaced; eviction skips worktrees whose lock owner is alive; dist/.size-base-complete marks a finished build. Orchestration tests run the real script against a throwaway git repo with pnpm/npm shimmed on PATH (build once, reuse, live lock, stale lock, interrupted build, guarded vs idle eviction). Also fixes the /tmp → /private/tmp realpath mismatch those tests surfaced (git lists worktrees by real path, so the registration check removed a live worktree). * fix(tooling): symlink-identity locks with compare-then-unlink; evict under the victim's lock; pr:evidence registers worktrees on add and cleans up exhaustively Review (three P1s): - Lock creation/takeover races: the lock is now a symlink whose target is the owner identity (pid:nonce), created with its identity in one syscall (no empty-file window), taken over only by compare-then-unlink on the exact identity judged stale, and verified after creation; release unlinks only a link that still names this run. Real overlapping-process tests: two runs on one base (exactly one builds, the other fails fast), and a takeover race against a simulated other taker across delays straddling the acquire window (a live lock is never unlinked, both never proceed). - Cross-base eviction: a victim is removed only while holding its own lock, acquired through the same path, so a run wanting it after the check finds it locked rather than half-removed; a live-locked victim is skipped. - pr:evidence worktrees: withWorktrees registers each worktree the moment its add succeeds and sweeps every resource on the way out, collecting failures instead of stopping at the first; planted reds for both (second add fails → first removed; removal of the middle one throws → the others still go). * test(size): serialize the size --base orchestration file with the other real spawners Caught running the full unit suite on the rebased branch: the file passed in isolation but intermittently failed under broad file parallelism, where it took 14s versus ~5.5s alone. It spawns node scripts/size-report.mjs per case, which spawns git and the shimmed package managers under it — the SUBPROCESS_STUB_TESTS class exactly (starved spawns surface as a vitest test timeout instead of the orchestration assertion the case is about), so it joins that serialized project with its spawn named at the entry, per docs/agents/testing.md. No rerun layer is involved: the flake is removed, not retried. Two full-suite runs green after. * refactor(size): extract the base-cache claim protocol and make stale takeover atomic Review (P1 + architecture): Stale-claim removal was compare-then-unlink (readlink then unlink; lstat then rm for a stray file), so another taker could replace the observed entry with its live claim between the two syscalls and this run would delete the replacement. Removal now happens only while holding the entry's takeover mutex — an atomically created directory — and re-verifies the claim inside it. A replacement can appear only by creating one on a free path (the abandoned claim occupies it until the unlink) or by another takeover (needs the mutex), so removal cannot delete a replacement. A mutex leaked by a process killed inside its sub-millisecond critical section is reclaimed by age, and even a wrong reclamation is contained: both takers re-verify inside, and the winner is still decided by the atomic symlink() that follows. The protocol moves out of size-report.mjs into scripts/size-base-cache.mjs (AGENTS.md: extract past 500 LOC) — 719 → 536, with the entry lifecycle (claim → evict others → ensure worktree → build if unstamped → measure → release) owned by the module behind withPreparedBaseWorktree. Mirrored tests in scripts/__tests__/size-base-cache.test.ts plant every dangerous interleaving directly on the filesystem: replacement-after-observation, a takeover held by another run, age reclamation, release-after-retarget, and a stray non-symlink. They need no subprocess and run in 9ms, so the raced single-process case was dropped from the orchestration file, which keeps only what real processes can show. Planted red: removing the mutex makes the contended case delete the claim it must not touch. * ci(size): preserve the reporter's whole module graph, and gate that it stays whole The Size workflow measures the base commit with the PR's reporter, so it copies the reporter out of the tree before checking the base out. Extracting size-base-cache.mjs made the reporter a two-file graph while the step still copied one file, and the base measurement died with ERR_MODULE_NOT_FOUND — after every deterministic gate had passed, because nothing local reproduces that copy. The step now copies the scripts directory, so a further split cannot leave an import behind, and size-report-preserved-closure.test.ts holds it to the reporter's real relative-import closure and to running the preserved copy rather than the checked-out tree. Planted red: restoring the single-file copy fails both cases, naming scripts/size-base-cache.mjs. Verified by running the reporter from a copied directory exactly as the workflow does. * fix(size): the takeover mutex has one holder for life; split report publishing out of the reporter Review (P1 + architecture): Age-based reclamation of the takeover mutex reintroduced the split ownership the mutex exists to prevent: a holder that is merely slow — paused or SIGSTOPed past any threshold — could have its mutex force-removed and replaced, putting two takers inside the supposedly exclusive section, where either could unlink the claim the other had just created; the unconditional pathname-based release could also delete the replacement mutex. The mutex is now a symlink naming its holder, created in one syscall, never reclaimed at any age, and released only by the run that owns it. A mutex leaked by a process killed inside a three-syscall critical section wedges one cache entry with the path to clear in the message, rather than silently deleting another run's live claim. Planted red: restoring age reclamation displaces a day-old delayed holder, which the new case pins. Publishing the report to a PR is a separate question from measuring and formatting it, so it moves to scripts/size-report-comment.mjs with the marker and retry policy it owns; its existing regression drives it through the real script unchanged. scripts/size-report.mjs is 386 LOC — under the 500 tripwire and below the 512 it had on base. * test: prove delayed size cache holder is preserved * docs: keep size review in CI |
||
|
Michał Pierzchała
|
72d421fe36 |
docs(agents): ADR 0019 unit checklist, owning-seam mock rule, worktree and rebase guidance (#1836)
* docs(agents): ADR 0019 unit checklist, owning-seam mock rule, worktree and rebase guidance Retro follow-up (item 2). Adds docs/agents/adr-0019-unit.md — the order of operations for one command unit with the declaration site for each step, the evidence a unit review must carry, and what 'done' is not — so the pattern rediscovered during the snapshot unit (#1779) is written down once. testing.md: mock the seam the code under test consumes (fake inspectFacts / bindDevice), not the generic dispatchCommand mock; a migrating command moves its tests off the dispatch mock in the same PR. AGENTS.md: fresh-worktree preflight (pnpm install + build in the worktree; layering scan reads tracked files only) and concurrent-agent hygiene (one full gate per host, verify subagent edits with git -C, one PR per worktree). pull-requests.md: two readiness claims (published-and-reported vs merge-ready) and the rebase rule — main has no up-to-date protection; rebase on conflict or when `check:affected --base <merge-base> --head origin/main` names your surface. * docs(agents): name the admitted-plan token in the ADR 0019 unit checklist (#1841) * docs(agents): merge-ready owes live evidence only for changed device-facing paths * docs(agents): the unit checklist documents the admission API on main; #1841 updates the row when it lands |
||
|
Michał Pierzchała
|
fa9a350361 |
docs: prefer design fixes over regression-only guards (#1722)
* ci: require simplicity review for large tooling changes * docs: prefer design constraints over regression-only fixes * docs: simplify design-first guidance |
||
|
Michał Pierzchała
|
53e4be5f86 |
Remove SkillGym suite and repo-health snapshot infrastructure (#1480)
* chore: drop SkillGym and the repo-health aggregator (#1412 descope) Remove the SkillGym harness (test/skillgym/), its check-affected lane, package scripts, and devDependency — the help-conformance bench is now the single non-gating small-model oracle. skills/ markdown classifies as docs in the affected-check selector instead of failing open. Remove scripts/repo-health: its only gating assertion duplicated the Layering Guard job, its case-count metric imported the deleted SkillGym suite, and its sole planned consumer (#1424 / PR #1477) was closed with the Track C descope on #1412. Verified: check-affected node --test suites, oxfmt, oxlint, tsc, check:layering, fallow audit vs origin/main, and the full unit suite (unit-core + subprocess-stub) all pass. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FUv7bvbWNryuXgSBuqTtep * fix(scripts): fold slow-test budgets into the reporter for production-exports The Fallow production-exports gate flagged all three budget exports: their in-file consumer (SLOW_TEST_RATCHET) and the repo-health entry point that kept the module reachable were both removed in the descope, leaving the config-loaded reporter as the only consumer — invisible to --production analysis. The data-only module's second consumer is gone, so per the boundaries-are-earned norm the constants move into the reporter instead of gaining a suppression. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FUv7bvbWNryuXgSBuqTtep * docs: align skills/ format policy and purge last SkillGym mention Address both P2 review findings on #1480: the testing-matrix row and the selector's formatGate both still claimed oxfmt covers skills/, while selectChecks classifies skills/*.md docs-only (oxfmt ignores **/*.md, so the claim was a no-op even before). The matrix now states the docs-only policy and formatGate drops the dead underSkills fact. The merged examples/README.md index (from #1469) loses its skillgym mention. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FUv7bvbWNryuXgSBuqTtep --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
Michał Pierzchała
|
1a76344685 |
docs: restructure AGENTS.md and CONTEXT.md for progressive disclosure (#1402)
* docs: restructure AGENTS.md and CONTEXT.md for progressive disclosure
Apply the Claude 5 context-engineering guidance to the repo's agent docs:
keep the always-loaded file to gotchas and invariants, and move situational
guidance one hop away behind a routing table.
AGENTS.md 315 -> 229 lines. Cut generic agent-behavior boilerplate, three-way
duplication (Common Mistakes restated Hard Rules; Finding Source Owners
restated the registry section), and facts visible from the repo itself.
Kept verbatim: the expensive-lessons principles, enforcement gates, Hard
Rules, and environment traps.
Split out docs/agents/{cli-flags,pull-requests,device-verification}.md and
folded the Testing Matrix into docs/agents/testing.md, reframed around
pnpm check:affected so the prose stops duplicating the selector.
CONTEXT.md keeps all 50 terms, now grouped under a section index so a task
loads one section instead of the whole glossary.
* fix(check-affected): move the selector-owning sentinel to the Testing Matrix
The Testing Matrix moved from AGENTS.md to docs/agents/testing.md, but the
affected-check selector still treated only AGENTS.md as selector-owning. A
later matrix edit would have been classified as inert docs and skipped the
fail-open, so the selector could keep deriving gates from a spec that had
changed underneath it.
Move the sentinel with the prose, as a named SELECTOR_OWNING_DOCS set so the
next move is one line, and fix the two in-code comments plus the testing.md
paragraph that still pointed at the AGENTS.md matrix.
* docs: restore two rules dropped by the AGENTS.md split
Review caught two repo-specific rules that did not survive the move. Both are
prose without any backticked identifier, so the identifier-diff used to verify
the split could not see them.
- "Test through public interfaces; do not add unrelated production exports
solely to enable tests" returns next to the behavioral-tests rule in
docs/agents/testing.md, with the reason it exists.
- The guidance-ownership rule (decide whether new guidance/schema/metadata
belongs to the command surface, CLI grammar, CLI help, MCP projection, or
daemon runtime) returns to the always-loaded Docs & skills section, since it
governs all command-surface work and not just the flag case.
Also point the ADR routing row at docs/adr/README.md, which is already the
"read when you touch…" index, rather than at the bare directory.
|