Files
callstack__agent-device/docs/agents/device-verification.md
Michał Pierzchała 1a76344685 docs: restructure AGENTS.md and CONTEXT.md for progressive disclosure (#1402)
* docs: restructure AGENTS.md and CONTEXT.md for progressive disclosure

Apply the Claude 5 context-engineering guidance to the repo's agent docs:
keep the always-loaded file to gotchas and invariants, and move situational
guidance one hop away behind a routing table.

AGENTS.md 315 -> 229 lines. Cut generic agent-behavior boilerplate, three-way
duplication (Common Mistakes restated Hard Rules; Finding Source Owners
restated the registry section), and facts visible from the repo itself.
Kept verbatim: the expensive-lessons principles, enforcement gates, Hard
Rules, and environment traps.

Split out docs/agents/{cli-flags,pull-requests,device-verification}.md and
folded the Testing Matrix into docs/agents/testing.md, reframed around
pnpm check:affected so the prose stops duplicating the selector.

CONTEXT.md keeps all 50 terms, now grouped under a section index so a task
loads one section instead of the whole glossary.

* fix(check-affected): move the selector-owning sentinel to the Testing Matrix

The Testing Matrix moved from AGENTS.md to docs/agents/testing.md, but the
affected-check selector still treated only AGENTS.md as selector-owning. A
later matrix edit would have been classified as inert docs and skipped the
fail-open, so the selector could keep deriving gates from a spec that had
changed underneath it.

Move the sentinel with the prose, as a named SELECTOR_OWNING_DOCS set so the
next move is one line, and fix the two in-code comments plus the testing.md
paragraph that still pointed at the AGENTS.md matrix.

* docs: restore two rules dropped by the AGENTS.md split

Review caught two repo-specific rules that did not survive the move. Both are
prose without any backticked identifier, so the identifier-diff used to verify
the split could not see them.

- "Test through public interfaces; do not add unrelated production exports
  solely to enable tests" returns next to the behavioral-tests rule in
  docs/agents/testing.md, with the reason it exists.
- The guidance-ownership rule (decide whether new guidance/schema/metadata
  belongs to the command surface, CLI grammar, CLI help, MCP projection, or
  daemon runtime) returns to the always-loaded Docs & skills section, since it
  governs all command-surface work and not just the flag case.

Also point the ADR routing row at docs/adr/README.md, which is already the
"read when you touch…" index, rather than at the bare directory.
2026-07-25 12:13:09 +02:00

3.1 KiB

Manual Device Verification

Read this before running agent-device by hand against a simulator, emulator, or physical device.

Before the run: defeat staleness

Dev-loop staleness has three layers, and each produces a convincing false negative.

  • After changing runtime code reached through bin/agent-device.mjs or the daemon: pnpm build, then pnpm clean:daemon — the daemon does not self-reload.
  • Before any Android verification from source: pnpm build, pnpm build:android, pnpm clean:daemon. build:android refreshes and verifies both bundled Android helper artifacts for the current package version.
  • shutdown deliberately HANDS OFF a healthy simulator runner. The adopted runner keeps serving the old Swift binary until you kill its process or the source fingerprint changes, so "my change did nothing" measured against an adopted runner is a classic false negative. If Swift runner code changed, run pnpm build:xcuitest.

Prove the path under test was actually active

  • Android: capture snapshot -i --json and require androidSnapshot.backend to be android-helper with helperVersion equal to package.json's version. A stock UIAutomator fallback is not valid verification unless the fallback itself is the behavior under test.
  • For repo-owned Agent Device Tester work, examples/test-app/README.md is the source of truth for simulator, physical-device, Metro/dev-client, and app-surface steps. An already-installed com.callstack.agentdevicelab is not sufficient — the README's Metro/dev-build and snapshot -i checks must prove the expected app surface is running.
  • For Android RN/Expo/dev-client apps on any local Metro port, adb reverse tcp:<port> tcp:<port> is harmless and should be run before opening the app or URL.

Session hygiene

Every manually opened session is a resource that must be closed — including exploratory sessions and failed verification attempts.

  • Every agent-device open needs a matching close with the same --session, --platform, --udid, and --state-dir before the agent finishes.
  • Use a purpose-specific session name for experiments, and an isolated --state-dir under /private/tmp when you need cleanup isolation beyond the current worktree's default daemon.
  • Track opened sessions in working notes; close each one before the final response.
  • If close is blocked by stale daemon metadata, inspect processes first with ps -ax | rg "agent-device|xcodebuild test-without-building". Stop only exact stale PIDs belonging to this verification run, then pnpm clean:daemon.
  • If cleanup cannot be completed, report the remaining session name, state dir, PIDs, and metadata paths as a blocker.

Sandboxed environments

Start the daemon outside the sandbox with escalation. The daemon binds localhost, and sandboxed runs fail before any product code executes, with listen EPERM: operation not permitted 127.0.0.1 or repeated Failed to start daemon/metadata cleanup messages. Those are not agent-device regressions — rerun with escalation. Unit tests, typecheck, lint, and build can stay sandboxed unless they need devices or listener access.