Commit Graph

73 Commits

Author SHA1 Message Date
devin-ai-integration[bot] d585d74172 chore: close out architecture experiments (#1213)
* chore: close out architecture experiments

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* docs: record unavailable live experiment

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* ci: make Android perf script atomic

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* docs: explain atomic perf workflow

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* chore: simplify back-edge diagnostics

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Michał Pierzchała <thymikee@gmail.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-07-11 15:07:32 +02:00
Michał Pierzchała e2bfed5f9f feat(replay): ADR 0012 migration steps 5+6 — resume + --update retirement (#1211)
* feat(replay): ADR 0012 migration steps 5+6 — resume + --update retirement

Step 5 (decision 4, resume): replay --from <n> --plan-digest <sha256>
resumes at a 1-based plan step, skipping 1..n-1 without executing them.
Every divergence report now carries a real resume object (allowed, from,
planDigest, reason?) computed by a preflight that rejects INVALID_ARGS
before any action when: the plan digest no longer matches the current
script (edits/includes/platform-conditioned expansion), --from is out of
range, a skipped step can produce outputEnv values, or the skipped range
or resume target is runtime control flow (retry/runFlow.when — these are
single plan entries, never individually addressable). `test` rejects
--from/--plan-digest both at the CLI-schema layer and at the daemon
dispatch layer (the original command name is only visible before test
rewrites its nested request to `command: 'replay'`).

New modules: src/replay/plan-digest.ts (canonical SHA-256 plan digest)
and src/daemon/handlers/session-replay-resume.ts (preflight + the
report's resume object), kept out of src/replay/ to avoid a
replay<->compat import cycle.

Step 6 (decision 1, retirement): --update/-u no longer rewrites .ad
files. The ADR mandates a no-op, not an error or flag removal: --update
now runs identically to a plain replay and returns the same bounded
suggestions every divergence already carries. Removed: healReplayAction's
retry-and-rewrite arm and its exclusive helpers (collectReplaySelectorCandidates
stays — decision 1's suggestions still use it), the write call from the
runtime loop, and the env/${VAR}-interpolation/compat-flow refusal guards
that existed only to protect that rewrite. writeReplayScript itself keeps
its own round-trip tests but is otherwise unused now; deleted after the
production-exports gate flagged it as dead.

Docs: cli-help.ts workflow topic + --update/--from flag help, AGENTS.md
selector pipeline note, maestro-compat-debt-map.md, website replay-e2e.md
and commands.md updated for the retired rewrite and the new resume loop.

* fix(ci): classify resume flags + provider-scenario resume coverage

The Integration Tests job's architecture-progress gate
(test:integration:progress:check) requires every public CLI flag to be
classified; --from/--plan-digest (replayFrom/replayPlanDigest) were
unclassified. Classify them as device-observable workflow flags and add
real provider-backed coverage to the Android lifecycle scenario: a full
replay diverges on a missing selector, the report's resume object is
asserted (allowed/from/planDigest), and resuming at the next index
replays only the tail. Also refresh the stale replayUpdate reason
("selector-healing replay update" -> the retired no-op).

* fix: bind replay resume digest to execution plan

* test: align replay runtime module topology

* fix: clear replay CI regressions

* docs: clarify replay repair and resume paths

* docs: clarify replay resume step semantics

* docs(replay): note that ${VAR} values stay out of the plan digest (ADR 0012 + workflow help)

Settled decision from the PR #1211 re-review (maintainer-approved): interpolated
${VAR}/--env/AD_VAR_* VALUES are deliberately NOT part of the resume plan digest.
Substitution happens after the digest is computed over the still-unsubstituted
${VAR} text, so re-running the same script with different variable values keeps
the same digest and stays resumable — supplying the right values on resume is the
caller's responsibility. The digest still binds the script/includes, the effective
--platform/--target, and per-action runtime hints + target-v1 identity. Documented
in ADR 0012 decision 4 and the `help workflow` resume topic.

* docs: clarify replay digest interpolation
2026-07-11 15:06:49 +02:00
devin-ai-integration[bot] 0a8ea3a57b refactor: consolidate architecture ownership and client results (#1210)
* refactor: consolidate architecture ownership and client results

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix: keep selector parse chunk grouping current

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix: update moved architecture breadcrumbs

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix: enforce moved selector architecture

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix: keep selector guarantee ownership current

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* docs: update selector ownership references

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Michał Pierzchała <thymikee@gmail.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-07-11 09:40:24 +02:00
devin-ai-integration[bot] 952bc3704a refactor: keep command and daemon-route owner-file claims tooling-only (#1178) (#1192)
* refactor(command-descriptor): keep owner-file claims tooling-only

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(daemon): keep daemon-route owner-file claims tooling-only

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(daemon): guard against re-adding owner-file paths to the production route chain

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(command-descriptor): derive owner-file projection from colocated RAW_COMMAND_DESCRIPTORS

- Keep ownerFiles on each RAW_COMMAND_DESCRIPTORS entry as the source of truth.
- Add tooling-only __OWNER_FILES__ build flag so production bundles omit the
  ownerFiles properties entirely.
- Derive COMMAND_OWNER_FILES from RAW_COMMAND_DESCRIPTORS instead of a
  hand-maintained parallel table.
- Guard command-explain tests against leaking ownerFiles into production
  descriptor objects.
- Enable treeshake.propertyReadSideEffects: false in tsdown to help drop the
  dead ownerFiles branch from production bundles.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* chore: apply oxfmt formatting

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(build): guard tooling metadata exclusion

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(command-descriptor): drop global treeshake option and add bundle guard

- Remove treeshake.propertyReadSideEffects from tsdown.config.ts; the
  __OWNER_FILES__ define + conditional spread already keeps owner files out
  of the bundle, so the global DCE lever is unnecessary and scope-creeping.
- Add a comment on the __OWNER_FILES__ global declaration explaining the
  deliberate type-versus-runtime mismatch.
- Add test/output-economy/owner-files-no-leak.test.ts to build dist and
  assert that no command or daemon-route owner-file path appears in the
  emitted JS.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(build): remove owner metadata property reads

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(command-descriptor): enforce owner claim totality

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Michał Pierzchała <thymikee@gmail.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-07-11 07:46:00 +02:00
Michał Pierzchała d4146c7f1b feat: add Android test IME helper for deterministic text entry (#1198) (#1201)
* feat: add Android test IME helper for deterministic text entry (#1198)

Ships a headless InputMethodService (android-ime-helper) as a third Android
helper APK, replacing the visible system keyboard during automated sessions.
Renders zero accessibility nodes and accepts Unicode/CJK/emoji text over a
base64-encoded broadcast channel, fixing both the settle-diff IME-chrome
flood and the ASCII-only adb-shell text entry limit in one structural fix.

- android-ime-helper/: InputMethodService + build/package scripts on the
  existing helper-APK toolchain (javac+d8+aapt2+zipalign+apksigner).
- src/platforms/android/ime-helper.ts, ime-lifecycle.ts: install/version
  lifecycle (shared with the other two helpers via the new
  helper-package-install.ts), activation on session open, and on-device
  restore-hygiene (previous IME persisted to a device settings key so any
  daemon/state-dir can recover it; restored on close, daemon teardown, and
  daemon startup for orphans left by a crashed run).
- input-actions.ts: fill/type route through the helper's broadcast channel
  when active, unicode-safe; unchanged ASCII-shell fallback otherwise.
- doctor: new android-test-ime check flags a stuck helper IME with a
  copy-pasteable `adb shell ime set` remediation command.
- Gating: default-on for emulators, opt-in via `open --test-ime` on real
  devices.
- Dead-weight: rewrote the manual ADBKeyBoard workaround doc, dropped the
  now-provably-live skillgym non-ASCII eval case, updated the ASCII
  fallback's error message to point at the helper instead of dead-ending.

🤖 Generated with Claude Code

Co-Authored-By: Claude <noreply@anthropic.com>

* fix(#1201 review): permission-gate the IME receiver, fix CI, add opt-out

Addresses the independent review's blockers and should-fixes.

SECURITY (blocker 1): the text-injection receiver was RECEIVER_EXPORTED with
no gate — any co-installed app could inject text into the focused field while
the test IME was active. Fixed by requiring the WRITE_SECURE_SETTINGS sender
permission on the (in-process, dynamically-registered) receiver: adb shell
holds it, third-party apps cannot. The reviewer's suggested exported=false +
explicit-component approach was tried first but empirically breaks delivery on
API 36 (adb shell cannot reach a non-exported receiver there) — documented in
the helper README. Live-verified: a purpose-built rogue APK's broadcasts
(implicit and package-scoped, no permission) are silently dropped, field
unchanged; adb shell's bare broadcast still injects. Added
ime-helper-security.test.ts asserting the permission gate and that no
permissionless exported registration returns.

CI (blocker 2): (a) added `testIme` to integration-progress-model flag buckets
(Integration Tests was red on the unclassified flag). (b) mocked
resolveAndroidImeHelperArtifact in session-doctor-android / ime-lifecycle /
input-actions-test-ime tests so they no longer depend on android-ime-helper/dist
existing on disk (Coverage was red on a fresh checkout); verified by running
them with dist removed.

Should-fixes: added `--no-test-ime` to opt out on emulators (tri-state gating,
parser-tested); PR body's "byte-identical" claim corrected to size/CRC-match.

🤖 Generated with Claude Code

Co-Authored-By: Claude <noreply@anthropic.com>

* docs(#1201): pin the API-36 exported-receiver constraint in a comment

The RECEIVER_EXPORTED flag cannot express why it must stay exported. Add a
one-line note so a future hardening pass doesn't switch to RECEIVER_NOT_EXPORTED
and silently break the CLI (adb shell can't deliver explicit broadcasts to
non-exported components on API 36+; WRITE_SECURE_SETTINGS is the actual gate).

🤖 Generated with Claude Code

Co-Authored-By: Claude <noreply@anthropic.com>

* fix(#1201 review): harden IME restore lifecycle (blockers 1 & 2)

Blocker 1 — a failed restore no longer deletes the recovery value. restore now
reads back default_input_method after `ime set` and only clears the persisted
previous-IME record on a confirmed-successful restore; a failed set keeps the
value so a later retry / startup recovery / doctor remediation can still
un-strand the user off the helper IME.

Blocker 2 — startup orphan-recovery no longer overwrites/races user state.
It only restores when the device's CURRENT default IME is still our helper
(so a user who legitimately switched away is left alone), and skips any device
a live session in this process owns (the fire-and-forget startup vs. concurrent
`open` race — activate now marks the device active BEFORE the `ime set`, so any
recovery pass that could observe the helper active also observes the flag and
skips). Never persists the helper itself as the previous IME. activate also
verifies its own switch via read-back.

Exported ANDROID_IME_HELPER_SERVICE_COMPONENT so restore compares the active IME
without reading the packaged artifact from disk. Tests: failed-restore keeps the
value (+ later recovery succeeds), startup no-op when current != helper, startup
skips a live-owned device.

🤖 Generated with Claude Code

Co-Authored-By: Claude <noreply@anthropic.com>

* chore(#1201): delete unused ACTION_ENTER path, baseline test-only export seams

Rebased onto main (#1202 production-unused-exports gate). Two follow-ups:

- Deleted the unused ACTION_ENTER broadcast end-to-end (TS sendAndroidImeHelperEnter
  + its test, Java handler, README): nothing routes through it — `keyboard enter`
  uses the keyevent ENTER path — so the new production-exports gate flagged it as
  dead production code. Removed rather than grandfathered.
- Added the three legitimate test-only seams (resetAndroidImeHelperInstallCache,
  resetAndroidTestImeActivationCacheForTests, setAndroidTestImeActiveForTests) to
  fallow-baselines/production-unused-exports.json, matching how the sibling helper
  reset functions (resetAndroidMultiTouchHelperInstallCache, ...) are already
  grandfathered there.

🤖 Generated with Claude Code

Co-Authored-By: Claude <noreply@anthropic.com>

* fix(#1201): stop daemon-startup adb spawn on non-Android hosts (macOS Smoke)

Root cause of the red macOS Smoke shard (proven, not hand-waved): the
fire-and-forget restoreOrphanedAndroidTestImeOnDaemonStartup ran `adb devices`
at EVERY daemon startup, on every platform. GitHub macOS runners ship the
Android SDK, so this cold-started the adb server mid-replay and destabilized the
macOS System Settings replay timing — the failed job's cleanup shows
"Terminate orphan process: pid (N) (adb)"; main's green runs spawn no adb.

Fix: gate the startup orphan scan behind a host-side marker written in the
daemon state dir when a session activates the test IME (mirrors the
managed-web-browser orphan-cleanup `installed` gate). A host that never uses the
Android test IME — the macOS CI runner included — never writes the marker and so
never spawns adb at startup. The marker is cleared once nothing is left stuck.

Adds SessionStore.resolveStateDir(); tests: startup recovery does not scan adb
when no marker exists (+ marker cleared after a clean scan).

🤖 Generated with Claude Code

Co-Authored-By: Claude <noreply@anthropic.com>

* chore(#1201): suppress fallow class-member false-positive on state-dir accessor

CI's Fallow audit flags SessionStore.resolveDaemonStateDir as an unused class
member, but it is called via sessionStore.resolveDaemonStateDir() in
session-open.ts — fallow's class-member tracer just doesn't resolve a method
call sited inside a call argument. Renamed for clarity (avoids the collision
with config.ts's free resolveStateDir) and added the localized
fallow-ignore-next-line unused-class-member suppression.

🤖 Generated with Claude Code

Co-Authored-By: Claude <noreply@anthropic.com>

* fix(#1201 review): durable persist before switch + device-scoped recovery markers

Addresses devin-ai-integration's two P1 restore-safety blockers on 19cbce79d.

P1.1 — durably persist the restore target BEFORE the global IME switch.
writePersistedPreviousIme now checks the `settings put` exit code AND reads the
value back, returning a boolean. activate persists first and, if it cannot be
persisted, fails open to the existing input path WITHOUT switching — a rejected
`settings put` can no longer strand the user on the helper with no restore
target. Regression test added.

P1.2 — close the marker crash/offline blind spot. Recovery intent is now
recorded per device, BEFORE the switch (ordering: durable record -> marker ->
ime set), eliminating the post-switch/pre-marker crash window. Markers are
device-scoped and each is retained until that device is actually observed clean:
an offline/disconnected-but-stuck device keeps its marker and is recovered on
reconnect instead of being cleared because the current `adb devices` scan saw no
set-failed. Close-time restore clears only that device's marker (stateDir plumbed
through teardown/close). Tests cover the persist-failure, post-switch/pre-marker
crash, offline-then-reconnect, live-session-owned, and user-switched-away cases.

🤖 Generated with Claude Code

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-07-10 19:38:37 +02:00
devin-ai-integration[bot] 47134bf764 feat: add derived fail-open check:affected selector (#1195)
* feat: add derived fail-open check:affected selector

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor: simplify selector for complexity gate; add docs

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix: fail open on ambiguous non-source fixtures; guard catalog against real package.json/vitest.config

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor: use src/utils/exec.ts process helpers in check:affected runner

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(check:affected): SkillGym ownership, honest catalog, working-tree discovery

- Add SkillGym ownership for skills/ and test/skillgym/; stop short-circuiting
  their Markdown as docs-only (findings 2 & 4).
- Drop the fabricated GitHub 'SkillGym' job: it is a local-only gate, now
  localRunnable with no CI job, guarded by a workflow-existence self-test (3).
- Fold working-tree (staged/unstaged/untracked) state into local discovery and
  disable rename detection so both rename paths classify (1).
- Add run.test.ts entrypoint regressions (real diff/status/rename discovery,
  --run order/skip/stop-on-failure).

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(check:affected): union staged + unstaged diffs so they cannot cancel

A single `git diff HEAD` nets index against working tree, so a staged add
and an unstaged delete of the same file cancel and hide it. Collect
`--cached` (staged) and unstaged diffs separately and union them; add a
cancellation regression test.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(check:affected): cover required suite gates

* refactor(check:affected): delegate tests to vitest

---------

Co-authored-by: Michał Pierzchała <thymikee@gmail.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-07-10 17:53:52 +02:00
devin-ai-integration[bot] db492cbaed test(output-economy): add routine-workflow output-behavior oracle (#1190)
* test(output-economy): add routine-workflow output-behavior oracle

Add a deterministic routine-workflow measurement (#1180) that pairs
response bytes with follow-up behavior: fallback-observation count,
retry count, and whether an actionable failure preserves the session.
Refs chain across one recorded checkout session and counts derive from
the real formatters, so dropping settled-diff refs, the unchanged-
interactive tail, or a recovery handle fails the suite. Adds a matching
non-gating help-conformance next-command case. Response defaults
unchanged.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(output-economy): make routine-workflow ref-surfacing depend on rendered output

Address review on #1190:
- Drop the raw mutation-confirm/failure MCP data samples that leaked e4/e5
  and mislabeled the surface; e4/e5 now surface only from the rendered CLI
  settled-diff, so dropping added refs genuinely raises the fallback count.
- Track the failure once as its projection-invariant normalized payload
  (workflow.failure.shared.json) instead of duplicate cli/mcp raw copies.
- Reuse the shared REF_TOKEN_PATTERN from economy-metrics.ts.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(output-economy): reuse shared fixtures in routine-workflow oracle

Address review finding #2 on #1190: chain the workflow onto the shared
per-surface fixtures instead of copy-pasting them.
- orient/recheck reuse SNAPSHOT_RESULT + SNAPSHOT_DAEMON_RESULT; the
  first mutation reuses SETTLE_ADDED_REF_RESULT, so session identity and
  ref generations come from ./fixtures.ts and the two suites cannot drift.
- Only the genuinely workflow-specific pieces remain local: the unchanged
  recheck, a tail retargeted onto a settled-diff ref (SETTLE_TAIL_RESULT
  taps an unsurfaced @e6 and can't chain), the in-session timeout failure,
  and its recovered retry. routine-workflow.ts drops ~110 LOC.
- Rendered-output ref guard preserved: @e5 (settled diff) and @e7 (tail)
  surface only from formatter output; recovery semantics unchanged.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Michał Pierzchała <thymikee@gmail.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-07-10 15:27:24 +02:00
devin-ai-integration[bot] ae74c51abd chore: add agent-efficiency regression guards (#1174)
* chore: ratchet architecture dependency graph

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test: ratchet agent-facing output economy

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* feat: derive command navigation explanations

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix: keep efficiency checks fallow-clean

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(layering): enforce back-edge ceiling monotonicity and cover root src files

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* ci(layering): pin back-edge-ceiling ratchet to PR merge-base

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(output-economy): baseline-independent actionability floors, policy-derived error, like-for-like screenshot surfaces

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(explain): resolve true CLI aliases, canonical usage, and derived owners

Surface true CLI aliases from parser normalization (long-press, metrics,
tap, launch, relaunch) distinct from catalog keys, preserving implied-flag
semantics (relaunch => open --relaunch). Extract the canonical single-line
usage builder to src/utils/cli-usage.ts so schemas without usageOverride
include positionals and flags. Replace guessed handler paths with a
completeness-checked daemon-route owner map keyed by the closed
DaemonCommandRoute union, fixing silently-dropped non-kebab routes
(reactNative, recordTrace) and generic dispatch. Add table-driven coverage
for aliases, synthesized usage, split-family/route-variant/dispatch owners,
structured output, and explain:command CLI exit/stdout/stderr.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix: enforce exact ratchets and compact command explain

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix: colocate command ownership metadata

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix: bind daemon owners to production routes

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix: preserve generic dispatch bundling

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix: enforce monotonic output budgets

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Michał Pierzchała <thymikee@gmail.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-07-10 11:54:09 +02:00
Michał Pierzchała 983625fc5d feat: fix codex runner, add --override-doc grading, port skillgym quiz cases to help-conformance bench (#1176)
* feat: fix codex runner, add override-doc grading, port skillgym quiz cases

The 2026-07-09 evaluation of scripts/help-conformance-bench.mjs found it
structurally right but broken for the codex runner (two bugs), thin on
coverage (4 cases), sequential, and unable to grade a draft help rewrite
without a rebuild.

- Fix runCodex: (1) codex exec reads stdin until EOF when not attached to
  a TTY, and execFile never closes the child's stdin, so every codex call
  hung until RUN_TIMEOUT_MS with empty output — close stdin right after
  spawn. (2) `-o outFile` writes the same final JSON that codex also
  prints to stdout, so concatenating both produced two back-to-back JSON
  objects that broke every JSON.parse candidate and silently zeroed
  extractCommands() — prefer the clean -o payload, fall back to stdout
  only when it's empty.
- Add `--override-doc <topicId>=<path>` (repeatable): loads a topic's
  text from a file instead of shelling out to
  `node bin/agent-device.mjs help <topic>`, so a draft help rewrite can
  be A/B graded with zero rebuild.
- Port three cases from
  test/skillgym/suites/agent-device-smoke-suite.ts
  (settle-diff-is-observation, sample-output-settled-diff-next-target,
  sample-output-not-settled-needs-observe) as self-contained
  "next-command quiz" cases, generalizing the scorer to support regex
  matchers/forbidden patterns alongside the existing named expectations.
  Fixture output text matches the CURRENT settle rendering in
  src/commands/interaction/output.ts, including the "unchanged
  interactive (N):" tail added by #1167/#1172.
- Parallelize the runner x case matrix with a concurrency cap
  (HELP_BENCH_CONCURRENCY, default 4); results still print in the
  original matrix order.
- Extend test/skillgym/README.md's existing pointer to this bench with
  the new flags.

Validated with real LLM calls (both runners, all 7 cases, 14 calls,
~$0.25 total): 13/14 pass; the one fail (claude-haiku-4-5 on
dogfood-mode) is a genuine model miss (returned an empty command plan
asking for the app name instead of committing to a generic plan), not a
bench bug. `--override-doc` demonstrated live: stripping the dogfood
doc's evidence-command examples regresses codex:gpt-5.4-mini from 3/3 to
2/3 on the same case, showing the flag both loads and changes grading.

* fix: apply live-doc post-processing to --override-doc, fail fast on bad overrides

Review findings on the initial version (all reproduced):

- HIGH: an override for the --help:first30 doc id skipped the live path's
  firstLines(text, 30) cap, so a 49-line draft leaked lines 31-49 into the
  prompt — grading content a live run never shows, on the doc id every case
  uses. loadDoc now splits source (live shell-out vs override file) from
  post-processing, and the post-processing applies to both, so an override
  differs ONLY in where the text comes from.
- MEDIUM: an --override-doc topic id no selected case uses was silently
  ignored (exit 0, real doc graded). Now fails fast listing the valid doc
  ids for the selection.
- LOW: a missing override file threw a raw ENOENT stack trace; expected
  failures now print one clean Error line. Added --help usage text that
  documents last-wins semantics for repeated same-topic overrides and the
  post-processing parity.

Guard tests (scripts/__tests__/help-conformance-bench.test.ts, wired into
the unit-core vitest project by explicit path): a 49-line fixture whose
prompt must keep line 30 and drop line 31, unknown-topic fail-fast with
valid ids listed, clean no-stack error for a missing file, and last-wins
for repeated overrides. All spawn the script in --dry-run with every
required doc overridden, so they need no LLM calls and no built CLI.

Live re-validation: a 33-line override of --help:first30 whose lines
31-33 instruct the model to emit a sentinel command; neither
claude-haiku-4-5 nor codex:gpt-5.4-mini emitted it (both scored 4/4,
matching the live-doc baseline), proving the cap applies end-to-end.
2026-07-10 09:06:58 +02:00
Michał Pierzchała 888984169b fix: make record app-scoped by default (#1163)
* fix: reject recording for failed iOS simulator session

* fix: make record app-scoped by default
2026-07-08 21:48:35 +02:00
Michał Pierzchała 2717047b86 fix: normalize iOS simulator screenshot density (#1160)
* fix: normalize iOS simulator screenshot density

* fix: avoid density metadata after screenshot downscale

* fix: satisfy screenshot density CI gates

* fix: harden screenshot metadata collection

* refactor: centralize screenshot density policy

* refactor: reuse screenshot density support check
2026-07-08 21:05:24 +02:00
Michał Pierzchała f18d0b2e92 fix: improve settle observation guidance (#1154) 2026-07-08 17:12:19 +02:00
Michał Pierzchała 7f61df30ae feat: add TV remote command (#1147)
* feat: add TV remote command

* feat: improve TV remote ergonomics

* test: cover tv-remote provider scenario

* fix: preserve focused Android TV nodes

* docs: tighten PR description guidance

* fix: remove d-pad command alias

* docs: clarify tv-remote hold syntax

* feat: add tv-remote longpress CLI sugar
2026-07-08 10:59:48 +02:00
Michał Pierzchała ef4b66d4dc test: remove slow-test ratchet pins (#1143) 2026-07-07 14:46:02 +02:00
Michał Pierzchała 5c5fa012f7 feat: --settle returns the settled diff in the interaction response (#1101) (#1106)
* feat: --settle returns the settled diff in the interaction response (#1101)

press/click/fill/longpress --settle executes the action, waits for the UI
to go quiet (wait stable's loop, shared via stable-capture.ts), and returns
the settled diff vs the pre-action tree in the same response — one round
trip instead of the interact -> observe pair.

- payload: changed lines only (bounded), summary counts, added-line refs,
  refsGeneration; best-effort (settled:false + hint on never-quiet content,
  never an action failure); --verify shares the settle captures
- ref issuance: the settled tree becomes the session snapshot; a
  diff-carrying settle response clears snapshotRefsStale and the MCP layer
  merge-only re-pins added-line refs at the settle generation
- grammar: --settle + --settle-quiet <ms> + --timeout <ms> (flag-sourced
  descriptor budget with new envelope:'widen' semantics mirroring wait)
- ADR 0011: new settleObservation guarantee classified on every path with
  contract scenarios per enforced/delegated cell

* test: give the two contention-flaky doctor scenarios explicit budgets

The doctor provider scenarios sit at ~5s of real daemon-harness work on a
loaded host and flake at vitest's 5s default during full-suite runs (the
known contention flake AGENTS.md documents). Same in-file precedent as the
Metro-probe scenario's 10s budget.

* fix: move SettleParams to contracts to satisfy the layering DAG

daemon/handlers/interaction-flags.ts imported the type across the
daemon -> commands boundary (R2 commands-floor). The tuning params are
part of the interaction contract like SettleObservation, so they live
in contracts/interaction.ts and both layers import from there.

* feat: keep settle diffs content-first — drop Key nodes, added lines win the cap

Bluesky dogfood: a fill that summons the iOS keyboard spent 49 of the 80
capped diff lines spelling out QWERTY keys, and a screen transition with
269 removals could starve out the added lines entirely. Key-type nodes
are now filtered from both diff sides (the [keyboard] container line
still signals presence), and under truncation added lines — the ones
carrying fresh refs — win slots over removals.

* docs: state the core loop in the top-level help starting point

Benchmarked with headless haiku/sonnet agents given only --help: both
models skipped the help-workflow pointer and started with plain
snapshot (38KB payloads they then had to re-read from files). One
core-loop line at the starting point is what teaches snapshot -i and
--settle to models that never read a second help page.

* fix: preserve settle digest refs for mcp

* fix: reduce settle fallow complexity

* fix: surface settle output in CLI text

* fix: complete settle handling for longpress

* refactor: localize daemon timeout envelopes

* refactor: deepen post-action observation

* refactor: centralize post-action observation planning

* refactor: derive settle capability from descriptors

* refactor: trim settle descriptor helpers
2026-07-06 20:18:44 +02:00
Michał Pierzchała 54f6d45b32 refactor: extract host process primitives (#1134) 2026-07-06 19:01:32 +02:00
Michał Pierzchała 7475415ac5 build: strip runner unit-test blocks from package (#1128)
* build: strip runner unit-test blocks from package

* ci: fix package size and fallow checks

* fix: resolve packaged recording scripts

* fix: keep recording script resolver internal
2026-07-06 12:52:05 +02:00
Michał Pierzchała f9721e7e8b test: split the Android platform test aggregation and share the scripted adb stub (#1103)
* test: split the Android platform test aggregation and share the scripted adb stub

AGENTS.md names the platform index.test.ts aggregations as offenders to
shrink opportunistically; this splits the 2,735-line Android one along
its (already well-factored) source modules, every test moved verbatim
(92 tests before and after):

- ui-hierarchy.test.ts (22): parseUiHierarchy/androidUiNodes
- app-lifecycle-install.test.ts (13): install/resolve/infer/launch
  component parsing
- app-lifecycle-open.test.ts (19): open/close, deep links, launch args,
  TV category, fallback resolve-activity
- input-actions.test.ts (11): type/fill/swipe/scroll/rotate
- settings.test.ts (14): appearance/clear-app-state/fingerprint/
  permissions
- notifications.test.ts (2), app-parsers.test.ts (1)
- keyboard state/dismiss tests (10) appended to the existing
  device-input-state.test.ts

Consistency fix folded in: the file carried a local withMockedAdb fork
because it needs scripted per-subcommand adb responses, which the shared
arg-recorder helper cannot express. The fork now lives in
src/__tests__/test-utils/mocked-binaries.ts as withScriptedAdb next to
withMockedAdb, and hands each call a fresh copy of the shared
ANDROID_EMULATOR fixture.

The copy matters: the Android TV test mutated the callback's device
(device.target = 'tv'), which the old per-call object literal absorbed
silently. With a shared fixture that mutation leaked into the next test
and flipped its launch to LEANBACK. The helper now clones per call and
the TV test builds { ...device, target: 'tv' } instead of mutating.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqeW8sA2ZnvnftdvpCqFMS

* test: serialize the scripted-adb group and repoint its slow-test pins

Review follow-up for the android index.test.ts split: the monolith
implicitly serialized the env-mutating adb-stub tests (PATH,
AGENT_DEVICE_TEST_ARGS_FILE) in one worker, and the split let vitest
run them across parallel files. Make the contract explicit:

- new android-adb vitest project runs the six scripted-adb test files
  in a single fork (singleFork), keeping the pre-split execution
  semantics; ui-hierarchy and app-parsers stay in the parallel unit
  project (pure parsing, no env mutation)
- test/test:unit scripts run both projects
- the five slow-test ratchet pins that referenced index.test.ts keys
  now point at the split file names, so the pinned real-time offenders
  keep their exemption instead of failing at 2x budget under load; the
  reporter's own pinned-key fixture updated to match

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqeW8sA2ZnvnftdvpCqFMS

* test: use vitest 4 android adb serialization

* docs: update unit project readiness guidance

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-07-06 11:23:03 +02:00
Michał Pierzchała 2557670193 test: slow-test ratchet and speed rules from measured experiments (#1099)
* test: slow-test ratchet, budget-derived emulator poll, speed guidance from experiments

Measured (2026-07-04, full unit suite: 340 files / 3,210 tests / 48s wall):
wall clock was bounded by the slowest FILE (44.6s android monolith at ~7x
file-level parallelism), and the slowest tests were sleeping through real
production budgets (10.8s proving 'times out' by waiting the constant out,
8s emulator polls at 1Hz, real retry backoff). Two config experiments
rejected with data: --no-isolate exploded the suite to 205s (module state
thrashes across files sharing workers) and --pool=threads changed nothing.

- scripts/vitest-slow-test-reporter.ts: the slow-test ratchet. Unit budget
  2.5s / integration 15s; failure at 2x budget (the band between reports
  without failing so host-load variance cannot make the gate cry wolf);
  36 pinned offenders, exact keys, ratchet-only pin (tracking #1098).
- waitForAndroidEmulatorByAvdName: poll cadence derives from the caller's
  budget (min 1s, floor 50ms, ~timeout/20) — devices.test.ts 25.6s -> 2.8s
  (9x) in isolation, and short-budget production calls stop sampling at
  1Hz against small budgets.
- vitest.config: slowTestThreshold 500 for local visibility; reporter
  wired; isolation/pool decisions documented with the measurements.
- docs/agents/testing.md 'Speed rules' + AGENTS.md testing bullet: the
  three conversion patterns in preference order (budget-derived cadence,
  budget-wiring assertion, fake clocks), the no-seam constraint, and the
  file-granularity Amdahl argument that makes the monolith test split a
  wall-clock fix, not just navigation.

* fix: fallow findings on the slow-test gate — import edge, factory reporter, unit tests

The string-path reporter wiring read as a dead file (fallow cannot see
vitest's reporter loading); the config now imports the factory, making
the edge real and type-checked. The class shape tripped the
unused-class-members rule (framework callbacks are invisible to
reference analysis) — converted to a factory returning the Reporter
object, with the classification and rendering logic extracted as pure
exported functions. Those functions now carry their own unit tests
(budget bands, integration budgets, pin matching, warn-vs-fail
rendering), which also grounds the CRAP estimate in real references.
Canary re-verified: unpinned 5.2s sleeper fails the run with exit 1;
clean runs exit 0.
2026-07-04 19:13:06 +02:00
Michał Pierzchała 664b713130 feat(interaction): opt-in --verify evidence for press/click/fill (#1064)
* feat(interaction): opt-in --verify evidence for press/click/fill (#1047)

Adds an opt-in --verify flag that returns cheap post-action evidence
(foregroundApp, nodeCount, interactiveNodeCount, digest,
changedFromBefore) instead of requiring a full follow-up snapshot to
confirm a mutating command had an effect. The digest hashes the
(type, label, identifier) multiset of an interactive-only capture,
order-independent so it doesn't flip on harmless re-ordering; the
node tree is never serialized back to the client, only the digest and
counts. Default behavior (no --verify) is byte-identical to today.

Implements the approved design from the #1047 issue comment:
- src/utils/ax-digest.ts: new standalone digest module.
- Pre-action digest reuses the snapshot the resolution path already
  captures for ref/selector targets (zero extra cost); point targets
  opt into one extra baseline capture only when --verify is set.
- Post-action: one interactive-only capture through the same capture
  helper, digested and discarded.
- --verify threaded through the CLI flag schema, MCP input schema,
  interactionResultExtra allowlist, and MCP output schemas, following
  the same plumbing as --double-tap and the #1040 targetHittable
  precedent.
- The native-ref/direct-iOS-selector fast paths are skipped when
  --verify is set, since they bypass the resolution/capture path
  evidence depends on.

* test: cover press --verify with a provider scenario; classify the flag

CI's architecture-progress gate failed on 1 unclassified public flag.
--verify drives real device captures, so it belongs in the
device-observable list backed by an actual provider scenario rather than
the intentionally-outside bucket: the new scenario asserts evidence
(changedFromBefore, digest, nodeCount) on press @ref --verify, that the
verify capture's tree is never serialized into the response, and that
the transcript completes (snapshot -> tap -> verify snapshot; the @ref
path reuses the session snapshot as its baseline, so no extra
resolution capture entry exists).

* fix: type the verify scenario transcript entries

* fix: include interaction extras in the fill @ref response branch

The ref branch of dispatchFillViaRuntime rebuilt responseData from
backendResult/coordinates, dropping interactionResultExtra(result) — so
fill @ref --verify returned no evidence even though the post-action
capture ran (live E2E gap found in PR #1064 review). Spreading the
extras also gives fill @ref the same ref/refLabel/selectorChain (and
conditional targetHittable/hint) fields press @ref already returns.

Adds daemon tests for fill @ref --verify evidence and for the no-verify
path staying evidence-free with no post-action capture.
2026-07-04 10:48:54 +02:00
Michał Pierzchała fcb7e32f1c fix: clarify clean-xcuitest output (#1018)
* fix: clarify clean-xcuitest output

* fix: simplify clean-xcuitest formatter

* fix: clarify clean-xcuitest failure output
2026-07-02 13:53:42 +02:00
Michał Pierzchała b6128c0088 docs: retire plans/perfect-shape.md — roadmap complete (#1003)
* docs: retire plans/perfect-shape.md — roadmap complete

The perfect-shape roadmap (two-registry thesis: CommandDescriptor +
PlatformPlugin, typed-result spine, folder DAG + layering lint, agent-cost,
and the Apple apple+appleOs platform model with a non-breaking leaf wire) is
substantively complete and merged. Per its own §5 retirement note, the durable
decisions now live in ADR-0008 (command descriptor) and ADR-0009 (Apple/AppleOS),
and current-state terms in CONTEXT.md; this removes the last plan file.

- Delete plans/perfect-shape.md (plans/ is now empty and gone).
- CONTEXT.md: add "Architecture (perfect-shape refactor, completed 2026-07)"
  end-state summary plus a "Deferred / next-minor" note (Phase 2c client-types
  narrowing, b.3 recording/providers facets, strict DAG back-edge inversion,
  legacy alias drops) so nothing is lost.
- Repoint every remaining perfect-shape.md/§ reference (ADRs 0003/0008/0009,
  ci.yml, scripts/layering/check.ts, and the platform-plugin/apple comments)
  to ADR-0008/0009 or CONTEXT.md. No dangling references remain.

Docs/comment-only; tsc, oxlint, oxfmt, and the layering DAG check all pass.

* docs: repoint dangling perfect-shape section refs before retiring the roadmap

Removing plans/perfect-shape.md left three comments citing bare section numbers
with no surviving target. The rationales are already inlined, so drop the numbers
(and point the do-not-flatten note at the durable ADR):
- src/platforms/apple/plugin.ts: `(§7)` -> "do-not-flatten; see docs/adr/0009".
- src/core/interactors/register-builtins.ts: "the §5.1 ... sketch" -> "an ... sketch".
- scripts/layering/check.ts: drop `(§5.5 ...)`, keep the inline "re-export barrels only".
2026-07-02 07:19:37 +02:00
Michał Pierzchała a3e967526a refactor: rename ios-runner -> apple-runner (#981) (#996)
Finish the cosmetic ios-runner -> apple-runner rename now that the
top-level XCTest runner is the OS-agnostic Apple engine
(iOS/iPadOS/tvOS/macOS/visionOS from one Xcode project).

Cosmetic only, no behavior change:
- git mv ios-runner/ -> apple-runner/ (AgentDeviceRunner, README, RUNNER_PROTOCOL)
- Update repo project-path consumers: build-xcuitest-apple.sh, package.json
  files globs, .fallowrc.json, write-xcuitest-cache-metadata.mjs,
  runner-xctestrun.ts fingerprint/project paths, recording overlay + test,
  daemon-client-timeout kill pattern, setup-apple-replay hashFiles glob,
  ci.yml swift-compat scan, AGENTS.md.
- Rename runtime home cache/derived/lease dir default
  ~/.agent-device/ios-runner -> ~/.agent-device/apple-runner (build script,
  package/clean scripts, runner-xctestrun RUNNER_DERIVED_ROOT, runner-lease,
  runner-contract hint, cli-help/commands.md docs) and the tests asserting it.
- Rename OS-agnostic runner symbols: runIosRunnerCommand ->
  runAppleRunnerCommand, prewarmIosRunnerCache -> prewarmAppleRunnerCache,
  createIosRunnerCachePrewarmOnColdBoot / createIosRunnerCacheColdBootPrewarmForOpen
  -> createAppleRunner* (+ call sites, type aliases, test mocks).

Intentionally left as ios-runner (out of scope / would change behavior):
- prepare ios-runner CLI subcommand (user-facing command name)
- AGENT_DEVICE_IOS_RUNNER_* env var names and .tmp/ios-runner-derived CI values
- ios-runner-prebuilt cache-key-prefix, ci.yml job id, workflow/ADR filenames
- agent-device-ios-runner-<version> release artifact basenames

Part of #972 (Phase 3 - Apple PlatformPlugin).
2026-07-01 14:24:48 +02:00
Michał Pierzchała d21b8ce32e feat: add doctor command (#883)
* feat: add doctor command

* fix: reduce doctor command complexity

* fix: classify doctor integration flags

* fix: simplify doctor setup

* refactor: split doctor checks

* fix: simplify doctor check set

* fix: include stopped android avds in devices

* fix: report doctor device inventory

* refactor: reuse device inventory selectors

* fix: summarize doctor inventory by platform

* fix: show metro cwd in doctor

* refactor: simplify metro doctor lookup

* fix: update doctor imports after apple consolidation

* feat: make doctor Metro probe controllable and surface hidden toolchain failures

Two gaps found while verifying the doctor command on a real environment:

- Metro host/port were uncontrollable from the CLI: --metro-host/--metro-port
  were rejected by allowedFlags, and readDoctorOptions only read them from
  req.runtime (populated by remote/connection profiles, never a plain CLI
  flag). The Metro check's own hint told users to 'pass the correct
  --metro-host/--metro-port', which did not exist. Declare the flags and read
  them from req.flags (runtime kept as fallback) so the probe can target any
  endpoint, e.g. from outside an RN/Expo project directory.

- A broken per-platform toolchain was silently hidden: readDoctorDeviceInventory
  dropped inventory failures whenever any other platform returned devices, so a
  broken Xcode or Android SDK still reported a green 'pass'. Keep the failures
  and surface each as a warn (device-<platform>) when other platforms have
  devices; scoped --platform runs stay quiet.

* fix: align doctor CI expectations

* feat: extend doctor preflight checks

* fix: keep doctor checks within ci gates

* fix: simplify doctor metro surface

* refactor: trim doctor bundle impact

* fix: restore useful doctor diagnostics

* refactor: reuse doctor output helpers

* refactor: share device inventory grouping

* refactor: keep doctor focused on preflight checks

* refactor: simplify doctor toolchain probes

* fix: keep scoped simulator hint generic

* fix: clarify doctor Xcode selection context

* fix: recognize provider scope in remote doctor

* fix: address doctor review gaps

* fix: keep doctor metro checks inferred
2026-07-01 14:24:32 +02:00
Michał Pierzchała a707348b9b feat: add hosted WebDriver provider support (#948)
* feat: add cloud webdriver artifacts

* fix: clean up local session after provider release failure

* fix: tag cloud webdriver provider requests

* feat: connect hosted webdriver providers

* docs: document hosted provider credentials

* refactor: tighten cloud webdriver provider internals

* refactor: consolidate cloud webdriver provider definitions

* refactor: collapse hosted webdriver runtime wrapper

* refactor: reduce cloud webdriver smell surface

* docs: clarify hosted provider interfaces

* fix: avoid regex slash trimming in webdriver urls

* docs: rename hosted providers to device clouds

* fix: align provider profile imports with remote modules

* fix: skip local android recovery for provider devices

* test: classify cloud provider integration flags

* fix: close active cloud connection session

* test: cover provider disconnect cli flow

* fix: make cloud webdriver sessions launchable

* fix: align cloud webdriver input gestures

* fix: avoid keyboard input during cloud scroll

* fix: constrain cloud webdriver scroll gestures

* refactor: isolate cloud webdriver scroll frame

* refactor: deduplicate cloud webdriver helpers

* refactor: tighten cloud webdriver action types

* fix: harden cloud webdriver release

* fix: polish provider disconnect diagnostics

* refactor: group connection profile helpers

* fix: repair rebased internal paths

* fix: satisfy cloud webdriver CI guards
2026-07-01 13:01:47 +02:00
Michał Pierzchała 2e0879a260 fix: exempt sdk/ public barrels from R3 platforms-seam (unblock main) (#987)
#984 added the R3 platforms-seam layering rule and #986 moved the public SDK
entry barrels into src/sdk/; the two merged mutually inconsistent, so the
Layering Guard is failing on main. sdk/ are public re-export barrels that
legitimately expose platform symbols and are off the CLI cold path (not imported
by bin.ts), so they are a correct R3 exemption alongside core/interactors and the
daemon server — not a cold-start regression.
2026-07-01 10:21:03 +02:00
Michał Pierzchała 3d70943550 feat: enforce import-direction DAG (Phase-5 layering lint) (#984)
Generalize the inline CI "Layering Guard" grep into a structured
import-direction lint (scripts/layering/check.ts) over the resolved
import graph, per plans/perfect-shape.md §5.5.

The full target DAG (kernel ◄ platforms ◄ core ◄ commands ◄ {cli,
client, daemon/server}; client ◄ daemon/client) is only partly realized
— the client/remote/metro extraction, the daemon/server split, and the
utils dissolution are still pending Phase-5 moves, so the tree still
holds legitimate back-edges (platforms→core, commands→cli, utils→*).
Enforcing the whole DAG today would need a mass import rewrite that
Phase 5 defers. The lint therefore enforces the three invariants the
completed moves (kernel/, daemon/client/) already guarantee and that are
green today:

  R1 kernel-sink      — nothing under src/kernel/ imports another zone,
                        except the one type-only kernel→contracts re-export.
  R2 commands-floor   — nothing below the command surface (kernel,
                        platforms, core, daemon) imports src/commands/.
                        Generalizes the former guard (daemon + platforms).
  R3 platforms-seam   — platforms/ is statically imported only at the
                        core interactor seam (src/core/interactors/) and by
                        the daemon server; elsewhere use a dynamic import()
                        or a type-only import, preserving CLI cold-start.

Dynamic import('../platforms/*') and `import type` stay allowed.

Fixes the three pre-existing R3 violations by converting static
platforms value imports to dynamic imports (all in already-async call
sites, behavior-preserving and cold-start-improving):
  - src/client/client.ts        debug.symbols → lazy symbolicateCrashArtifact
  - src/cli/commands/web.ts      setup/doctor → lazy agent-browser-tool
  - src/core/dispatch-interactions.ts  runner-sequence → lazy (matches the
                                       file's own dynamic-import pattern)

Wire the check into the Layering Guard CI job and add a check:layering
package.json script (also folded into check:tooling). scripts/layering/**
is excluded from fallow (untested CI script, like scripts/perf/**).
2026-07-01 10:02:45 +02:00
Michał Pierzchała 26ac865c63 refactor: consolidate Apple platform internals (#968) 2026-06-30 21:30:46 +02:00
Michał Pierzchała edc8dd059b ci: automate iOS runner request-count gate for the Apple runner unwind (Phase 3 step c prep) (#966)
Replaces the manual "run with --debug, hand-count the runner phases" check with
an automated, committed assertion so the Phase 3 step (c) runner relocation (and
future runner refactors) can prove byte-identical runner request behavior.

- src/daemon/runner-request-count.ts: pure, unit-testable counter. Parses the
  daemon --debug diagnostics ndjson and counts the iOS-runner round-trip phases,
  plus baseline parse/compare logic. Owns RUNNER_ROUND_TRIP_PHASES as the single
  source of truth, now imported by request-router.ts (was a local const) so the
  in-process cost graft and the external counter never drift.
- src/daemon/__tests__/runner-request-count.test.ts: 13 unit tests over synthetic
  ndjson fixtures (tolerant parse, counting, baseline parse/compare). Run in the
  normal unit suite; no hardware.
- scripts/runner-request-count/: assertion harness (run.ts) + committed baseline
  (expected-counts.json). Drives the existing smoke-ios replay scenario with
  --debug in an isolated --state-dir, counts runner round-trips from daemon.log,
  and asserts against the baseline. --update regenerates the baseline. Infra
  hiccups are inconclusive (don't fail); only a real count drift fails.
- .github/workflows/ios.yml: new "Assert iOS runner request count" step in the
  smoke-ios job, reusing the booted simulator.
- package.json: `validate:runner-count` script. .fallowrc.json: harness entry.

The baseline ships unarmed (established=false); the harness records observed
counts (printed + uploaded as a test/artifacts artifact) without failing, so the
maintainer arms it once from a real CI run.
2026-06-30 17:26:01 +02:00
Michał Pierzchała 65227c6719 refactor: absorb CLI parser into cli/parser/ — Phase 5 (#958)
Move the CLI argument/flag/help parser out of utils/ into a dedicated
src/cli/parser/ folder, per plans/perfect-shape.md §5.5 (utils/ hosts a 3k
CLI parser among its buried subsystems).

Files moved (3): args, cli-flags, cli-help (args->cli-help intra-set import
stays relative).

- git renames; importers repointed via the resolve-based codemod
  (64 importers; staying-utils/kernel deps recomputed to ../../)
- no public-export/rslib impact
- update scripts/integration-progress-model.ts import + fallow-baselines/
  health.json keys (args incl. :high impact variant)

Behaviorless path codemod. typecheck/lint/format/build/tests green;
integration-progress model still runs.
2026-06-30 13:56:04 +02:00
Michał Pierzchała 43d0a639d0 feat: leveled response views + --level knob with snapshot digest — Phase 4 (#942)
* feat: leveled response views + --level knob, with a snapshot digest — Phase 4

Add the agent-cost leveled-response system: a responseLevel knob
(digest | default | full) plumbed end to end behind a global --level flag
(mirroring --cost), and a per-command ResponseView registry applied in the
router on the success path.

- contracts: RESPONSE_LEVELS/ResponseLevel + meta.responseLevel + boundary
  schema whitelist. Plumbing mirrors --cost: cli-flags FlagDefinition +
  GLOBAL_FLAG_KEYS, AgentDeviceClientConfig + overrides, buildClientConfig,
  buildMeta. ResponseLevel exported from the public root.
- src/daemon/response-views.ts: the ResponseView registry. Seeds the snapshot
  digest — the full node tree (the dominant token sink) collapses to
  { nodeCount, refs: first 12 hittable/non-occluded refs with labels } plus the
  cheap top-level signals (truncated/visibility/snapshotQuality). full returns
  today's shape (nothing richer is computed yet).
- router graft (applyResponseLevelView + applyAgentCostGrafts): composes with
  the existing cost block. With responseLevel default (or unset) AND no
  registered view AND no --cost, the original response is returned UNCHANGED —
  byte-identical to today (Maestro .ad recompare safe). cost.nodeCount reads the
  original node tree so it stays accurate even after a digest.

Tests: snapshot view unit test (digest filters hittable/occluded, drops the
tree, keeps cheap signals; default/full passthrough); router graft test via an
injected view (default identity byte-identical, digest applies, full passthrough,
digest+cost composition, unregistered-command passthrough, boundary parse).

Verified: tsc, oxfmt + oxlint --deny-warnings, fallow audit clean, rslib build,
Layering Guard empty, 1106 daemon/contracts/client tests pass (incl. the
existing cost/typed-error grafts after the restructure).

* fix: repoint MCP output-schemas import to kernel/device (rebase fixup)

The kernel move (#940) deleted src/utils/device.ts; #941's
command-output-schemas.ts (merged after #940's codemod ran) still imported the
old path. Same one-line fix as #943; de-dups once that lands.

* fix: re-classify responseLevel flag in integration-progress model

The --level/responseLevel flag is a diagnostics/output flag (not device-
observable), classified in the exclusion bucket alongside --cost. (Lost in an
earlier rebase; re-applying.)
2026-06-30 07:53:28 +02:00
Michał Pierzchała 6ae0612ebc fix: clean up maestro test reporter output (#935)
* fix: clean up maestro test reporter output

* chore: enable expo build disk cache

* refactor: simplify replay progress detail formatting

* fix: surface replay runner recovery hints

* fix: prioritize ios runner recovery hint

* fix: avoid trailing punctuation in runner state hint

* fix: keep internal cleanup scripts out of runner hints

* chore: remove redundant maestro test app open flag

* fix: make test app maestro flow self-contained

* fix: simplify maestro test duration output

* fix: refine maestro test summary output

* fix: dim maestro live progress counters

* test: clear test app state before maestro flow

* test: update maestro reporter progress expectations

* chore: remove maestro app open flag handling

* fix: apply maestro reporter cleanup to default reporter
2026-06-29 21:02:30 +02:00
Michał Pierzchała 8cc6bb8b92 feat: add replay test reporters (#936)
* feat: add replay test reporters

* feat: support custom replay test reporters

* fix: satisfy reporter CI guards

* refactor: modularize replay test reporters

* refactor: trim replay reporter parsing

* refactor: simplify custom replay reporters
2026-06-29 20:37:41 +02:00
Michał Pierzchała 54178319d8 perf: reduce iOS simulator screenshot overhead (#927) 2026-06-29 15:53:19 +02:00
Michał Pierzchała 9dc07cc56e perf: reuse Apple runner cache across version bumps (#900)
* perf: reuse Apple runner cache across version bumps

* perf: remove unused Apple runner symbols

* perf: keep Swift runner unit tests out of runtime builds

* perf: skip Apple runner asset catalog in runtime builds

* perf: use concrete simulator for xcuitest script builds

* perf: show cold Apple runner startup progress

* perf: prewarm Apple runner cache during simulator boot

* refactor: dedupe Apple runner option plumbing
2026-06-29 15:43:15 +02:00
Michał Pierzchała 62f923dec4 feat: opt-in agent-cost wallClockMs behind --cost — Phase 4 slice 1 (#922)
* feat: opt-in agent-cost wallClockMs behind --cost

Add per-command wall-clock latency as a purely additive, opt-in response
field (cost.wallClockMs) gated behind a new global --cost flag.

The flag plumbs end to end mirroring --debug: cli-flags definition +
GLOBAL_FLAG_KEYS, AgentDeviceClientConfig/overrides, buildClientConfig,
buildMeta (meta.includeCost), the DaemonRequestMeta contract, and the
boundary parse in daemonCommandRequestSchema so it survives the HTTP edge.

The graft lives in request-router handleRequest (the seam that owns the
outer wall-clock incl. lock + execute + finalize). It mirrors the
conditional registerDownloadableArtifacts spread: when --cost is off OR the
response is an error, the response is returned untouched. Only on an opted-in
successful response is cost appended, so the default serialized DaemonResponse
is byte-identical to today (Maestro .ad recompare safe). Proven by the parity
test (flag-off identity, flag-on additive-only, error path, boundary survival).

Additive / semver-minor. MCP exposure and richer signals (roundTrips,
nodeCount) are deferred to follow-up slices.

* test: classify --cost flag as outside provider-backed integration

The integration progress guard (test:integration:progress:check) treats every
public CLI flag as either device-observable (requiring provider-backed coverage)
or intentionally excluded. --cost is a diagnostics/output flag (a purely additive
response field, not device-observable), so it joins json/help/version/verbose in
the 'config, output, diagnostics, and transport' exclusion bucket.
2026-06-29 13:50:12 +02:00
Michał Pierzchała 5a67623585 perf: reduce Apple runner build overhead (#898) 2026-06-27 13:48:38 +02:00
Michał Pierzchała 2d6fc98222 refactor: tighten daemon output helpers (#892)
* refactor: tighten daemon screenshot typing

* refactor: share screenshot result typing

* refactor: inline record shape checks

* refactor: inline record readers

* refactor: type batch and network output

* refactor: type batch run response

* refactor: narrow batch and observability output types

* refactor: validate typed batch runtime hints

* refactor: restore record boundary guard

* refactor: split daemon output parsers

* refactor: narrow screenshot overlay parser

* refactor: type screenshot overlay data

* refactor: type shutdown result payload
2026-06-26 16:32:52 +02:00
Michał Pierzchała 24cb2b622c fix: refine Apple provider pressure reporting (#877) 2026-06-25 17:34:50 +02:00
Michał Pierzchała ced61ce382 fix: disable nested sandboxing for ios runner builds (#869) 2026-06-25 10:18:00 +02:00
Michał Pierzchała ba825d8df2 fix: use desktop scroll events on macOS (#863)
* fix: use desktop scroll events on macOS

* fix: support paced macOS desktop scroll

* fix: clear scroll CI quality gates

* fix: address macos scroll review feedback

* refactor: simplify macos scroll plumbing

* fix: tighten scroll duration contract

* fix: limit apple scroll duration reporting
2026-06-25 07:38:21 +02:00
Michał Pierzchała 7739b71a25 refactor: centralize command family facets (#849) 2026-06-24 12:40:03 +02:00
Michał Pierzchała d47cd30117 feat: add agent-device proxy command (#844) 2026-06-23 17:20:08 +02:00
Michał Pierzchała 47fadf6925 fix: clean daemon-owned ios runner leases (#829) 2026-06-19 16:01:11 +02:00
Michał Pierzchała bc5726e8b0 test: cover web provider scenario (#827) 2026-06-19 13:39:39 +02:00
Michał Pierzchała feb5309e38 fix: classify external xctest runner flags (#810) 2026-06-16 15:54:00 +01:00
Michał Pierzchała 5c83fe4e50 fix: improve daemon diagnostics and remove compact snapshots (#786)
* fix: improve daemon startup diagnostics

* perf: relax query-sweep recovery budget

* fix: make compact snapshot flag a no-op

* fix: remove compact snapshot mode

* test: align tvOS remote provider expectation

* test: drop compact snapshot progress target
2026-06-12 17:32:56 +02:00
Michał Pierzchała fa1c1d55da refactor: localize command surface modules (#772)
* refactor: localize command surface modules

* refactor: localize command runtime modules

* refactor: tighten localized command exports

* refactor: address command localization review

* refactor: deepen batch command policy

* refactor: split provider progress model

* refactor: collapse command client facets

* refactor: run progress metrics as TypeScript

* refactor: remove obsolete command shims

* fix: update localized snapshot output import

* fix: preserve debug command localization
2026-06-12 14:02:04 +02:00
Michał Pierzchała cba020de21 fix: add iOS private AX snapshot fallback (#758)
* fix: add iOS private AX snapshot fallback

* fix: add public iOS snapshot query recovery

* fix(ios): make private AX snapshot fallback recover deep React Native trees

Four fixes that turn the #758 private AX fallback from
works-on-one-tree-shape into reliable on Bluesky Home:

- Depth ladder: the AX server rejects bulk snapshot requests outright
  (kAXErrorIllegalArgument) once requested depth crosses a
  tree-size-dependent limit that moves with live content. Retry at
  56/40/24/12 instead of giving up after one attempt at 64.
- Real attribute identifiers: the server silently ignored the raw
  keypath strings the bridge passed, so every node came back with a
  zero frame (breaking ref taps and the interactive/compact filters,
  which is why 'snapshot -i -c' stayed sparse). Map keypaths through
  XCElementSnapshot.axAttributesForElementSnapshotKeyPaths (it returns
  an NSSet) and drop the mapper's expensive extras (automation type,
  window display id, base type) that pushed deep requests past the 30s
  main-thread watchdog.
- Viewport from the private root frame when the public windows query
  degrades to an infinite viewport, so off-screen drawer content stops
  passing the visibility filter.
- Runner source fingerprint now includes .m/.h, so bridge edits stop
  reusing stale cached runner builds.

Also hardens the bridge per review: UInt(exactly:) for untrusted
element types, pid_t-sized objc_msgSend for process id matching, and
objCType-checked NSValue frame decoding.

* fix(ios): recover deadline-truncated near-empty compact snapshots

The all-structural sparse detector misses the common large-RN-tree case
where the typed-query sweep resolves one or two stray controls before
its 1s deadline: the payload has 'content', so recovery never fires,
yet 2 nodes is useless in practice. Treat deadline-truncated payloads
with <= 8 nodes as needing recovery, and only replace the original
payload when the recovered tree actually carries more nodes. Completed
sweeps on legitimately minimal screens stay untouched (not truncated).

* chore: fix CI for the AX snapshot fallback branch

- Sync the setup metadata script's fingerprint extension list with the
  runtime (.m/.h were added for the ObjC bridge), fixing the cache
  metadata parity test.
- Reduce find.ts complexity flagged by fallow: hoist the node fetcher
  into createFindNodeFetcher with a recoverSparseInteractiveSnapshot
  helper, split match disambiguation and resolution scoring into
  narrowMultipleMatches/resolvedTouchScore, extract rectsMatch.

* feat(ios): make accessibility fallbacks and collapsed containers visible in snapshot output

Two transparency gaps from #701's 'no silent fallback' requirement:

- Runner-attached snapshot messages now surface as snapshot warnings
  (readAppleSnapshotResult previously dropped them), so every recovery
  through the fallback accessibility backend or query tier is announced,
  states what it usually means (the app publishes an unhealthy
  accessibility tree - fixing the app is the real cure), and points to
  screenshot as visual truth.

- A leaf whose label merges many comma-joined segments is flagged as a
  collapsed accessible container: the app marks a container accessible,
  hiding every descendant from assistive tech and automation alike.
  Nothing can be recovered below it (VoiceOver sees the same merged
  element), so the warning names the node, estimates the merged label
  count, and gives the app-side fix plus the screenshot/coordinate-tap
  workaround.

Validated live on the lab stress fixture (adlab://stress?accessible=1):
the 6-node tree now carries '@e5 [Other] merges ~126 labels...'.

* fix(ios): detect sparse trees with labeled roots and surface warnings through the daemon

Validated against a real-world repro (a production React Native app's
login screen, simulator build provided privately by the reporter): a
full-screen accessibilityViewIsModal overlay leaves the public snapshot
with just Application+Window. Two gaps kept recovery off:

- The sparse detector counted the Application label (the app's display
  name) as content and the full-screen root as hittable, so the app
  name alone defeated recovery. Application/Window labels and root
  hittability say nothing about tree health and no longer count.
- Interactor-level snapshot warnings were dropped by the daemon capture
  chain (only the runtime/commands layer kept them); they now thread
  through CaptureSnapshotResult into BackendSnapshotResult.

With both fixes that login screen recovers through the public query
tier: 16 nodes with every control addressable (fill @ref + read-back
verified), and the output carries the recovery warning. Bluesky-class
trees still ladder into the private fallback unchanged.
2026-06-12 07:55:17 +02:00
Michał Pierzchała 3a02e514e1 refactor(ios): consolidate series batching onto the sequence runner command (#768)
* refactor(ios): consolidate series batching onto the sequence runner command

Closes #767

Routes every Apple multi-press variant (plain, double-tap, hold, jitter)
and swipe series through budget-chunked sequence requests, retiring the
daemon-side tapSeries and dragSeries senders:

- Add a doubleTap step kind to the sequence allowlist on both ends,
  mirroring the retired tapSeries doubleTapAt branch.
- The single doubleTap interactor sends a one-step sequence and parses
  the result, surfacing step failures as errors.
- Swipe series unroll ping-pong daemon-side into per-step endpoints;
  the runner's coordinate-drag path ignores durationMs exactly as the
  daemon-sent (non-synthesized) dragSeries did.
- Extract runIosSequenceChunks so press and swipe share the chunking,
  aggregation, and global step-index rebasing.
- Keep tapSeries/dragSeries runner handlers for wire compatibility with
  older daemons, annotated like interactionFrame; remove both from the
  preflight-skip allowlist (daemon never sends them) and update ADR
  0005 / protocol-optimizations docs.

This also closes the latent watchdog exposure where press --count N
--interval-ms M routed to tapSeries and executed all pauses inside one
30s-watchdog main-thread block with no chunking.

Behavior note: plain tap series now use the synthesized HID tap path on
iOS non-tv (with runner-side tapAt fallback), matching the individual
tap command instead of the retired tapSeries' XCUICoordinate taps.

https://claude.ai/code/session_01VokBZWESTDgcnbYwS4DkJo

* refactor(ios): drop dead series wire surface from the daemon

- Remove chunkRunnerSequenceSteps: superseded by the budget-aware
  chunker; no production callers remained.
- Remove tapSeries/dragSeries from the RunnerCommand union along with
  their orphaned fields (count, intervalMs, doubleTap, pauseMs,
  pattern) and protocol fixtures: this type is the send surface of the
  current daemon, which no longer sends either command. The Swift
  runner keeps serving both for wire compatibility with older daemons.
- Retarget the ready-mutation preflight test from tapSeries to
  sequence.

https://claude.ai/code/session_01VokBZWESTDgcnbYwS4DkJo

* refactor(ios): remove retired series and frame wire commands entirely

Drops the runner-side wire compatibility for tapSeries, dragSeries, and
interactionFrame now that no daemon path sends them (series fuse into
sequence since this branch; interactionFrame was fused into scroll in
#760):

- Swift: delete the three handler cases, performDragSeries, runSeries
  (no remaining callers), the CommandType enum cases, journal-retention
  and traits entries, and the Command fields (count, intervalMs,
  doubleTap, pauseMs, pattern) that existed only for them. The
  never-sent synthesized dragSeries branch goes with it.
- TS: drop interactionFrame from the RunnerCommand union and
  isReadOnlyRunnerCommand, and its protocol fixture.
- Update stale perf scenario labels referencing the retired commands.

Verified dead before removal: no dynamic command construction anywhere
(runner-command-recovery only echoes in-flight command ids), no
raw-string references in Swift, no docs references. Helpers shared with
live paths (synthesizedDragAt, doubleTapAt, keyboardAvoidingDragPoints,
sleepFor) all retain callers.

Compat: an old daemon paired with a runner built from these sources
gets a CommandType decode rejection; the source-fingerprint check
rebuilds a matching runner on the next session.

https://claude.ai/code/session_01VokBZWESTDgcnbYwS4DkJo

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-06-11 14:08:30 +02:00