mirror of
https://github.com/callstack/agent-device.git
synced 2026-09-14 20:06:34 +08:00
v0.20.3
1205 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
f474f0784e |
feat: unify gesture planning and multi-touch execution (#1212)
* feat: unify gesture planning and multi-touch execution * fix: correct unified gesture helper behavior * refactor: tighten unified gesture architecture * fix: preserve gesture routing contracts * test: account for fresh gesture viewport * refactor: remove retired gesture series * fix: preserve example app navigation targets * test: reconcile unified gestures with helper ownership * docs: update Android helper gesture protocol * fix: refresh Maestro percentage swipe frames * refactor: remove stale Maestro frame cache * fix: harden unified gesture execution * fix: model gesture viewport in providers * refactor: remove legacy gesture paths * fix: remove unused swipe preset parser * refactor: tighten unified gesture boundaries * fix: close gesture review gaps * fix: preserve gesture compatibility contracts * fix: preserve multi-touch recording semantics * fix: refresh Apple runner state after app relaunch * test: lock Apple fling fallback route * fix: close Apple runner review gaps * refactor: tighten unified gesture seams * refactor: consolidate gesture planning policy * fix: preserve swipe response compatibility * fix: keep gesture lab aligned with replay coordinates |
||
|
|
4e06304b12 |
fix: align layering claims, remove app-log ineffective dynamic import, refresh architecture records (#1222) (#1227)
* fix: align layering claims, remove app-log ineffective dynamic import, refresh architecture records (#1222) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * docs: fix layering spine ordering (cli top), ADR 0009 deferred-scope wording, terse app-log-request-scope comment (#1222) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * docs: show full rank groups in layering spine diagrams; clarify back-edge order vs literal imports (#1222) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * docs: mark replay ADR context historical Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * docs: refresh replay repair and app-log boundary status Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Michał Pierzchała <thymikee@gmail.com> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
|
6275ed00c5 | docs: remove ready-for-human guidance (#1237) | ||
|
|
ddae892430 |
feat(replay): ADR 0012 decision 6 — agent-supervised re-record repair (#1228)
* feat(replay): ADR 0012 decision 6 — agent-supervised re-record repair Implements "heal-by-doing": `replay --save-script` arms recording from the first attempt (R1), a boundary watermark scopes the healed `.ad` to the repair run (R6), the daemon computes a total `repairHint` enum at divergence time via genuine ancestor-containment over the daemon's own tree (R3), and the writer fails loud on a bare `@ref` in a repair-armed session instead of emitting an unreplayable script (R4). - session-replay-runtime.ts: `armReplaySaveScript` arms `session.recordSession` + `saveScriptBoundary` before every step; `--save-script`'s absence leaves replay byte-identical to today. - session-replay-repair-hint.ts (new): `computeReplayRepairHint` — a total mapping over the 4 divergence kinds, with two fail-safes to `manual` (no recorded evidence; sparse/unavailable capture). Wired into both divergence-minting sites (session-replay-divergence.ts, session-replay-target-verification.ts) and threaded through divergence.ts's bounding/minimal-fallback/text-report paths so it survives every response level and projection. - session-script-writer.ts: `saveScriptBoundary` slices the healed script to the repair run; a bare unresolved `@ref` throws in a repair-armed session (ordinary open/close --save-script keeps its existing refLabel fallback). - cli-help.ts: documents the repairHint-routed repair sub-flows and the close --save-script / <stem>.healed.ad promotion step. Tests: repairHint mapping (all 4 kinds x container-present/absent x fail-safes, with genuine-containment fixtures), R1 fresh evidence, R6 sticky boundary across --from legs, --from ordering with no prefix duplication, opt-in byte-identical behavior, no-record/thrown-failure exclusion, R4 fail-loud, and a full fresh-session replay acceptance test. * fix(replay): adversarial-review fixes for ADR 0012 decision 6 Addresses P0-P2 semantic holes + CI found in review of the repair loop: P0 (session-replay-repair-hint.ts): the scrollRegion container test was a near-universal false positive. An UNIDENTIFIED scrollRegion (RN's default ScrollView/FlatList with no testID) matched "any anonymous scrollview exists," routing unrelated screens to record-and-heal (the exact mis-binding --update retirement prevents). Now an identified region must ALSO satisfy the recorded ancestry containment (AND); an unidentified/absent region falls back to ancestry alone, walking the WHOLE recorded chain (not just ancestry[0]). P1 (session-replay-runtime.ts): the boundary watermark was captured pre-loop from the current session, but step-1 open --relaunch can REPLACE the session with a fresh actions:[] one, amputating the healed open/prefix. Reworked into a per-step armer that stamps the boundary on the session that actually accumulates the run (firstArm captures prior actions; a later arm on an unset boundary means open replaced the session → boundary 0). Added R2 preflight: a fresh full replay --save-script on an already-armed session is rejected (INVALID_ARGS) rather than silently duplicating the recorded prefix. P1 (session-replay.ts): test now rejects --save-script (replay-only), matching the sibling --from/--plan-digest guard, so buildNestedReplayFlags can't fan it across a suite run. P2 (session-script-writer.ts): the default <stem>.healed.ad is never silently clobbered — a second repair against the same original fails loud unless an explicit --save-script=<path> is given, so an unreviewed diff survives. CI (session-replay-runtime.ts): extracted the per-step verify/dispatch/guard body into resolveReplayStepResponse (runReplayScriptFile 205→163 lines) and the arming into preflight/armer helpers; refactored the branchy test mock invokes into a shared makeRecordingReplayInvoke fixture — fallow complexity is green. New/updated tests: unidentified-scrollRegion false-positive (state-repair/ manual) + labeled positive + full-chain ancestry; R6 no-amputation (open replaces) + preserved-session prior-action exclusion; R2 full-replay rejection; test --save-script rejection; P2 clobber refusal + explicit-path overwrite. * fix(replay): close three residual ADR 0012 decision 6 review blockers B1 (session-replay-runtime.ts): R2 was bypassable by OMITTING --save-script — the session stays repair-armed (recordSession + saveScriptBoundary), so a plain full replay re-appended the prefix. preflightReplayAgainstActiveRepair now fires for ANY full replay (entryIndex 0) on an armed session, regardless of the flag; --from resumes stay allowed. B2 (session-replay-runtime.ts + session-action-recorder.ts): an explicit --save-script=<path> at BOTH the replay-arming and close (recordActionEntry) sites now clears saveScriptDefaultedHealedPath, so the clobber guard no longer wrongly refuses a caller-directed overwrite. Invariant: marker set iff the current saveScriptPath was defaulted. B3 (session-replay-repair-hint.ts): a root-level target with NO usable structural container signal (empty ancestry AND no identified scrollRegion) was reading as "present" on any non-empty screen -> record-and-heal on an unrelated screen. computeReplayRepairHint now fails such no-signal cases to manual (same class as the unidentified-region P0); empty ancestry + IDENTIFIED region still uses the region test. Regressions: plain full replay on an armed session -> INVALID_ARGS + zero dispatches + no prefix duplication; close --save-script=<explicit existing path> overwrites (marker cleared) while a defaulted path still fails loud; no-ancestry/no-region on an unrelated screen -> manual (selector-miss and action-failure) with the identified-region positive kept. |
||
|
|
e61edf753a |
docs: ADR-0012 Decision 6 (agent-supervised re-record repair) (#1226)
* docs: ADR-0012 Decision 7 — agent-supervised re-record repair Adds "heal-by-doing": when replay diverges on selector drift, the agent performs the failed step's intent with ordinary interactive commands against blessed refs, and the CLI emits the healed .ad from the session's actual successful execution path (session.actions) instead of hand-edited selector text. Folds in five normative protocol rules (R1-R5) from an external design review (verdict: SOUND-WITH-FIXES) covering record-arming timing, --from continuation semantics, mechanical repairHint routing, the writer's bare-@ref fail-close, and the recorded-open requirement. Makes explicit that this reintroduces an EXPLICIT, opt-in heal and is therefore consistent with (not a reversal of) decision 1's retirement of --update's SILENT auto-rewrite. * docs: renumber to Decision 6 and make repairHint the mechanical primary router - Rename 'Decision 7' -> 'Decision 6' (ADR has decisions 1-5; new one is the 6th, placed after 5. Mandatory validation; existing decisions unchanged). - Reframe the two-sub-flows router: the CLI-computed repairHint (mechanical, in-scope, ships with this decision per R3) is the PRIMARY router; the agent follows the hint and uses screen.refs only as an ambiguity override, not as the default router. Removes the agent-judgment-vs-R3 contradiction. * docs: ADR-0012 Decision 6 — daemon-side repairHint (4 kinds) + repair-run boundary (R6) Addresses two P1 review gaps: - P1-A: state that repairHint is computed daemon-side at divergence time from the recorded targetEvidence + the daemon's own full pre-action capture (only the enum crosses the wire, so the flat/capped screen.refs never gate routing); define repairHint for all four divergence kinds (selector-miss, identity- mismatch, identity-unverifiable, action-failure) with the sparse-capture fail-safe to manual. - P1-B: add R6 — --save-script records a boundary watermark (session.actions. length at invocation); the healed script serializes only the post-watermark slice, so a reused session's earlier actions don't pollute it. Clarifications: R4 fails loudly (non-zero exit, never swallowed); exact default output path = <original-stem>.healed.ad sibling; arming sets recordSession AND the watermark before step 1. * docs: ADR-0012 — declare repairHint in the wire contract + make R3 total Addresses three protocol blockers: - Blocker 1: add repairHint to Decision 4's details.divergence field list and spec it as a single bounded enum (record-and-heal|state-repair|caution|manual), present on every divergence, carried at every level, surviving all four projections (text/JSON/client/MCP). - Blocker 2: make R3's mapping total — no recorded targetEvidence (reachable for unannotated action-failure per #1223, or any kind on a legacy script) => manual, generalizing the sparse-capture fail-safe. - Blocker 3: correct capture-timing — target-binding kinds use their PRE-action tree; action-failure uses its POST-response tree (adequate for the container presence test); no new pre-action tree is stored for action-failure. Validation extended for the projection-survival and no-evidence/post-response cases. |
||
|
|
70bb9b6701 |
fix: wrap replay selector-miss as REPLAY_DIVERGENCE (repair-loop regression) (#1223)
* fix: wrap replay selector-miss as REPLAY_DIVERGENCE (repair-loop regression)
A dispatch failure thrown by a replay action (e.g. a selector-miss during
press/click/fill/longpress) escaped runReplayScriptFile's `if (!response.ok)`
divergence wrapping and fell through to the outer catch, returning a bare
COMMAND_FAILED with the legacy diagnostics shape instead of the ADR 0012
REPLAY_DIVERGENCE report (no resume, no screen refs, no suggestions) —
breaking the interactive repair loop for the common drift case.
Fix at the invocation layer: invokeReplayAction (the single choke point every
replay action dispatch funnels through — the top-level loop, retry blocks,
and nested Maestro runFlow invocations) now normalizes a thrown AppError into
a `{ok:false}` response before returning, matching DaemonInvokeFn's own
never-reject contract. This is more localized than wrapping the per-action
loop body in session-replay-runtime.ts and additionally closes the same gap
for retry/control-flow and Maestro nested dispatch, which funnel through this
exact function and previously had no throw protection either.
Adds regression coverage for the exact gap: end-to-end replay tests asserting
REPLAY_DIVERGENCE (not COMMAND_FAILED) with resume + screen refs for an
annotated press, an unannotated press, and a fill selector-miss thrown at
dispatch.
* style: wrap over-width line in replay selector-miss test (oxfmt)
* fix: narrow replay dispatch catch to AppError, normalize via normalizeError
invokeReplayAction's catch previously coerced ANY thrown dispatch failure
(via asAppError) into a {ok:false} response so the replay loop's divergence
wrapping applies — including plain TypeError/ReferenceError from programmer
bugs, which asAppError silently reclassified as a repairable
REPLAY_DIVERGENCE ("heal a selector"), masking a real crash.
Narrow the catch to only wrap actual AppError dispatch failures; anything
else re-throws to the outer internal-error path. Switch the wrap itself from
errorResponse(appErr.code, appErr.message, appErr.details) to
{ ok: false, error: normalizeError(dispatchErr) } — the canonical daemon
thrown->response pattern (request-generic-dispatch.ts, selector-runtime.ts)
— so retriable/supportedOn and hint/diagnosticId/logPath hoisting survive
on the thrown path instead of being dropped.
|
||
|
|
ae21067920 |
fix(mcp): advertise public Apple platform leaves in boot/shutdown/prepare output schemas (#1224)
The boot and shutdown MCP outputSchemas (and the non-exposed prepare schema) enumerated the internal PLATFORMS vocabulary (apple | android | linux | web), while the command contracts and runtime emit PublicPlatform (ios | macos | android | linux | web). Point the schemas at PUBLIC_PLATFORMS so a valid ios/macos result validates and the internal apple identity is never advertised on these public result fields. Add a focused, non-strict JSON Schema validator and use it to validate representative ios/macos structured content against the complete advertised schema (enums/consts + nested required), with negative coverage for the internal apple value, unknown enum values, the booted const, and a missing nested field. Closes #1219 Co-authored-by: Michał Pierzchała <thymikee@gmail.com> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
|
260ec45b34 |
feat(replay): ADR 0012 migration step 4 — target-binding verification enforcement (#1209)
* feat(replay): ADR 0012 step 4 target-binding verification + post-resolution guard + step-5 resume wiring Rebuilt onto current main (#1210 selector relocation, #1211 resume + --update retirement, #1213, #1215) as one coherent commit — the prior 4-commit history collided structurally with #1211's own session-replay-runtime split. Step 4 (decision 3, enforcement): every annotated resolved-target replay/test action is verified against a fresh pre-action snapshot BEFORE dispatch (verifyReplayActionTarget → classifyReplayTarget, decision 3's six paths). Only a verified outcome sends the action; selector-miss / identity-mismatch / identity-unverifiable each return a complete target-binding REPLAY_DIVERGENCE built from its own capture, with targetBinding {classification==kind, matchCount per the presence rule, recorded/observed, mismatches, candidates} through #1211's shared failure-response + bounding/sanitizer pipeline. Post-resolution guard: a verified outcome mints internal.replayTargetGuard (the verified member's identity); dispatch's own resolution (occlusion/ visibility guards verification does not replicate) must land on the SAME element or assertExpectedResolvedTarget refuses pre-action, which the loop converts to an identity-mismatch divergence. Guarded dispatch skips the direct-iOS and native-ref fast paths so a tree node exists to check. Step-5 resume wiring (this rebase): the target-binding divergence resume stub is replaced with #1211's buildReplayDivergenceResume — a pre-action divergence resumes AT the failed step (from = step index, real SHA-256 planDigest, allowed unless preflight rejects the skipped range). buildTargetBindingDivergenceResponse is the single resume site; planActions/planDigest are threaded from the loop. --update healing is retired by #1211, so the in-branch "verify healed action" work is dropped (its module and test removed as moot). Tests: classification (13), guard (4), token (5), runtime end-to-end (9, incl. old-script pass-through, guard threading + mismatch conversion, and a real computed resume assertion), plus target-binding wire tests in divergence.test.ts (classification==kind, matchCount presence, overflow-fallback targetBinding, computed-resume-not-stub, text report). Also repoints daemon-http-disconnect.test.ts already handled upstream by #1215. * test(replay): assert real computed resume on identity-mismatch and guard-mismatch paths Non-blocking review follow-up: only the selector-miss runtime test asserted the computed resume {allowed, from, planDigest}. All three target-binding paths funnel through buildTargetBindingDivergenceResponse (one shared resume site), so this is coverage not a bug — but pin resume on the identity-mismatch and guard-mismatch tests too so no path can regress its resume wiring silently. * fix(replay): carry structural denotation into the post-resolution guard (P1) The guard carried only LOCAL identity {id, role, label}. ADR path 6 isolates ONE member among several SAME-local-identity nodes via sibling / region-scoped viewportOrder — a discriminator lost at the guard. So if verification denoted duplicate A but dispatch's occlusion/visibility filter selected duplicate B (identical id/role/label), the guard passed and SENT the action on the wrong element — the exact verified-but-taps-different-node mis-binding step 4 exists to prevent. Fix: replayTargetGuard now carries a structural denotation of the verified member — its pre-order document index (decision 3's canonical total order, `node.index` after the shared reindex pipeline) plus its same-parent sibling ordinal. `assertExpectedResolvedTarget` compares BOTH local identity AND structural denotation against dispatch's pre-promotion capture, refusing pre-action when EITHER differs. Two distinct duplicates always differ in document order, so a different-member resolution can no longer pass. The promotion exemption is preserved (comparison is on the verified LEAF, and duplicates are distinct leaves). New helpers live in target-identity-node.ts (replay zone) so writer/classifier/guard share one sibling-ordinal implementation; computeSiblingOrdinal now delegates to it. The guard-mismatch divergence surfaces a `position:` mismatch line (from the structural denotations) so `mismatches` is never empty when local identity is identical. END-TO-END regression (session-replay-target-guard.test.ts): two "Save" buttons with IDENTICAL id/role/label, A covered+deeper (verification's winner), B visible (dispatch's winner) — driven through the real interaction runtime (`device.interactions.press`), the guard refuses pre-action with ZERO backend taps AND zero native tapTargets, the refusal driven purely by the structural denotation (both observedStructural/expectedStructural asserted). Plus a same-structural passing case, a runtime-level `position:`-mismatch divergence test, and updated guard-threading assertions for the {identity, structural} shape. Wire contract unchanged: replayTargetGuard is DaemonRequestInternal (never on the wire); decision 4's targetBinding {recorded, observed} stay LOCAL identity per decision 3's Identity tier — the structural denotation is an internal guard concern, surfaced only as a `mismatches` string. No ADR change needed. |
||
|
|
c23d951a58 | fix: preserve Maestro coordinate swipes on Android (#1207) | ||
|
|
d585d74172 |
chore: close out architecture experiments (#1213)
* chore: close out architecture experiments Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * docs: record unavailable live experiment Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * ci: make Android perf script atomic Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * docs: explain atomic perf workflow Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * chore: simplify back-edge diagnostics Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Michał Pierzchała <thymikee@gmail.com> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
|
e2bfed5f9f |
feat(replay): ADR 0012 migration steps 5+6 — resume + --update retirement (#1211)
* feat(replay): ADR 0012 migration steps 5+6 — resume + --update retirement
Step 5 (decision 4, resume): replay --from <n> --plan-digest <sha256>
resumes at a 1-based plan step, skipping 1..n-1 without executing them.
Every divergence report now carries a real resume object (allowed, from,
planDigest, reason?) computed by a preflight that rejects INVALID_ARGS
before any action when: the plan digest no longer matches the current
script (edits/includes/platform-conditioned expansion), --from is out of
range, a skipped step can produce outputEnv values, or the skipped range
or resume target is runtime control flow (retry/runFlow.when — these are
single plan entries, never individually addressable). `test` rejects
--from/--plan-digest both at the CLI-schema layer and at the daemon
dispatch layer (the original command name is only visible before test
rewrites its nested request to `command: 'replay'`).
New modules: src/replay/plan-digest.ts (canonical SHA-256 plan digest)
and src/daemon/handlers/session-replay-resume.ts (preflight + the
report's resume object), kept out of src/replay/ to avoid a
replay<->compat import cycle.
Step 6 (decision 1, retirement): --update/-u no longer rewrites .ad
files. The ADR mandates a no-op, not an error or flag removal: --update
now runs identically to a plain replay and returns the same bounded
suggestions every divergence already carries. Removed: healReplayAction's
retry-and-rewrite arm and its exclusive helpers (collectReplaySelectorCandidates
stays — decision 1's suggestions still use it), the write call from the
runtime loop, and the env/${VAR}-interpolation/compat-flow refusal guards
that existed only to protect that rewrite. writeReplayScript itself keeps
its own round-trip tests but is otherwise unused now; deleted after the
production-exports gate flagged it as dead.
Docs: cli-help.ts workflow topic + --update/--from flag help, AGENTS.md
selector pipeline note, maestro-compat-debt-map.md, website replay-e2e.md
and commands.md updated for the retired rewrite and the new resume loop.
* fix(ci): classify resume flags + provider-scenario resume coverage
The Integration Tests job's architecture-progress gate
(test:integration:progress:check) requires every public CLI flag to be
classified; --from/--plan-digest (replayFrom/replayPlanDigest) were
unclassified. Classify them as device-observable workflow flags and add
real provider-backed coverage to the Android lifecycle scenario: a full
replay diverges on a missing selector, the report's resume object is
asserted (allowed/from/planDigest), and resuming at the next index
replays only the tail. Also refresh the stale replayUpdate reason
("selector-healing replay update" -> the retired no-op).
* fix: bind replay resume digest to execution plan
* test: align replay runtime module topology
* fix: clear replay CI regressions
* docs: clarify replay repair and resume paths
* docs: clarify replay resume step semantics
* docs(replay): note that ${VAR} values stay out of the plan digest (ADR 0012 + workflow help)
Settled decision from the PR #1211 re-review (maintainer-approved): interpolated
${VAR}/--env/AD_VAR_* VALUES are deliberately NOT part of the resume plan digest.
Substitution happens after the digest is computed over the still-unsubstituted
${VAR} text, so re-running the same script with different variable values keeps
the same digest and stays resumable — supplying the right values on resume is the
caller's responsibility. The digest still binds the script/includes, the effective
--platform/--target, and per-action runtime hints + target-v1 identity. Documented
in ADR 0012 decision 4 and the `help workflow` resume topic.
* docs: clarify replay digest interpolation
|
||
|
|
8157b37efd |
fix: repoint tests to relocated request-progress/cancel modules (#1215)
main was red: this test still imported src/daemon/request-progress.ts and src/daemon/request-cancel.ts, which no longer exist after the relocation to src/request/progress.ts and src/request/cancel.ts. Import-only fix, no behavior change. |
||
|
|
7571b226a3 |
fix(macos): allow XCTest teardown before runner kill (#1206)
* fix(macos): allow XCTest teardown before runner kill * fix(macos): abort runner on HTTP disconnect |
||
|
|
d3adea4002 |
Project navigation contracts and add a network digest (#1208)
* test: record contract and digest spike selection Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * feat: project navigation client and MCP contracts from executable definitions Collapse the three independent per-command projection declarations (facet clientMethod, public client method signature, MCP output schema) for the typed system navigation subset (home, back, rotate, app-switcher, tv-remote) onto a single colocated projection in src/contracts/navigation.ts. The family builder, public client type, and MCP schema map now derive from those five projections. Refs #1185 Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * feat: add opt-in network response digest Preserve every network entry and top-level recovery/actionability signal while dropping only verbose per-entry header, body, and raw-log fields at digest response level. Record deterministic output-economy baselines and parity tests. Refs #1186 Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Michał Pierzchała <thymikee@gmail.com> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
|
0a8ea3a57b |
refactor: consolidate architecture ownership and client results (#1210)
* refactor: consolidate architecture ownership and client results Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix: keep selector parse chunk grouping current Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix: update moved architecture breadcrumbs Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix: enforce moved selector architecture Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix: keep selector guarantee ownership current Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * docs: update selector ownership references Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Michał Pierzchała <thymikee@gmail.com> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
|
c93dcdbc90 |
feat: disclose selector resolution in interaction responses (#1193)
* feat: disclose selector resolution in interaction responses Implements ADR-0012 migration step 1 (decision 2). Adds an additive `resolution` field to press/click/fill/longpress responses: runtime-selector carries the full pre-action diagnostic shape (unique or disambiguated with matchCount/winnerDiagnostic/tiebreak/bounded alternatives), runtime-ref and native-ref carry the exact ref-provenance shape, direct-ios-selector carries the explicit not-observed marker, and coordinate/maestro-non-hittable-fallback stay inapplicable (no field). The comparator in selectors-resolve.ts now records which criterion (visible/deepest/smallest-area) decided each disambiguation without changing resolveSelectorChain's winner. Extends the ADR-0011 guarantee matrix with the resolutionDisclosure guarantee across all six dispatch paths, wires the shared response builder and MCP output schema, adds digest-level trimming (drops alternatives, keeps the verdict/counts), and proves via contract tests that resolution diagnostics are never ref-issued or MCP-pinned and cannot be reused as @ref targets. * fix: address resolution disclosure review findings * refactor: make resolution-disclosure choices self-evident Replace the direct-iOS/maestro message-sniffing (and its justification paragraph) with an explicit maestroFallback flag passed from the dispatch site that already owns the path decision, and shrink every why-this-is-OK paragraph to one-line constraint statements per the maintainer directive. * fix: usage-based maestro fallback disclosure + spec label-fallback Blocker 1: the runner-payload source now carries maestroFallbackUsed derived from the runner's actual execution outcome (the usedNonHittableFallback message bit RunnerTests+CommandExecution.swift reports, the same signal directIosSelectorFallbackDetails already keys on) instead of the permission flag. A fallback-allowed dispatch that hit its element normally discloses direct-ios/not-observed; only an actually-executed coordinate fallback is the inapplicable maestro cell. Contract tests cover both sides. Blocker 2: ADR-0012 decision 2 now defines the ref/label-fallback disclosure (runtime-ref trailing-label recovery via tryResolveRefNode's fallbackLabel; native-ref stays exact because the backend receives only the ref handle), amends the matrix-cell enumeration, layer-3 coverage list, and validation bullet, and the runtime-ref contract suite proves the label-fallback shape. * fix: honest runtime-ref registry cells for label recovery The disambiguation cell no longer claims refs identify exactly one node by construction — trailing-label recovery is a first-match lookup without the ranking, now an intentional waiver whose outcome the label-fallback disclosure surfaces per-response. resolutionDisclosure.via points at tryResolveRefNode (now exported), the resolver producing both exact and label-fallback, with direct unit coverage of both outcomes. * docs: correct native-ref exactness rationale and tiebreak doc Native-ref forwards fallbackLabel to the backend; exact is justified by non-observability of any backend-side label recovery, not by non-forwarding. The tiebreak doc now states the derived winner-vs-runner-up decisive margin. * fix: disclose Maestro fill fallback usage |
||
|
|
952bc3704a |
refactor: keep command and daemon-route owner-file claims tooling-only (#1178) (#1192)
* refactor(command-descriptor): keep owner-file claims tooling-only Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * refactor(daemon): keep daemon-route owner-file claims tooling-only Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(daemon): guard against re-adding owner-file paths to the production route chain Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * refactor(command-descriptor): derive owner-file projection from colocated RAW_COMMAND_DESCRIPTORS - Keep ownerFiles on each RAW_COMMAND_DESCRIPTORS entry as the source of truth. - Add tooling-only __OWNER_FILES__ build flag so production bundles omit the ownerFiles properties entirely. - Derive COMMAND_OWNER_FILES from RAW_COMMAND_DESCRIPTORS instead of a hand-maintained parallel table. - Guard command-explain tests against leaking ownerFiles into production descriptor objects. - Enable treeshake.propertyReadSideEffects: false in tsdown to help drop the dead ownerFiles branch from production bundles. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * chore: apply oxfmt formatting Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(build): guard tooling metadata exclusion Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * refactor(command-descriptor): drop global treeshake option and add bundle guard - Remove treeshake.propertyReadSideEffects from tsdown.config.ts; the __OWNER_FILES__ define + conditional spread already keeps owner files out of the bundle, so the global DCE lever is unnecessary and scope-creeping. - Add a comment on the __OWNER_FILES__ global declaration explaining the deliberate type-versus-runtime mismatch. - Add test/output-economy/owner-files-no-leak.test.ts to build dist and assert that no command or daemon-route owner-file path appears in the emitted JS. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(build): remove owner metadata property reads Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(command-descriptor): enforce owner claim totality Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Michał Pierzchała <thymikee@gmail.com> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
|
973df8a8a8 |
feat(replay): ADR 0012 migration step 2 — structured replay divergence report (#1197)
* feat(replay): thread runFlow include provenance through Maestro parsing
ADR 0012 migration step 2. The Maestro parser previously lost per-action
source location in two stages: convertRootCommands stamped every action
produced by inlining a runFlow with the PARENT runFlow: command's own line,
and parseRunFlowFile's callers discarded the included file's line table and
path entirely. This is the live-reproduced bug: "Replay failed at step 5
(__maestroTapOn ...)" with no file or line anywhere in the failure.
ParsedReplayScript gains an optional actionSourcePaths array parallel to
actionLines (undefined means "the top-level replay file"). The Maestro
conversion pipeline now carries {path, line} through runFlow inlining,
including multi-level (include-of-include) nesting, via a transient
SessionAction.replaySource field that convertRootCommands reads back into
the flat arrays and strips before an action leaves the parser.
Regression coverage: a flow including ../launch.yml where a failure inside
the include must report the include's own path+line, plus a nested
include-of-include case.
* feat(replay): structured REPLAY_DIVERGENCE report for step failures
ADR 0012 migration step 2 (report only; kind: "action-failure"). Every
replay/test step failure now returns ok:false with code REPLAY_DIVERGENCE
and a bounded, redacted details.divergence v1 object instead of the
underlying failure's own code:
- step: 1-based plan index + source {path, line}, now accurate through
Maestro runFlow includes (previous commit) and already-correct for .ad.
- cause: the original code/message/hint, preserved verbatim.
- screen: a fresh post-failure interactive snapshot digest, captured and
blessed exactly like settle's stored-tree pattern (setSessionSnapshot +
markSessionSnapshotRefsIssued + session.snapshotGeneration), so refs are
immediately actionable. Capture/sparse failure degrades to
{state:"unavailable", reason, hint} and never masks the original cause.
- suggestions: collectReplaySelectorCandidates + resolveSelectorChain
re-resolution (session-replay-heal.ts), reused READ-ONLY to rank up to 5
candidates by identity-component strength (id > role+label > label >
other) then document order. The same-scrollRegion-as-recorded tier from
decision 1's total order is not evaluated here — it depends on decision
3's recorded target evidence, which this step does not consume.
--update's actual rewrite behavior is unchanged.
- resume: always {allowed:false, reason}, no planDigest/from key until
step 5 lands.
- Bounds: 8/24/64 KiB (digest/default/full) on the serialized
details.divergence object, 256-UTF-8-byte per-field caps, 8/20 screen
refs, 5 suggestions. Overflow writes the fuller detail to a session-scoped
artifact and returns {omittedBytes, artifactPath}, or artifactUnavailable
if the write itself fails — the cause is never dropped either way.
- Success path gains a one-line text summary ("Replayed N steps in X.Xs"),
closing the silent-success gap (text replay previously printed nothing).
Existing replay-failure test fixtures that didn't mock dispatchCommand now
do, since every failure attempts a post-failure snapshot capture.
* feat(cli): render replay divergence as a compact text report
ADR 0012 migration step 2, end-to-end preservation (daemon -> Node client ->
CLI leg). printHumanError now renders a compact divergence summary (step
location, screen availability, ranked suggestions or an omitted-count hint,
overflow artifact pointer) unconditionally — not gated behind --debug, which
previously only showed step+action+selector+a generic hint with no screen
evidence. --json already carried the full structured error via
normalizeError, which was already preserving details.divergence through
stripDiagnosticMeta/redactDiagnosticData (only hint/diagnosticId/logPath/
retriable/supportedOn are stripped there); this commit adds contract tests
proving that survival, plus the throwDaemonError -> AppError leg.
* feat(mcp): treat a replay divergence error as a ref-issuing result
ADR 0012 migration step 2, MCP leg. Previously any command failure (daemon
or otherwise) reached callTool's outer catch, which built a text-only
ToolResult with no structuredContent at all — a divergence's repair data
(screen refs, suggestions, cause) was completely lost over MCP.
The tool executor's own execute() now catches command failures directly, so
it can run the same ref-merge/pin pipeline successful ref-issuing responses
already get: an error's details.divergence.screen (when "available") is
merged and pinned at refsGeneration exactly like a snapshot/find response,
merge-only like every other pin path (an unrelated command error, e.g.
INVALID_ARGS with no divergence, never clears existing pins). The result is
isError:true, structuredContent is the full normalized error, and content
carries the same compact text summary the CLI renders. router.ts's own
catch is now only the fallback for failures outside a resolved command call
(unknown tool name, malformed params) and reuses the same normalize/format
helpers (extracted to mcp/tool-error.ts) rather than duplicating them.
* fix(replay): carry include provenance through retry/runFlow.when wrappers
Review blocker on #1197 (empirically probed): a `runFlow: file:` include
nested under `retry:` or a runtime `runFlow.when: {visible|notVisible}`
conditional broke provenance on both sides. Parse side: the transient
SessionAction.replaySource field was only stripped by the top-level
convertRootCommands path — nested actions under replayControl.actions kept
it, contradicting the field's own "never reaches dispatch" doc comment.
Runtime side: invokeReplayActionBlock invoked every nested action with the
WRAPPING control action's line, so a failure inside such a wrapped include
reported the retry:/runFlow.when: line of the root file — reproducing
exactly the provenance bug this PR fixes, on the single most common real
failure site (the RN suite's launch include is retry-wrapped).
Fix, both sides:
- Parse: stripNestedActionSources removes replaySource from every action a
control-flow wrapper carries and records it as the new parallel
replayControl.actionSources array (undefined entry = "the wrapper's own
file"). Deep leak check test proves no replaySource survives anywhere in
parsed output.
- Runtime: invokeReplayActionBlock/invokeReplayRetryBlock consult
actionSources per nested action; invokeReplayAction threads a sourcePath
(falling back to the wrapper's, then the root file) and attaches the
failing action's resolved {path, line} to the error as a
transport-internal details.replaySource marker — deepest failure wins, an
outer wrapper never overwrites it. Trace events gain sourcePath when it
differs from the root replay file.
Regression tests are the reviewer's probe scenario: a retry-wrapped include
and a runtime-notVisible-wrapped include, each failing INSIDE the include,
asserting divergence.step.source.path/line point into the include (and that
the internal marker is stripped from the flat details). Parser-level tests
assert the strip + actionSources shape for both wrapper kinds.
* fix(replay): single divergence capture; consume nested failure provenance
Review findings on #1197 (needs-work review, findings 1-3 + 5):
- ONE post-failure snapshot now serves both the screen digest and the
suggestion re-resolution. The previous per-purpose double capture had the
second setSessionSnapshot advance the generation past the one the report
had just advertised (screen.refsGeneration=G1, live session at G2), so
MCP pinned the caller's next ref at a stale generation and the daemon
warned "re-run snapshot" — defeating the ADR's ref-issuing contract in
exactly the case a selector suggestion exists. Single capture = one
generation, refs and suggestions minted from the same stored tree, and
one fewer device round trip per failure (heal path drops from 3 captures
to 2, plain failure from 2 to 1).
- Blessing follows the settle/find choke-point sequence with the store
ordered explicitly after markSessionSnapshotRefsIssued, replacing the
ineffective flag flip the review flagged; session-snapshot.ts's
caller-list doc gains the new caller.
- Sparse captures no longer write back (per the selector-capture
reliability contract): a sparse verdict degrades the whole observation to
screen unavailable + no suggestions.
- Capture flavor: interactive-only, except for the non-rect selector reads
(get/is/wait) where heal's suggestion semantics always used a full tree.
- withReplayFailureContext consumes the transport-internal
details.replaySource marker (deepest nested failure wins over the
wrapper's top-level source) and strips it from the flat details; the wire
shaping moved into a pure helper to stay under the complexity gate.
- step.source.line fallback is now ?? 1 (matching the parsers' own
fallbacks) instead of ?? 0, which rendered as "path:0" in the CLI report.
Runtime regression tests cover the reviewer's probe scenario end-to-end
(retry-wrapped and runFlow.when-wrapped includes failing inside the
include) plus a construction-path redaction assertion.
* fix(replay): redact divergence fields before truncation, per the ADR
Review should-fix on #1197: ADR 0012 decision 4 specifies "all rendered
strings and any overflow artifact pass through the central diagnostics
redactor BEFORE truncation"; the implementation truncated at construction
time and relied on normalizeError's later whole-details redaction — i.e.
truncate-then-redact, whose safety depended on the regex redactor's
incidental robustness to secrets split across the cut.
sanitizeReplayDivergenceField (redactDiagnosticData -> truncateUtf8Field ->
marker) is now the sanctioned field sanitizer, used at every divergence
string construction site (cause message/hint, action summary, source path,
screen ref roles/labels, suggestion selectors/labels) and by the minimal
overflow fallback. Tests: a password assignment whose value straddles the
256-byte boundary must yield no secret fragment and a [REDACTED] marker,
plus a no-truncation bearer-token case.
Also picks up the outstanding format-only diff on output.ts (the CI
format:check blocker).
* refactor(replay): one provenance representation through the Maestro pipeline
Maintainer directive on #1197: a workaround needing a paragraph-long
justification comment marks a redesign spot. The named suspect was the
transient SessionAction.replaySource field — attach in runFlow conversion,
strip at two different exits (top-level convertRootCommands, control-flow
stripNestedActionSources), with a doc comment narrating that lifecycle.
The conversion pipeline now returns provenance in a parallel structure
everywhere instead: MaestroConvertedActions { actions, sources } is the
uniform result of convertMaestroCommandWithLine / convertCommandList /
convertRunFlow / convertRepeat / convertRetry. runFlow includes produce
concrete sources; everything else produces undefined ("this file");
control wrappers put nested sources straight into
replayControl.actionSources. SessionAction.replaySource is deleted along
with both attach/strip helpers and their lifecycle docs — one
representation, nothing transient to strip, nothing to narrate.
Comment self-audit across the PR diff: acceptability arguments trimmed to
one-line constraint statements (redaction ordering, single-capture
rationale, --debug gating history, minimal-fallback defensiveness, resume
scope); the arguments live in the PR thread. Behavior unchanged — the
parser/runtime provenance regression tests (retry-wrapped and when-wrapped
includes, deep no-per-action-provenance leak check) pass unmodified in
substance, renamed to describe the carried-structure design.
* fix(replay): categorical fill-text exclusion, repair data on all text surfaces, sanitize/dedupe gaps
Maintainer review blockers on #1197, all three:
1. Fill text never serialized (categorical, not redact-if-secret-shaped).
formatDivergenceActionLabel replaces formatScriptActionSummary (now
deleted — no other consumer remained) for the divergence action field and
the top-level failure message: typing commands (fill/type) drop the typed
value for a `<text>` marker, keeping only the identifying target token
(selector/@ref/point). The flat details.positionals now goes through the
event log's categorical buildDisplayPositionals (`<text:N chars>`).
divergence.cause carries only {code, message, hint} — the fill
verification detail that legitimately holds `expected` is a nested cause
detail and is structurally never serialized, and platform fill failure
messages are static strings. Tests: a sentinel-bearing fill divergence at
every response level asserts the sentinel appears nowhere in the
serialized divergence, flat details, or message; unit matrix for the
label across fill selector/@ref/point and type.
2. Every text surface carries the repair data. The compact divergence
report moved from a CLI-private helper into the shared
formatReplayDivergenceReport (src/replay/divergence.ts) and is now
rendered by all three text surfaces: CLI printHumanError (as before),
MCP text content (tool-error.ts — previously code+hint only), and the
`test` default reporter's failure body (previously message+hint only).
--json and MCP structuredContent were already complete. Tests assert
step location, screen refs line, and ranked suggestions on the MCP text
and test reporter surfaces.
3. Sanitization + dedupe gaps. (a) The capture-failed screen hint
interpolated the raw capture error message — every `unavailable` screen
field now passes through sanitizeReplayDivergenceField (audited all
construction sites; cause/action/source/screen refs/suggestions were
already covered). Test: a secret in the capture error is redacted in
screen.hint. (b) Suggestion dedupe now keys on the node's unique tree
index and keeps the STRONGEST basis per the ADR (previously first-seen
basis won and ref-less nodes could collide/miss). Test: a node reachable
via a label-basis and an id-basis candidate appears once, tagged id.
Also: rebased onto origin/main (ReplaySuiteTestFailed.session fixture
field); the new production-unused-exports gate is clean.
* fix(replay): strip arbitrary cause details from the public divergence error
Maintainer P1 on #1197 (STRIP decision on the escalated scope question):
the flat REPLAY_DIVERGENCE details previously spread the underlying
command error's own details verbatim — and a real fill-verification
failure carries the entered text there (`expected`, unmasked fields, per
the fill-diagnostics contract). Per the ADR ("arbitrary nested cause
details are never serialized"), the transport now drops the cause's
details categorically; only the machine-dispatchable signals
(reason/retriable/supportedOn) survive alongside the transport's own
fields. The documented details-borne meta keys (hint/diagnosticId/
logPath) are hoisted onto the error fields first so repair guidance is
not lost (one pre-existing test updated to the hoisted location).
Privacy regression strengthened to the leak class the selector-miss test
could not see: the failing fill now returns details.expected/actual
carrying the sentinel, and the assertion covers the WHOLE serialized
public error at every response level.
* fix(replay): render repair refs on text surfaces; scrub expanded variables
Review blockers (devin-ai-integration on #1197), both:
1. Text-only repair data was incomplete: formatReplayDivergenceReport
rendered only the ref COUNT for an available screen and dropped the
unavailable hint, so a text-only CLI/MCP/test caller still needed a
follow-up snapshot or a switch to JSON — against the ADR's "no caller
gets a text-only divergence that loses its repair data". The report now
lists a bounded ref/role/label subset (8 lines, matching the digest ref
cap, with a "... N more" remainder) and carries the unavailable-screen
hint. Covered on all three text surfaces (CLI printHumanError, MCP text
content, test default reporter) plus report-level bound/hint tests.
2. Expanded replay variables were not categorically excluded: the generic
redactor cannot know replay-scope values, so `press 'label="${SECRET}"'`
failing with the daemon echoing the RESOLVED selector serialized the
expanded value in divergence.cause and the top-level message (and the
Maestro path bakes expansions into positionals at parse time, reaching
the action label the same way). Every divergence string now passes
through a per-report sanitizer that scrubs all non-builtin replay-scope
values (file env, -e CLI entries, AD_VAR_* shell env, and runtime
outputEnv merges — collected at failure time) to a `<var:NAME>` marker
BEFORE generic redaction/truncation; the top-level message/hint get the
same scrub. Marker replacement, not a drop: the caller still sees which
variable was interpolated. Regression: a selector error echoing an
expanded sentinel (message + hint) asserts the sentinel absent from the
whole public error and `<var:SECRET>` present; plus scrub unit tests
(longest-value-first, non-secret-shaped values).
Fallow follow-ups: the three failure call sites collapsed into one
failStep closure (duplication), divergenceScreenLine split per state
(complexity).
* fix: preserve replay divergence recovery and privacy signals
|
||
|
|
d4146c7f1b |
feat: add Android test IME helper for deterministic text entry (#1198) (#1201)
* feat: add Android test IME helper for deterministic text entry (#1198) Ships a headless InputMethodService (android-ime-helper) as a third Android helper APK, replacing the visible system keyboard during automated sessions. Renders zero accessibility nodes and accepts Unicode/CJK/emoji text over a base64-encoded broadcast channel, fixing both the settle-diff IME-chrome flood and the ASCII-only adb-shell text entry limit in one structural fix. - android-ime-helper/: InputMethodService + build/package scripts on the existing helper-APK toolchain (javac+d8+aapt2+zipalign+apksigner). - src/platforms/android/ime-helper.ts, ime-lifecycle.ts: install/version lifecycle (shared with the other two helpers via the new helper-package-install.ts), activation on session open, and on-device restore-hygiene (previous IME persisted to a device settings key so any daemon/state-dir can recover it; restored on close, daemon teardown, and daemon startup for orphans left by a crashed run). - input-actions.ts: fill/type route through the helper's broadcast channel when active, unicode-safe; unchanged ASCII-shell fallback otherwise. - doctor: new android-test-ime check flags a stuck helper IME with a copy-pasteable `adb shell ime set` remediation command. - Gating: default-on for emulators, opt-in via `open --test-ime` on real devices. - Dead-weight: rewrote the manual ADBKeyBoard workaround doc, dropped the now-provably-live skillgym non-ASCII eval case, updated the ASCII fallback's error message to point at the helper instead of dead-ending. 🤖 Generated with Claude Code Co-Authored-By: Claude <noreply@anthropic.com> * fix(#1201 review): permission-gate the IME receiver, fix CI, add opt-out Addresses the independent review's blockers and should-fixes. SECURITY (blocker 1): the text-injection receiver was RECEIVER_EXPORTED with no gate — any co-installed app could inject text into the focused field while the test IME was active. Fixed by requiring the WRITE_SECURE_SETTINGS sender permission on the (in-process, dynamically-registered) receiver: adb shell holds it, third-party apps cannot. The reviewer's suggested exported=false + explicit-component approach was tried first but empirically breaks delivery on API 36 (adb shell cannot reach a non-exported receiver there) — documented in the helper README. Live-verified: a purpose-built rogue APK's broadcasts (implicit and package-scoped, no permission) are silently dropped, field unchanged; adb shell's bare broadcast still injects. Added ime-helper-security.test.ts asserting the permission gate and that no permissionless exported registration returns. CI (blocker 2): (a) added `testIme` to integration-progress-model flag buckets (Integration Tests was red on the unclassified flag). (b) mocked resolveAndroidImeHelperArtifact in session-doctor-android / ime-lifecycle / input-actions-test-ime tests so they no longer depend on android-ime-helper/dist existing on disk (Coverage was red on a fresh checkout); verified by running them with dist removed. Should-fixes: added `--no-test-ime` to opt out on emulators (tri-state gating, parser-tested); PR body's "byte-identical" claim corrected to size/CRC-match. 🤖 Generated with Claude Code Co-Authored-By: Claude <noreply@anthropic.com> * docs(#1201): pin the API-36 exported-receiver constraint in a comment The RECEIVER_EXPORTED flag cannot express why it must stay exported. Add a one-line note so a future hardening pass doesn't switch to RECEIVER_NOT_EXPORTED and silently break the CLI (adb shell can't deliver explicit broadcasts to non-exported components on API 36+; WRITE_SECURE_SETTINGS is the actual gate). 🤖 Generated with Claude Code Co-Authored-By: Claude <noreply@anthropic.com> * fix(#1201 review): harden IME restore lifecycle (blockers 1 & 2) Blocker 1 — a failed restore no longer deletes the recovery value. restore now reads back default_input_method after `ime set` and only clears the persisted previous-IME record on a confirmed-successful restore; a failed set keeps the value so a later retry / startup recovery / doctor remediation can still un-strand the user off the helper IME. Blocker 2 — startup orphan-recovery no longer overwrites/races user state. It only restores when the device's CURRENT default IME is still our helper (so a user who legitimately switched away is left alone), and skips any device a live session in this process owns (the fire-and-forget startup vs. concurrent `open` race — activate now marks the device active BEFORE the `ime set`, so any recovery pass that could observe the helper active also observes the flag and skips). Never persists the helper itself as the previous IME. activate also verifies its own switch via read-back. Exported ANDROID_IME_HELPER_SERVICE_COMPONENT so restore compares the active IME without reading the packaged artifact from disk. Tests: failed-restore keeps the value (+ later recovery succeeds), startup no-op when current != helper, startup skips a live-owned device. 🤖 Generated with Claude Code Co-Authored-By: Claude <noreply@anthropic.com> * chore(#1201): delete unused ACTION_ENTER path, baseline test-only export seams Rebased onto main (#1202 production-unused-exports gate). Two follow-ups: - Deleted the unused ACTION_ENTER broadcast end-to-end (TS sendAndroidImeHelperEnter + its test, Java handler, README): nothing routes through it — `keyboard enter` uses the keyevent ENTER path — so the new production-exports gate flagged it as dead production code. Removed rather than grandfathered. - Added the three legitimate test-only seams (resetAndroidImeHelperInstallCache, resetAndroidTestImeActivationCacheForTests, setAndroidTestImeActiveForTests) to fallow-baselines/production-unused-exports.json, matching how the sibling helper reset functions (resetAndroidMultiTouchHelperInstallCache, ...) are already grandfathered there. 🤖 Generated with Claude Code Co-Authored-By: Claude <noreply@anthropic.com> * fix(#1201): stop daemon-startup adb spawn on non-Android hosts (macOS Smoke) Root cause of the red macOS Smoke shard (proven, not hand-waved): the fire-and-forget restoreOrphanedAndroidTestImeOnDaemonStartup ran `adb devices` at EVERY daemon startup, on every platform. GitHub macOS runners ship the Android SDK, so this cold-started the adb server mid-replay and destabilized the macOS System Settings replay timing — the failed job's cleanup shows "Terminate orphan process: pid (N) (adb)"; main's green runs spawn no adb. Fix: gate the startup orphan scan behind a host-side marker written in the daemon state dir when a session activates the test IME (mirrors the managed-web-browser orphan-cleanup `installed` gate). A host that never uses the Android test IME — the macOS CI runner included — never writes the marker and so never spawns adb at startup. The marker is cleared once nothing is left stuck. Adds SessionStore.resolveStateDir(); tests: startup recovery does not scan adb when no marker exists (+ marker cleared after a clean scan). 🤖 Generated with Claude Code Co-Authored-By: Claude <noreply@anthropic.com> * chore(#1201): suppress fallow class-member false-positive on state-dir accessor CI's Fallow audit flags SessionStore.resolveDaemonStateDir as an unused class member, but it is called via sessionStore.resolveDaemonStateDir() in session-open.ts — fallow's class-member tracer just doesn't resolve a method call sited inside a call argument. Renamed for clarity (avoids the collision with config.ts's free resolveStateDir) and added the localized fallow-ignore-next-line unused-class-member suppression. 🤖 Generated with Claude Code Co-Authored-By: Claude <noreply@anthropic.com> * fix(#1201 review): durable persist before switch + device-scoped recovery markers Addresses devin-ai-integration's two P1 restore-safety blockers on 19cbce79d. P1.1 — durably persist the restore target BEFORE the global IME switch. writePersistedPreviousIme now checks the `settings put` exit code AND reads the value back, returning a boolean. activate persists first and, if it cannot be persisted, fails open to the existing input path WITHOUT switching — a rejected `settings put` can no longer strand the user on the helper with no restore target. Regression test added. P1.2 — close the marker crash/offline blind spot. Recovery intent is now recorded per device, BEFORE the switch (ordering: durable record -> marker -> ime set), eliminating the post-switch/pre-marker crash window. Markers are device-scoped and each is retained until that device is actually observed clean: an offline/disconnected-but-stuck device keeps its marker and is recovered on reconnect instead of being cleared because the current `adb devices` scan saw no set-failed. Close-time restore clears only that device's marker (stateDir plumbed through teardown/close). Tests cover the persist-failure, post-switch/pre-marker crash, offline-then-reconnect, live-session-owned, and user-switched-away cases. 🤖 Generated with Claude Code Co-Authored-By: Claude <noreply@anthropic.com> --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
|
ee117c4080 |
fix: scope Android settle diffs to app content and collapse IME chrome (#1200)
* fix: scope Android settle diffs to app content and collapse IME chrome Android --settle diffs were flooded by system chrome: status-bar churn (clock/signal/battery) appeared as +/- lines in every settled diff, and summoning the keyboard added dozens of IME ref-bearing nodes that suppressed the unchanged-interactive tail (it fired zero times across the 2026-07-10 Android benchmark, regressing haiku to 2/3 success). Every Android accessibility node carries its owning package (bundleId), so settle chrome is now classified per node: IME-owned nodes collapse to one surviving line per contiguous run (the iOS [Keyboard] analog), and with a session appBundleId all other foreign-package nodes are dropped from both diff sides, mirroring record --scope app (#1163). Classification deliberately never walks parentIndex chains: Android's interactive-only pruning can chain an app node's parent through another window's node (locked in by a real-capture regression test). Inert on iOS/macOS, whose nodes never set bundleId. Benchmark (haiku, settle arm, 3 runs): 2/3 -> 3/3 success, snapshots per run 21/5/6 -> 7/1/3, settle uses 3/9/8 -> 10/11/11. * fix: keep unknown-foreign Android packages in settle output PR #1200 review blocker (live-reproduced with a real Sharesheet): the initial blanket foreign-package drop made settle blind to blocking system dialogs — a ResolverActivity over the app produced an empty diff and a tail pointing at covered buttons, relocating the death-spiral failure mode from IME noise to system dialogs. Invert the default to three per-node tiers: IME packages still collapse to one line per contiguous run; persistent system chrome (exactly com.android.systemui, the status-bar churn that motivated the fix) drops from both diff sides; every other foreign package is kept in full — a permission prompt's or share sheet's buttons stay actionable. appBundleId is no longer a classification input, only the never-drop-the-app-under-test guard, which also defuses the pre-action staleness noted in review. Regression tests use the real captured Sharesheet (raw 41-node + interactive 10-node shapes, package "android" throughout). * fix: drop only status/nav-bar systemui runs from settle output Second review round on PR #1200: com.android.systemui hosts actionable overlays (volume panel, media/output pickers, some system prompts), so package-level dropping reintroduced dialog blindness for that class. Key the drop on window-runs instead, live-verified on the emulator: the status-bar window carries com.android.systemui:id/status_bar* markers throughout while the VolumeDialog window carries only volume_dialog* ids. A systemui run (contiguous same-package parent chain) drops only when a member matches the status_bar*/navigation_bar* marker prefixes; every other systemui surface is kept with actionable refs. Run grouping never crosses packages, preserving the per-node cross-window guarantee. Also corrects the tracking-issue references (#1178 was an unrelated registry issue; the Android settle-noise issue is #1198) and adds a real-capture VolumeDialog regression test: status bar drops, volume dialog survives in both diff and tail. |
||
|
|
d968a27dc3 |
fix: honor session metro hints and expo dev-client bundle urls (#1199)
* fix: honor session metro hints and expo dev-client bundle urls - metro reload now resolves against the dev server the session's last metro prepare bound (via a per-session hint file), instead of silently defaulting to localhost:8081 and reloading an unrelated project. Explicit --metro-host/--metro-port/--bundle-url still win. - metro prepare --metro-kind expo (detected or forced) now hints the virtual-metro-entry bundle URL instead of index.bundle, which 404s against Expo dev servers in monorepos (live-verified against react-navigation's example app). Package-manager detection for --install-deps now walks up from --project-root to the nearest lockfile so Yarn/pnpm workspace monorepos don't wrongly fall back to npm and hit EUNSUPPORTEDPROTOCOL on workspace: deps; install failures now hint at --no-install-deps and the detected PM. - open now accepts --metro-host/--metro-port/--bundle-url/--launch-url as session-hint setters (folded into the same runtime object the daemon already persists for open), so a fresh session doesn't need a throwaway reload-first call just to seed hints. Updates help text for metro and open to match. * fix: make the metro-sessions file the single reload hint store Review follow-up (PR #1199): - open's --metro-host/--metro-port/--bundle-url now also record the session's dev-server binding in the metro-sessions file (the one local store metro reload resolves against), so a later plain reload actually reuses what open set. The daemon runtime-hints write stays for device-native dev-server prefs. - The binding now carries bundleUrl (prepare persists the local-flow bundle URL; bridge runtimes are excluded). - clearMetroSessionHints is wired: session close drops the binding (teardown intent — even when the daemon call fails), and a hintless open that creates the session clears a leftover same-name binding. The open result carries sessionReused so the client can tell fresh from reused sessions. Regression test pins prepare -> close -> flagless reload resolving to the default, never the stale port. - Package-manager detection recognizes bun.lock (text lockfile, default since Bun 1.2) and bounds the lockfile walk-up at the nearest .git entry. - Comment audit per maintainer directive: multi-line acceptability arguments trimmed to one-line constraint statements; the store's lifecycle is documented once on MetroSessionHints. * fix: preserve bundle-url mount prefix and broadcast expo reloads over /message Maintainer review follow-ups (PR #1199): - Reload/message endpoint URLs keep the bound bundle URL's mount prefix (e.g. /tenant-42/index.bundle -> /tenant-42/reload) instead of collapsing to the host root. The Expo virtual entry (.expo/.virtual-metro-entry.bundle) is an entry-module path, not a server mount, so it maps to the server-root endpoints (verified live: Expo serves /message at the root). - When the dev server has no HTTP /reload route and answers with the app page (Expo), metro reload now broadcasts {"version":2,"method":"reload"} over the server's /message websocket (the channel dev-server CLIs use for the r key) instead of reporting the app-page 200 as a successful reload. The result carries a transport field (http | message-socket). Live-verified: a flagless metro reload against the running Expo server made the app re-fetch its JS bundle. - Endpoint resolution moved to src/metro/metro-reload-endpoints.ts so the seam tests use is production-imported (keeps the new production-unused-exports ratchet clean). - Docs reconciled: help metro and website commands.md describe the single session binding store, open's hint flags, prefix preservation, and the message-socket fallback. * fix: preserve Expo reload mount prefixes |
||
|
|
58995bf51b |
feat: parse and preserve .ad target-v1 evidence (ADR 0012 migration step 3) (#1196)
* feat: parse and preserve .ad target-v1 evidence (ADR 0012 migration step 3)
Recording now emits a `# agent-device:target-v1 {...}` comment immediately
before every click/press/longpress/fill action that resolves through the
tree (ref or selector), carrying the identity/ancestry/sibling/scrollRegion/
viewportOrder tuple from decision 3's record-time write algorithm plus a
record-time self-check (verified/unverifiable). The parser accepts known
fields in any order, NFC-normalizes, rejects malformed/oversized annotations
with INVALID_ARGS, binds an annotation only to the immediately-next action
line, and treats unknown future target-vN comments as ordinary comments.
writeReplayScript's read-then-rewrite (heal) path preserves v1 annotations
in canonical form. Nothing consumes the parsed evidence yet beyond
preservation — replay-time enforcement is a later migration step.
The identity tuple's own node/tree only ever lived on the internal
visualization/session-history payload (never the public response); it is
converted to the compact target-v1 evidence and stripped before anything is
persisted or returned.
* fix: bound candidate identity before self-check comparison
The identity-set scan in computeTargetEvidence compared an untruncated
candidate identity against the 256-byte-bounded recorded identity, so a
node whose own id/label exceeds the field cap could fail to match itself,
corrupting the record-time self-check. Compare through the same bounding
on both sides instead.
* fix: address PR #1196 review — contract boundary, formatting, path-6 reasons, worst-case sizing
- native-ref contract test: deliberately updated to assert the new ADR 0012
boundary — the preflight's node/preActionNodes ride the INTERNAL runtime
result and visualization payload only; the public responseData never
carries node/preActionNodes/targetEvidence (asserted directly against
buildInteractionResponseData).
- oxfmt formatting on all touched files.
- classifyTargetBindingMatch path 6 now distinguishes decision 3's two
spec-distinct outcomes: a signal isolating a member that differs from the
winner ('signal-isolated-wrong', the paths-4/5 comparison class, future
identity-mismatch) vs true fall-through ('no-signal-isolation', future
identity-unverifiable), so step 4 can consume the reasons directly.
- writer reduction loop sizes every candidate against the worst-case
verification value ("unverifiable" is 4 bytes longer than "verified"), so
a fail-closed self-check downgrade can never push an accepted payload over
the 4 KiB cap; pinned by a 1-byte-granularity sweep across the boundary
that fails against the old placeholder-sized check.
* fix: reject missing role keys in target-v1 annotations, correct data-flow comment
Second-review nits: the writer emits `role` unconditionally (top level,
ancestry entries, scrollRegion) — possibly as the empty string for typeless
nodes, which stays accepted — so a MISSING role key can only come from a
hand-edited/adversarial annotation and is now rejected with INVALID_ARGS
instead of silently parsing as an implicit empty role, which step-4
enforcement could otherwise match against anonymous wrapper nodes. Also
corrects the interaction-touch-response comment that overstated where the
raw node/tree flows (finalizeTouchInteraction strips it before both session
history and touch overlay telemetry).
* fix: record iOS selector target evidence
* refactor: make the record-time evidence channel structural, drop argument-comments
Maintainer directive: comments that argue a workaround is acceptable mark
code to redesign. Applied to the whole PR diff:
- The record-time node/tree now travel on a typed `recordedTarget` side
channel of InteractionResponsePayloads instead of being smuggled through
the visualization Record and stripped later. The construction site routes
them there exclusively, finalizeTouchInteraction consumes the channel
directly, and extractTargetEvidenceForRecording (the strip helper and its
justifying doc) is deleted — the public/internal split is now enforced by
the type shape, so the contract test asserts it in two lines instead of a
paragraph.
- findNearestScrollableContainer/findNearestAncestor made generic over the
node type, removing a cast plus its safety-argument comment.
- computeLocalIdentity/boundedLocalIdentity collapsed into one always-capped
identity reader — the raw/bounded split was the root of the earlier
self-match bug and existed only to be explained.
- parseReplayScriptDetailed's rejectUnbound takes the pending annotation as
a parameter, removing a non-null assertion and its comment.
- Remaining paragraph-length argument-comments trimmed to one-line
constraint statements (module docs, sizing/floor comments, regex/role
rationales, test comments); spec-mapping docs stay.
* chore: oxfmt the selector-evidence test files
* fix: record get target evidence, fail closed on broken parent linkage
Maintainer blockers on ADR 0012 step 3:
- get text/attrs now records target-v1 evidence: GetCommandResult carries
the resolution tree (preActionNodes, internal — neither the recorded
result nor the public payload copies it), recordIfSession gains the typed
recordedTarget channel, and dispatchGetViaRuntime threads the capture
through. The direct-iOS get query is gated during recording so the
snapshot path supplies the evidence tree, mirroring the tap/fill gating.
find/is stay uncovered: their results are match-set shaped (found flags,
predicate booleans over possibly-many matches), not a single resolved
winner — noted in the PR.
- buildAncestryChain now reports a broken parent walk (dangling parentIndex
or cycle) instead of silently producing a root-like chain; the writer
fails the annotation closed to 'unverifiable' per decision 3's
capture-anomaly rule, and broken candidates cannot prove an ancestry
prefix. Regression tests for both anomaly shapes.
- New ADR 0012 recording tests live in focused files
(interaction-target-evidence.test.ts) instead of growing the
interaction.test.ts aggregation; the earlier additions moved out.
* refactor: extract direct-iOS get selector guards below the complexity threshold
|
||
|
|
46d2931bf0 | refactor: remove redundant facade exports (#1204) | ||
|
|
fb7dbfe6fe | 0.19.3 v0.19.3 | ||
|
|
6e21fedc08 | refactor: remove production-unused exports (#1203) | ||
|
|
47134bf764 |
feat: add derived fail-open check:affected selector (#1195)
* feat: add derived fail-open check:affected selector Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * refactor: simplify selector for complexity gate; add docs Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix: fail open on ambiguous non-source fixtures; guard catalog against real package.json/vitest.config Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * refactor: use src/utils/exec.ts process helpers in check:affected runner Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(check:affected): SkillGym ownership, honest catalog, working-tree discovery - Add SkillGym ownership for skills/ and test/skillgym/; stop short-circuiting their Markdown as docs-only (findings 2 & 4). - Drop the fabricated GitHub 'SkillGym' job: it is a local-only gate, now localRunnable with no CI job, guarded by a workflow-existence self-test (3). - Fold working-tree (staged/unstaged/untracked) state into local discovery and disable rename detection so both rename paths classify (1). - Add run.test.ts entrypoint regressions (real diff/status/rename discovery, --run order/skip/stop-on-failure). Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(check:affected): union staged + unstaged diffs so they cannot cancel A single `git diff HEAD` nets index against working tree, so a staged add and an unstaged delete of the same file cancel and hide it. Collect `--cached` (staged) and unstaged diffs separately and union them; add a cancellation regression test. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(check:affected): cover required suite gates * refactor(check:affected): delegate tests to vitest --------- Co-authored-by: Michał Pierzchała <thymikee@gmail.com> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
|
fb1117f229 |
ci: ratchet against production-unused exports (#1202)
* ci: ratchet against test-only exports Three exported-and-unit-tested-but-unreferenced-in-production incidents this week (#1166 getNearestCommandNames, #1167 buildSettleTail, #1199 clearMetroSessionHints) — the first two were caught by fallow's dead-code check because they had zero importers anywhere; #1199 was missed because a test file imports the export, and fallow's default reachability graph counts a test import as "used". Adds a second, stricter pass reusing fallow's own --production mode (entry.exclude test/story/dev files) via scripts/test-only-exports/check.ts: an export alive in fallow's default graph but dead in its production graph, with no other reference anywhere in its own file, has no production call site — exactly the #1199 shape. Ratchets against a checked-in baseline (scripts/test-only-exports-baseline.json, 77 entries); new findings fail `pnpm check:test-only-exports` (wired into CI's Fallow job and check:tooling). A `// test-seam: <reason>` comment above an export is the escape hatch for intentional test seams. Also extends .fallowrc.json's ignoreExports for seven daemon route handlers (src/daemon/handlers/*.ts) that are genuinely production-reachable through request-handler-chain.ts's `typeof import()` lazy-load pattern, which fallow's static import graph can't trace as a named-export consumer — without this they were false positives in the production-mode pass. * fix: harden test-only-exports ratchet per review Addresses the two should-fixes and all five minors from the independent review of #1202: - Replace the regex own-file occurrence count with an oxc-parser AST walk (typescript@7 ships no JS scanner API, so the review's fallback tool suggestion is the primary): identifiers are counted as AST nodes deduped by source span, so mentions in JSDoc/block comments, strings, and template-literal text no longer masquerade as call sites (review finding 1, both constructed cases re-verified fixed), and a `//` inside a string no longer hides real usages (finding 6). Span dedupe keeps barrel re-exports (`export { x } from`) counting once. The sharper count surfaced one organic false negative on main: `selector` in src/commands/index.ts was previously exempted because the regex matched "selector" inside the './...selector-read.ts' import path string; it is now baselined alongside its sibling `ref` (same re-export line). - Make the baseline shrink-only (finding 2): --update-baseline refuses new findings with the same wire/delete/annotate message, so the `// test-seam:` annotation in the reviewed source diff is the only acceptance path; CONTRIBUTING no longer documents baseline regeneration as an acceptance option and now describes baseline growth as a deliberate manual edit. - Stale baseline entries now emit a `::warning` CI annotation (finding 3). - Commit a re-runnable fixture test (finding 4): check.test.ts mirrors scripts/layering/model.test.ts, builds a synthetic package with a clearMetroSessionHints-shaped export (JSDoc self-mention included), asserts it is flagged, and asserts the annotated twin passes; wired before the check in pnpm check:test-only-exports. - Mark the unreadable/unparseable-file fallbacks CONSERVATIVE: per CONTRIBUTING's convention (finding 5). - Document the dynamic property access (obj[name]) blind spot in the script header and CONTRIBUTING (finding 7). * fix: harden test-only export ratchet * refactor: use native Fallow export gate * chore: refresh production export baseline |
||
|
|
f53d572f87 |
fix: align Maestro swipe semantics across platforms (#1179)
* fix: preserve explicit Android Maestro swipe lanes * fix: align Maestro swipe semantics across platforms * fix: avoid replaying iOS Maestro gestures * refactor: make swipe coordinate policies explicit |
||
|
|
db492cbaed |
test(output-economy): add routine-workflow output-behavior oracle (#1190)
* test(output-economy): add routine-workflow output-behavior oracle Add a deterministic routine-workflow measurement (#1180) that pairs response bytes with follow-up behavior: fallback-observation count, retry count, and whether an actionable failure preserves the session. Refs chain across one recorded checkout session and counts derive from the real formatters, so dropping settled-diff refs, the unchanged- interactive tail, or a recovery handle fails the suite. Adds a matching non-gating help-conformance next-command case. Response defaults unchanged. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(output-economy): make routine-workflow ref-surfacing depend on rendered output Address review on #1190: - Drop the raw mutation-confirm/failure MCP data samples that leaked e4/e5 and mislabeled the surface; e4/e5 now surface only from the rendered CLI settled-diff, so dropping added refs genuinely raises the fallback count. - Track the failure once as its projection-invariant normalized payload (workflow.failure.shared.json) instead of duplicate cli/mcp raw copies. - Reuse the shared REF_TOKEN_PATTERN from economy-metrics.ts. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(output-economy): reuse shared fixtures in routine-workflow oracle Address review finding #2 on #1190: chain the workflow onto the shared per-surface fixtures instead of copy-pasting them. - orient/recheck reuse SNAPSHOT_RESULT + SNAPSHOT_DAEMON_RESULT; the first mutation reuses SETTLE_ADDED_REF_RESULT, so session identity and ref generations come from ./fixtures.ts and the two suites cannot drift. - Only the genuinely workflow-specific pieces remain local: the unchanged recheck, a tail retargeted onto a settled-diff ref (SETTLE_TAIL_RESULT taps an unsurfaced @e6 and can't chain), the in-session timeout failure, and its recovered retry. routine-workflow.ts drops ~110 LOC. - Rendered-output ref guard preserved: @e5 (settled diff) and @e7 (tail) surface only from formatter output; recovery semantics unchanged. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Michał Pierzchała <thymikee@gmail.com> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
|
1f14e224d3 |
refactor(daemon): route perf metrics sampling body through PlatformPlugin facet (#1191)
* refactor(daemon): route perf metrics sampling body through PlatformPlugin facet Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(daemon): cover the shipped perf sampler dispatch path via the facet Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * ci: re-trigger checks (flaky settle-observation integration test) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(daemon): pin perf sampler selection to the facet tag, not the platform Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Michał Pierzchała <thymikee@gmail.com> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
|
ae74c51abd |
chore: add agent-efficiency regression guards (#1174)
* chore: ratchet architecture dependency graph Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test: ratchet agent-facing output economy Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * feat: derive command navigation explanations Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix: keep efficiency checks fallow-clean Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(layering): enforce back-edge ceiling monotonicity and cover root src files Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * ci(layering): pin back-edge-ceiling ratchet to PR merge-base Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(output-economy): baseline-independent actionability floors, policy-derived error, like-for-like screenshot surfaces Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(explain): resolve true CLI aliases, canonical usage, and derived owners Surface true CLI aliases from parser normalization (long-press, metrics, tap, launch, relaunch) distinct from catalog keys, preserving implied-flag semantics (relaunch => open --relaunch). Extract the canonical single-line usage builder to src/utils/cli-usage.ts so schemas without usageOverride include positionals and flags. Replace guessed handler paths with a completeness-checked daemon-route owner map keyed by the closed DaemonCommandRoute union, fixing silently-dropped non-kebab routes (reactNative, recordTrace) and generic dispatch. Add table-driven coverage for aliases, synthesized usage, split-family/route-variant/dispatch owners, structured output, and explain:command CLI exit/stdout/stderr. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix: enforce exact ratchets and compact command explain Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix: colocate command ownership metadata Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix: bind daemon owners to production routes Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix: preserve generic dispatch bundling Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix: enforce monotonic output budgets Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Michał Pierzchała <thymikee@gmail.com> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
|
66fe801377 |
docs: ADR for interactive replay and resolution disclosure (#1177)
* docs: add ADR 0012 for interactive replay, resolution disclosure, and retiring --update healing Records the decision to retire --update healing as a silent actor (repurposing its candidate machinery as ranked suggestions), disclose selector disambiguation in every interaction response, verify replay steps against record-time identity evidence, and add an interactive replay --from loop with a structured divergence report for all callers. * docs(adr-0012): ground in live replay evidence; require step provenance for --from Adds hands-on evidence from driving replay on the RN playground (silent text-mode success, app-state divergence heal cannot fix, Maestro step-index shift from runFlow flattening, per-format hint/code inconsistency, recordings carrying zero observation steps), makes step provenance (source file + line, including through Maestro runFlow inlining) a requirement of the divergence report plus an optional replay --list-steps dry-run, and adds a one-line text-mode success summary as decision 4d. * docs: make interactive replay ADR implementable * docs: tighten interactive replay contracts * docs(adr-0012): demote geometry to disambiguation signal, fix matchCount, define matching algorithm Reworks the target-v1 contract per review: identity is recorded id, else role + normalized label, plus a leaf-anchored ancestry prefix (K=8, nearest ancestors kept, root-side truncation only); absolute rects are demoted to never-compared diagnostics with the ±8 tolerance removed rather than tuned; duplicates disambiguate by recorded sibling order among the matching set, then viewport-relative order within the recorded scroll region — never absolute pixels; ties are identity-unverifiable divergences with candidates listed. matchCount is redefined as the replay-time recorded-selector match count (0..N, always present), with selector-miss (0) and identity-mismatch (>=1, no identity candidate) as distinct classes in an explicit six-path verification classification. Also inlines the quantitative benchmark numbers (3.67->1.00 snapshots, 14.3 vs 23.3/26.7 commands, 38/38 in 539s) so the evidence is durable without the external harness directory. * docs(adr-0012): unify positional-signal candidate domains between record and replay Fixes the P1 domain mismatch: sibling becomes a genuine same-parent child index (parent already captured as ancestry[0], no new field; identical by definition on both sides, non-isolating when the same index recurs under different parents); viewportOrder gets one region-scoped domain — the identity set partitioned by scroll region, ordinal within the recorded partition on both sides, unavailable (never compared cross-region) when the recorded region no longer exists; document order (pre-order index) is the canonical total order making every ordering deterministic, including equal rect centers. Residual ties stay identity-unverifiable with candidates listed. Record-time write, replay verification, and mandatory validation updated in lockstep; the six-path classification is unchanged. * docs(adr-0012): conditional matchCount, dependency-ordered migration, writer invariant, suggestion ranking contract |
||
|
|
b3b0f590b3 |
fix: add curated suggestions for open-url and close-session (#1175)
Last night's help-conformance benchmark showed agents guessing open-url and close-session with no "did you mean" hint (the Levenshtein fallback doesn't cover either token — both exceed the edit-distance threshold for their length). Add curated entries pointing them at `open <url>` and `close`. |
||
|
|
983625fc5d |
feat: fix codex runner, add --override-doc grading, port skillgym quiz cases to help-conformance bench (#1176)
* feat: fix codex runner, add override-doc grading, port skillgym quiz cases The 2026-07-09 evaluation of scripts/help-conformance-bench.mjs found it structurally right but broken for the codex runner (two bugs), thin on coverage (4 cases), sequential, and unable to grade a draft help rewrite without a rebuild. - Fix runCodex: (1) codex exec reads stdin until EOF when not attached to a TTY, and execFile never closes the child's stdin, so every codex call hung until RUN_TIMEOUT_MS with empty output — close stdin right after spawn. (2) `-o outFile` writes the same final JSON that codex also prints to stdout, so concatenating both produced two back-to-back JSON objects that broke every JSON.parse candidate and silently zeroed extractCommands() — prefer the clean -o payload, fall back to stdout only when it's empty. - Add `--override-doc <topicId>=<path>` (repeatable): loads a topic's text from a file instead of shelling out to `node bin/agent-device.mjs help <topic>`, so a draft help rewrite can be A/B graded with zero rebuild. - Port three cases from test/skillgym/suites/agent-device-smoke-suite.ts (settle-diff-is-observation, sample-output-settled-diff-next-target, sample-output-not-settled-needs-observe) as self-contained "next-command quiz" cases, generalizing the scorer to support regex matchers/forbidden patterns alongside the existing named expectations. Fixture output text matches the CURRENT settle rendering in src/commands/interaction/output.ts, including the "unchanged interactive (N):" tail added by #1167/#1172. - Parallelize the runner x case matrix with a concurrency cap (HELP_BENCH_CONCURRENCY, default 4); results still print in the original matrix order. - Extend test/skillgym/README.md's existing pointer to this bench with the new flags. Validated with real LLM calls (both runners, all 7 cases, 14 calls, ~$0.25 total): 13/14 pass; the one fail (claude-haiku-4-5 on dogfood-mode) is a genuine model miss (returned an empty command plan asking for the app name instead of committing to a generic plan), not a bench bug. `--override-doc` demonstrated live: stripping the dogfood doc's evidence-command examples regresses codex:gpt-5.4-mini from 3/3 to 2/3 on the same case, showing the flag both loads and changes grading. * fix: apply live-doc post-processing to --override-doc, fail fast on bad overrides Review findings on the initial version (all reproduced): - HIGH: an override for the --help:first30 doc id skipped the live path's firstLines(text, 30) cap, so a 49-line draft leaked lines 31-49 into the prompt — grading content a live run never shows, on the doc id every case uses. loadDoc now splits source (live shell-out vs override file) from post-processing, and the post-processing applies to both, so an override differs ONLY in where the text comes from. - MEDIUM: an --override-doc topic id no selected case uses was silently ignored (exit 0, real doc graded). Now fails fast listing the valid doc ids for the selection. - LOW: a missing override file threw a raw ENOENT stack trace; expected failures now print one clean Error line. Added --help usage text that documents last-wins semantics for repeated same-topic overrides and the post-processing parity. Guard tests (scripts/__tests__/help-conformance-bench.test.ts, wired into the unit-core vitest project by explicit path): a 49-line fixture whose prompt must keep line 30 and drop line 31, unknown-topic fail-fast with valid ids listed, clean no-stack error for a missing file, and last-wins for repeated overrides. All spawn the script in --dry-run with every required doc overridden, so they need no LLM calls and no built CLI. Live re-validation: a 33-line override of --help:first30 whose lines 31-33 instruct the model to emit a sentinel command; neither claude-haiku-4-5 nor codex:gpt-5.4-mini emitted it (both scored 4/4, matching the live-doc baseline), proving the cap applies end-to-end. |
||
|
|
b8ac75893a |
fix: make settle tail list real actionable targets (#1172)
* fix: make settle tail list real actionable targets Post-merge benchmark of #1167 (React Navigation prevent-remove flow, iOS sim, claude-haiku/sonnet) found the unchanged-interactive tail regressing to chrome-only noise in exactly the cases it was built for: - buildSettleTailEntries required `hittable === true`, stricter than what `snapshot -i` itself shows for the same interactive-only capture. Right after a dismiss animation, real buttons commonly report `hittable: false`/undefined while application/window containers pass, so the tail surfaced two useless chrome lines and dropped the actionable button. Fixed by dropping the hittable requirement and excluding structural application/window roles instead. - withoutKeyboardKeys only stripped `Key` nodes; real keyboard chrome (shift/Emoji/return/Dictate/Next keyboard) are XCUIElementTypeButton nodes and leaked through as fresh added-line refs, which suppressed the tail trigger for exactly the post-fill case it exists for. Fixed by detecting the whole keyboard subtree structurally (via parentIndex, not a locale-fragile label list): descendants collapse out of the diff, and the keyboard container's own ref no longer counts as a "meaningful" added ref for the trigger decision. Nobody presses shift via settle diff refs, so collapsing keyboard chrome does not block a user from explicitly targeting the keyboard itself. * fix: classify the whole iOS keyboard window as settle chrome Live-device review of the first Bug B fix found "Next keyboard" and "Dictate" still leaking into the settled diff as added refs: on a real iPhone 17 Pro simulator they live in a SIBLING subtree of the [Keyboard] container (the candidate bar), not under it, so the container-descendant walk missed them. A raw hierarchy capture shows the software keyboard in its own dedicated window hosting both the container and the candidate bar, so the structural rule is now: every node inside a window that has a [Keyboard] descendant is keyboard chrome. Conservative guard: a window also hosting an editable text node outside the container (iOS puts inputAccessoryView composers in the keyboard window) is never window-classified — those fall back to the container-descendant walk. The live run also showed the filled field re-labeling itself with its new value (ancestor wrappers inherit it), which produced added refs that suppressed the tail even with chrome fixed. The trigger now also ignores self-echo refs — added lines whose settled node rect contains the action point — since they re-describe the acted-on element, not a new target. The fill-keyboard provider fixture is now a trimmed REAL capture from the benchmark flow (sibling candidate bar, main app window absent from the interactive settled capture, self-echo relabels) instead of a hand-built tree that hid the sibling-branch shape. |
||
|
|
c78bfd1e62 |
fix: guide agents past iOS keyboard dismissal and get-text guesses (#1173)
* fix: guide agents past iOS keyboard dismissal and get-text guesses Tonight's benchmark leaderboard showed two recurring agent-UX misses: keyboard dismiss failing after fill (5x, now the #1 failed command) and get-text guessed as a command name (1x). - iOS keyboardDismiss now returns a hint explaining the on-screen keyboard does not block agent-device interactions, so agents should press the next target directly instead of retrying dismiss, and use keyboard enter only when submission is actually wanted. - help manual-qa and help workflow recovery text no longer teach the false "dismiss to unblock the target" pattern. - The command-suggestion curated map now maps get-text/gettext/get_text to the real get text command shape; the generic edit-distance fallback did not produce any suggestion for these guesses. * fix: soften keyboard guidance for genuinely covered targets Review on #1173 flagged the unconditional "does not block" claim as false for targets visually covered by the keyboard: direct-selector press resolution is isHittable-gated and falls through to ELEMENT_NOT_FOUND, the tree path allows non-hittable taps with only a no-visible-effect hint, and bottom-pinned-button-under-keyboard (#291/#469/#957) is a real case where dismissal was the remedy. All three strings (runner hint, help manual-qa, help workflow) now say the keyboard USUALLY does not block presses and name concrete fallbacks: scroll the target into view, or keyboard enter when submission is wanted. |
||
|
|
5c08f1664f | 0.19.2 v0.19.2 | ||
|
|
a3885351c2 |
fix: stabilize android maestro gestures (#1171)
* fix: stabilize android maestro gestures * fix: address maestro android gesture review |
||
|
|
8878399272 |
feat: include unchanged interactive refs in settle output (#1167)
* feat: include unchanged interactive refs in settle output Benchmarks (gpt-5.4-mini + claude-haiku, July 2026) showed 27% of --settle actions were followed by a fallback snapshot -i because a change-only diff omits refs for elements that did not change: after a modal dismiss the diff shows only removals, so the next button to press is invisible. Add an unchanged-interactive tail to SettleObservation, attached only when the diff's added lines carry zero refs (the modal-dismiss/toast-only signature). It lists the settled tree's remaining hittable, uncovered elements so the response stays actionable without an extra round trip. Rides the CLI text, MCP digest view, ref pinning, and output schema the same way the diff's added-line refs already do. * refactor: address fallow audit findings on the settle tail - drop the unused export on buildSettleTail (tests exercise the trigger through the public interaction path and the filter via buildSettleTailEntries) - extract the digest tail capping from interactionSettleView into a module-private helper to stay under the complexity gate |
||
|
|
9a2277c045 |
fix: alias launch/relaunch to open and suggest canonical commands for unknown names (#1166)
* fix: suggest canonical commands for unknown command names Agents commonly guess command names that don't exist, e.g. relaunch/launch instead of `open <app> --relaunch`, burning turns on Unknown command errors that only say "run --help". Add a curated alias-to-canonical-shape map for the most common guesses (launch/relaunch/start/restart, touch, input/ settext/entertext, screencap/capture, dismiss), backed by a nearest-name edit-distance fallback derived from the live command registry so suggestions can't drift. Also hint that `open` takes the app/bundle id as a positional when an unknown flag looks like a bundle-id guess (e.g. --bundle-id), and apply the same suggestion to `help <unknown>`. Suggestions are display-only; nothing auto-executes and the error code stays INVALID_ARGS. * fix: address review — dead export, case-insensitive suggestions, tighter nearest-name matching - Drop the export on getNearestCommandNames (module-private; only suggestCommandFor uses it) to satisfy the Fallow unused-export gate. - Lowercase the input token before both the curated-map lookup and the nearest-name pass, so RELAUNCH/Relaunch/TAP/Touch get the same hint as their lowercase forms. Added a curated `tap` entry: lowercase `tap` is normalized to press before the unknown-command check, so the entry only catches case variants like TAP. - Tighten the nearest-name fallback: exact prefix matches win outright (`clos` now suggests only `close`, not "one of: close, logs"), otherwise only ties at the minimum edit distance are kept, and 1-2 character tokens never get a suggestion (`ls` no longer suggests `is`). - Share the "open <app> --relaunch" example string between the curated map and the unknown-flag hint, and extend the registry-drift tests to parse each curated example end-to-end (validates open --relaunch as a registered flag) plus assert keyboard dismiss is a real keyboard action. * feat: promote launch and relaunch to true open aliases Follow the tap -> press precedent: `relaunch <app>` now runs `open <app>` with --relaunch injected, and `launch <app>` runs a plain `open <app>` (no forced restart — that would silently destroy app state). Both are normalized in normalizeCommandAlias before parsing, so command identity stays `open` for daemon requests and telemetry, all other args/flags pass through to open's normal validation (URL targets still get the daemon's existing --relaunch guidance), and an explicit --relaunch stays idempotent. Alias matching is now case-insensitive (TAP, RELAUNCH, Launch), so the curated tap suggestion entry is dead and removed along with launch/relaunch; start/restart and the rest of the map stay suggestion-only since start is genuinely ambiguous. |
||
|
|
9dbecd02a6 |
fix: make manual-qa help self-sufficient and slim routine help paths (#1168)
* fix: make manual-qa help self-sufficient and slim routine help paths July 2026 benchmarks (gpt-5.4-mini/haiku/sonnet driving the CLI) showed agents burning 14-41KB of tool output on help before a routine QA flow: help react-native routed "generic navigation, selectors, refs, verification" to help workflow (31.7KB), one run read the bare `agent-device help` mid-task (16.3KB), and several fell back to per-command help because manual-qa lacked concrete command shapes. - manual-qa gets a "Command shapes" block covering the full routine QA loop (open --relaunch, deep-link open, snapshot -i, press/fill --settle, wait text, close) plus a quoting note for apostrophe/quote labels, so agents don't need to read help workflow or per-command help for a normal pass. - react-native's routing block no longer sends routine QA to help workflow; it now points routine flows at manual-qa and reserves workflow for deep exploration/debugging. - physical-device and the shared topic-footer routing also point at manual-qa alongside workflow. - Trimmed the bare `agent-device help` output: cut Agent Quickstart lines that duplicated topic-specific detail already owned by react-native/workflow/remote/web/tv, dropped 3 redundant Examples, and fixed an alignment bug where one long usage string (tv-remote) forced padding whitespace onto every other Commands: row. The "Default app loop" line (shown to flip Haiku from 0/3 to 3/3 in earlier benchmarks) stays intact and early. * fix: satisfy oxfmt and restore pinned-ref staleness nuance in help workflow - pnpm format:check failed on the new renderAlignedSection columnWidth line; reformatted with pnpm format. - Reviewer noted the only content genuinely lost from the bare-help Quickstart trim was the "pinned refs get exact staleness warnings" nuance; restored it as one line in help workflow's Snapshots and refs section (its natural owner), keeping the bare-help line short. |
||
|
|
c325842f62 |
fix: reap idle daemons and take over stale runner leases (#1169)
* fix: reap idle daemons and take over stale runner leases Each AGENT_DEVICE_STATE_DIR spawns its own daemon that never exits on its own; deleted codex/claude sandboxes leave orphaned daemons accumulating (10+ observed). A stale-but-still-running orphan also keeps holding its iOS runner lease, so a fresh daemon for the same device fails with COMMAND_FAILED "already owned by another agent-device daemon". - Daemon self-reaps after an idle window (default 5 minutes, matching the iOS runner idle-stop default) once it has no open sessions, no in-flight requests, and no active recording. AGENT_DEVICE_DAEMON_IDLE_TIMEOUT_MS overrides the window; 0 disables it. - A runner lease whose owner PID is dead, or whose owner AGENT_DEVICE_STATE_DIR no longer exists, is now reclaimed automatically instead of erroring; a genuinely live owner with an existing state dir still gets the existing rejection + hint. * fix: gate lease takeover on proven-dead owners and fail closed on stat errors Review follow-ups on #1169: - Replace fs.existsSync (which never throws and swallows EACCES/IO errors into "gone") with fs.statSync + error-code inspection: only ENOENT/ ENOTDIR count as proof the owner state dir is gone; any other stat error fails closed and classifies the owner as alive. - Split stale classification by reason (owner-process-dead vs owner-state-dir-gone). Adoption (readStaleRunnerLease -> tryAdoptRunnerSessionFromLease) is now strictly PID-dead-gated: a dir-gone-but-alive owner may still hold a live runner connection, so its lease routes through the force-stop path (kill leased runner processes, rebuild) instead of being silently adopted - no two masters. - Tests: EACCES stat error keeps the busy rejection; dir-gone+PID-alive refuses adoption before probing; dir-gone force-stop asserts a fresh runner launch instead of adopting the old runner pid. * test: make idle reap tests deterministic |
||
|
|
3667f6ece5 |
fix: point unknown selector keys at role=/label= forms (#1165)
* fix: point unknown selector keys at role=/label= forms press 'button="Push Article"' errored with a nonsense suggestion (text="button=\"Push Article\"") because isSelectorToken rejects `button` as a key, splitSelectorFromArgs returns null, and the point fallback wraps the whole raw token in text=. Add detectUnknownSelectorKeyToken to spot a key=value token whose key isn't a recognized selector key, and use it in readPointTarget to throw a targeted error before the numeric parse: role=<key> label=<quoted value> when the key looks like an accessibility role word (isRoleHintWord, mirroring ROLE_LABELS), otherwise label=<quoted value>. wait-positionals.ts has no analogous point-fallback path, so it needs no change. * fix: fold unquoted multi-word values into the unknown-key suggestion Review follow-up: readPointTarget only inspected positionals[0], so an unquoted multi-word value split across positionals (press 'button=Push' 'Article') dropped the trailing tokens and confidently suggested the wrong completion (label="Push"). Fold trailing positionals into the suggested value like mergeRestIntoSelectorValue does, unless the value was fully quoted (button="Push Article") and therefore complete — that distinction keeps fill's trailing text argument out of the suggestion. Also: drop the dead `text` entry from ROLE_HINT_WORDS (valid selector key, short-circuits in ALL_KEYS first) and note the set is a superset of ROLE_LABELS rather than a mirror; reject whitespace-only values in detectUnknownSelectorKeyToken. |
||
|
|
477c684cf6 | fix: read runner cache package version from project root (#1170) | ||
|
|
ff7fc5ba61 | 0.19.1 v0.19.1 | ||
|
|
888984169b |
fix: make record app-scoped by default (#1163)
* fix: reject recording for failed iOS simulator session * fix: make record app-scoped by default |
||
|
|
0d2b0353ff | docs: clarify agent setup and text entry guidance (#1164) | ||
|
|
cfef0a4bca |
feat: add session event timeline (#1032)
* feat: add session event timeline * fix: support cursor-only event reads * refactor: simplify event log formatting * refactor: trim event log helpers * docs: document session event timeline * refactor: tighten session event log internals * fix: redact event log action positionals by default * fix: align event log after rebase * test: cover events in provider output guard * fix: harden session event privacy * fix: harden event message redaction * fix: harden session event logging --------- Co-authored-by: Michał Pierzchała <thymikee@gmail.com> |
||
|
|
cf31fb3f7b | fix: harden iOS XCTest recovery paths (#1158) | ||
|
|
2717047b86 |
fix: normalize iOS simulator screenshot density (#1160)
* fix: normalize iOS simulator screenshot density * fix: avoid density metadata after screenshot downscale * fix: satisfy screenshot density CI gates * fix: harden screenshot metadata collection * refactor: centralize screenshot density policy * refactor: reuse screenshot density support check |