mirror of
https://github.com/callstack/agent-device.git
synced 2026-09-14 20:06:34 +08:00
codex/2178-interaction-facade
301 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Michał Pierzchała
|
2d2aa6c670 |
feat: add stale device claim release and dead-end recovery guidance (#2162)
* feat: add stale device claim release and dead-end recovery guidance Close the #1320 recovery loop for claims no daemon can settle on its own: - agent-device device release --stale settles a provably dead owner's durable resources through the same exact-owner reconciliation open and daemon startup use, then clears the claim last — daemonlessly, composing a local-only platform gateway in the CLI process. Live, uncertain, PID-reused, and corrupt claims always fail closed and are reported with the reason. - DEVICE_IN_USE conflicts whose recorded owner provably cannot release (dead or superseded) now carry the exact release command as their recovery instead of a status inspection that dead-ended. - device status --stale now offers the matching release command when provably dead owners are listed. - daemon stop now warns in text output when a claim was orphaned (previously visible only via --json) and names the status/release commands. Part of #1320. * test: cover release refusal branches, text rendering, and orphan warnings Changed-line coverage on the stale-release slice was 69.33% against the 70% gate: the refusal-reason branches, the text-mode outcome rendering with the live-owner hint, and the daemon stop orphaned-claim warning had no tests. Cover them directly; the misnamed-claim-file refusal is also pinned. * test: prove resources gate stale release and the scan-to-lock race stays closed Review follow-ups on #2162: - Two end-to-end CLI regressions run device release --stale through the real local gateway against a dead owner whose state dir holds an attributable durable screen-recording/app-log record: an owner-mismatched record and unreadable resource evidence both retain the claim (app-log-owner-mismatch, app-log-descriptor-invalid), proving cleanup must reach a terminal state before the claim can be deleted. - A deterministic race regression holds the per-device claim lock while a release is in flight, replaces the claim with a successor before releasing the lock, and proves the transaction reports changed without reconciling or touching the successor's claim. * fix: bind stale-claim recovery to the dead owner's state dir Review P1 on #2162: the CLI composed one gateway from the caller's state dir, so recovery for a foreign stale claim could clear the caller's live owned-process record when both used the same session name — Apple recording cleanup clears by session id through the gateway-composed store. Recovery is now composed per claim, with the owned-process record store and session artifact paths bound to the stale claim's recorded state dir, and disposed after each transaction. The regression writes two dead claims with one shared session name in different state dirs and proves each recovery is composed from that claim's own state dir, never the caller's. |
||
|
Michał Pierzchała
|
b7e4b5aa7b | fix: improve ref-based interaction recovery (#2167) | ||
|
Szymon Dziedzic
|
d9677301f4 |
feat: add human takeover controls (#2078)
* feat: add human takeover controls * fix: harden human takeover controls * fix: align host XCTest selection count * fix: address takeover readiness feedback * fix: handle macos runner permission prompt in smoke tests * fix: detect background macos permission dialog * fix: dismiss inaccessible macos privacy sheet visually * fix: map macos privacy prompt without accessibility * refactor: own human-control holds in lease registry * fix: cancel pending human takeover on disconnect |
||
|
Michał Pierzchała
|
315a0ac679 |
fix(daemon): fail closed when the auth hook is silent about tenant (#2104)
An auth hook that ran but returned no tenantId opted the deployment into tenant attestation; falling back to the client's own claim (RPC body meta.tenantId, aux-route x-agent-device-tenant header) let a holder of one valid shared token impersonate any tenant on /rpc and on the diagnostics/ upload/download routes. resolveTrustedTenant() in the new src/daemon/server/tenant-trust.ts is now the single seam both surfaces go through and the only place that computes the resulting identity: hook attests -> use it; no hook configured -> keep today's client-declared behavior (loopback/dev unchanged); hook configured but silent with a client-declared tenant -> refuse (401) instead of trusting the claim, and no raw client-declared metadata survives into the dispatched request in that case either. Fixes #2095 |
||
|
Michał Pierzchała
|
74a70f1764 | refactor: remove next-major compatibility surfaces (#2046) | ||
|
Michał Pierzchała
|
17cfd8ca8a |
feat: add deterministic device selection resolver (#2020)
* feat: add deterministic device selection resolver * test: adapt open selection harnesses * chore: keep context glossary within budget * fix: separate device identity from selection filters * refactor: make the selection resolver the sole owner of selection provenance Simplifies the deterministic device-selection resolver (net -114 lines vs the previous head) while fixing the outstanding app-aware provenance finding: - Move the booted-simulator app-affinity narrowing into the resolver behind an appleSimulatorAppTarget param, with its own typed reason 'single-app-installed-local' (candidateCount 1). This removes the selectedDevice escape hatch that reported 'preferred-local' with candidateCount 2 for the app-narrowed pick, and gives the app-match errors the same platform-aware retry selectors as every other selection failure. - Delete dead code: the allowBootableLocal param (no caller ever passed it, so the eligibleDevices filter was unreachable), the hasExplicitProviderIdentity alias, the duplicated deviceCandidateDetails in dispatch-resolve, and the double candidate computation. - Shrink the public selection contract to what #1777 specifies: drop `booted` (it contradicted its own doc comment once markSelectionBootedAfterPreparation flipped it; bootOccurred plus the reason codes carry the same information) and drop `retrySelectors` from success metadata (the daemon only ever emits retry selectors inside error details). DeviceSelectionRetrySelector leaves the contracts facade. - Replace the typeof-import lazy seam and resolver threading through four context objects with one lazy forwarding wrapper; the dispatch eager closure stays at 83. - Collapse the Apple path to resolve -> optional simulator fallback; the provider branch goes through the generic resolver call directly. - Consolidate the five copy-pasted resolveTargetDeviceSelection test mocks into one shared stub (selectionFromResolveTargetDevice). - Add the requested regression: two booted simulators with the app on one now assert typed selection metadata through resolveTargetDeviceSelection, plus a resolver-level app-affinity provenance test. Validation: typecheck, oxlint, oxfmt, layering (184-check guard OK), DI seams, fallow changed-files, eager-closure 235/235, daemon suite 323 files / 2287 tests, core/commands/mcp/client suites 255 files / 2133 tests. |
||
|
Michał Pierzchała
|
021fe2aa1d |
feat(maestro): support assertTrue phase 1 - literal/${VAR} truthiness (#1295) (#2010)
* feat(maestro): support assertTrue phase 1 - literal/${VAR} truthiness (#1295)
Adds the assertTrue command, scoped to literal values and bare ${VAR}
lookups per the #1292 lookup-only decision; JS expressions keep
failing loud at parse time with a runScript hint. Truthiness on a
looked-up value is evaluated against a pinned falsy-string table
("", "false", "0", "null", "undefined") since flow config/env/
runScript-output values are always stored as strings, rather than
native JS truthiness (which would treat "false" as truthy).
Wires assertTrue through the parser, interpreter, optional/warning
composition, and the layer-1 conformance oracle, narrowing the
067_assertTrue_pass divergence to the JS-expression case and removing
the now-satisfied 076_optional_assertion entry. Also materializes
scrollUntilVisible's default direction in the conformance canonical
projection, a latent gap only exposed once 076 could fully compare.
* fix(maestro): correct assertTrue truthiness claim in CLI help/docs
The support-matrix text said assertTrue is "evaluated with JS
truthiness", but the engine actually uses a pinned falsy-string table
("", "false", "0", "null", "undefined") since looked-up values always
arrive as strings — native JS truthiness would treat "false" as
truthy. Spell out the actual rule instead of the misleading claim.
* fix(maestro): fix oxfmt quote-style violation in expected-divergence.ts
CI's format gate failed on a single-quoted string containing an
apostrophe; oxfmt prefers double quotes there.
* fix(maestro): bump eager-closure-budget pin for the new truthiness module
engine-truthiness.ts is a genuinely new module on the core interpreter
path (assertTrue is dispatched unconditionally by
replay-plan-step-execution.ts), so packages/maestro/src/index.ts now
eagerly evaluates 105 modules instead of 104 — a deliberate growth,
not a laziness regression.
---------
Co-authored-by: Claude <noreply@anthropic.com>
|
||
|
Michał Pierzchała
|
957a6727f8 |
fix(android): publish covered state from exact order evidence (#1981)
* fix(android): unify snapshot occlusion across API levels * fix(android): preserve exact occlusion evidence * fix(android): restore collective occlusion coverage * fix(android): preserve snapshot evidence across consumers |
||
|
Michał Pierzchała
|
3e06884fcd |
fix(ios): verify fill's synthesized-replacement route before reporting success (#1995)
* fix(ios): verify fill's synthesized-replacement route before reporting success
The channel-penalized fill route (runSynthesizedReplacementRoute, taken when
the XCTest accessibility channel is already penalized under load) posted the
synthesized keystrokes and returned ok:true without ever reading the field
back, so a dropped or still-in-flight character was indistinguishable from
success. This is the same class of bug already fixed for bare `type`
(#1676/#1924), but that fix never covered this fill-only route.
Reusing type's append-mode commit-wait verbatim would have been wrong: its
"observed value isn't a prefix of expected -> trust the app" rule exists to
tolerate legitimate transforms (autocomplete, formatters) during append, but
it also waves through a dropped-middle-character corruption, since a value
with a hole in it is neither a matching prefix nor an exact match. Verified
against the real corruption strings ("Ada Lovelace" -> "Avelace", "ada@example"
-> "aexample") that the old rule would classify both as "trust it" and never
fail. Added a separate replacement-mode outcome function with no such
escape hatch, consistent with how isRepairableTextEntryMismatch already
treats every .replacement-mode mismatch as failing/repairable unconditionally.
Verified on a real iOS Simulator via xcodebuild test-without-building, not
just a build: the new regression test proves the old model accepts both
corruption strings while the new one correctly reports commit-not-observed,
and the full pre-existing append/type test suite passes unchanged.
* fix(ios): restore labeled observe-closure call sites for the redaction guard
The previous shared-plumbing refactor passed each route's outcome function
as a stored closure parameter, which erases Swift argument labels at the
call site. That broke the CI "Coverage" check's static content-redaction
test (apple-runner-log-redaction.test.ts), which locates the commit wait's
observe closure by its literal `observe: {` label to verify it only logs
polled field content through the value-free logCommitCadence boundary,
never a raw NSLog.
Restructured so the shared placeholder/deadline/observe/pacing ingredients
are still factored into one place, but each of the two public entry points
(append/type, replacement/fill) now calls its own named outcome function
directly with real argument labels, restoring the labeled closure shape the
guard depends on. Verified locally: the TS redaction test passes, and the
full on-device unit test set (19 tests, iOS Simulator) still passes with
zero regressions.
|
||
|
Michał Pierzchała
|
766d42e124 |
fix(ios): report an unobserved text commit instead of a partial success (#1924)
* fix(ios): report an unobserved text commit instead of a partial success awaitSynthesizedFirstResponderCommit returned Void, so its three exits were indistinguishable to the caller: the expected text committed, the app transformed the input, or the 3s deadline expired with a strict prefix still outstanding. The caller returned dispatched-with-no-failure in all three, and `type` answered ok with textEntryRoute synthesized-first-responder over a field holding part of the requested text. The wait now returns a SynthesizedTextCommitOutcome and an expired deadline becomes TEXT_INPUT_COMMIT_NOT_OBSERVED, whose hint points at fill rather than a type retry — type appends, so retrying it concatenates onto whatever committed. The tail is still not re-synthesized: #1676 rejected that because a stalled prefix cannot be told apart from a suffix still queued, so repair double-posts. Reporting is what the runner does instead. typeIntoCurrentTarget loses its `dispatched` flag, which was exactly `failure == nil` and could not express the new state — characters posted, commit unconfirmed, command must refuse. Failure is now the single discriminator. The decision moves behind an injected clock/observer so the deadline branch runs in the macOS host lane on every PR instead of needing a simulator. Refs #1874, #1844 * fix(ios): close false-failure windows in the commit wait Adversarial review found two deterministic false failures in the wait added by the previous commit, plus a message that asserted a field state never read. The deadline was checked before observing, so a commit landing during the final poll sleep was condemned as never observed — under exactly the loaded-host timing the wait exists for. The check now runs after an observation, so the last thing before condemning is a read. `treatingPlaceholderAsEmpty` maps a value equal to the field's placeholder to "", a prefix of every expected value. `type "0.00"` into a field placeheld "0.00" committed instantly, read as pending for the full 3s, and failed. The observation now settles on an exact raw match; the normalized read still drives the prefix walk. The outcome-to-failure mapping moves to textEntryFailure(forCommitOutcome:) so the branch the command refuses on is pinned by a test rather than living only in a ternary. `.unobservable` staying a success is what keeps `type "...\n"` working, and it now has an assertion. Message and hint no longer claim the field holds a partial value: under both fixed windows it may hold all of it. The docs sentence no longer implies every text-entry route verifies its result — the replacement and keyboard-visible routes have no resolvable element to observe and are unchanged. Refs #1874, #1844 * test(ios): pin the placeholder fix at the boundary it actually lives on Review [P1]: testValueEqualToThePlaceholder… injected an observe closure that already returned "0.00", so it never supplied the normalized "" that causes the failure. The raw-value short-circuit lived in the production observe closure, which that test bypassed entirely — reverting the fix left it green. The raw-exact/normalized-prefix choice moves into commitObservation, and the test drives it with (raw: "0.00", normalized: "", expected: "0.00"). Reverting commitObservation to always return the normalized reading now fails the exact-match assertion. normalizedValue is a closure rather than a value so an exact match still costs one accessibility read instead of two, on a path that polls every 20ms for up to three seconds; a second test pins that laziness. The old test is deleted rather than kept: its remaining assertion (an exact match settles without polling) is already covered by testSynthesizedCommitStopsAtTheFirstSettledObservation. Refs #1874 * fix(ios): never treat placeholder equality as commit evidence Review [P1]: an empty text field renders its placeholder AS its accessibility value, which is why editableTextValue(treatingPlaceholderAsEmpty:) classifies that value as empty. The previous revision's raw-exact short-circuit therefore matched BEFORE anything committed whenever the requested text was the placeholder: `type "0.00"` into a field placeheld "0.00" settled on the first read and returned ok with zero characters delivered — reintroducing the success-misdescribes-the-device failure this PR exists to remove. The state is structurally indeterminate. element.value is identical whether the placeholder is rendering or the committed text happens to equal it, and placeholderValue does not disambiguate, so no read resolves it and waiting the deadline out discovers nothing. placeholderMakesCommitUnobservable detects it up front and reports the commit unobserved, which the caller refuses on. commitObservation is deleted rather than narrowed: the raw match was only ever consulted in this exact case, and in this exact case it is not evidence. The failure message drops its deadline reference — this refusal never waits. Refs #1874 * fix(ios): scope the placeholder refusal to an empty baseline Review [P1]: the guard took only the placeholder and the expected text, so it refused any append whose result happened to equal the placeholder. Value "0" + `type ".00"` against placeholder "0.00" was refused before a single read, even though the non-empty pre-dispatch value proves the placeholder is not what is rendering and a later "0.00" is genuine commit evidence. The baseline is what decides it, so it is now an input. placeholderCommitEvidence returns three states rather than a boolean: normalRead expected differs from the placeholder; the placeholder never enters into the observation indistinguishable expected IS the placeholder and the field was empty, so the placeholder was what rendered and no read can resolve it rawValueIsEvidence expected IS the placeholder but the field held content, so a raw match is real Only .indistinguishable refuses, and it still refuses before the wait, since no read resolves it. .rawValueIsEvidence reaches the observation and settles on the raw match, which the normalized read would otherwise hide. commitObservation returns for that third state, now scoped by evidence rather than applied unconditionally as in the revision that made raw equality a false success. Both readings stay closures, so normalRead — the ordinary case — never pays for the raw read. Refs #1874 * fix(ios): keep placeholder-equal commits conservative |
||
|
Michał Pierzchała
|
4137e4275e |
feat(ios): split raw and regular snapshot projections behind one capture hint (#1926)
The private-AX backend interpreted `--raw` as the regular projection: it folded the viewport and scroll clips and dropped sub-pixel decorations before returning, so a raw capture that recovered onto it answered with viewport-pruned nodes labeled raw (#1797 D4). Nothing related the two copies of that decision. Presentation now exposes the two projections it always implied. `presentRegular` folds visibility, eligibility, scope and scroll hints; `presentRaw` is the acquired tree, normalized, narrowed only by a scope or depth the request asked for. Acquisition reads one derived `CaptureHint` instead of the request itself, so what a capture may skip is stated once, beside the reason skipping it keeps the projection complete: scope and its relative depth never narrow, raw depth does (raw depth is traversal depth), and the raw projection never carries `interactiveOnly` — `--raw -i` is the acquired tree. Two structural rules replace the hand-synchronized ones. The raw plan is derived from `SnapshotBackendKind.supportsRawProjection`, so the query sweep — an interactive element query with no hierarchy to return — cannot be planned for a raw request. And presentation compares the requested projection with the hint the acquisition was captured under, dropping that tier with a structured `IOS_SNAPSHOT_PROJECTION_MISMATCH` failure rather than presenting it under the requested label. Declared residue: a regular `--depth` request still cuts the traversal at that depth while regular presentation emits collapsed depth, so a node that would present within the limit can be dropped. The cut is what keeps `--depth 1` probes cheap; making it complete is the open visible-depth frontier obligation. Refs #1797 (migration step 3, raw-projection delta). |
||
|
Michał Pierzchała
|
be51870118 |
fix: make iOS scroll release controlled (#1906)
* fix: reduce iOS scroll overshoot * fix: make iOS scroll release controlled * fix: make controlled iOS scrolls deterministic * fix: preserve continuous drag sampling |
||
|
Michał Pierzchała
|
5df5ec469d |
feat(maestro): add positional selectors (#1911)
* feat(maestro): add positional selectors * perf(maestro): resolve selectors once per snapshot |
||
|
Michał Pierzchała
|
0f05fa38a5 |
feat(maestro): add recursive tree selectors (#1910)
* feat(maestro): add recursive tree selectors * perf(maestro): resolve scroll selectors once |
||
|
Michał Pierzchała
|
f065e6aeb4 |
fix(maestro): align label metadata ownership (#1909)
* fix(maestro): align label metadata ownership * fix(maestro): centralize command label parsing * test(maestro): cover runFlow label ownership |
||
|
Michał Pierzchała
|
48c34a686b |
docs: simplify AI SDK integration guide (#1891)
* docs: simplify AI SDK integration guide * docs: add AI SDK prerequisites * docs: use supported dotenv code fence |
||
|
Michał Pierzchała
|
2eeb9c644f | fix(maestro): refresh target after selector miss (#1886) | ||
|
Michał Pierzchała
|
9ce1ef7587 |
feat(snapshot): move scope into presentation (#1855)
* feat(snapshot): move scope into presentation Use one preorder label/identifier/value policy across Swift and TypeScript, keep scoped iOS acquisition conservative, and remove the daemon's second scope pass. Non-vacuity: label-only matching failed identifier/value parity fixtures; Android pass-through failed its boundary test; disconnecting Swift applyScope produced eight scope/depth/projection failures. * fix(snapshot): select scopes with presented content * docs(snapshot): describe presentation-owned scope * docs(snapshot): record contribution-aware scope * refactor(snapshot): drop unrelated provider churn * fix(snapshot): accept healthy empty scoped capture * refactor(snapshot): isolate empty-scope admission * fix(snapshot): align scope ownership across runtimes * test(snapshot): pin post-wire scope owner * test(snapshot): retain find test shrink |
||
|
Michał Pierzchała
|
84e6f9bf2d |
refactor(android): raw is the acquired tree; one presentation for dialog recovery; residues declared (#1832 C3) (#1865)
* refactor(android): raw is the acquired tree; one presentation for dialog recovery; residues declared (#1832 C3) - C3: the three regular-projection pruners (invisible subtrees, stale application windows, covered same-window surfaces) move out of parseUiHierarchyTree into the projection as a non-mutating classification (collectAndroidHiddenNodes in ui-hierarchy-visibility.ts). --raw presents the acquired tree; interactive ⊆ regular ⊆ raw by construction. Hidden-content hints and the scope root are derived per projection over retained children, which is what the mutating pruners implied. Property-checked identical to main for regular/-i/depth/scope over 12,000 random tree × projection pairs; raw grew on 2,514/3,000 and never shrank. - Android blocking-dialog recovery routes through buildSnapshotState (the one presentation), which moves to src/daemon/snapshot-state.ts below the daemon-server type cycle; importers repointed, its tests mirror the module. - Freshness route signature drops role/selected (Android never carries them). - Residues declared at their sites and in CONTEXT.md; docs + CHANGELOG. - ui-hierarchy.ts split by question: node predicates (ui-hierarchy-node.ts), regular-projection visibility (ui-hierarchy-visibility.ts), scope (ui-hierarchy-scope.ts); 974 → 634 LOC. * test: lower the snapshot.test.ts size pin to its new length * fix(android): dialog recovery acts on the presentation's occlusion result Review P1 on #1865: routing blocking-dialog recovery through buildSnapshotState made the occlusion result available but nothing consumed it. containsBlockingDialog scanned every node and findCloseAppButton returned the first text match with a rect, so a stale ANR surface left under the foreground one could still trigger recovery, and a covered "Close app" could be tapped ahead of the visible top button — the disagreement the routing was supposed to remove. Both decisions now filter through isSnapshotNodeInteractionBlocked, the shared predicate over the annotator's structured result. Two regressions cover it, both proven red against an unfiltered selection: a covered Close app preceding a visible one (asserts the visible center is tapped) and a fully covered dialog signal (asserts recovery does not trigger, no tap dispatched). Rebase reconciliation: screenshot-runtime.ts arrived on main (#1878) importing buildSnapshotState from its old home; repointed to src/daemon/snapshot-state.ts with the other importers. |
||
|
Michał Pierzchała
|
d29dc22861 |
fix(android): warn when a permission revoke kills the session app (#1856)
* fix(android): warn when a permission revoke kills the session app settings permission deny|reset maps to pm revoke, and Android kills the app's process whenever a runtime permission it currently holds is revoked, so a grant -> deny/reset sequence silently left the session on the launcher and the next selector failed with no hint. The revoke path now reads the prior grant state from dumpsys package first and, when it was granted, returns wasGranted: true plus a warning naming open <app> --relaunch; the settings CLI output renders response warnings, and commands.md documents the behavior next to the pm revoke mapping. Closes #1796 * fix(android): state the revoke-kill consequence conditionally Review finding: the warning asserted the app had been killed, inferred only from the prior grant state. dumpsys reports granted=true for any user profile while pm revoke acts on the current one, and the app need not have been running, so the claim could be false. State the platform rule and make the consequence conditional; the relaunch guidance is unchanged. * fix(android): model the prior grant state as granted/not_granted/unknown A failed or unparseable dumpsys read as "nothing granted", so the response asserted the app was untouched when the state was simply unknown, and the grant scan matched every granted=true line — install permissions and other users' blocks included — so another profile's grant could claim a kill that never happened. Both directions of the same defect. The read now resolves the acting user (am get-current-user) and walks the dump's nesting (Packages: > User <id>: > runtime permissions:), and reports priorGrantState: granted | not_granted | unknown. unknown carries the same relaunch guidance without claiming what the state was; only not_granted is silent. * fix(android): address the foreground user in every permission mutation The tri-state read scoped state to am get-current-user, but the mutations ran bare pm grant/revoke and clear-permission-flags. PackageManagerShellCommand defaults those to UserHandle.USER_SYSTEM, so on a device whose foreground user is nonzero the command read one user's state and edited user 0 — leaving the running app's permission untouched while reporting on a user it did not change. Proven on a Pixel 7 / API 36 emulator with the foreground user switched to 10: a bare pm revoke flipped User 0 to granted=false and left User 10 granted=true. The foreground user is now resolved once and passed as --user to pm grant/revoke, pm clear-permission-flags, and appops set, and the state read takes that same id. When it cannot be resolved the mutation keeps the platform default and the state is reported unknown rather than guessed. * test(android): pin the user-scoped permission argv in the provider scenario The scripted ADB provider answered only the unscoped pm grant/revoke form, and the Settings contract asserted the unscoped transcript entry, so the provider lane could not see which user a permission mutation addressed. * test(android): extract the settings contract out of the lifecycle monolith The user-scoped argv assertions pushed android-lifecycle.test.ts past its size ratchet, whose instruction is to extract rather than grow a file over the tripwire. assertAndroidSettingsContract moves to a sibling module and the pin drops 1597 -> 1559. * refactor(android): shrink the permission path to one concept per file Size/design pass on the #1796 change: - settings.ts was 505 lines (past the 500 extract-before-adding tripwire); the permission family moves to settings-permission.ts and the dispatcher drops to 265. - permission-grant-state.ts loses topLevelSection (a nestedBlock with an indent-0 header), its single-use line reader, and androidPriorGrantState (one map lookup at its only production call site). - the grants map narrows to 'granted' | 'not_granted': unknown was never a value, absence is what carries it, so the tests read the map directly. - the permission tests move to settings-permission.test.ts and consolidate into argv/tri-state/photos/rejection tables; the parser tests fold seven cases into two. Every red-proof re-run after the consolidation: dropping --user reds 7 argv/photos cases, and the pre-fix state model reds 13 across both files. * fix(android): refuse permission mutations that cannot name their user The fallback issued bare pm/appops commands when am get-current-user did not answer, which is the #1796 defect itself: those default to UserHandle.USER_SYSTEM, so a session running as user 10 had user 0 edited while the response reported only priorGrantState: unknown. It was also a fallback added without approval, and the docs' claim that every mutation names its user was false on that path. Resolving the acting user is now a prerequisite: setAndroidSetting permission fails with COMMAND_FAILED and a recovery hint, issuing no pm, appops or clear-permission-flags call at all. The test that locked the fallback in is replaced by one asserting the empty mutation call list for grant, deny and reset. |
||
|
Michał Pierzchała
|
67ce19b50c |
fix(daemon,kernel): device-selection safety — identity conflicts and ambiguity fail instead of retargeting (#1880)
* fix(daemon): session-lock identity conflicts fail instead of changing device identity
`--session-lock strip` resolved every conflict by deleting the offending selector, including
--udid/--serial/--device. The command then ran against the BOUND session's device rather than the
one the caller named, and the error hint that produced this state actively recommended strip. A
wrong-device tap that reports success is worse than any loud failure, so:
- a conflict on a device IDENTITY selector now fails under both reject and strip; strip keeps
resolving platform/scope selectors (--platform, --target, --ios-simulator-device-set,
--android-device-allowlist), which is what it exists for;
- the structured error carries both sides (requestedDevice, boundDevice) so a caller can choose a
recovery without parsing prose;
- the fresh-session hint offers the two real recoveries — close the bound session if the requested
device is intended, drop the selector if the bound device is — and never mentions strip for an
identity conflict. The existing bound-session hint already stated both and is unchanged.
Admission-layer only: this is device/session resolution policy, evaluated before any interaction
dispatch path is selected, so no ADR 0011 guarantee cells move.
One table pins the crossing: fresh vs existing session, matching/conflicting/absent identity,
reject vs strip, binding vs inventory command, Android serial / Android udid / Apple udid, with the
exact error details and hint asserted. 7 of its 13 rows are red against the previous policy.
* fix(kernel): validate platform-specific device flags before resolution
`--udid` enters Apple resolution unconditionally, so `--platform android --udid emulator-5580`
answered "No Apple device with UDID emulator-5580" — an answer about a platform the request had
explicitly excluded, which reads as a missing device rather than a mistyped flag. Both directions
now fail as INVALID_ARGS naming the correct flag (--serial for Android/HarmonyOS, --udid for Apple).
Requests that name no platform keep the existing DEVICE_NOT_FOUND behavior, since nothing
contradicts the selector there.
* fix(kernel): refuse ambiguous singular device resolution instead of picking one
resolveDevice answers with exactly one device, so every caller of it needs one concrete device. When
the request carried no identity and several candidates were equally preferred, it returned the first
by discovery order (or alphabetically for Apple) — a successful response describing a device the
caller never selected. That is worse than any loud failure, and reads are not safer than writes: a
snapshot of the wrong emulator is a wrong answer that looks right.
Ambiguity is now a refusal at the resolver boundary, not a command-kind allowlist:
- established preference tiers are preserved (virtual over physical, the Apple kind/target rank,
then booted over offline); only what survives them equally is ambiguous, since the comparator's
remaining tie-breaks are name and discovery order, which encode nothing about intent;
- one booted emulator beside offline candidates still resolves, as do explicit --device/--udid/
--serial and any command inside an existing session, whose identity is already fixed;
- multi-device commands ("devices") never enter singular resolution;
- the error reuses the declared device-candidate details domain (AMBIGUOUS_MATCH + "devices"), so
CLI and MCP already render the bounded list, with a hint naming the platform-appropriate selector.
* fix(kernel): drop the mismatched article from the selector-flag hint
"Use --serial X to select a android device by serial" — the platform name is interpolated, so no
article fits every value. Names the flag's platform family instead.
|
||
|
Michał Pierzchała
|
8c06965d28 |
fix(daemon): cap events.ndjson with cursor-safe rotation (#1867)
* fix(daemon): cap events.ndjson with cursor-safe rotation Rotate events.ndjson to events.ndjson.1 once it reaches AGENT_DEVICE_EVENT_LOG_MAX_BYTES (default 5 MB), keeping one rotated generation. Cursors stay absolute across rotation through a sidecar window offset, so a persisted nextCursor still names the same event; a cursor older than the retained window fails with COMMAND_FAILED and details.reason EVENT_LOG_CURSOR_EXPIRED instead of returning a wrong page. Closes #1788 * fix(daemon): verify the events.ndjson window against the files on disk Rotation recorded only a dropped-line offset, written after the rename, so a reader landing in that window mapped every absolute cursor a whole generation too far (reproduced: 5 of 58 reads returned event 17 for cursor 9), and a missing rotated file or stale sidecar shifted cursors permanently and silently. The sidecar now records each retained generation's first absolute line index, line count, and first-line digest, and is written before the rename it describes. The reader identifies each file on disk by digest, derives its start from the matching record, and checks the recorded line count and generation contiguity; anything unverifiable raises a typed EVENT_LOG_WINDOW_UNVERIFIED instead of a guessed offset. A torn snapshot (rotation landing mid-read from the threadpool) is retried, not interpreted. A corrupt sidecar fails reads typed and never blocks appends, and rotation no longer does synchronous whole-file I/O. * refactor(daemon): split event-log window placement and share one line splitter |
||
|
Michał Pierzchała
|
294654a3a5 |
fix(android): resolve snapshot scope once and disclose the API 23 occlusion-scan gap (#1832 C1/C2) (#1846)
* fix(android): resolve snapshot scope once and disclose the API 23 occlusion-scan gap (#1832 C1/C2) - Android resolves --scope inside its projection only, under the shared scope specification (matchesSnapshotScope in @agent-device/contracts/snapshot: first document-order match over label/value/identifier, empty on no match). The daemon post-wire scopeSnapshotNodes pass skips the android backend, so scope has one owner and one no-match semantics instead of BFS+fallback followed by document-order+empty. - Golden table contracts/fixtures/snapshot-scope-policy.json is asserted against the predicate, the Android projection, and the daemon pass; the Swift runner twin (#1797) consumes the same table. - androidSnapshot.occlusionScanUnavailable discloses helper trees without drawing-order (API 23), where the covered-sibling pruner cannot run. Disclosure only; C1 stays open until occlusion moves to the daemon annotator. * fix(android): resolve scope over the presented tree and stop dropping it on interaction captures Adversarial review findings on the first commit: - BLOCKER: captureSnapshotData spread `snapshotScope: undefined` over flags, so an interaction capture (press/click/fill/longpress/hover --scope, --settle observation) reached the Android platform unscoped while buildSnapshotState still saw the scope. The post-wire pass used to rescue it; after skipping android it returned the unscoped tree. One effective scope now feeds both. - Scope resolves over the PRESENTED nodes of the requested projection, not the acquired tree, so an acquired match that membership drops no longer empties the snapshot, and Android matches the domain iOS's pass uses. - Slicing after the walk keeps ancestor context (hittable/collection/chrome) above the scope root, so scoped -i is a subset of unscoped -i; --depth stays scope-relative. - Shared findSnapshotScopeRange/reindexSnapshotNodes so the daemon pass and the Android projection run one implementation; scope slice extracted to ui-hierarchy-scope.ts (mirrors its test). - parseUiHierarchy moved to a test fixture module (it had no production caller left). - Golden rows sharpened (value row no longer matches via label on Android); the Android leg runs raw AND regular. CHANGELOG entry; docs wording corrected for iOS/@ref. * fix(layering): keep the contracts snapshot façade exhaustive over snapshot-scope * fix(android): scope to the first match whose subtree still has presented content Review P1 on #1846: with scope resolved strictly over presented nodes, `snapshot -i --scope panel` answered "no nodes" whenever the matched container was a structural view membership drops — even though the button inside it was exactly what was asked for — and `--depth 0` then hid a node the response prints at depth 0. The scope root is now the first document-order match whose subtree contributes at least one node to the requested projection, and the result is that subtree's presented nodes re-rooted at depth 0. Both failure modes die: a decorative match membership drops no longer empties the snapshot, and a dropped container still scopes to its content. `--depth` under scope filters the depths the response emits, so a node shown at depth 0 survives `--depth 0`. Tests: the golden legs stay raw+regular (bare TextViews cannot survive -i, so an -i leg would measure membership, not scope) with the projection interplay pinned by two dedicated tests on actionable shapes; the 'not re-scoped after the wire' case now runs a real parsed scoped tree instead of fabricated depth-0 siblings. |
||
|
Michał Pierzchała
|
6a8beb653e |
feat(mcp): compact server instructions in both eras + MCP-only help tool (#1839)
* feat(mcp): compact server instructions in both eras + MCP-only help tool (#1833) MCP-only clients got no workflow guidance: server/discover carried two sentences, legacy initialize carried nothing, and the CLI guides (agent-device --help, help <topic>) were unreachable over MCP. - MCP_SERVER_INSTRUCTIONS: one MCP-phrased workflow card (<2 KB, the Claude Code truncation limit) returned by server/discover and legacy initialize alike. - help tool, router-owned (not a command descriptor): no topic -> the CLI decision card; topic -> agent-device help <topic|command> text, prefixed with the one-line CLI->tool-property mapping; unknown topic -> isError listing the topics. listCommandTools() stays descriptor-only for the AI SDK; the router composes descriptors + help. - Move src/cli/parser/cli-help{,-overview}.ts to src/cli-schema/ so src/mcp (rank 3) can import the renderers without a layering back-edge into src/cli (rank 6). * fix(mcp): name terminal-only commands in help guides; colocate cli-help tests with their sources - The MCP guide preamble claimed every `agent-device <command>` line is a tool of that name; `help web` tells the reader to run `web setup` / `web doctor` and no `web` tool exists. The preamble now lists the exact CLI-only set (listCliCommandNames minus listMcpExposedCommandNames) — derived, not scanned out of prose where `device`/`web` are ordinary words. Regression: help web names `web` as terminal-only, and the listed set equals the registry difference. - cli-help-*.test.ts move from src/cli/parser/__tests__ to src/cli-schema/ to mirror the moved sources. * perf(mcp): tighten the guide card, tool description, and preamble Instructions card 1572 -> 1378 bytes (paid every session), tool description and preamble trimmed, HELP_TOOL built once as a const. Bundle delta vs main 3189 -> 2715 bytes; the remainder is the guide text itself, which the bundle carried in no MCP-phrased form before. |
||
|
Michał Pierzchała
|
d0d5c8594c | fix: serve remote daemon request diagnostics to the caller (#1801) (#1814) | ||
|
Michał Pierzchała
|
60f6356b04 |
fix: read replay scripts on the caller and ship them with the request (#1810)
* fix: read replay scripts on the caller and ship them with the request Closes #1802 * test: assert the caller-side replay path as a substring, not a hand-escaped regex * perf(cli): load the Maestro engine only when a replay entry is a flow The command registry evaluates every command family on CLI startup, so the replay script-source builder's static @agent-device/maestro import put the YAML parser on the --help path. It now loads on demand behind the format check, and the startup import-closure guard covers the engine the way it already covers node:http. * refactor: share the replay request field vocabulary across the CLI and client views The new replay script-source flags appear in both CliFlags and CommandExecutionOptions, which fallow flagged as a clone; ReplayRequestFields declares them once. The test-suite handler's missing-sources rejection now travels the typed-error path its sibling rejections already use, so the fix adds no branch to handleSessionReplayCommands. |
||
|
Michał Pierzchała
|
801734d433 |
feat(ai-sdk): add agent-device/ai-sdk tool set and document the MCP zero-code path (#1804)
* feat(ai-sdk): add agent-device/ai-sdk tool set and document the MCP zero-code path
Adds `createAgentDeviceTools()` under a new `agent-device/ai-sdk` subpath,
built from the same command registry the MCP server uses so both stay in
lockstep without a hand-maintained tool list. Introduces a `frameworkTier`
descriptor facet ('core' | 'extended') so the factory can default to a
curated perceive/act loop instead of handing a model dozens of tools.
`ai` is wired as an optional peer dependency, imported lazily inside the
factory rather than at module scope, so importing the subpath itself never
requires `ai` to be installed - only calling it does. The package's own
publishing gate (scripts/lib/shipped-imports.ts) is extended to recognize
peerDependencies as a valid resolution source, since this is the first
optional peer this package has shipped.
Also restructures the AI SDK doc around three tiers (zero-code via
@ai-sdk/mcp, the new typed tool set, hand-written tools) and fixes a stale
`needsApproval` reference in favor of the current `toolApproval` API.
* fix(layering): classify src/ai-sdk as a rank-4 zone
The layering guard requires every src/<folder>/ to be explicitly ranked or
unranked; the new src/ai-sdk/ subpath (added in the prior commit) was left
unclassified, failing CI's Layering Guard job. It sits at the same tier as
client/compat/daemon-server/metro/remote/sdk - a public integration surface
consuming mcp (3) and core (2), imported by nothing else in the tree.
* fix(ci): cover, exempt, and pack the new ai-sdk subpath
Fixes the remaining CI failures on the ai-sdk subpath commit:
- Coverage: src/ai-sdk/index.ts had no dedicated unit test (only manual/
integration verification), so changed-line coverage sat at 6.9% against
the 70% gate. Adds src/ai-sdk/__tests__/index.test.ts (core vs 'all' tool
filtering, session/platform pinning and schema hiding, error
normalization, toolApproval passthrough) with createCommandToolExecutor
and createAgentDeviceClient mocked the same way command-tools.test.ts
does, plus a dedicated missing-peer-dependency.test.ts that mocks `ai`
itself to throw, isolated to its own file so it doesn't affect the other
tests' use of the real, installed `ai` package. Changed-line coverage is
now 29/29 (100%).
- Fallow Code Quality: src/ai-sdk/index.ts and examples/sdk/ai-sdk-tools.ts
are entry points with no in-repo importer (reached only via package.json
exports / run directly), and the new subpath's exports are unused
internally by design - both need the same treatment src/sdk/*.ts and its
examples already have in .fallowrc.json.
- Integration Tests: test/integration/installed-package-metro.test.ts and
src/__tests__/package-exports.test.ts each hand-list every published
subpath and smoke-check it from a real packed install; added ./ai-sdk to
both so the new subpath is actually exercised, not just silently passing.
* fix(ai-sdk): hide MCP transport/config fields from the model too
createAgentDeviceTools() only removed session and mcpOutputFormat from tool
schemas. stateDir was still model-visible and reached the shared executor
as client configuration, letting a tool call redirect into a different
daemon state directory - defeating the "one pinned session" guarantee the
factory exists to provide. includeCost and responseLevel are MCP
tool-config knobs in the same category, irrelevant to this adapter.
Widens the hidden-field set to session/stateDir/mcpOutputFormat/
includeCost/responseLevel, and now strips them from the runtime input
inside execute() too (not just the schema), so the guarantee holds even if
a caller bypasses schema validation. The schema-properties filter and the
input filter now share one omitHidden() helper instead of two near-
duplicate implementations.
Addresses the P1 review comment on #1804.
|
||
|
Michał Pierzchała
|
8db36299e4 |
feat(web): add hover command for hover-gated UI (#1783) (#1786)
* feat(web): add hover command for hover-gated UI (#1783) Add a first-class `hover <x y|@ref|selector> [--settle]` verb, admitted on web only, that moves the pointer without pressing via the agent-browser backend (mouse move). It rides the existing targeted-touch pipeline (ref/selector/coordinate resolution, occlusion/off-screen guards, settle observation, response builder, recording) through a new optional Interactor/backend `hover` op that only the web provider implements. Touch platforms have no hover state: capabilities advertise it on web only and iOS/Android/Linux reject it at admission with a --platform web hint; longpress stays the mobile hold-gesture verb. Closes #1783 * fix(hover): native hoverRef route for web @ref, android coverage pin, revert skill edit Review follow-ups on #1786: - hover @ref on web now dispatches through the provider's own element handle (agent-browser `hover <ref>`) via a new backend hoverTarget, mirroring click/fill's ADR 0011 native-ref path — web ref frames carry no rects, so the coordinate route could never resolve them. The shared preflight + exact-ref dispatch is extracted into dispatchNativeRefInteraction and used by tap/fill/hover; the guarantee matrix native-ref row now lists hover. - Daemon regression test is production-faithful: rect-less web ref frame, scoped provider, asserts no coordinate dispatch. Selector→coordinate and provider hoverRef tests added. - Android emulator coverage summary pin 2/53 → 3/54. - skills/agent-device/SKILL.md reverted (out of scope, AGENTS.md rule). - Docs/help disclose that --settle with @ref on web shares click's existing limitation; use a selector or coordinates for the settled diff. * test: drive hover through the apple output guard; cover direct hover dispatch The provider-integration apple-leak guard partitions every public command into driven/skipped; hover was neither, which failed Integration Tests and took Coverage down with it. Drive it (it reaches the Apple capability refusal, which is scanned like any other error response). Also cover the direct-dispatch handleHoverCommand seam. * test(web): drive hover @ref in the provider-backed web scenario The integration-progress gate requires every public command to be referenced by a provider-backed scenario. Add hover @ref to the web desktop flow: it must reach the provider's hoverRef handle (never a coordinate) and be recorded on the session without fabricated x/y, like click @ref. |
||
|
Michał Pierzchała
|
4cfa34bd22 |
docs: reposition README around mobile app automation for AI agents (#1780)
* docs: reposition README around mobile app automation for AI agents Lead the README with the category (mobile app automation, testing, and verification for AI coding agents) and the three surfaces (CLI, MCP server, Node.js API) so humans and search engines can classify it, then keep verification and evidence as the differentiator. - Move per-platform transport/caveat sentences (HarmonyOS HDC/uitest, Vega VVD-only) out of the intro into "How it works" and add an inline guard comment plus an AGENTS.md rule so new platforms only add a name to the intro list. - Add MCP and Node.js quick starts next to the CLI walkthrough. - Add a works-with/proof line, a "What to ask your agent" prompt list, a product-ladder sentence, and two AEO-shaped FAQ entries. - Point the cloud/remote row at the remote proxy and device clouds docs. - Align npm, MCP registry, and docs-site descriptions with the same category phrasing. - Remove em dashes and evaluative filler per the humanizer skill. * docs: tighten README hero, link proof points, move install above capabilities * docs: add mobile-MCP FAQ distinction, sharpen hero support line, drop coming-soon * docs: make the build-on-top audience explicit in README * docs: trim README repetition and signposting * docs: close the session in finally in the README Node.js snippet |
||
|
Michał Pierzchała
|
f378050586 |
feat(snapshot): attach a fallback screenshot to sparse captures (#1764)
A sparse verdict already tells the caller to use a screenshot as visual truth, which made that screenshot the guaranteed next command on every unreadable screen — a second round trip to obey advice we authored. The user-facing `snapshot` dispatch now takes the shot itself and links the path in its warnings. The fallback is deliberately hung off `dispatchSnapshotViaRuntime` and skipped for internal observations: selector resolution, settle, and wait polling reach `captureSnapshot` directly, so a wait polling an unreadable screen cannot turn into a screenshot per poll. A failed shot is swallowed — the verdict's own warning still carries the manual remedy, so the fallback can never fail the snapshot that was asked for. Sparse captures also say when the screen is the app's problem. Only the `sparse-tree` reason code is evidence about the app: every backend reached the screen and it published no semantic content, which is the same emptiness assistive tech gets. `ax-rejected`, `budget`, `no-nodes` and `capture-failed` are limits of this tool and stay unattributed, so readers are not sent to file bugs against code that is not broken. |
||
|
Michał Pierzchała
|
f5d9789764 |
feat: enforce local device claims and reconcile stale owners (#1735)
* feat: enforce local device claims * fix: address device claim review feedback * fix: persist canonical daemon claim state directory |
||
|
Michał Pierzchała
|
602b7a2995 |
refactor: narrow perf API to actionable evidence (#1731)
* refactor: narrow perf API to actionable evidence * fix: address perf API review feedback * fix: preserve deprecated Android CPU metrics |
||
|
Michał Pierzchała
|
b0d4b40467 |
chore: stop publishing skills to npm (#1730)
* chore: stop publishing skills to npm * fix: align simulator skill startup * docs: align agent setup with open-first workflow |
||
|
Michał Pierzchała
|
c2c81549d9 |
feat: add simulator verification skills (#1716)
* feat: add simulator verification skills * chore: simplify simulator skills * docs: refine simulator skill guidance * test: guard simulator skill workflows * style: format simulator skill contract test |
||
|
Michał Pierzchała
|
4279d4c580 |
docs: align snapshot fallback and actions guidance (#1713)
* docs: align snapshot fallback and actions guidance
The snapshot guide claimed a zero-node XCTest result fails without ever
switching to AX, but regular iOS capture has an explicit recursive-tree →
query-sweep → private-AX recovery plan (ADR 0004,
RunnerTests+SnapshotCapturePlan.swift). The public CLI also exposes
`--actions`, `--force-full`, and `--timeout`, while both website reference
pages published a three-flag snapshot usage line.
- Add a schema-derived gate: `commands.md` must publish the exact usage
`buildCommandUsage('snapshot', getCliCommandSchema('snapshot'))` produces,
so the canonical invocation cannot drift from the command schema again. The
flag list is never restated in the test. Proven red against the pre-fix
`commands.md`.
- Extract the fence walker both doc checks now share, and prove the new gate
fails on a planted usage drift.
- Publish the canonical snapshot usage in the command reference and describe
`--actions` as iOS-simulator-only and planning-only.
- Replace the "Backends (iOS)" list with an iOS capture behavior section
written from ADR 0004 and the live capture plan: regular visible strategy
with a bounded recovery ladder, raw diagnostic strategy preserving strict
capture failures, and recovered/sparse/degraded output staying observable
through quality warnings. Capture tiers are documented as internal, not as
user-selectable backends.
Custom-action discovery stays separate from invocation: the runner can read
names but cannot trigger them (RunnerAXSnapshotBridge.h), so neither page
implies otherwise.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014VzyMim5q4jbp1xDMm3jVo
* docs: state that --actions and --raw are rejected as a pair
Both pages described the combination as a silent no-op ("returns a raw
tree without them"), which cannot happen: `customActionFlagsResponse` in
src/daemon/request-router.ts rejects `snapshotCustomActions` + `snapshotRaw`
with INVALID_ARGS at the shared request seam, before any session or device
work, so CLI, Node client, and MCP all get the same answer. Pinned by
src/daemon/__tests__/request-router-custom-action-flags.test.ts.
The underlying reason was right and is kept — custom actions are only
readable through the private-AX capture path, which the raw diagnostic
strategy does not take — but the user-visible outcome is a rejection, not a
degraded capture, so both pages now say to choose one flag or the other.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014VzyMim5q4jbp1xDMm3jVo
---------
Co-authored-by: Claude <noreply@anthropic.com>
|
||
|
vw2x
|
4b432fb59b |
feat: add HarmonyOS support (#1683)
* feat: add HarmonyOS device automation foundation Add HDC-backed discovery, snapshots, application lifecycle, and core mobile interactions. Route HarmonyOS through the platform registry and client contracts. Cover parsing and capability parity with focused tests. * feat: support HarmonyOS HAP deployment Install and reinstall signed HAP archives through HDC. Resolve bundle identities from module metadata and relaunch after package replacement. Extend deploy routing and capability coverage for HarmonyOS. * feat: add HarmonyOS single-pointer gestures Execute pan, fling, and swipe plans through HDC uiInput primitives. Derive scroll coordinates from the live ArkUI viewport. Keep unsupported multi-touch gestures explicitly rejected. * refactor: split session inventory command handling Separate session, device, capability, and app inventory response paths. Preserve the public inventory response contract while reducing handler complexity. * feat: support HarmonyOS keyboard actions Route HarmonyOS enter, return, and dismiss through HDC key events. Expose supported keyboard actions through the system command metadata. Keep keyboard visibility inspection explicitly unsupported. * fix: reject unsupported HarmonyOS drag gestures Keep drag unavailable until HDC can preserve source and destination hold semantics. * feat: add HarmonyOS app log streaming 1. Stream HarmonyOS app logs through PID-scoped hilog sessions.\n2. Record HarmonyOS app identity during bundle-id opens for app-scoped commands.\n3. Cover backend routing and bundle identity resolution. * feat: report HarmonyOS foreground app state 1. Read the foreground HarmonyOS mission through aa dump.\n2. Expose HarmonyOS appstate with package and ability metadata.\n3. Add parser coverage for foreground and missing-state cases. * fix: advertise appstate through capabilities 1. Classify appstate in the command descriptor capability matrix.\n2. Surface supported appstate commands in capability inventory.\n3. Cover the advertised Android capability contract. * feat: sample HarmonyOS process performance 1. Sample HarmonyOS process CPU and resident memory through HDC.\n2. Expose the verified metrics through the shared perf command.\n3. Keep frame and memory snapshot collection explicitly unavailable. * feat: clear HarmonyOS app state 1. Add HarmonyOS settings clear-app-state through bundle cleanup.\n2. Force stop the app before clearing data and cache.\n3. Reject all unverified HarmonyOS settings explicitly. * docs: document HarmonyOS support 1. Describe HarmonyOS HDC prerequisites and HAP installation.\n2. Add HarmonyOS to platform discovery and product documentation.\n3. Document verified performance limits for the public HDC surface. * fix: preserve HarmonyOS deploy session identity 1. Bind a resolved HarmonyOS bundle after install or reinstall.\n2. Keep app-scoped logs and observability available after deployment.\n3. Cover session identity preservation for HarmonyOS reinstall. * test: lock HarmonyOS capability boundary 1. Add an independent HarmonyOS capability-matrix oracle and exact advertised-command regression test. 2. Document current HDC-backed support and evidence-based unsupported command boundaries. * refactor: simplify HarmonyOS shared platform boundaries 1. Split device selection and settings dispatch into focused helpers without changing behavior. 2. Keep HarmonyOS serial selection and lock-policy classification covered by regression tests. 3. Remove Fallow complexity findings from the HarmonyOS diff against upstream main. * fix: bound default HarmonyOS HDC commands 1. Apply a 15 second timeout to ordinary HDC operations. 2. Preserve operation-specific timeout budgets for installation and capture paths. 3. Add regression coverage for default and overridden HDC timeouts. * feat: add HarmonyOS screen recording Implement physical-device whole-screen recording through the system recorder and HDC media transfer. Reject unsupported HarmonyOS recording scopes and export flags. Cover capability routing, media retrieval, cleanup, and simulator rejection. * feat: report HarmonyOS HDC readiness Add an HDC version check to the HarmonyOS doctor flow. Document HarmonyOS as a supported doctor platform and cover the result. * refactor: simplify HarmonyOS recording checks Reduce recording validation and test complexity without changing behavior. * test: cover HarmonyOS platform contracts Synchronize public platform expectations across CLI, MCP, replay, and inventory tests. Mock HarmonyOS inventory probes to preserve concurrent test behavior. * test: model HarmonyOS recording capability Require a physical HarmonyOS device in the independent capability parity oracle. * test: cover HarmonyOS input and lifecycle paths Exercise HDC input, lifecycle, installation, and relaunch command sequences. * test: cover HarmonyOS device observability paths Exercise discovery, screenshot validation, and process performance sampling. * docs: define HarmonyOS CI hardware policy Keep HDC hardware validation local and require mocked CI contract tests. * fix: honor HarmonyOS app inventory filters * fix: bound HarmonyOS app inventory classification 1. 限制应用元数据分类并发并为默认清单设置整体时限. 2. 将请求取消信号传递给 HarmonyOS 应用清单读取. 3. 补充失败时中止在飞读取且不继续排队的回归测试. * fix: preserve HarmonyOS inventory failure causes 1. 保留触发应用元数据分类失败的原始错误, 避免被取消同级任务覆盖. 2. 补充总时限中止在飞读取且不启动排队任务的回归测试. 3. 验证后序任务失败时保留默认筛选的恢复提示. |
||
|
Michał Pierzchała
|
fcf429e0bd |
docs: split device cloud integration guides (#1695)
* docs: split device cloud integration guides * docs: refine device cloud integration copy |
||
|
Michał Pierzchała
|
e18a183ac9 |
fix: harden artifact ingestion boundaries (#1692)
* fix: harden artifact ingestion boundaries * fix: bound archive inspection and upload expiry * fix: preserve upload preflight expiry |
||
|
Michał Pierzchała
|
e6b4fa2810 |
fix: isolate concurrent remote connections (#1675)
* fix: isolate concurrent remote connections * refactor: harden remote connection state * fix(cli): scope every emitted connect command to its own session `scopeNextSteps` only reached `ConnectReadiness.nextSteps`, so two command-bearing outputs still shipped unscoped: - `providerArtifactNotes()` emitted `agent-device artifacts --json` as a prose note, which never passes through that helper. - `buildDeferredRuntimeNotice()` emitted `agent-device metro prepare --remote-config <path>` independently in connection.ts. On the shared-host concurrency path this branch fixes, following either one resolves against the host-global active connection, so the artifacts instruction can return another job's provider video and log URLs (#1659). Both producers now take the connection state and format through one exported `scopeCommand` helper, which is the single place a suggested command is bound to its originating session. The metro config path is shell-quoted alongside the session name. Coverage: the BrowserStack route asserts human and JSON shapes carry one `--session` per suggested command and that each emitted session resolves back through `readRemoteConnectionState` to the connection that printed it; `connection status` pins the scoped deferred-metro `nextStep`. --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
Michał Pierzchała
|
e14c9d8d7b |
fix(cloud): unblock iOS snapshot and gate cloud fill on text-entry focus (#1658) (#1666)
* fix(cloud): unblock iOS snapshot and gate cloud fill on text-entry focus (#1658)
Two bugs isolated to the cloud-webdriver iOS path.
`snapshot`/`diff` refused every capture on a live BrowserStack session with
SESSION_NOT_FOUND, instantly and without a driver round trip. The app-session
guard they ran belongs to the local XCUITest runner, which must attach to a
target app; a cloud capture reads the provider's own driver session and needs
no app identity, so it now applies to local Apple targets only. The session
was empty in the first place because the provider open path skips local app
resolution wholesale — no simctl/devicectl reaches a hosted device — and
dropped an explicitly spelled bundle id along with it. A dotted, non-deep-link
target is the bundle id under the same convention resolveIosApp applies
locally, so a cloud `open com.example.app` now records it.
`fill` tapped and sent its keys in back-to-back requests. A WebView input —
an OAuth page in a Safari view controller — takes first responder
asynchronously, so the keys landed with nothing focused while the command
still answered "Filled N chars"; tapping and filling as two separate commands
worked only because the round trip between them gave the field time to focus.
The cloud interactor now waits on the same signal the Apple runner uses, the
software keyboard going from hidden to shown after its tap, and discloses what
it observed as `textEntryReadiness` so a fill with no witness cannot pass for
a filled field. Where keyboard visibility cannot witness the focus move —
back-to-back fills into one form, the shape that failed most often — it spends
the runner's full readiness budget rather than racing the app with a short
settle.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015Zjbzf7HdziX9SzFpNP7WW
* fix(cloud): let a new bundle-id open replace the tracked cloud iOS app
Adopting an explicitly spelled bundle id on a provider-backed open (the
fix that makes snapshot/diff work at all) also made a previously dead
precedence rule live: the provider branch returned currentAppBundleId
first, so once a first open had populated it, `open com.a` followed by
`open com.b` left the session still reporting com.a to every
appBundleId-gated command.
The local path does the opposite, and is the convention this branch is
meant to mirror: resolveIosApp returns a dotted target unchanged and
never consults the session's current app. Only its deep-link branches
prefer the tracked id. Flip the provider branch to match — an explicit
bundle-id target wins, and everything the branch cannot name (deep
links, display names, bare open) still falls back to the tracked id.
* fix(cloud): fail a witness-less cloud fill instead of reporting it filled
Review follow-ups on #1658.
`not-observed` was still a success: it sent the keys and answered "Filled N
chars", and nothing renders `textEntryReadiness` in default CLI output — so the
exact silent success this branch exists to remove survived whenever focus never
happened. A tap that raises no keyboard now fails with
`text_entry_focus_not_observed` and sends no keys, leaving the field untouched
rather than half-written, and the readiness vocabulary keeps only outcomes that
describe a fill that did type.
The readiness budget was advertised but not enforced at the request boundary:
each keyboard probe inherited the client's 30s default, so one hung probe could
hold a 2s wait for far longer. Probes now carry their own bound, threaded
through the client as a per-request timeout override.
The probe also swallowed every error as "this driver cannot answer", which
degraded a dead session, an auth rejection, or a grid outage into a blind text
entry. Only a positively classified unimplemented route counts as unsupported
now — classified on the W3C error code rather than the status, since `unknown
command` and `invalid session id` share HTTP 404 — and everything else
propagates.
The provider scenario proved request ordering against a stub that always
accepted keys. Its fake now models the device: focus lands a beat after the
tap, and keys arriving while the keyboard is down are accepted and dropped,
exactly as an unfocused field does. The tests assert the field's own value, and
both go red against the pre-fix `fill`.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015Zjbzf7HdziX9SzFpNP7WW
* fix(cloud): witness the tapped field's focus before a cloud fill types
Review of cc23f2b found two ways a fill could still report success
without evidence that OUR tap focused the field it was aimed at.
P1. `settled-keyboard-up` and `settled-unknown` both typed and returned
normal success. Keyboard visibility can only witness that *a* field took
focus, never *which*: filling a second field in an already-open form
reads the same before and after, so a missed tap left the first field
focused and `POST /keys` — which the driver routes to whatever holds
first responder — appended to it while every request returned 200.
Failing those closed outright would have broken ordinary multi-field
form fills, which do work: a live AWS Device Farm run types both fields
of a WebView login correctly. So witness focus properly instead. W3C
`GET /element/active` answers the question keyboard visibility cannot —
is the thing focused now the thing I tapped — and answers it whether or
not the keyboard was already up. That becomes the primary signal
(`focused-element`); the keyboard transition stays as the fallback for
drivers without the route, and a keyboard already up on such a driver
now refuses rather than typing.
The test is identity, not geometry. Containment of the tap point looks
like the obvious rule and is wrong: focusing a field can re-lay it out.
On a live iPhone 16, tapping Safari's collapsed address bar expands it
into a taller field that no longer covers the tapped point, and a
containment-only rule refused a fill that plainly worked. So a tap that
MOVES focus counts, with containment as the second half of the test —
re-filling the already-focused field moves nothing, and only geometry
tells that from a tap that missed. Both readings are taken before the
tap, since each is evidence only as a change.
P2. The 2s budget bounded the loop but not the calls inside it: every
probe got a fixed 1500ms, so one begun near the deadline finished well
past it. Both the probe timeout and the sleep are now capped by the
remaining budget.
Also fixes a related escape the review did not name: the poll loop had
no catch, so one transient grid error aborted a fill the next poll would
have satisfied. Probe failures are now tolerated within the budget, but
a budget that expires without a single answered probe rethrows, so a
dead session surfaces as itself rather than as "the tap missed".
The provider scenario gains the two-field case the review asked for: it
begins keyboard-up with the email field focused, misses the password
tap, and asserts no keys reach the email field.
Verified on AWS Device Farm iPhone 16 / iOS 18.0 at this exact tree:
address bar (the re-layout case) and both WebView login fields all
report `focused-element`, the second with the keyboard already up, and
the device reads back `tomsmith` and a 20-character password.
* fix(cloud): refuse a cloud fill no focus probe can witness, and bound the composite probe
Two blockers from the review of
|
||
|
Michał Pierzchała
|
f4ebd6f14f |
fix(ios): synthesize hidden-keyboard text through responder (#1657)
* fix(ios): synthesize hidden-keyboard text through responder * docs: document iOS text synthesis failure |
||
|
Michał Pierzchała
|
b19ee116b5 |
fix(remote): stop persisting the daemon bearer token, and authenticate forced-reconnect release correctly (#1648)
* fix(remote): stop persisting the daemon bearer token in connection state ADR 0007 requires generated connection profiles to strip daemon and Metro bearer tokens; only the Metro half was honored. `connect` was writing the daemon bearer token into the 0600 connection-state file, and every later command read it back out. Stop writing `authToken` into `RemoteConnectionState['daemon']` and resolve it at each reader from the existing flag -> environment (AGENT_DEVICE_DAEMON_AUTH_TOKEN) -> remote-config-profile chain instead, matching src/cli/auth-session.ts's precedence. Behavior change: a user who ran `connect --daemon-auth-token <value>` and relied on later commands picking the token back up from the state file will now get an auth failure. They must export AGENT_DEVICE_DAEMON_AUTH_TOKEN, set daemonAuthToken in their remote config, or pass --daemon-auth-token on each command. website/docs/docs/remote-proxy.md is updated to show the supported env-var workflow. * fix(remote): authenticate forced-reconnect lease release with the previous endpoint's own credential connect --force released the previous connection's lease using the new connection's ambient daemonAuthToken instead of the previous endpoint's own credential, and swallowed the resulting auth failure — silently orphaning the old lease when replacing a connection with a differently-authenticated one. Resolve the release token from the previous connection's own remote-config profile first, fall back to the ambient token only when the two connections share the same daemon endpoint, and otherwise skip the release and surface an actionable notice (tenant, run id, lease id, endpoint) through the existing connect notice channel instead of hiding the failure. * fix(remote): stop merging ambient env defaults into the previous lease's own token resolvePreviousOwnDaemonAuthToken read the previous connection's profile through resolveRemoteConfigProfile, which folds AGENT_DEVICE_DAEMON_AUTH_TOKEN (and other env defaults) into the result. When the previous config file declared no token and the new connection's credential came from that same global env var, it was misclassified as belonging to the previous endpoint and sent there on forced-reconnect release — recreating the credential leak the prior fix was meant to close, just via env instead of --daemon-auth-token. Read the previous profile with the new readRemoteConfigFile (a provenance- preserving, file-only load with no ambient env/CLI merging), so only a token the previous config file itself declares can satisfy rule 1. Rules 2 and 3 are unchanged. * fix(remote): verify the previous config file still speaks for its endpoint Rule 1 reads the previous connection's own config file to recover a credential that provably belongs to the previous endpoint. It re-read `previous.remoteConfigPath` and trusted whatever token that file holds *now* — but a config path is routinely reused, so "connect to A from ./remote.json, re-point ./remote.json at B, connect --force" classified B's token as A's own and sent it to A during lease release. Same cross-endpoint leak the env-merge fix closed, arriving through the file instead of the environment. The file must now still vouch for the previous endpoint, by either of two independent facts: its bytes still hash to the `remoteConfigHash` recorded at connect time (so it is literally the declaration that stood up the previous connection), or — if it changed — it still declares the same daemon base URL. The second is what keeps an ordinary credential rotation releasing its lease instead of orphaning one; endpoint equality, not the fact of an edit, is what separates rotation from re-pointing. Endpoint comparison runs both sides through `buildRemoteConnectionDaemonState`, the same normalizer that produced the stored `daemon.baseUrl`. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017Rva4YGtSCAKJqH5PbpcCU * fix(remote): bind previous config token to its endpoint --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
Michał Pierzchała
|
3937036e5e |
feat: support --settle on scroll and back (#1638) (#1650)
* feat: support --settle on scroll and back (#1638) Scroll-then-observe and back-then-observe are legitimate agent pairs, but the post-action observation registry never grew past the touch commands, so `--settle` on either was rejected with INVALID_ARGS — burning a tool call each in AppControlBench's bsky-16. Both commands now carry the `settle` descriptor trait, and every surface derives from it rather than a hand list: CLI allowed flags, MCP/SDK input fields, the flag-sourced timeout envelope, and MCP ref-pinning. The CLI flag/metadata helpers moved out of the interaction family into post-action-observation-grammar.ts (back is a system command), and SETTLE_REF_ISSUING_TOOLS became a derivation — a hand list would have silently stopped pinning the new commands' refs. settleAfterInteraction and the new settleObservationCommand are two entry points over one engine: same loop, storage, hints, and diff bounds, with the target-less path supplying its own baseline and no proximity point. The daemon reaches that command through the runtime surface, never by importing `commands/` (R2) — the same seam the touch handlers use for press/fill — and generic-settle.ts is loaded through a lazy `await import` returning a closure, so the interaction runtime subgraph stays out of this dispatcher's static graph (a static edge folded ~18 files into the daemon-server type cycle; R10 caught it). Both of generic-settle's orderings are load-bearing and tested: the baseline is frozen before dispatch (and before the Android dialog preflight), and the observation runs after markDeferredInteractionOutcome so settle's first capture folds in the #1542 stabilization rather than racing it. The ADR 0014 "a settled diff publishes refs" rule moved to settle-ref-issuance.ts, shared by both routes. One divergence is deliberate: scroll/back resolve no element, so the diff baseline is the session's STORED pre-action tree — "settled tree vs the last tree you observed" — not press's freshly resolved pre-action capture. Both commands also switch to preserve-daemon on timeout, which changes the non-settle path too: with --settle their dominant hang mode is now a wedged accessibility bridge, and a timed-out capture must not reset the daemon and lose every session (#1105). The reviewed-set gate records it. Live-validated on an iOS 26.2 simulator (Settings): scroll --settle settled in 1786ms with a +6/-6 diff carrying fresh refs; back --settle in 771ms with +15/-6. Alternating cost runs, one call vs the pair it replaces: scroll 2.9-3.0s vs 5.3-5.6s, back 3.1-3.2s vs 4.7-5.1s. Those include the #1627 deep-capture extension. * fix: render settled-diff refs paste-ready in CLI output A settled diff activates a PARTIAL ref frame (ADR 0014), which admits only the pinned `@eN~s<gen>` form of the refs it issued. The unchanged-interactive tail already rendered that way, but the diff's own added lines rendered the bare `@eN` embedded in the snapshot line — so a CLI caller who copied the ref the diff just handed them got `plain_ref_requires_complete_frame` and had to append the generation by hand. Added lines now render pinned when the response carries `refsGeneration`, exactly like the tail. Removed lines render verbatim: they name elements that just left the screen, and `SettleDiffLine` never gives them a ref. This is not new to scroll/back — press/click/fill/longpress had the same gap since #1101. MCP was never affected: its ref-pin store rewrites plain refs on the way in, which is why the model never sees a suffix. Live: `scroll down --settle` now emits `+ @e14~s218078 [cell] "Game Center"`, and `press @e14~s218078` copied straight out of that line taps successfully. * test: record the pinned-diff-ref bytes in the output-economy baseline Rendering added diff-line refs pinned costs 8 bytes in the two settle CLI text samples (two `~s<gen>` suffixes). The output-economy baseline is the tripwire for exactly this, so the increase takes an explicit reviewed waiver rather than a silent baseline bump — the same one the settled TAIL's pins already carry, for the same ADR 0014 reason. Only `bytes` moves: lines, refs, hints, and shape are unchanged, which is the evidence that this is a suffix on existing refs and not a new payload. Caught by CI, not locally: `pnpm test:unit` runs unit-core and subprocess-stub only, while the Coverage lane runs every vitest project. * test: prove the generic settle degrades when its runtime cannot be built `createGenericSettleRuntime` catches and returns undefined so an observation that cannot even start does not fail an action that already succeeded. That was a claim in a docstring with nothing behind it — the one changed line the coverage gate reported uncovered (95/96). The test puts the session in the state the catch exists for: the router handed us a session that is no longer in the store, so building the settle runtime throws SESSION_NOT_FOUND. The response keeps its scroll result and simply carries no settle payload. Removing the try/catch fails it. * build: teach fallow that vi.mock reaches pinOwnProcessStartTime dynamically Not from this PR: #1642 added `pinOwnProcessStartTime` on main, and its three consumers reach it the only way a Vitest module mock can — `vi.mock(path, async (importOriginal) => (await import('...')).pinOwnProcessStartTime(...))`. Dependency analysis cannot follow that dynamic import to a consumer, so the export reads as dead the moment any PR pulls that file into its audit scope. This PR is the one that did. The entry records the consumers by path and the reason, matching the daemon route-handler entry directly above it, which exists for the same dynamic-`import()` limitation. * refactor: adopt the best of the parallel #1653 implementation Two sessions independently built #1638 (PR #1650 and PR #1653) and converged on the same architecture — trait in the registry, one engine with two entry points, runtime-command seam, lazy import, preserve-daemon, stored-baseline honesty. #1650 continues; this folds in what #1653 did better: - The agent-facing help core loop (cli-help.ts) now names scroll and back as settle-capable. Without this, the benchmarked closed-grammar help line kept instructing agents that --settle is only for press/click/fill/longpress — actively steering the AppControlBench models away from what #1638 shipped. - issueSettleRefs moves into session-snapshot.ts, beside the partial-frame primitive it wraps, deleting the single-function settle-ref-issuance module. - Their seam tests: back reader→writer settle plumbing, back CLI settle rendering, and a trait-less generic command (home) ignoring a stray settle flag rather than observing or rejecting. What #1650 had that #1653 lacked, for the record: the SETTLE_REF_ISSUING_TOOLS registry derivation (without it, MCP never pins a scroll/back settle diff's refs and the partial frame rejects every follow-up), BackCommandResult.settle in contracts, back's MCP output schema, paste-ready pinned diff refs, and the docs/changelog/baseline surfaces. * bench: help-conformance case for settled scroll-to-find planning The #1638 extension of the closed --settle grammar to scroll/back is the feature's entire payoff — collapsing scroll-then-observe into one call — and the closed command list is an enumerated N whose enumerator is this bench. The regex over the help text proves the sentence exists; this case checks whether a model plans differently because of it. One focused case, deliberately not coached: a pinned visible-first snapshot (rendered by formatSnapshotText, pinned by the sample-producers gate) whose wanted row is summarized off-screen with no ref anywhere in the output. The tempting pre-#1638 plan is `scroll` plus a separate `snapshot -i`; acceptance is the single settled call. Scoring was verified against eight plan shapes in both directions before recording. Model-backed record (claude-haiku-4-5, 3 trials, current help): 0/3 — but the decomposition is the finding. Settle eligibility GENERALIZED (3/3 trials put --settle on scroll unprompted; the mutation-suffix framing concern did not materialize) and the two-call habit is residual (1/3). All three trials failed on `scroll @e3 down --settle` — the pre-existing #1366 scroll-takes-no-target confusion, which the live CLI recovers with a dedicated hint but a single-shot bench cannot. The recorded gap is therefore a first-30 doc gap (nothing teaches that scroll takes no target), not a settle-eligibility gap; tuning the case until it passes would just delete the evidence. |
||
|
Michał Pierzchała
|
870d12c406 |
fix: honest find contract — press/tap aliases, read-only list action, selector uniqueness (#1637)
* fix: honest find contract — press/tap aliases, read-only list, selector uniqueness (#1625) Three defects in find's contract, fixed together because they are one vocabulary: press/tap are the same action as click everywhere else in this CLI, yet find rejected them — agents using the vocabulary the tool itself established burned a tool call per attempt (four in one bench run). Both parsers now normalize press/tap to click; longpress/swipe stay real exclusions. The #1602 recovery hint told agents to run bare find to 'list matches', but bare find CLICKS a unique match — inspection guidance pointing at a mutation (the #1625 report: 'find Dictionary' navigated into Dictionary). find <q> list is the read-only surface that guidance needed: every match with its @ref, unique match included, never a tap. Captured UNSCOPED (the label-scope optimization narrows to the first match, exactly wrong for listing), published as an ADR 0014 partial frame authorizing every listed ref. Selector-shaped queries skipped the ambiguity check and took the first match silently — the mis-binding path the AMBIGUOUS_MATCH recovery advice itself pointed agents at, while --first/--last were documented as explicit opt-ins. Selector and text queries now share one contract: multiple matches reject with the #1597 candidates listing unless --first/--last narrows explicitly. The hint is rewritten around the new contract; docs and the MCP find output schema follow. Regressions at every layer: both parsers (alias, list token, unsupported-action hint shape), the daemon handler (selector ambiguity with candidates, --first opt-out, list returns all matches with zero action dispatches, unique-match list does not tap). * refactor: single-home the find read result and flatten parseFindArgs (fallow) The daemon's DaemonFindResult had drifted into an identical structural twin of the engine's FindReadCommandResult — the two grew the list variant in parallel and crossed the clone threshold. The shape now lives in contracts as FindReadResult (below both zones, per R2's own remedy) with the engine and daemon both aliasing it. parseFindArgs collapses the four bare single-token actions into one membership check and extracts the get sub-action parser, bringing it back under the complexity threshold instead of waiving it. * style: merge duplicate contracts import (lint) * fix: accept list on the MCP input surface and pin every listed ref (review) - FIND_ACTION_VALUES gains 'list' so field-metadata/MCP input no longer rejects the action the CLI parser accepts - FindCommandResponseData types 'matches' (public client response) - MCP mergeFindRefPins learns every matches[] ref, so a plain @eN press after find-list forwards pinned and the partial frame admits it - CLI/MCP text renders every listed match as its own pinned line via the snapshot-line role/label normalizers - regressions: daemon partial-frame scope, pin store, CLI output, MCP schema + find-list->press chain, typed client list response |
||
|
Szymon Dziedzic
|
3e4828d68d |
feat: add scale-only screenshot sizing (#1617)
* feat: add scale-only screenshot sizing
* fix: refuse retired --max-size inputs on every released surface
Released sizing inputs must fail closed with migration guidance instead of
silently producing native-size artifacts:
- contracts: RETIRED_SCREENSHOT_MAX_SIZE declaration + SCREENSHOT_SCALE_LIMITS
as the single source for the scale bounds and migration messages
- .ad parser: released 'screenshot ... --max-size N' and 'record start ...
--max-size N' lines now refuse at parse time (frozen replay-compat witnesses)
- daemon: screenshot rejects old-client screenshotMaxSize like recording does;
the recording guard now shares the same contract data
- Node client: screenshot/record daemon writers refuse the removed { maxSize }
option before transport
- CLI: --max-size unknown-flag error carries the migration guidance
- config/env: stale screenshotMaxSize config keys and the retired
AGENT_DEVICE_SCREENSHOT_MAX_SIZE env var are refused for sizing commands
(other commands keep working)
Quality: numberField now reuses the canonical readOptionalNumber contract
helper (AppError bounds instead of plain Error); png-resize inlines one-use
wrappers and restores the worker-thread rationale; docs typo fixed.
* test: drop retired maxSize entries from the MCP undocumented-input allowlist
* fix: refuse retired maxSize at the MCP field-projection seam + release-provenance corpus witnesses
- readFieldInput silently dropped undeclared keys before the daemon writers
could refuse them, so an MCP call carrying { maxSize } reached transport and
returned native-size success. New retiredField() combinator declares the
removed key in the field map: the projection seam refuses it with the
canonical migration message and the JSON schema no longer advertises it.
Real-route MCP executor regressions cover screenshot and record.
- replay-compat corpus: derived v0.20.5 witnesses for the released screenshot
and record --max-size forms (SHA-256 pinned, new retired-capture-size
coverage surface) so check:replay-compat proves the shipped syntax refuses
with migration guidance instead of degrading silently.
---------
Co-authored-by: Michał Pierzchała <thymikee@gmail.com>
|
||
|
Michał Pierzchała
|
4269ca6d88 |
fix: update MCP registry namespace (#1618)
* fix: update MCP registry namespace
* feat: inherit MCP descriptions from CLI help
* refactor: project command guidance per surface
* refactor: make command guidance a single canonical description
The guidance type carried seven fields, but only three were ever set, and all
twenty call sites used it the same way: to hold a second, hand-written MCP
string next to a near-identical CLI one. That is the drift the abstraction was
meant to remove, so the type no longer offers a per-surface description at all.
A command now has one canonical description plus an optional tail per surface:
guidance: {
description: 'Shared body.',
cliDetail: 'Flags, positional syntax, terminal examples.',
mcpDetail: 'When-to-use and sequencing hints.',
}
Because a surface can only append, CLI help and MCP tool text cannot diverge —
the guard against CLI syntax in MCP descriptions becomes structural rather than
a review tripwire, since flag vocabulary only lives in cliDetail. All twenty
commands that previously carried two descriptions now share one body.
Also:
- Drop `summary` from the description fallback chain. It is the short list-view
line, so falling back to it replaced the full description with a fragment on
both surfaces: artifacts, boot, and shutdown each lost their real description.
- Stop writing the MCP variant back over `metadata.description`. That field
feeds CLI help, `explain`, and docs; `explain` was printing MCP-only text.
MCP now reads a separate `mcpDescription`.
- Drop `mcp.parameters`. It restated inputSchema property descriptions inside
the tool description — 1232 characters duplicated verbatim across six tools,
and three of sixteen declared hints silently rendered nothing because the
property had no description. Those properties are documented in the schema
instead, which serves MCP, --help, and docs at once.
- Drop `cli.flags`. Its one use appended "Relevant flags: --surface,
--launch-console." to help text that already named both flags inline.
Tests assert the structural property (both surfaces share a canonical prefix)
and the summary-fallback regression, alongside the existing CLI-syntax guard.
CLI help wording assertions follow the new copy.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dp3J8UUgYxtw5vzJzvjkSf
* test: gate undocumented MCP tool inputs
Guidance no longer restates input fields in prose, so a tool's inputSchema is
the only place its inputs are documented — for the model, for --help, and for
the docs site. An undescribed property is a silent gap rather than a cosmetic
one, which is exactly the failure mode the removed `mcp.parameters` selection
had: it dropped hints for properties that carried no description and reported
nothing.
Describe the two trigger-app-event inputs that mechanism used to name, and add
a ratcheting gate over every MCP tool input. A property key that is not already
in the budget fails immediately; the total may never grow, and lowering it is
required once properties gain descriptions, so the 132 remaining stay visible
instead of settling in as permanent debt.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dp3J8UUgYxtw5vzJzvjkSf
* fix: project the canonical description to every surface
Storing only the MCP variant left the shared body unpropagated: `metadata.description`
and the executable definition kept their pre-guidance text, so `explain click` reported
"Click or tap a semantic UI target..." while CLI help and the MCP tool both used the
canonical "Activate a UI target...". 53 commands were affected — the CLI schema base,
`explain`, and docs all read `metadata.description`.
`projectCommandGuidance` now returns the canonical body plus the MCP-only tail, and
`defineCommandFacet` writes the body to both metadata and the definition. Only the tail
is stored apart, as `mcpDetail`, so the body has exactly one home instead of a second
full copy that could drift; `composeMcpDescription` joins them for the tool surface.
The surface gate pins the invariant: definition, metadata, and `explain` must report the
identical body, and neither CLI help nor the MCP description may do anything but extend
it. Both arms verified by breaking them.
Also replace the undocumented-input ratchet's bare-key allowlist plus aggregate budget
with exact `tool.property` identities. The old shape stayed green while a gap migrated:
describing `foo.text` and adding an undescribed `bar.text` left both the allowed-name set
and the total of 132 unchanged, and stale names kept authorizing later gaps. Verified
with that exact scenario — `app` was already an allowed name via push/reinstall/settings
and the total held at 132, yet a newly undescribed `open.app` now fails. Recording a fix
requires deleting its baseline entry.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dp3J8UUgYxtw5vzJzvjkSf
* refactor: drop guidance.description in favour of the command's own
`guidance.description` restated what `metadata.description` already is. Setting it
shadowed the metadata literal rather than replacing it, so every command that used it
shipped two bodies: the canonical one and a terse original that no surface could
observe — 768 bytes of unreachable strings across 20 commands.
Move each canonical body to the metadata literal where it belongs and delete the field.
Guidance is now tails only, `cliDetail` and `mcpDetail`, which also removes the question
of where a body is written: there is one place, and no chain to consult. Three guidance
blocks held nothing else and are gone entirely.
registry.js drops 1117 bytes.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dp3J8UUgYxtw5vzJzvjkSf
* refactor: give every command one text block with a mandatory summary
The CLI carried four prose fields with a fallback chain between them, and one of
them — `helpDescription` — was authored on 45 commands and generated on the rest.
That ambiguity is why surfaces drifted: whichever field a reader looked at, some
other field might be the one actually rendered.
Prose now lives in a single `CommandText`, and `CommandSchema` keeps only grammar:
summary what is this command, in a list of ninety? (mandatory)
description what does it do, and when do I reach for it? (mandatory)
cliDetail flags, argument shapes, terminal examples
mcpDetail sequencing and cross-tool hints
`--help`, the command list, the MCP tool description and `explain` are projections
computed where they render, so nothing derived is stored and no field can be both
input and output. The four-field model was validated against the whole surface
before the migration: all 67 commands reproduce their MCP text exactly and derive
their help body from `description`, so none needed a fifth field.
Making `summary` mandatory fixes a regression this branch introduced. 23 commands
had none, so the command list fell back to the full detail paragraph; lengthening
those descriptions earlier turned `click`'s list entry from 61 characters into 267,
`fill`'s into 214, `devices`' into 159. Every command now states its own line, and
a gate holds them under 72 characters, non-empty, period-free, and distinct from
the description.
Two duplications go with it: the per-command help printed its synopsis twice, once
as a header and again under `Usage:`, and `press` said "use longpress" in both its
body and its tail.
* refactor: tighten the command text plumbing
Self-review follow-ups on the text model, all quality-only:
`command-text.ts` moves from `cli-schema/` to `commands/`. It is a command concept
that MCP reads as much as the CLI does; living under `cli-schema` made the MCP
surface import a CLI module to render its own tool descriptions.
`defineCommandFacet` no longer casts. It took a facet and returned it with the
schema completed, but claimed to return the input type, which needed
`as unknown as` — a double cast is the type system reporting that the signature
was wrong. Splitting `CommandFacetInput` from `CommandFacet` states the completion
in the return type, so both that cast and the registry's `as CommandSchema` go.
`push`'s summary duplicated its description apart from a trailing period, which the
gate missed by comparing exact strings; it now compares normalised text, and the
summary says something the description does not. `install-from-source`'s summary
loses a clause it did not need.
---------
Co-authored-by: Claude <noreply@anthropic.com>
|
||
|
Thiago Brezinski
|
a13a6832ee |
feat: add selector-targeted drag gestures (#1567)
* feat: add selector-targeted drag gestures * fix: address drag gesture review feedback * fix: satisfy drag review quality gates * fix(android): lower drag trajectories piecewise * test(replay): validate drag fixture selectors * fix(ios): ignore full-viewport chrome containers * test(drag): prove destination on live devices |
||
|
Michał Pierzchała
|
4c7a899a05 |
fix: prevent iOS text entry runner wedge (#1604)
* fix: prevent iOS text entry runner wedge * fix: preserve iOS hardware-keyboard text entry * fix: expire iOS text-entry tap witnesses * fix: fail interrupted iOS text entry * test: run interrupted iOS typing regression |
||
|
Michał Pierzchała
|
4f8dc3f31e |
refactor: move selector engine into workspace package (#1589)
* refactor: move selector engine into workspace package
* refactor(selectors): trim the package façade to its real consumers
Follow-up to the selector-package cutover, from a structural review of it.
- Drop 15 façade symbols with no consumer anywhere in the repo:
selectorUsesKey (added by the cutover, never called), isNodeVisible /
isNodeEditable (the real helpers are contracts/snapshot's), normalizeText,
splitIsSelectorArgs, IS_PREDICATE_REQUIRED_MESSAGE, four nested Replay
types, SelectorDisambiguationDisclosure, and the four kernel type
re-exports every consumer already imports from kernel directly.
- Delete SelectorCapturePolicyInput.selectorExpression, which
deriveSelectorCapturePolicy never read; the policy varies only by
predicate, so it takes one now. Two of the four tests asserted that the
unread parameter had no effect and could not fail; they go with it.
- Return the Maestro export vocabulary to the maestro package. The cutover
inlined MAESTRO_TEXT/STATE_SELECTOR_KEYS' values into the CLI call site,
leaving both constants dead in the package that owns the concept and no
gate over the two copies. MAESTRO_SELECTOR_PROJECTION is now the one
statement of it.
- Dedupe SelectorDiagnostics and SelectorDisambiguationDisclosure, declared
character-for-character twice across the AST/string seam, and name the two
shared option shapes once instead of five inline copies. The parser-side
resolution types take an Ast prefix so the twins read as twins.
- Delete three identity wrappers: parsePrivateSelector,
selectorExpressionToMaestro, and the formatSelectorFailure forwarder —
nothing passes it a chain any more, so the SelectorChain | string union
and its branch go too.
- Delete internal/index.ts, an AST barrel whose only consumer was one test
in the same directory (renamed to engine.test.ts), and the match.ts
pass-through that existed to feed it.
- ReplaySelectorGrammar had three variants for two behaviors; 'wait' and
'ordinary' were the same path. It is 'is' | 'positional' now.
- Drop the deleted src/sdk/selectors.ts from .fallowrc.json's entry list.
Behavior unchanged. pnpm check green: 598 unit files / 5278 tests, smoke
35 passed / 3 live skipped, layering 71/71, depgraph 22/22, mutation config
45/45, fallow clean, package smoke sound. Counterfactual: pointing
MAESTRO_SELECTOR_PROJECTION.textKeys at the state keys turns three
replay-maestro-export cells red; restored before commit.
* test(selectors): split the engine aggregation test by source concept
`internal/index.test.ts` (renamed `engine.test.ts` when its barrel went away)
was a 708-line aggregation over the whole engine — past the 500-line tripwire
and mirroring no source module, so it also ran as one serial unit.
It becomes five files that each mirror what they test, plus the parser cells
folded into the existing parse test:
resolve.test.ts alternative fallback, strict uniqueness,
first-match existence
resolve-disambiguation.test.ts ADR 0012 ranking: deepest, smallest-area,
winner-vs-challenger disclosure, tie fallback
resolve-viewport.test.ts the visibility half: on-screen beats
off-screen, including inside an off-screen
scroll container
match.test.ts per-key matching semantics (text, role,
focused, appname/windowtitle, decoded
newline labels)
arguments.test.ts where the selector ends and the command's
positionals begin, both grammars
parse.test.ts +6 grammar/escape cells beside the existing
property tests
The login-form tree shared by resolve.test.ts and match.test.ts moves to
`__tests__/login-form-nodes.ts` rather than being copied into both.
All 27 cells are carried over unchanged and still pass; no file now exceeds
224 lines. pnpm check green: 602 unit files / 5278 tests, layering 71/71,
depgraph 22/22, mutation config 45/45, fallow clean over 127 changed files.
* revert(selectors): keep agent-device/selectors public, behind one AST subpath
The cutover removed the `agent-device/selectors` public subpath as part of
tightening the API. It is in use, so the removal is reverted: the subpath ships
the same ten symbols v0.20.5 shipped, with the same signatures.
That has to coexist with the reason the package façade is string-only, so the
AST leaves through one named door instead of the main one:
@agent-device/selectors string-in/string-out; every in-repo consumer
@agent-device/selectors/ast the published parser surface; one consumer,
src/sdk/selectors.ts
`packages/selectors/src/ast.ts` re-exports parseSelectorChain,
tryParseSelectorChain, isSelectorToken, the AST-taking findSelectorChainMatch
and resolveSelectorChain, isNodeVisible, isNodeEditable, and types
SelectorChain / SelectorDiagnostics. `formatSelectorFailure` keeps its
published `SelectorChain | string` first parameter as a shim here rather than
widening internal/resolve.ts back to a union — the compatibility obligation
sits at the boundary that owes it.
This is strictly narrower than main, where the AST was reachable from anywhere
in src/ via src/selectors/*. Two gates hold it there: facade-symbols.ts pins
./ast to exactly the v0.20.5 list, and package-boundaries.test.ts asserts
src/sdk/selectors.ts is the only file outside the package that imports it.
Restored alongside: the ./selectors export and tsdown entry/chunk group, the
.fallowrc.json entry, the package-exports supported-subpath list, and both
client-api.md sections. No CHANGELOG entry — nothing is removed any more.
pnpm check green: 602 unit files / 5278 tests, smoke 35 passed / 3 live
skipped, layering 71/71 (10 packages, 32 subpaths), depgraph 22/22, mutation
config 45/45, fallow clean over 129 changed files, package smoke imported all
12 published entry points with publint and attw passing. Verified functionally
against the built dist: the doc's parse -> findSelectorChainMatch example
returns the same shapes as before, resolveSelectorChain still returns an AST
`selector`, and formatSelectorFailure still accepts a chain.
* fix(selectors): correct the two expectations that still assume the removal
Review P1s on a792415a: restoring the public subpath left two gates asserting
it was gone.
- installed-package-metro.test.ts moved `agent-device/selectors` into the
blocked-specifier list. It goes back to the subpath smoke set, running the
same `isSelectorToken('||')` + `parseSelectorChain` check it ran before the
removal, so the file's only remaining delta from main is a formatter reflow.
- owner-files-no-leak.test.ts asserted `dist/src/sdk-selectors.js` was absent.
It requires the stable named chunk again, and still rejects an auto-numbered
`selectors2.js` fallback — the pair is what proves the restored tsdown chunk
group is doing its job, verified against a clean build.
PR body corrected: the removal is no longer described as intentional API
tightening.
* refactor(selectors): satisfy the widened fallow scope after rebase
main's #1591 (the follow-up filed from this review) removed `packages/**` from
.fallowrc.json's ignorePatterns, so the new package is audited for the first
time. Everything below is a finding fallow could not previously see.
Dead surface, all confirmed consumer-free:
- 12 type re-exports from the `.` façade whose shapes consumers only ever
reach structurally.
- MAESTRO_TEXT_SELECTOR_KEYS / MAESTRO_STATE_SELECTOR_KEYS, orphaned by this
branch's own MAESTRO_SELECTOR_PROJECTION change, and the test-util
SELECTOR_VALUE_HAZARDS. All three are module-local now.
- IS_PREDICATE_USAGE_HINT fails --production because its only consumer is the
is-argument-surface parity test. It gets a commented `ignoreExports` entry
rather than deletion: the constant is what makes the daemon and CLI raise
ONE hint instead of two copied strings (ADR 0010), so the test asserting
that is the point, not an accident.
`fast-check` is now declared by the package that imports it.
Duplication, split by what could be proven:
- `isUsefulVisibilityAnchor` existed character-for-character in both
packages/selectors and packages/maestro. Moved to
@agent-device/contracts/snapshot, which both already depend on and which
already owns this vocabulary. Safe because the `normalizeType` each copy
called is itself character-identical to the contracts one — checked before
moving, since a different normalizer would have silently changed which
nodes anchor.
- maestro additionally reimplemented `normalizeType`, `buildSnapshotNodeMap`
(as `buildSnapshotNodeByIndex`) and `findSnapshotAncestor`, all
character-identical to contracts'. Deleted in favour of the shared ones.
- The three scroll-ancestor walks are NOT deduped. They are structurally the
same walk but each uses a different scrollable predicate, and I have no
evidence the three agree; collapsing them would be a Maestro-conformance
change, not a cleanup. Both maestro sites now say so, and the work is filed
separately.
`projectSelectorExpression` (15 cyclomatic / 22 cognitive, written by the
cutover) splits into a dispatcher plus `readAgreedTextValue` and
`projectSelectorTerms`; all three are under threshold.
Rebase note: the one conflict, in package-boundaries.test.ts, resolved to
NEITHER side — #1591 had already deleted `AdReplayVerifiedTargetGuard` as an
unused export, and this branch deletes the seven ReplaySelectorPort names, so
the conflicting block is empty.
* build: record fast-check for packages/selectors in the lockfile
Declaring the dependency in packages/selectors/package.json without
regenerating pnpm-lock.yaml made every CI job fail in its install step with
ERR_PNPM_OUTDATED_LOCKFILE. My local `pnpm install --frozen-lockfile` printed
"+ 1 dependencies were added: fast-check@^4.9.0" and exited 0, which read as
success but was the same mismatch CI refuses.
Regenerated with the pinned pnpm 11.17.0, not the 11.5.3 on this machine:
11.5.3 rewrites peer-dependency resolution keys repo-wide (dropping
`(supports-color@7.2.0)` suffixes) and produced a 222-line diff. With the
pinned version the diff is the 4 lines this change actually needs, plus
pnpm's alphabetical re-sort of the root selectors entry.
|
||
|
Michał Pierzchała
|
6baa5d97e1 | fix: make wait verdicts evidence-based (#1570) |