mirror of
https://github.com/callstack/agent-device.git
synced 2026-09-14 20:06:34 +08:00
t3code/cleanup-claim-tests
13 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Michał Pierzchała
|
e3880f10c8 |
fix(android-helper): build with the pinned build-tools version, fail on CI when unset (#2568)
build-android-helper.sh compiled with whichever build-tools directory was newest on the image while every lane that builds a helper installs exactly build-tools;36.0.0. That selection feeds d8 and aapt2, so a newer package on the runner changed the helper's bytecode and resources rather than just its packaging, and nothing said so. The script now takes the version as an input, its last positional or AGENT_DEVICE_ANDROID_BUILD_TOOLS, resolves it under $SDK_ROOT/build-tools, and checks the four tools the build actually runs instead of aapt2 alone. An unpinned build is a hard failure on CI; locally the newest-installed fallback stays and is announced on stderr. Each lane that builds a helper declares the version it installs and interpolates that same value into its sdkmanager line and the build environment, so an install and a build inside one lane cannot drift. setup-android-replay-host exports it to the packaging gate and names it in both helper cache keys, which are keyed on APK bytes; size.yml and release-android-snapshot-helper.yml declare it at job level. The fixture-repack cluster keeps its own pin: a different command family that ships no helper APK. Closes #2527 |
||
|
Michał Pierzchała
|
3394d5b89c |
fix(android): keep a scroll's swipe out of the IME window (#2514)
* fix(android): keep a scroll's swipe out of the IME window Android is the case the clip exists for beyond iOS: an `adjustPan` or `adjustNothing` activity keeps a window whose recorded bounds already run under the IME, so a plan built from them aims at the keys. The helper now reports the largest input method window beside the application window, in absolute screen pixels like the window next to it, and `scroll` clips its band with the shared rule or refuses when the keyboard owns the surface. An older helper reports no keyboard keys, and a provider-supplied viewport has no IME channel at all. Both read as "nothing to avoid", which is what the shared rule already does with a missing frame; neither turns into a refusal. `UiAutomation.getWindows()` answers with an empty list until the service asks for interactive windows, so the read applies the seam the tree capture already uses rather than depending on a snapshot capture having run first in the same instrumentation; the one-shot fallback below it has no such neighbour. Measured on a Pixel 7 emulator with an `adjust=pan` contact editor, the application window keeps its full 2400px height while the IME window reports `[0,1517][1080,2400]`, and `scroll down` answers with `referenceHeight: 1505`, `keyboardMinY: 1517`, `keyboardAvoided: true` and a swipe ending at 301 instead of starting at 1920 under the keys. * fix(android): clear the composer, not just the key plane, before a scroll swipes The helper kept the largest `TYPE_INPUT_METHOD` rectangle as the keyboard. A composer bar and its key plane can arrive as separate windows and the key plane is the larger one, so the earlier top edge was discarded and the clipped band still ended inside the composer: the swipe landed on keys the rule exists to keep it off. The read now copies every input method window and unions the ones the swipe's centre line crosses, which is the same line the shared clip rule tests. A candidate strip at the edge of the screen that the swipe can never reach no longer shortens the band either. The selection runs on plain window edges, because `Rect` is a device type whose constructors throw off-device, so the two-window case is a unit test rather than a simulator-only path. * refactor(android): pass the measured occlusion explicitly and drop the duplicate rect check The clip rule already fails open on a keyboard frame it cannot measure, so the helper reader no longer needs its own copy of the check, and the gesture-viewport validator goes back to its original body. The refusal names its three numbers instead of spreading the clip variant, so the discriminator never leaks into error details. |
||
|
Michał Pierzchała
|
37d67de776 |
fix(android): carry accessibility selected state into snapshots (#2515)
The snapshot helper never serialized `selected`, and the host reads only the
helper's XML, so no later layer could recover it: `get attrs` had no `selected`
field, no snapshot node was marked selected, `is selected` could not match, and
a Maestro `assertVisible {id, selected: true}` failed with "Maestro visible
condition did not match" for a visible element while `selected: false` matched
every Android node (#2462).
The helper now emits both answers, like `enabled` and `password`, so an
unselected control answers `false` and a helper older than the attribute answers
nothing. The parser, the Android hierarchy node, and the published snapshot node
carry it to `get attrs` and the `[selected]` marker.
Snapshot lines render that marker on the default formatter path too: `--settle`
and `diff` already compared selection, and a line that weighs a fact it cannot
print turns a tab tap into a changed pair whose two lines look identical.
|
||
|
AdzeB
|
f30328d086 |
fix(android): preserve editable-field metadata in snapshots (#2290)
* fix(android): preserve editable-field observation metadata * fix(android): carry field facts through the attrs digest and selection offsets past editability - `get attrs --level digest` kept only the pre-#2288 semantic fields, so `editable`/`password`/`hintShowing`/`selectionStart`/`selectionEnd` vanished on the token-cheap route. The digest now keeps them, with a regression covering explicit false/zero/empty and omission when unavailable. - The helper emitted selection offsets only inside `isEditable()`, but read-only selectable text exposes a selection too. Each nonnegative offset is now emitted independently; -1 stays absent. Parser-to-snapshot regression for a non-editable selectable node. - Docs: the field-metadata notes get their own section instead of leading the efficiency tips. - Dropped the test-isolation commit: main already mocks notifyIosRunnerAppRelaunched, and the lifecycle test passes without it. --------- Co-authored-by: Michał Pierzchała <thymikee@gmail.com> |
||
|
Michał Pierzchała
|
c7f42ccedc |
refactor: move Android family behind package exports (#2117)
* refactor: move Android family behind package exports * fix: address Android W5 review feedback * fix: update relocated routing fixture assertion |
||
|
Nicolas Bataille
|
4b8bcaca60 |
feat(interaction): accept fill <target> "" as the clear-field primitive (#2066)
* feat(interaction): accept fill <target> "" as the clear-field primitive
Emptying an input was not expressible: `fill` refused the empty string
("Expected text to be a non-empty string"), `type` only appends, and `keyboard`
has no delete verb. Clearing a field before typing is a routine QA step, so the
only route was the app's own clear button or N locale-dependent keyboard delete
presses read out of a snapshot.
`fill <target> ""` now means "replace with nothing". Both platforms already own
the clear half of replace, so this is the validation and reporting that stood in
front of it, not a new interaction:
- `stringField` takes an opt-in `allowEmpty`, used only by `fill`'s `text`.
`requiredField` still refuses a MISSING text, so `fill @e57` stays an error
rather than silently erasing the field — `readFillTargetFromPositionals` now
reports `undefined` for "no text argument" instead of collapsing it to `''`.
`type` keeps refusing an empty text: appending nothing is not a clear.
- The Apple runner's empty-text early return skipped the clear while reporting
"typed". For a replacement it now runs `clearTextInput` and verifies the field
came back empty (secure fields stay unverifiable, as elsewhere).
- Android already clears before typing and skips an empty shell/IME write, but
its verifier read a cleared field's absent `text` attribute as a mismatch
against `''`. An empty expectation now accepts null or "".
Whitespace-only text keeps its established per-shape rules; only `''` is new.
Closes #2063
* fix(interaction): fail the empty-fill clear closed on every backend
Addresses the P1 review on #2066, then closes the same fail-open class
on the backends the PR did not reach:
- Android: an empty expectation no longer matches when the verification
scan observed NO input node at all — actual is null both for a cleared
field and for a wrong point/lost focus, and three empty samples of
nothing were a stable success for a clear that never touched a field.
- Apple runner: when the empty-replacement path cannot resolve a clear
target (including the synthesized first-responder route, whose target
carries no element), it returns the typed TEXT_INPUT_NOT_FOCUSED
failure instead of falling through to the vacuous-typing
verified-success return. Regression runs in the ios.yml XCTest lane.
- webdriver: fill is tap + sendKeys and owns no clear mechanism, so an
empty fill refuses as UNSUPPORTED_OPERATION before touching the
device, instead of reporting a clear it cannot perform.
- linux + web coordinate fill: typing zero characters over the
select-all selection left the old value intact; the empty fill now
deletes the selection.
- recording: an empty --record-as literal matches inside every string;
it now parameterizes only the fill's own text field instead of
rewriting every empty field and empty evidence label in the entry.
(The session-wide echo registry already excluded empty literals.)
- help: the text-entry topic taught agents that fill "" is not a
clear-field command; it now states the new contract.
Each new test was observed red against the pre-fix code.
* fix(android): read hint-showing from the helper so a cleared field verifies
Live Pixel 9 emulator, adb-shell channel: clearing the Settings search
field succeeded on the device but reported 'Android fill verification
failed', because a cleared EditText dumps its HINT as text — getText()
returns the hint for an empty field on modern Android, so 'Search
settings' read back as a residual value. This is the same
placeholder-as-value trap the Apple runner already handles with
treatingPlaceholderAsEmpty.
The helper now emits hint-showing (isShowingHintText, API 26+), the
hierarchy parser carries it, and fill verification matches against the
field's VALUE — hint-only text is an empty value, for empty and
non-empty expectations alike. A field whose real value equals its hint
string keeps failing the clear check: only the authoritative flag, never
the text, says it is a hint. Raw uiautomator dumps carry no such fact
and keep the fail-closed behavior.
Live evidence, both admission channels, after this fix: test-ime and
adb-shell clears both report Filled 0 chars with the field back on its
placeholder; the pre-fix adb-shell run failed closed (never a false
success).
* fix(interaction): close the adversarial-review findings on the empty-fill clear
- android adb-shell: the delete burst is sized from the value being
REMOVED (pre-mutation read; the attempt's cap when unreadable), not
from the empty incoming text, which sent the 12/24-delete minimums and
could never empty a field longer than 36 characters.
- android: the unconfirmed soft-success no longer applies to an empty
expectation — nothing app-formats the empty value, so residue after a
clear is a failed clear, and the soft-success also skipped the second,
bigger delete burst.
- android masked fields: an empty expectation accepts an observed masked
node with no dump text (a masked field WITH content dumps its bullet
run), so clearing a password field no longer fails after the clear
worked — matching iOS, where a secure-field clear succeeds unverified.
- find: 'find <q> fill ""' now reaches the fill leaf as the clear
request on both the CLI reader and the daemon positional parse; a
MISSING value keeps its refusal at each producer, so the typed
value: string contract is unchanged.
- maestro export: a recorded clear exports as tapOn + eraseText instead
of a vacuous inputText: "" (with the 50-character-default warning).
- the missing-text refusals teach the clear form: (use "" to clear
the field).
Full unit suite green (1061 files); each behavioral fix carries a test
observed red against the prior code.
* refactor(interaction,android): extract the fill parse and shell-attempt branches
The review commits pushed parseFillTarget and fillAndroid over the
complexity gate (13 cyclomatic each). Each fill target shape parses in
its own function sharing one missing-text response, and the adb-shell
attempt (clear sizing + clear + type + verify) moves out of the fill
loop. Behavior-preserving; the existing tests cover every branch.
* refactor(interaction,android): one owner per empty-fill fact
Design pass after review: the missing-vs-empty rule and the observed-
value rule each had several owners; now each has one.
- parseFillTarget decodes ONCE through readFillTargetFromPositionals —
which already owns shape detection and documents the undefined-vs-''
contract on DecodedFillTarget — and keeps only what the wire owns:
versioned-ref admission, the selector whitespace rule, and the daemon
responses. This deletes the point branch's duplicated slicing, the
hasFillText guard, and the three per-shape parse functions.
- observedAndroidValue() is the single statement of Android's value
rule (absent attribute and hint-only text are the empty value); the
text branch, the match rule, and the masked branch all consume it.
The masked branch thereby gains the hint-showing collapse it was
missing, and isAcceptableAndroidFillMatch narrows to plain strings.
- The empty-text-is-clear contract is stated once, on Interactor.fill
in contracts, instead of implied per backend.
Behavior-preserving except the masked+hint gain; the existing tests
cover every branch (494 Android, 15 fill-target).
---------
Co-authored-by: Michał Pierzchała <thymikee@gmail.com>
|
||
|
Michał Pierzchała
|
957a6727f8 |
fix(android): publish covered state from exact order evidence (#1981)
* fix(android): unify snapshot occlusion across API levels * fix(android): preserve exact occlusion evidence * fix(android): restore collective occlusion coverage * fix(android): preserve snapshot evidence across consumers |
||
|
Michał Pierzchała
|
84e6f9bf2d |
refactor(android): raw is the acquired tree; one presentation for dialog recovery; residues declared (#1832 C3) (#1865)
* refactor(android): raw is the acquired tree; one presentation for dialog recovery; residues declared (#1832 C3) - C3: the three regular-projection pruners (invisible subtrees, stale application windows, covered same-window surfaces) move out of parseUiHierarchyTree into the projection as a non-mutating classification (collectAndroidHiddenNodes in ui-hierarchy-visibility.ts). --raw presents the acquired tree; interactive ⊆ regular ⊆ raw by construction. Hidden-content hints and the scope root are derived per projection over retained children, which is what the mutating pruners implied. Property-checked identical to main for regular/-i/depth/scope over 12,000 random tree × projection pairs; raw grew on 2,514/3,000 and never shrank. - Android blocking-dialog recovery routes through buildSnapshotState (the one presentation), which moves to src/daemon/snapshot-state.ts below the daemon-server type cycle; importers repointed, its tests mirror the module. - Freshness route signature drops role/selected (Android never carries them). - Residues declared at their sites and in CONTEXT.md; docs + CHANGELOG. - ui-hierarchy.ts split by question: node predicates (ui-hierarchy-node.ts), regular-projection visibility (ui-hierarchy-visibility.ts), scope (ui-hierarchy-scope.ts); 974 → 634 LOC. * test: lower the snapshot.test.ts size pin to its new length * fix(android): dialog recovery acts on the presentation's occlusion result Review P1 on #1865: routing blocking-dialog recovery through buildSnapshotState made the occlusion result available but nothing consumed it. containsBlockingDialog scanned every node and findCloseAppButton returned the first text match with a rect, so a stale ANR surface left under the foreground one could still trigger recovery, and a covered "Close app" could be tapped ahead of the visible top button — the disagreement the routing was supposed to remove. Both decisions now filter through isSnapshotNodeInteractionBlocked, the shared predicate over the annotator's structured result. Two regressions cover it, both proven red against an unfiltered selection: a covered Close app preceding a visible one (asserts the visible center is tapped) and a fully covered dialog signal (asserts recovery does not trigger, no tap dispatched). Rebase reconciliation: screenshot-runtime.ts arrived on main (#1878) importing buildSnapshotState from its old home; repointed to src/daemon/snapshot-state.ts with the other importers. |
||
|
Michał Pierzchała
|
cd9a7ce41b |
test(android): add comprehensive emulator E2E coverage (#1482)
* test(android): add catalog emulator smoke coverage * test(android): use stable snapshot diff mutation * test(android): assert actual back destination * test(android): separate keyboard and fill IMEs * fix(ci): keep Android timing report in one shell * refactor(test): simplify simulator e2e coverage * test(android): assert stable diff landmarks * fix(android): release snapshot helper gracefully * fix(android): fully release snapshot helper runtime * test(android): report coverage classifications * fix(android): stabilize accessibility root capture * fix(android): bound UiAutomation connection * ci: upload worktree daemon diagnostics * fix(android): cancel stalled wait captures * fix(android): bound helper fallback lifecycle * fix(android): harden emulator e2e lifecycle * fix: align e2e changes with kernel package * test(android): prove alert helper reuse directly * fix(android): cancel stalled settle captures * fix(android): separate helper retirement budgets |
||
|
Michał Pierzchała
|
a0aa02579b |
build(android): unify the 4 helper build/package scripts behind one parameterized pair (#1466)
* build(android): unify snapshot/ime helper build+package scripts Replace the four ~75%-duplicated shell scripts with one parameterized build script and one parameterized package script, mirroring scripts/build-xcuitest-apple.sh's env-var-driven pattern. The helper is selected via AGENT_DEVICE_ANDROID_HELPER or a first positional arg; per-helper differences (HELPER_DIR/PACKAGE_NAME, snapshot's test-compile+run step, ime's aapt2 resource-compile step, and the manifest JSON fields) live in small case blocks. All package.json entry points keep their names and output paths. Verified byte-level equivalence between main and this branch: identical unzip -l listings, identical classes.dex SHA-256 for both helpers, and identical manifest fields (only the per-signing-run sha256 differs). Fixes #1461 * chore: drop stale android/multitouch-helper .gitignore entries The multitouch helper was consolidated away in #1281; these two lines were never cleaned up. |
||
|
Michał Pierzchała
|
f087a5938e |
fix: align Android Maestro gesture dispatch (#1356)
* fix: align Android Maestro gesture dispatch * fix: preserve Android gesture guarantees |
||
|
Michał Pierzchała
|
1a1ef7c419 |
feat(android): one persistent automation helper owning snapshot + viewport + canonical injection (#1281)
* feat(android): consolidate touch injection and gesture viewport into the persistent snapshot helper (#1275)
One Android automation helper now owns snapshot capture, gesture viewport
resolution, and canonical one-/two-pointer plan injection. A live persistent
helper session executes gesture/viewport commands over its socket protocol;
without a session the same APK runs one-shot via am instrument. The separate
one-shot multitouch helper APK is deleted (atomic replacement, no fallback).
Touch scheduling/injection is extracted into focused Java classes
(TouchPlan, TouchPlanInjector, PointerEventSchedule, GestureViewportReader)
instead of growing SnapshotInstrumentation. ADR 0013 amended.
* fix(android): stop a structurally-failed helper session before the one-shot viewport retry
A structured ok=false viewport response leaves the session process alive, and
Android permits only one instrumentation owner of UiAutomation - running the
one-shot fallback against a still-live helper contends with it and masks the
original structured failure. Stop the session first; regression pins that the
one-shot retry only executes once the session is gone.
* refactor(android): extract helper touch dispatch into focused classes; split session tests; document helper API v2 (PR #1281 review)
Addresses findings 2 and 3 from PR #1281 review (finding 1, viewport
session-stop ordering, was already fixed in
|
||
|
Michał Pierzchała
|
e58cbcdb5f |
refactor: colocate native platform sources under android/, apple/, linux/ (#1273)
Move the scattered root-level native projects into per-platform folders and drop
the now-redundant platform prefix:
- android-ime-helper/ -> android/ime-helper/
- android-multitouch-helper/ -> android/multitouch-helper/
- android-snapshot-helper/ -> android/snapshot-helper/
- apple-runner/ -> apple/runner/
- macos-helper/ -> apple/macos-helper/
- src/platforms/linux/atspi-dump.py -> linux/atspi-dump.py
Only repo source paths move. Identity surfaces stay frozen so no user's runner
cache is invalidated on upgrade: the derived-cache key hashes source paths
relative to AgentDeviceRunner and excludes packageVersion, and the
~/.agent-device/{apple-runner,macos-helper} namespaces, the
agent-device-android-*-helper artifact/manifest/protocol names, the
AgentDeviceRunner Xcode project, and the `prepare ios-runner` CLI command are
unchanged. Updates build/package scripts, CI, package.json files+scripts,
ignore/attr/fallow configs, runtime path resolvers, and test fixtures.
Also: re-base repo-root-relative refs inside the moved apple/runner for the
added nesting level (gated XCUITest fixture walk + two doc links), and clean the
legacy dist/apple-runner packaged output so the relocated runner can't
double-ship into the wholesale-included dist (with a regression test).
|