2 Commits

Author	SHA1	Message	Date
Ignacy Łątka	44b38d490e	fix(tool-server): inject Android keyboard/button over adb input (#449) (#455) ... ## What & why `keyboard` and `button` drive Android through the simulator-server's USB-HID transport (`pressKey` / `cmd:"button"`). The guest only receives those HID events when the AVD exposes a hardware keyboard (`hw.keyboard = yes`) — the default, but CI / headless AVDs are routinely created with `hw.keyboard = no` (and `hw.mainKeys = no`), where the guest **silently drops** them. Because that transport is fire-and-forget, both tools returned `{ typed }` / `{ pressed }` **success while injecting nothing**. This still affects current `main`: after the keyboard refactor, `makeAndroidImpl` routes Android **TV** to `adb input` but Android **phones/tablets** still to `typeSimulatorServer` (the HID path). Fixes #449 (found during 0.13.0 RC E2E testing on a `hw.keyboard=no` AVD — `argent-test-ci-repro`, the config CI uses). ## Change Route Android **phone/tablet** key / text / button injection through `adb shell input text` and `input keyevent` (new `utils/android-input.ts`), the same on-device transport Android TV and Vega already use: - **`keyboard`** — `platforms/android.ts` now sends the non-TV branch to a `typeAndroidPhone` helper (adb `input`); the Android-TV and iOS branches are untouched. - **`button`** — the Android branch presses via `injectAndroidKeycode`; iOS keeps the simulator-server HID path. `adb input` lands regardless of `hw.keyboard`, on emulators (any config) and physical devices alike, and a non-zero exit surfaces as a **thrown error** instead of a silent no-op — directly addressing the fire-and-forget concern already called out in the `button` tool. Touch gestures are unchanged. ## Verification - **Unit tests — 11/11 pass** (rebased on current `main`): new `keyboard-android.test.ts` (vocabulary parity vs `NAMED_KEYS`, `input text` shell-quoting incl. spaces, `input keyevent` mapping, empty-text no-op, newline rejection) + updated `button.test.ts` (Android `back` now asserts `KEYCODE_BACK` over adb). - **eslint** on the changed files: clean. - **Live tool-server** on a `hw.keyboard=no` AVD: the `keyboard` tool now issues `adb … shell input text '…'` (vs the RC's HID `pressKey`), and a failed inject **raises an error** rather than a fake success. The `adb input` path was independently observed typing/deleting on this exact AVD during RC testing (where the HID path silently no-op'd). - Note: a full `tsc --build` / whole-suite run couldn't be completed in my environment (the `argent-private` submodule isn't checked out, which breaks the npm workspace install, and `main` added upstream deps my base lacked). `platforms/android.ts` mirrors the existing `platforms/ios.ts` branch structure one-for-one, and the pre-rebase revision of this change passed `tsc --build` + eslint. CI here is authoritative for the full typecheck. ## Reviewer notes - `input text` receives the string as one shell-quoted argv token, so spaces are preserved verbatim; newlines are rejected (they'd truncate the `input` line) — callers use `key: "enter"`. - `keys` counts by codepoint, matching the tv/vega/simulator-server backends.	2026-07-10 12:51:24 +02:00
Mateusz Ścianek	b8afbed183	feat(tv): Apple TV (tvOS) and Android TV support (#374) ... Adds end-to-end support for driving **Apple TV (tvOS)** and **Android TV (leanback)** targets through argent, using a focus-driven interaction model (TVs have no touchscreen — interaction is moving a focus highlight with the remote). This PR was reworked after review to **reuse the existing cross-platform tool surface and the `tv-remote` tool from the Vega/Fire TV work (#366)** rather than ship a separate family of `tv-*` tools. ## What this delivers TV targets are driven by the **same cross-platform tools as every other platform**, routed through the existing `dispatchByPlatform` fork mechanism — there are no dedicated `tv-describe`/`tv-navigate`/`tv-type`/`tv-set-focus` tools: - **`describe`** — on a `runtimeKind: "tv"` target, returns the focus-driven view (focused + focusable elements) instead of a tap tree. On Android TV it auto-falls-back to the full `uiautomator` tree when the RN focus engine exposes nothing. - **`tv-remote`** — the cross-platform remote/D-pad tool from #366, now extended with `ios` (Apple TV HID daemon) and `android` (Android TV adb keyevents) branches alongside the existing `vega` branch. Single button, a `repeat`, or a whole path (`["up","right","select"]`) in one call. - **`keyboard`** — types into the focused field on a TV target (named keys are rejected there — they're navigation, which belongs to `tv-remote`). - **`button`** stays hardware-only (phones/tablets); it is not a TV tool. **Apple TV** runs two native daemons (in-sim AX service + host-side HID daemon), shipped via the `argent-private` submodule. **Android TV** reuses the same tool surface, adb-backed (`input keyevent`, `uiautomator dump`, `input text`). ## Remote vocabulary parity `tv-remote` exposes the full 16-button vocabulary (`up`/`down`/`left`/`right`/`select`/`back`/`home`/`menu`/`playPause`/`rewind`/`fastForward`/`next`/`previous`/`volumeUp`/`volumeDown`/`mute`): | | Apple TV | Android TV | Vega | |---|---|---|---| | D-pad / select / back / menu / home / playPause | ✅ | ✅ | ✅ | | media-transport + volume/mute | ❌ rejected | ✅ | ✅ | Media-transport/volume keys **genuinely work on Android TV** (real keycodes — verified live: `volumeUp` moved `STREAM_MUSIC`, `mute` toggled per `dumpsys audio`). On the **Apple TV simulator** they are **rejected with a clear error**: on-device testing confirmed the tvOS sim's HID stack silently drops Consumer-Control events, so returning success would be a lie. ## Key design points - **tvOS UDIDs are UUID-shaped and indistinguishable from iOS by shape.** Handled with `runtimeKind` detection (tvOS runtime string for Apple TV; `pm list features` leanback/television for Android TV — *not* `ro.build.characteristics`, which lies on TV emulators). - Converted launch-app / restart-app / screenshot / screenshot-diff / run-sequence from **eager** to **lazy** service resolution, so a tvOS target never spins up (and hangs on) the iOS-only simulator-server / native-devtools blueprints. - Native injection selects the platform-matched (TVOSSIMULATOR) dylib slice; daemons recycle across sim reboots and app relaunches so injection and focus survive. - SKILL.md footprint minimized per review: the two TV skills were collapsed into a single lean **`argent-tv-interact`** (~40 lines). ## Verification - `npm run build`, ESLint, Prettier, and the full tool-server suite (**1565 tests**) all pass; CI green (incl. the Apple TV / Android TV / Vega e2e jobs). - Core flows verified live against real apps on both an Apple TV 4K simulator and a Google ATV emulator. ## Dependency / merge order Pins the `argent-private` submodule to the tip of its `feat/tv-support` branch (software-mansion/argent-private#20). **Merge that PR to `main` first, then bump the submodule pointer here to the merged SHA before merging this PR.** 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-01 14:59:37 +02:00