Files
software-mansion__argent/packages/tool-server/test/screenshot-tv-scale.test.ts
Mateusz Ścianek b8afbed183 feat(tv): Apple TV (tvOS) and Android TV support (#374)
Adds end-to-end support for driving **Apple TV (tvOS)** and **Android TV
(leanback)** targets through argent, using a focus-driven interaction
model (TVs have no touchscreen — interaction is moving a focus highlight
with the remote).

This PR was reworked after review to **reuse the existing cross-platform
tool surface and the `tv-remote` tool from the Vega/Fire TV work
(#366)** rather than ship a separate family of `tv-*` tools.

## What this delivers

TV targets are driven by the **same cross-platform tools as every other
platform**, routed through the existing `dispatchByPlatform` fork
mechanism — there are no dedicated
`tv-describe`/`tv-navigate`/`tv-type`/`tv-set-focus` tools:

- **`describe`** — on a `runtimeKind: "tv"` target, returns the
focus-driven view (focused + focusable elements) instead of a tap tree.
On Android TV it auto-falls-back to the full `uiautomator` tree when the
RN focus engine exposes nothing.
- **`tv-remote`** — the cross-platform remote/D-pad tool from #366, now
extended with `ios` (Apple TV HID daemon) and `android` (Android TV adb
keyevents) branches alongside the existing `vega` branch. Single button,
a `repeat`, or a whole path (`["up","right","select"]`) in one call.
- **`keyboard`** — types into the focused field on a TV target (named
keys are rejected there — they're navigation, which belongs to
`tv-remote`).
- **`button`** stays hardware-only (phones/tablets); it is not a TV
tool.

**Apple TV** runs two native daemons (in-sim AX service + host-side HID
daemon), shipped via the `argent-private` submodule. **Android TV**
reuses the same tool surface, adb-backed (`input keyevent`, `uiautomator
dump`, `input text`).

## Remote vocabulary parity

`tv-remote` exposes the full 16-button vocabulary
(`up`/`down`/`left`/`right`/`select`/`back`/`home`/`menu`/`playPause`/`rewind`/`fastForward`/`next`/`previous`/`volumeUp`/`volumeDown`/`mute`):

| | Apple TV | Android TV | Vega |
|---|---|---|---|
| D-pad / select / back / menu / home / playPause | ✅ | ✅ | ✅ |
| media-transport + volume/mute | ❌ rejected | ✅ | ✅ |

Media-transport/volume keys **genuinely work on Android TV** (real
keycodes — verified live: `volumeUp` moved `STREAM_MUSIC`, `mute`
toggled per `dumpsys audio`). On the **Apple TV simulator** they are
**rejected with a clear error**: on-device testing confirmed the tvOS
sim's HID stack silently drops Consumer-Control events, so returning
success would be a lie.

## Key design points
- **tvOS UDIDs are UUID-shaped and indistinguishable from iOS by
shape.** Handled with `runtimeKind` detection (tvOS runtime string for
Apple TV; `pm list features` leanback/television for Android TV — *not*
`ro.build.characteristics`, which lies on TV emulators).
- Converted launch-app / restart-app / screenshot / screenshot-diff /
run-sequence from **eager** to **lazy** service resolution, so a tvOS
target never spins up (and hangs on) the iOS-only simulator-server /
native-devtools blueprints.
- Native injection selects the platform-matched (TVOSSIMULATOR) dylib
slice; daemons recycle across sim reboots and app relaunches so
injection and focus survive.
- SKILL.md footprint minimized per review: the two TV skills were
collapsed into a single lean **`argent-tv-interact`** (~40 lines).

## Verification
- `npm run build`, ESLint, Prettier, and the full tool-server suite
(**1565 tests**) all pass; CI green (incl. the Apple TV / Android TV /
Vega e2e jobs).
- Core flows verified live against real apps on both an Apple TV 4K
simulator and a Google ATV emulator.

## Dependency / merge order
Pins the `argent-private` submodule to the tip of its `feat/tv-support`
branch (software-mansion/argent-private#20). **Merge that PR to `main`
first, then bump the submodule pointer here to the merged SHA before
merging this PR.**

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 14:59:37 +02:00

52 lines
2.3 KiB
TypeScript

import { describe, expect, it, vi, beforeEach } from "vitest";
// `tvTargetLongSide` shells `sips -g pixelWidth -g pixelHeight` to read the
// capture's real dimensions, then returns the `sips -Z` target as
// longest-actual-side * scale. Mock child_process.execFile so we can feed it a
// 4K vs a non-4K (1920x1080) Apple TV capture and assert the target.
const execFileMock = vi.fn();
vi.mock("node:child_process", async () => {
const actual = await vi.importActual<typeof import("node:child_process")>("node:child_process");
return { ...actual, execFile: (...args: unknown[]) => execFileMock(...args) };
});
import { tvTargetLongSide } from "../src/tools/screenshot";
// promisify(execFile) appends a node-style callback as the last argument. Reply
// with the given sips `-g` stdout via that callback when present.
function mockSipsDims(stdout: string): void {
execFileMock.mockImplementation((...args: unknown[]) => {
const cb = args.find((a) => typeof a === "function") as
| ((e: Error | null, r: { stdout: string }) => void)
| undefined;
cb?.(null, { stdout });
});
}
describe("tvTargetLongSide — tvOS screenshot scaling", () => {
beforeEach(() => execFileMock.mockReset());
it("scales against the real long side for a 4K (3840x2160) capture", async () => {
mockSipsDims("pixelWidth: 3840\npixelHeight: 2160\n");
expect(await tvTargetLongSide("/tmp/cap.png", 0.3)).toBe(1152); // 3840 * 0.3
});
it("regression: scales against 1920 for a non-4K Apple TV capture (no 2x blowup)", async () => {
// The standard non-4K "Apple TV" sim captures at 1920x1080. The old code
// hardcoded 3840, so `-Z 1152` against a 1920-wide image yielded an
// effective 0.6x — twice the requested 0.3. The target must be 1920*0.3=576.
mockSipsDims("pixelWidth: 1920\npixelHeight: 1080\n");
expect(await tvTargetLongSide("/tmp/cap.png", 0.3)).toBe(576);
});
it("falls back to the 4K long side when the dimension probe fails", async () => {
execFileMock.mockImplementation((...args: unknown[]) => {
const cb = args.find((a) => typeof a === "function") as
| ((e: Error | null) => void)
| undefined;
cb?.(new Error("sips: command not found"));
});
expect(await tvTargetLongSide("/tmp/cap.png", 0.3)).toBe(1152); // 3840 * 0.3 fallback
});
});