mirror of
https://github.com/software-mansion/argent.git
synced 2026-09-14 19:27:14 +08:00
Filip131311
5a898091fe
Both wait tools fetched their tree through `describeIos`, which short-circuits
every tvOS read to an empty tree — the iOS accessibility service cannot drive an
Apple TV. `await-screen-idle` resets whenever the tree is empty, so on tvOS that
condition was permanently true: it was structurally incapable of ever returning
settled, and polled to the timeout to discover it. It also read only the tree and
never the hint, so it said nothing at all about why. No selector could match
either. The TV surface had no wait primitive, leaving `run-sequence` there
timing-tuned and never robust.
`describe` works on the same screens because it routes TV targets to the focus
view before the iOS dispatch. The wait tools now poll that same source.
The adapter turns a focus read into an ordinary describe tree, so the polling,
fingerprinting and selector matching engines are untouched. Two things it must
get right:
- It uses the daemon's REAL frames. They were always in the payload — undeclared
on `TvElement` and dropped by describe's rendering — so `isVisible` and reading
order mean here what they mean everywhere else. An index-derived band is only a
fallback for backends that genuinely report no bounds.
- An empty read carries a hint. That is load-bearing, not decoration:
`await-ui-element` treats an empty tree as untrustworthy only when something
says so, and without it `condition: "hidden"` would report success on the first
poll of a still-launching app — releasing an interaction that was deliberately
gated. Strictly worse than the bug being fixed.
The cursor is marked with a synthetic `focused` trait rather than only the
`focused` field, which buys two things a field cannot: `{role:"focused"}` becomes
selectable, so "wait until focus lands on X" is expressible — the wait a TV
sequence actually needs between `tv-remote` and `select` — and the cursor enters
the idle fingerprint, so a screen whose focus is still moving does not read as
settled.
Android TV deliberately keeps its full uiautomator tree. An empty focus set is
steady state there, not a transition, for react-native-tvos screens whose focus
engine the OS accessibility tree cannot see; routing it onto the focus source for
consistency would import this very bug onto a platform that works. There is a
test guarding that.
The wait path never repairs the read path: `describeTv` retries and can respawn
the tvOS ax daemon, which inside a poll loop is redundant at best and destructive
at worst — it would drop the state being waited on. It takes one bare read per
poll and points at `describe` for the repair.
`await-screen-idle` also gains a `note`, which is not tvOS-specific. Any unsettled
wait on an empty read now explains itself, so a device whose screen is simply off
no longer produces a silent full-budget stall on any platform.
Beyond the reported symptom, this removes a real tax: the MCP layer waits on this
tool before every auto-screenshot and treats a failure by sleeping the whole
budget, so on tvOS every launch-app cost 3s and every run-sequence 15s. It now
returns as soon as the screen holds.
The skill's claim that describe returns "normalized frames" is corrected — the
frames exist, but the focus view does not print them — along with the navigation
step that told the agent to count rows from them.
Verified on an Apple TV 4K simulator (tvOS 26.5, TVSettings): idle settles in
~780ms where it previously never settled, cursor and value waits resolve in
single-digit ms where every condition previously failed at full timeout.
Fixes #620