mirror of
https://github.com/software-mansion/argent.git
synced 2026-09-14 19:27:14 +08:00
Piotr Krawiec
ab35c5653e
## What & why Fixes #474. On Vega, `describe` collapsed a `WebView` into a single opaque node — only the page `<title>` surfaced, none of the DOM. Root cause: a `WebView` embeds a nested `<app toolkitName="Chromium">` whose text nodes carry **no `role`** and keep their string in a `<text>` **element** child. The native RN UIToolkit instead tags text nodes `role="text"` with **inline** text. `isMeaningful` only counted inline `node.text`, so every WebView text node looked empty → the whole web DOM (grid tiles, headings, labels) was judged unmeaningful and flattened away. Notably the underlying automation toolkit *does* publish the full Chromium a11y tree (confirmed by querying the same backend through the Appium kepler driver) — this was purely a parser-side traversal gap. ## The fix (`source-parser.ts`) 1. **`ownText(node)`** — a node's text is now its inline text **plus** the text of its direct `<text>` element children. Both `isMeaningful` and `labelOf` use this one notion (they were already asymmetric: `labelOf` read `<text>` children, `isMeaningful` didn't). 2. **Sub-span collapse in `convert`** — the Chromium a11y tree aggregates a container's accessible name **and** repeats it as child spans (a tile `"10Tile 10"` nests `"10"` + `"Tile 10"`). Children are dropped only when they are **all plain text spans whose in-order labels exactly reconstruct the parent's aggregated label** (Chromium joins inline spans with no separator). Exact reconstruction — not substring containment — means a distinct `"Play"` under `"Playlist"` (or `"Tile 1"` under `"Tile 10"`) is never swallowed, and a real list whose container name joins rows with spaces (`"Home Settings About"`) never reconstructs from its rows, so separately-navigable rows keep their own frames. ## Verification - New regression tests in `vega-source-parser.test.ts` (real-device fixture `vega-webview-grid.xml`): all 12 grid tiles surface; frames normalize against the window; focused/selected state preserved; exact sub-spans collapse; **distinct substring child kept** (`Play`/`Playlist`); **distinct list rows kept with their own frames** (`Home`/`Settings`/`About`); labelless structural container keeps all children; structural/interactive same-text child kept. - Full `tool-server` suite green (one unrelated pre-existing failure: `await-ui-element.test.ts`, fails on `main` too). - **End-to-end on a running VVD**: a 12-tile WebView grid now renders every tile with pixel-accurate frames and live `Selected`/`focused` state, where `describe` previously returned one opaque node. ### Before / after (live device) ``` # before view "Argent WebView Grid" [clickable,focused] (0,0,1,1) # after view "Argent WebView Grid" [clickable,focused] (0,0,1,1) header view "Argent WebView Grid" view "Selected:" view "Tile 3" view "| Activations:" view "1" view "1Tile 1" ... view "12Tile 12" (each with its own frame) ```