mirror of
https://github.com/software-mansion/argent.git
synced 2026-09-14 19:27:14 +08:00
Hubert Gancarczyk
5ac69349f0
## What
Adds the CSS combinators to flow selectors, read off frames. Any action
or condition can now scope its matches by where an element sits relative
to another:
```yaml
- tap: { text: Delete, within: { id: profile-card } } # CSS "A B" — inside that container
- tap: { role: Switch, next: { text: Wi-Fi } } # CSS "A + B" — the nearest follower
- assert: { visible: { role: Button, after: { text: Danger zone } } } # CSS "A ~ B" — any follower
- assert: { hidden: { any: true, within: { id: empty-state } } } # CSS "*" — the universal selector
```
| Scope | CSS | Reads as |
| --- | --- | --- |
| `within: <sel>` | `A B` | the match's frame sits inside the scope
element's frame |
| `after: <sel>` | `A ~ B` | the match follows the scope element in
reading order |
| `next: <sel>` | `A + B` | as `after`, narrowed to the nearest follower
of each anchor |
Plus `any: true` — the CSS `*` universal selector, accepted only
alongside a scope and never beside `text`/`id`/`role`.
The child combinator `>` has deliberately **no analog**: flow trees
flatten to leaves under one synthetic root, so parent/child structure
does not survive to replay.
## How
- **Geometric, not ancestry-based.** Frames are what survives
flattening, so every scope is resolved from them by `findAll` in
`ui-tree-match`. The synthetic root never counts as a scope element, and
every relation requires a *distinct* node (nothing scopes itself).
- **Reading order is row-band aware.** An element follows the anchor
when it starts below the anchor's bottom edge, *or* shares its row band
and sits entirely to its right. That band rule is what makes `{ role:
Switch, next: { text: Wi-Fi } }` resolve the Wi-Fi row's own switch even
though the taller switch's frame starts a couple of pixels *higher* than
the label's — a raw top-y comparison gets that backwards. `followKind`
is the single classification both the follower test and the nearest-pick
grouping read, so they cannot disagree.
- **`next` is the union over anchors**, exactly as CSS `A + B` is: three
rows on screen give three switches, one per label.
- **Scopes compose and nest**, capped at `MAX_SELECTOR_SCOPES` (6)
counted across the whole relation *tree* — a size bound, not a depth
bound, because three relations branch and the runner's loose-alternative
expansion is exponential in the bare-string scopes it finds.
- **Flow-YAML only.** `await-ui-element`'s strict schema rejects
`within`/`after`/`next`/`any`, so a scope can never be silently dropped
on that path.
- Adding a fourth relation is a **compile error**: `RELATION_RESOLVERS`
is a `Record` keyed by the relation union, so a new scope cannot be
wired through parse/serialize/reporting and then ignored by the match
engine.
## Testing
- **Unit:** full tool-server suite green — **3006 passed / 1 skipped**.
New coverage in `ui-tree-match.test.ts`, `flow-utils.test.ts`,
`flow-selector-scopes.test.ts` (renamed from
`flow-within-selector.test.ts`, now covering all scopes) and a new
`flow-skill-docs.test.ts` that parses every YAML snippet in the skill so
the docs cannot rot. Each fix is mutation-checked: reverting it fails at
least one test.
- **Real-device E2E** (Pixel 3a, Bluesky on Metro 8081, via a branch
tool-server): three flows, 20 steps, all green — geometric ordering both
ways, two `text` asserts that pin `next`'s pick *by identity* out of
five candidates, empty-scope `hidden` semantics, and two observable taps
(`next`-scoped and `any: true`-scoped) that navigated to the *adjacent*
topic rather than the anchor's own.
- **iOS not covered.** Native-devtools does not attach on either booted
simulator in this environment, so *every* flow selector fails there — an
unscoped `{ text: "Video" }` fails identically. Verified environmental,
not caused by this branch: the globally-installed argent reports the
same, direct `SIMCTL_CHILD_DYLD_INSERT_LIBRARIES` injection also fails
to connect, and it survives a forced simulator reboot on both iOS 18.3
and 26.5. Worth re-running the iOS leg once that is healthy.
## Notes for reviewers
Two deliberate deviations from CSS, both documented in the skill:
- **`next` is looser than `+`.** Where `A + B` matches nothing unless
the immediate sibling is a `B`, `next` keeps looking and returns the
nearest match further on. That is what makes it survive the wrapper and
spacer nodes a flattened tree is full of — but a row *missing* the
control you asked for resolves to the next row's rather than failing.
Happy to switch to strict `+` semantics if the loud failure is
preferred.
- **"follows" is not transitive.** Against a tall anchor an element can
follow something that follows the anchor without following it directly,
so nesting `after` scopes is not the same as chaining `~`.
Other things worth a look:
- `nearestAfter` and `afterTester` are direct scans, not indexed. An
earlier sorted index was a *second* implementation of the nearest rule
and the two disagreed on frames near the tolerance; it also measured
2-13x slower than the scan on realistic shapes, with a prune that
provably never fired. The quadratic tail is documented with measured
numbers: 1.2 ms for a single-anchor `next` at the 12k-node tree cap,
~555 ms only when the anchor selector matches essentially every node.
- A bare-string *anchor* cannot self-correct the way the loose fallback
usually does — a decoy `testID` matching your anchor text wins the
identifier pass whenever anything sits after it. The skill tells authors
to spell sibling anchors as maps.
- `MAX_SELECTOR_SCOPES = 6` replaces the previous
`MAX_SELECTOR_WITHIN_DEPTH = 10`, so a `within` chain deeper than 6 that
parsed on the earlier commits of this branch now rejects. Nothing
released depends on it.
---------
Co-authored-by: Hubert Gancarczyk <claude-hubert.gancarczyk@swmansion.com>
58 lines
2.4 KiB
TypeScript
58 lines
2.4 KiB
TypeScript
import { describe, it, expect } from "vitest";
|
|
import { readFileSync } from "node:fs";
|
|
import * as path from "node:path";
|
|
import { parseFlow } from "../../src/tools/flows/flow-utils";
|
|
|
|
/**
|
|
* The create-flow skill is the agent-facing reference for selector scopes, so
|
|
* a snippet that no longer parses is a live defect: an agent copies it and the
|
|
* flow fails at parse. Guards the examples against parser drift.
|
|
*/
|
|
const SKILL = path.resolve(__dirname, "../../../skills/skills/argent-create-flow/SKILL.md");
|
|
|
|
function scopesSection(): string {
|
|
const md = readFileSync(SKILL, "utf8");
|
|
const after = md.split("#### Scopes")[1];
|
|
expect(after, "the scopes section is missing from SKILL.md").toBeDefined();
|
|
return after!.split("\nSelectors resolve against")[0]!;
|
|
}
|
|
|
|
// `…` marks an illustrative fragment (`within: <sel>`, and the deliberately
|
|
// REJECTED spelling the `any` paragraph contrasts against) — not runnable YAML.
|
|
const runnable = (snippet: string): boolean => !snippet.includes("…") && !snippet.includes("<sel>");
|
|
|
|
describe("create-flow SKILL.md scope snippets", () => {
|
|
it("every step snippet in the scopes section parses", () => {
|
|
const snippets = [...scopesSection().matchAll(/`((?:tap|assert|await|type|scroll-to):[^`]*)`/g)]
|
|
.map((m) => m[1]!)
|
|
.filter(runnable);
|
|
expect(snippets.length).toBeGreaterThan(8);
|
|
for (const snippet of snippets) {
|
|
expect(() => parseFlow(`steps:\n - ${snippet}\n`), snippet).not.toThrow();
|
|
}
|
|
});
|
|
|
|
it("every bare selector map in the scopes section parses in a tap slot", () => {
|
|
const maps = [...scopesSection().matchAll(/`(\{ (?:role|text|id|any)[^`]*\})`/g)]
|
|
.map((m) => m[1]!)
|
|
.filter(runnable);
|
|
expect(maps.length).toBeGreaterThan(3);
|
|
for (const map of maps) {
|
|
expect(() => parseFlow(`steps:\n - tap: ${map}\n`), map).not.toThrow();
|
|
}
|
|
});
|
|
|
|
it("the paragraph's rejected `any` spelling really is rejected", () => {
|
|
// The docs tell agents to write `{ role: Switch, next: … }` and NOT
|
|
// `{ any: true, role: Switch, next: … }`. If the parser ever started
|
|
// accepting the second, the advice would be noise.
|
|
expect(scopesSection()).toContain("may **not** sit beside");
|
|
expect(() =>
|
|
parseFlow("steps:\n - tap: { any: true, role: Switch, next: { text: Wi-Fi } }\n")
|
|
).toThrow(/already matches every element/);
|
|
expect(() =>
|
|
parseFlow("steps:\n - tap: { role: Switch, next: { text: Wi-Fi } }\n")
|
|
).not.toThrow();
|
|
});
|
|
});
|