mirror of
https://github.com/software-mansion/argent.git
synced 2026-09-14 19:27:14 +08:00
5ac6805040
this is urgent
---
## Problem
Every `native-*` tool can get stuck telling the agent to do something
that will never work.
I hit this during an end-to-end pass over 0.17.0. After a mid-session
simulator reboot, `native-devtools-status` reported:
```json
{ "envSetup": true, "appRunning": true, "connected": false,
"requiresRestart": true, "nextLaunchWillBeInjected": true, "injectable": true }
```
and every `native-*` tool returned `restart_required` with "Call
restart-app then retry." Following that advice never helped, and the
message offered nothing else. `flow-execute` was blocked too, since its
`launch` step needs the connection, and flow recording silently degraded
a portable `tap: { selector }` to raw coordinates when it couldn't reach
native devtools.
The app was injected the whole time. All three dylibs were mapped into
the process and the simulator log showed a successful dial on every
launch, while the tool-server held the listening end of that exact
socket and still reported `connected: false`. `argent server stop &&
argent server start` cleared it instantly.
## The real defect is the state, not the wording
`requiresAppRestart` returned `true` for every app without a live
connection:
```ts
async requiresAppRestart(bundleId) {
if (connections.has(bundleId)) return false;
await reverifyEnv();
return true; // <- asserted, never measured
}
```
That is not a measurement. `restart_required` was a constant wearing a
state's clothes, and the advice built on it stays literally true
forever, so an agent obeying it restarts the app indefinitely.
Softening the message would have left that intact. This PR measures the
state instead.
## Fix — derive it from the running process
`restart-app` relaunches into the simulator's **current** launchd
environment, so it can only ever help a process launched under different
terms than a fresh launch would get. That is answerable: read the
process. `launchctl list` already gives its pid; `ps eww -p <pid> -o
etime=,command=` gives the environment it was exec'd with and its age.
Simulator apps are ordinary host processes owned by the same user, so
both are readable.
`appConnectionState` (replacing `requiresAppRestart`) returns:
| state | meaning | remedy |
| --- | --- | --- |
| `connected` | devtools are live | — |
| `not_running` | no process to read | `launch-app` |
| `stale_process` | no bootstrap dylib, a previous run's endpoint, or
older than this service's listener | `restart-app` |
| `unregistered` | injected against **this** endpoint, started after the
listener came up, still silent | restart the **tool-server** |
| `indeterminate` | process could not be read (ios-remote, unreadable
`ps`) | `restart-app`, with the loop warning |
Only `not_running` / `stale_process` / `indeterminate` report
`restart_required`. `unregistered` gets a new `service_stale` precheck
status that names the tool-server restart and explicitly says not to
relaunch the app — the launch a restart would perform has demonstrably
already happened.
The listener-age comparison is load-bearing: after a tool-server restart
the unix path is identical (it is derived from the udid) but the inode
is new, so an injected process older than the listener still needs
relaunching. Without that check this would send people off to restart a
tool-server they had just restarted — a new loop in place of the old
one. Both grace comparisons lean away from `unregistered` - the first
towards `stale_process`, the second towards `indeterminate`, and both of
those ask for a relaunch - so an uncertain read costs at most one wasted
`restart-app`.
Call sites follow the same rule rather than each inventing guidance:
- `describe` withholds `should_restart` for `unregistered` (setting it
would rebuild the restart-app → describe loop through a live service
instead of a system app) and carries the diagnosis as a `hint`, which
still marks the empty read untrustworthy for `await-ui-element`.
- `flow-run` raises the measured diagnosis instead of a blanket
"relaunch it" — both when a flow
resolves a selector against the full hierarchy, and at the `launch`
step's readiness gate, which
previously polled for the connection and then guessed at "a stale or
duplicate argent server".
- `native-devtools-status` reports `state`, and `requiresRestart` is now
false for `unregistered`.
- The six `native-*` tools type their block as the shared
`NativeDevtoolsPrecheckBlock` instead of restating it, so a new precheck
outcome cannot reach an agent through a tool that never mentions it.
## Scope
Why the service can stop registering a connection it is listening for is
still #561 and is not fixed here. What changes is that the state is now
measured, so that defect reports itself accurately instead of
masquerading as an app that needs restarting.
## Verification
**E2E, real iOS 18.6 simulator, Bluesky (RN 0.81.5).** Two tool-servers
were raced onto the same per-udid socket — the #561 shape, where the app
is genuinely injected and connected, just to the newer instance. Both
were then queried on the same app at the same instant:
| | `connected` | `requiresRestart` | `state` | `native-describe-screen`
|
| --- | --- | --- | --- | --- |
| main, stale instance | false | **true** | — | `restart_required` —
"Call restart-app then retry." |
| this branch, stale instance | false | **false** | `unregistered` |
`service_stale` — "Restarting the app cannot change that … restart the
tool-server" |
| healthy instance | true | false | `connected` | `ok`, 44 elements |
Restarting the app in main's row lands on the newer instance again,
leaving the queried one stale forever. That is the loop, reproduced on
demand.
The other states were walked on the same simulator: a pre-existing
process against a fresh listener → `stale_process` (`requiresRestart:
true`); `restart-app` → `connected`; terminate → `not_running` ("Call
launch-app (or restart-app)").
**Unit.** `native-devtools-app-state.test.ts` drives the real blueprint
factory with a stubbed process table and covers all five states plus the
`etime` and env-token parsers. Mutation-checked — dropping the
listener-age comparison, the endpoint match, or the connect grace each
fails exactly one test and nothing else.
**Suite.** `vitest run` in `packages/tool-server`: 296 files, 3113
passed, 1 skipped. `tsc --build` across the workspace, `tsc --noEmit -p
tsconfig.test.json`, `eslint` and `prettier --check` on every changed
file: clean. `origin/main` merged in first.
## Review round
A review pass over the derivation itself, with each finding reproduced
before it was acted on.
**`describe` computed the diagnosis and threw it away.**
`buildAppStateMessage` ran for every
unconnected state, but only `unregistered` carried it out; the rest
returned a bare
`should_restart: true`. That boolean reaches an agent as an undocumented
JSON field — `describe`'s
own description never mentions it — and the single place it is rendered
as prose is
`await-ui-element`'s timeout note, which spells it *"call restart-app
and retry"*. So the two states
whose message carries the escape hatch lost it exactly where it was
needed. `indeterminate` is the
sharp end: on `ios-remote`, whose app processes are out of reach of the
local process table, it is
the **only** reachable unconnected state, so that path had no exit at
all. The diagnosis now rides
out as a `hint` for every state; `should_restart` still marks only the
states a relaunch fixes.
**`native-devtools-status` measured the same device twice.**
`isAppRunning` ran its own
`launchctl list`, and `appConnectionState` ran another one after
re-verifying the launchd env — so
the two snapshots sat seconds apart and could contradict each other
(`appRunning: true` beside
`state: "not_running"`), with `requiresRestart` gated on the staler of
the pair. Both are derived
from one measurement now; only `indeterminate`, which genuinely leaves
running-ness unanswered,
pays for a second probe.
**Coverage.** Mutation testing found the 3 s grace term entirely
unpinned — the fixtures sat 200x
away from the boundary on either side, so both dropping the term and
weakening `>=` to `>` kept the
suite green while flipping a relaunchable process to `unregistered`, the
one verdict the file exists
to withhold. Also unpinned: the `reverifyEnv` call (swapping it for the
latching `ensureEnvReady`
the comment explicitly forbids passed the whole suite) and an
unparseable `ps` age staying "no
evidence" rather than becoming a measurement. All three are now covered,
and each new test was
confirmed to fail on the un-fixed code.
**Verified against the real thing.** The `launchctl` and `ps` contracts
were re-checked on an
iOS 18.6 simulator rather than reasoned about: row layout and pid
column, one-line `ps eww` output
with the argent tokens present in a live app's environment, `etime`
matched against `lstart` to the
second, and the four `etime` format strings read out of the `ps` binary.
Two corrections came out of
it — the environment can reach `kern.argmax`, which is exactly Node's
default `maxBuffer`, so the
probe now sets the same 16 MiB its siblings use; and `ps` suppresses the
environment of a
SIP-protected platform binary, which the docstring's "never as 'not
injected'" promise did not
account for (unreachable here, since `com.apple.*` is rejected as
non-injectable upstream, but the
promise now says so).
**Also corrected:** the `argent-device-interact` skill offered
`restart-app` as the only remedy for
a missing injection; the dylib-platform check in
`download-native-binaries.sh` and a
`requiresAppRestart` reference in `boot-device.ts` described states that
no longer exist; and three
tool descriptions still said *"Fails if native devtools are not
connected"* directly beside the new
sentence explaining that this case returns a status rather than failing.
**Known and deliberately left** (both pre-existing, neither introduced
here):
`describe` with no `bundleId` auto-targets from the connected list, so
an unregistered app throws
`NATIVE_TARGET_NO_CONNECTED_APPS` before any state is measured and the
message is swallowed by the
fallback's catch — surfacing it would newly mark those empty reads blind
for `await-ui-element`'s
`hidden` condition, which wants its own verification pass. And a dylib
that is inserted but silently
skipped by dyld (a platform mismatch) reads as injected-but-silent, so
`service_stale` → tool-server
restart → `stale_process` → `restart-app` is a two-step cycle;
distinguishing it needs a load
confirmation the process table cannot give.
## Review round 2
A second pass over the derivation, run as four independent sweeps
(claims-vs-code, nearest-twin
and reachability, non-happy paths and inputs, and absence). Three of the
four independently landed
on the same defect.
**The flow launch gate was the one caller that never asked whether the
app could be injected at
all.** Every other consumer of `appConnectionState` gates `com.apple.*`
first - the precheck
throws, `describe` returns its terminal hint, `native-devtools-status`
short-circuits,
`flow-ios-tree` only ever auto-resolves a connected app.
`waitForNativeDevtools` measured the raw
launch id. That matters because the launchd environment carrying the
bootstrap dylib is
simulator-wide, so a system app's process inherits the very tokens the
measurement reads.
Confirmed on an iOS 18.6 simulator: `com.apple.Preferences`, launched
under argent, reports
```
71940 0 UIKitApplication:com.apple.Preferences[9a63][rb-legacy]
etime=00:03
DYLD_INSERT_LIBRARIES=.../libArgentInjectionBootstrap.dylib
NATIVE_DEVTOOLS_IOS_CDP_SOCKET=/tmp/argent-nd-39B9C547.sock
```
Run through the shipped algorithm that is `processCarriesInjection:
true` against an older
listener, i.e. **`unregistered`** - so a flow with an `ios:` launch
entry of `com.apple.Preferences`
failed its launch step with *"restart the tool-server"*, for an app no
tool-server restart can ever
help. Library validation means it can never connect, so the gate now
says that terminally.
**`describe` dropped the entire diagnosis when the probe rejected.**
`appConnectionState` re-applies
the launchd env before it can answer anything, so it throws outright on
a sim that goes away
mid-call; that landed in the native fallback's outer `catch` and
returned an empty tree with no
hint. An agent saw "nothing on screen" rather than "could not be read",
and `await-ui-element` had
nothing to append to its timeout note. The two other consumers of the
same call already degraded to
`indeterminate` instead - this one now matches them.
**`native-devtools-status` re-ran the probe that had just failed.** One
route into `indeterminate`
is `inspectRunningApp` rejecting, i.e. `simctl spawn <udid> launchctl
list` failing. The
running-ness fallback then issues that same command unguarded, so a
shut-down simulator surfaced a
raw subprocess error rather than the structured `init_failed` guidance -
which the branch 40 lines
above it handles correctly, for exactly this reason.
**`native-devtools-status` was the only surface withholding the loop
escape.** Every other consumer
carries `buildAppStateMessage`, whose `indeterminate` text ends *"do not
keep restarting the app;
restart the tool-server"*. This tool returned booleans only, and
`requiresRestart: true` is the
whole of what an `indeterminate` app tells you. On ios-remote, where no
app process can ever be
inspected, that is the only state a running app reaches - so the tool
whose stated job is verifying
readiness answered with the restart loop and nothing else. It now
returns `message`.
**Also fixed:** both probes log their own failure (a broken probe
degrades every app to
`indeterminate`, indistinguishable at the tool surface from a genuinely
uninspectable one - the
sibling `ps` probe in `vega-process.ts` logs for the same reason); the
gate skips its
several-second measurement once the run is cancelled; and the gate no
longer repeats the bundle id
its caller's prefix already names.
**Prose that had drifted from the code:** `not_running` is reachable on
ios-remote too, so
`indeterminate` is only the sole unconnected state a *running* app
reaches there; the second grace
comparison leans to `indeterminate`, not `stale_process`; four of the
five states settle
running-ness, not four besides `not_running`; a null pid means a row
that did not parse, not a job
launchd reported as having no process (measured: a `UIKitApplication`
row is removed outright when
the app exits, and `-` appears only on daemon labels);
`waitForNativeDevtools`'s documented return
contract described the branch the change had rewritten; and a rationale
comment was attached to the
test above the one it described.
**Coverage.** The gate's measured reason had none at all - replacing the
whole diagnosis with a
literal left all 542 flow tests green. Eleven mutations were run against
the fixes and every one is
now killed: the injectability guard, the measured reason, the doubled
bundle id, `describe`'s reject
guard, `describe`'s `unregistered` hint merge, the `message` field, the
precheck fallback, the
`appRunning` guard on `requiresRestart`, a state renamed in the
description only, the connect grace
widened from above (the old fixtures all sat 10x clear of it, so it
could have grown to 25 s
unnoticed), and the `indeterminate` guidance line.
---
## Review round 3 — external review findings
Six findings from @hubgan's E2E review, each reproduced before it was
acted on. Fixed in b9d17437
and 3eb59ef7. **This round supersedes several claims made above — they
are listed at the end.**
**The launch gate refused every `launch:` of an Apple system app.** A
coordinate-only flow gave
`["launch:error","tool:skip"]` with `resolveService` called 0 times —
the guard short-circuited
before any measurement — and the reason it emitted said *"Drive it by
coordinate instead"*, from
the gate that had just killed a purely coordinate-driven flow.
`argent-create-flow`'s worked
example (SKILL.md:171, :199-209) is exactly that shape.
The premise underneath it did not survive checking. #453, which
motivated `isInjectableBundleId`,
recorded `connected: false` for `com.apple.Preferences` on an iOS 26.5
sim; the review recorded
`connected: true`, with both dylibs mapped, on 18.5. So injection into a
simulator system app
appears to be runtime-dependent rather than categorically impossible,
and neither reading is
checkable from CI. The gate therefore no longer decides: it polls
exactly as before — where the app
does connect, nothing changes — and only on timeout does a
non-injectable id return "no failure"
instead of a reason. The launch step's job is to start the app; it
succeeded. **Whether
`isInjectableBundleId`'s blanket prefix rule is correct at all is now an
open question, and a
separate one from this PR.**
**The flow tree source could not report it either, because its
measurement was unreachable.**
`resolveNativeTargetApp(api, undefined)` draws candidates from
`listConnectedBundleIds()` — the same
map `appConnectionState` reads to answer `connected` — so a
non-connected app threw the stock
auto-target error first, whose "Launch or restart the app first" is the
loop this PR exists to
break. Driving the real factory: `appConnectionState === "unregistered"`
and
`listConnectedBundleIds() === []` at the same instant. The race between
the two is closed, not
narrow: no macrotask boundary separates them, while the socket `close`
handler is one. The branch's
own test passed only because its stub let the two accessors disagree.
The flow now threads the id
its `launch` step named — which does not come from that map — through
`ActionEnv` → `fetchFlowTree`
→ `queryFullHierarchyTree`; the unreachable branch is gone.
**The gate emitted `buildAppStateMessage` verbatim for every state but
`unregistered`.** Those are
written for the native-* tool surface, whose reader has not launched
anything, so a crash-on-launch
app was answered with "Call launch-app (or restart-app) then retry" —
the action the step had taken
9.5 s earlier. `flowLaunchGateReason` now rewrites each state for a
caller that has just launched,
exhaustively, so a state added later cannot inherit the tool-surface
remedy.
**`describe` without a `bundleId` — the default form — lost the whole
diagnosis.** Auto-targeting
throws before anything is measured and the outer catch returned the tree
bare, so
`await-ui-element`'s blind-read guard (which keys off exactly
`hint`/`should_restart`) let a
`hidden` wait release in 0 ms against an element still on screen, where
the same call *with* a
`bundleId` correctly refused. An empty read the hierarchy could not
corroborate now says so. A
device with **no** native-devtools service at all is deliberately left
unqualified — that is an
absence of a second opinion, not a failed attempt at one, and marking it
blind broke a real
`await-ui-element` case.
**Three of the six `native-*` descriptions named the terminal system-app
rejection and three did
not**, though all six throw the identical error (verified: one
byte-identical message across all
six). The clause is inlined verbatim in all six —
`scripts/extract-tools.mjs` accepts only a single
plain literal and silently drops anything else from the SpiderShield
scan.
**`not_running` cannot tell "stopped" from "not installed"** — both
simply lack a
`UIKitApplication:<id>` row — and for the second reading its only remedy
is refused outright by
`simctl launch`. It now names both readings instead of asserting the one
it cannot see. A real
probe was considered and not taken: `get_app_container` needs a booted
sim and ios-remote has no
such verb, so it would answer `undefined` on exactly the host where
`indeterminate` is already the
only reachable state.
### Claims above that this round supersedes
- *"Library validation means it can never connect, so the gate now says
that terminally."* — the
gate now lets system-app launches through; the premise is contested (see
above).
- *"The flow launch gate was the one caller that never asked whether the
app could be injected at
all"* and *"`flow-ios-tree` only ever auto-resolves a connected app"* —
both inverted here.
- *"Known and deliberately left"* — the `describe` no-`bundleId` path
was closed, with tests, so
that paragraph now over-counts by one.
- The suite figures in **Verification** and **Review round 2** are
pre-round-3 counts.
---
## Review round 4 — reviewing round 3
Three lenses over 856efd8e's two parent commits (claims-vs-code,
non-happy
paths + reachability, absence + mutation). Round 3 did not survive
intact, and
what follows is what was wrong with it.
**Three statements were false in prose an agent acts on.** `connecting`
requires
a process inside the 3 s connect grace, while this gate takes its
verdict 9.5 s
after launching the app — so its remedy claimed the step had launched
the very
process it was measuring, next to a base message reading "it launched
moments
ago". The figure that arm and `not_running` quoted was the connect wait
alone,
omitting the 1.5 s post-launch settle that precedes it
(`LAUNCH_TO_VERDICT_MS`
now names the whole spend). And `stale_process` blamed the launchd
environment,
which is false for the producer reached only when
`processCarriesInjection` is
**true** — that process carries this endpoint and is merely older than
the
listener. The measured text it wraps names both producers, so the two
contradicted each other in one message.
**A `hidden` wait still released in 1 ms.** Round 3 carved out a failed
native-devtools resolution as "no service on this device at all". The
blueprint
is registered unconditionally, so on iOS that rejection only ever means
the
service failed to come up — a socket bind losing to a concurrent
same-udid
server, which this repo documents as the dominant crash source in 0.16.0
telemetry. Reproduced: `describe` → no hint, then `await-ui-element
hidden` →
`{"success":true,"elapsed":1}`. It is now as unexplained as its
siblings.
**The failure-code discrimination was defeatable.** The flow tree source
classified auto-targeting's throw by reading a code off a module-local
`Symbol`,
so a duplicate `@argent/registry` instance would read it as absent and
fall back
to the exact message the change replaces. It tests the empty connected
list
directly now — the same condition, structurally, with no symbol
involved. The
`connected` arm added in round 3 was unreachable for the same reason the
branch
it replaced was, and is gone.
**Coverage the mutation lens found missing.** Three of the four sites
threading
the launched bundle id were unpinned — including the `settleTree` read
every
`tap`/`type`/`scroll-to` uses, so an action step could have silently
lost the
terminal reason with the suite green. Also unpinned: `unexplainedHint`
dropping
a pre-existing degraded hint (where its exact twin fifty lines up **is**
pinned), the tree source's "Flows resolve selectors" half, and both of
the
gate's withholdings.
**Test cost.** The four per-state gate tests drove the whole runner in
pumped
fake time, ~9 s each, which is what pushed this file past vitest's
default under
parallel load. The mapping is a pure function and is tested as one; one
end-to-end test still proves the gate reaches it. 46 s → 16 s, with more
tests
than before. Widening the fake-timer step was tried and is worse (52 s)
— the
pumps are what let the run's disk I/O proceed.
**Also corrected:** comments the round-3 commits had falsified in the
very files
they edited; the `ActionEnv` claim that a fragment never carries a
launch step
(a flow is classified by its FIRST non-echo step, so a later launch is
legal and
does thread its id); `isBlindRead`'s enumeration of what sets its flags;
and
change narration that had leaked into permanent prose.
`argent-create-flow` now
states what a `com.apple.*` target costs a flow — its worked example is
selector-driven, and round 3's justification comment claimed the
opposite.
**Verification.** 17 mutations across both rounds, every one killed by a
specifically-named test. Two consecutive full runs of
`packages/tool-server`:
3181 passed, 1 skipped, and 6 failures in `boot-device-hotboot.test.ts`
that are
pre-existing on this host (it asserts `-gpu darwin`/`swiftshader` where
a Linux
box resolves `-gpu host`; the file is untouched by this PR). `tsc
--build`,
`tsc --noEmit -p tsconfig.test.json`, eslint and prettier clean on every
changed
file; `scripts/extract-tools.mjs` emits all 75 tools with zero warnings.
---
## Review round 5 — reviewing round 4
Round 4 left two of the defects it set out to fix and introduced one. It
also
broke CI.
**ESLint failed on the push.** Two `` \` `` escapes inside a
double-quoted string
(`no-useless-escape`). The pre-push check missed it because it was piped
through
`tail`, so the shell read the pipe's exit status rather than eslint's
and
reported success over a failure. Gates are run unpiped now, and
repo-wide —
`eslint .` is what CI runs, and a changed-files invocation covers a
subset of it.
**`unregistered` still quoted the connect wait alone** — the exact
understatement `LAUNCH_TO_VERDICT_MS` was introduced for, applied in
round 4 to
`not_running` only. And the end-to-end test asserted `8000 ms` while its
sibling
fifty lines down asserted the whole spend, so the file enforced two
contradictory rules and the *correct* figure would have failed CI. The
post-launch settle counts: the connection is read off the live map and
the poll
checks it once before its first sleep, so a dial landing inside those
1.5 s is
caught. All three arms that quote a figure now quote the same one,
pinned
together.
**`stale_process` named the launchd environment as fact** four lines
below a new
comment saying that is false for one of the state's two producers. It is
in fact
sound — but only where the message already scopes it, to a *second*
landing,
whose relaunch makes the process younger than any established listener
and so
rules that producer out (it needs `processAge + grace >= listenerAge`).
The
comment now says which producer and why. The test named
"does not blame the launchd environment" asserted only the phrase it
kept, so it
was green with the blame present; it now pins the ordering it is named
for.
**The `connected` arm is reachable — round 4's "unreachable" claim was
wrong.**
`appConnectionState` re-reads the live connections map *after* its env
re-apply
and process probe (`native-devtools.ts`), several simctl round-trips
past the
empty list that routed the call there, precisely so a dial landing in
that window
is not reported as unregistered. Folding the arm away left that case
throwing a
bare "Flows resolve selectors…" with no app named and no remedy. It is a
branch
again, and says the connection arrived mid-read.
**`unexplainedHint` dropped `screenshot` along with the `bundleId`
suggestion**,
so a caller that passed a bundle id received a hint with no next action
at all —
while `argent-device-interact` tells the agent to follow that hint.
`screenshot`
is unconditional now. Its JSDoc also claimed two of three call sites are
reached
only after an explicit id resolved and measured connected; all three are
reachable with an explicit id, as the file's own comment twenty lines
above says.
**Also:** `argent-create-flow` recommended `tool: gesture-tap` where the
same
file documents the first-class, equally tree-free `tap: { x, y }`; and
change
narration removed from four more comments.
### Standing caveats
- **The `com.apple.*` premise is still unsettled and is not settled by
this PR.**
#453 recorded `connected: false` on iOS 26.5; the review recorded
`connected: true` with both dylibs mapped on 18.5. Every change here is
correct
under either reading, but `isInjectableBundleId` remains a blanket
prefix
predicate and may be wrong on some runtimes. Worth its own issue.
- `boot-electron-spawn-error.test.ts` and `boot-device-hotboot.test.ts`
fail on
the Linux host used for this work; both were confirmed failing at the PR
base
`b2e5c54d` and neither file is touched here.
---
## Review round 6 — reviewing round 5
Round 5's own headline fix was both broken and unpinned.
**The hint stopped parsing.** Moving `screenshot` out of the conditional
clause
without rejoining the sentence produced, on the default no-`bundleId`
form:
> … not evidence that nothing is on screen. Pass `bundleId` to have the
connection state measured, or Take a `screenshot` to see what is there.
That is the exact string `argent-device-interact` instructs an agent to
act on.
Both forms are one sentence again, and both are now pinned by a test
anchored to
the end of the string rather than matching a substring.
**And deleting the action entirely left the suite green.** The defect
round 5
describes — "a caller that passed a bundle id got a hint with no next
action at
all" — could be reintroduced with 635 tests passing. Nothing asserted
the hint
named a next step. It does now.
**Two claims round 5 corrected in the source survived verbatim in
tests.** The
"two of the three call sites are reached only after an explicit id
resolved and
measured connected" partition (all three are reachable with an explicit
id), and
"waited only NATIVE_READY_TIMEOUT_MS" — sitting ten lines above the
assertion
that now forbids that figure.
**A guard test did not test its own name.** `does not blame the launchd
environment for a stale process` pinned only *where* the blame sits.
Restating
it as fact on a first landing — `It lands here twice, so the …` —
passed. It
asserts the conditional now, which is the whole distinction: only a
repeat rules
out the producer that carries this endpoint.
**The `connected` branch still had no coverage** — which is how round 4
came to
delete it as dead in the first place. It is exercised now, including
that its
message still names the app and an action.
**Also:** the terminal system-app reason recommended `tool: gesture-tap`
where
the same round-5 fix in `argent-create-flow` had moved to the
first-class,
equally tree-free `tap: { x, y }`. The twin was missed.
**Verification.** Four mutations against these fixes, all killed:
dropping the
`screenshot` action; reintroducing the broken join; stating the launchd
blame as
fact; folding the `connected` branch away. Gates run unpiped and read by
exit
code — `eslint .` 0, `tsc --build` 0, `tsc --noEmit -p
tsconfig.test.json` 0,
`prettier --check .` 0.
---
## Review round 7 — reviewing round 6
**The gate asserted the contested fact as settled.** Round 3's
justification for
keeping the 8 s wait read "a system app's process inherits the injection
tokens
**and does connect on some runtimes**". Eight other places in this repo
say the
opposite — `blueprints/native-devtools.ts` ("our dylib can never load",
and the
agent-facing precheck throw), `native-devtools-status`, `describe`'s iOS
hint,
and `test/describe-tool.test.ts` ("a com.apple.* app never connects"), a
file
these commits edit. `flow-ios-tree.ts` contradicted *itself*, its
docblock
saying "can never load" above a message saying "cannot be relied on to
inject".
The fact is not settleable here, so nothing asserts it now. The comment
cites
both observations — #453's `connected: false` on iOS 26.5, the review's
`connected: true` with both dylibs mapped on 18.5 — and justifies the
wait
without picking: polling costs this step its timeout under the
pessimistic
reading and is the only thing that makes selectors work under the
optimistic
one, so waiting spends time on an open question where skipping would
bake an
answer into behaviour. The tree source's docblock is hedged to match the
message
it already had.
**This PR does not resolve that inconsistency**, and
`isInjectableBundleId`
remains a blanket `com.apple.` prefix predicate. It deserves its own
issue.
**A guard let the thing it guards be deleted.** Round 6 made the
repeat-landing
assertion conditional (`if (blame !== -1)`), so removing the whole
clause —
leaving a reader who lands there twice with "re-run the flow" and no
escape —
passed 36/36. Unconditional again, wording pinned.
**A rationale that was false in both directions.** "Keeps the
`not.toContain`
guards non-degenerate: they would pass vacuously if the constants were
made
equal" — equal constants make them *fail*, and 500 vs 1500 makes them
fail for
the wrong reason (`"1500 ms".includes("500 ms")`). It pins the substring
property those guards actually need.
**Two more presence-only guards.** "keeps the ax-degraded boot guidance
**ahead
of** the unreadable-hierarchy note" asserted only that both strings
appear, and
passed with them swapped. And round 6's own `tap: { x, y }` twin fix had
no test
at all — the same absence that got the `connected` branch deleted as
dead two
rounds earlier. Both pinned, and the `tap: { x, y }` claim was verified
end-to-end: that step form parses, resolves through `resolveTargetPoint`
with no
tree read, and passes in a flow whose selector step fails with this very
message.
**Two test names promised more than they checked** — a remedy "that is
not the
action the step just took", asserted for states whose remedy is another
relaunch; and a crash-on-launch guard forbidding only the literal `Call
launch-app` while the arm legitimately suggests launch-app for watching
the exit.
**Verification.** Three mutations, all killed: deleting the
repeat-landing
clause, reverting the coordinate form to `tool: gesture-tap`, and
putting the
read-note ahead of the boot guidance. Gates read by exit code, unpiped:
`eslint .` 0, `tsc --build` 0, `tsc --noEmit -p tsconfig.test.json` 0,
`prettier --check .` 0. Suite: 3187 passed, 1 skipped, and the 6
pre-existing
`boot-device-hotboot` failures on this Linux host.
---
## Review round 8 — reviewing round 7
**The gate's own justification was false, and the disproof was already
in this
PR's review.** Round 7's comment claimed polling "is the only thing that
makes
selectors work at all" under the optimistic reading. It is not:
`settleTree`
retries a failed tree read for `SETTLE_TIMEOUT_MS` on top of
`POST_LAUNCH_SETTLE_MS`, so roughly the first 4.5 s of connect latency
is
absorbed with no gate at all — which is exactly why the review's
fragment (no
`launch` step, therefore no gate) resolved selectors 4/4 against
`com.apple.Preferences` on 18.5. Confirmed with an in-repo
counterfactual:
at 2000 ms connect latency the flow passes with or without the gate; at
6000 ms
only with it. What the poll adds is the band from ~4.5 s out to
`LAUNCH_TO_VERDICT_MS`, and the comment now says that.
**Hedging one surface left its twins absolute — the same failure mode, a
fourth
time.** The precheck throw, 230 lines from the message this PR hedged
and in the
same file, still told agents Argent "can never be injected into it"; so
did the
predicate's docblock, `native-devtools-status`'s description ("the dylib
can
never load into them", "TERMINAL state — the app can never be
injected"), and
`describe`'s terminal hint. An agent reading `argent-create-flow` was
told
"depends on the simulator runtime" and the status tool told it "can
never".
Every agent-facing surface now says injection cannot be RELIED on, and a
test
pins that none of them claims impossibility — the six native-*
descriptions, the
status description, the shared recovery text, the device-interact skill,
and
(separately, since it is module-private and reachable only through the
tool's
output) `describe`'s terminal hint. Each surface is asserted non-empty
first, so
a missing one cannot satisfy the negative match vacuously.
**The decision is untouched.** `isInjectableBundleId` still answers "not
injectable", the precheck still throws, and the description still says
`Do NOT restart/retry` — pinned by the same test, because softening the
claim
must not soften the behaviour or #453's restart loop returns. An app
that MIGHT
connect cannot be the basis of a retry loop either, which is now the
predicate's
recorded reason.
**Also:** the round-7 reword left "The flow / flow path" duplicated;
withholding
the verdict before the measurement (promoted to its own paragraph last
round)
was asserted nowhere, so moving it after `appConnectionState` left the
suite
green — a test now proves the measurement is not taken for an app whose
verdict
is withheld either way; and two more test names were narrowed to what
they
actually check.
**Verification.** Mutations run against every decision these commits
added.
Killed: re-asserting "can never load" in the status description;
dropping its
terminal instruction; re-asserting impossibility in `describe`'s hint;
moving
the withholding after the measurement; and the earlier rounds' decisions
re-checked and still pinned. One survivor is unpinnable by construction
— a
claim inside a code comment, which no test can assert. Gates read by
exit code,
unpiped: `eslint .` 0, `tsc --build` 0, `tsc --noEmit -p
tsconfig.test.json` 0
(this one rejected a tuple vitest had accepted), `prettier --check .` 0.
---
## Review round 9 — scope: the skill docs
This is a state-derivation fix. It should not be rewriting skill docs,
and the
earlier rounds had it doing exactly that.
**`argent-create-flow` is back to its base.** Nothing in it was
falsified by this
change. The `com.apple.*` caveat rounds 4-6 grew there documents the
question
this PR explicitly declines to settle — whether `isInjectableBundleId`'s
blanket
prefix rule is right at all — and that belongs to its own issue, not to
a
paragraph in a skill an agent reads before recording a flow.
**`argent-device-interact` only got smaller.** It said *"call
`restart-app` first
if needed"* — the unconditional-restart advice the measured states exist
to
replace — and that clause is deleted. The four-status routing table the
first
round put in its place is gone with it: the remedy already travels on
the tool's
own output and on the six `native-*` descriptions, both pinned by tests,
and a
second copy in a doc no code reads is one more surface to drift.
Tests follow the prose. The two `SKILL.md` reads in
`native-devtools-status.test.ts`
go with the text they pinned, and two comments that cited the reverted
skill text
are corrected.
### Claims above that this round supersedes
- The first round's *"the `argent-device-interact` skill offered
`restart-app` as
the only remedy for a missing injection"* — the clause is deleted, not
replaced.
- Round 4's *"`argent-create-flow` now states what a `com.apple.*`
target costs a
flow"*, round 5's *"`argent-create-flow` recommended `tool:
gesture-tap`"*, and
round 6's twin fix in the skill — all in the reverted paragraph. The
matching
source-side fix in `flow-ios-tree.ts` stands; only its stated twin is
gone.
- Round 8's cross-surface certainty test no longer reads the skill. It
covers the
status description, the six `native-*` descriptions and the shared
recovery text
— the surfaces that make the claim.
**`origin/main` merged in, and #519 needed one test to follow.** It
landed after
this branch last merged main and makes a `getFullHierarchy` reply with
no windows
an error — an untrustworthy read, not a blank screen. `does not measure
when
auto-targeting resolved an app` had been using an empty `windows` as a
don't-care
return, so the call now ended before it could observe anything. The stub
serves a
real window, which is the state the test names. Mutation-checked:
dropping the
empty-connected-list term from the guard makes the measurement fire and
fails
that test alone.
**Verification.** `packages/tool-server`: 304 files, 3242 passed, 1
skipped, on a
macOS host (the `boot-device-hotboot` failures noted in the standing
caveats are
Linux-only and do not reproduce here). Gates read by exit code, unpiped:
`eslint .` 0, `tsc --build` 0, `tsc --noEmit -p tsconfig.test.json` 0,
`prettier --check .` 0.
1511 lines
67 KiB
TypeScript
1511 lines
67 KiB
TypeScript
import { describe, expect, it, vi } from "vitest";
|
|
import { FailureError, FAILURE_CODES, getFailureSignal } from "@argent/registry";
|
|
import {
|
|
buildAppStateMessage,
|
|
isInjectableBundleId,
|
|
NON_INJECTABLE_NATIVE_WARNING,
|
|
NON_INJECTABLE_RECOVERY,
|
|
precheckNativeDevtools,
|
|
MAX_NATIVE_DEVTOOLS_INIT_ATTEMPTS,
|
|
type NativeDevtoolsApi,
|
|
type NativeDevtoolsAppState,
|
|
type NativeDevtoolsInitFailure,
|
|
} from "../src/blueprints/native-devtools";
|
|
// Both tools gate `execute()` behind `ensureDeps(["xcrun"])`, but the
|
|
// restart-guidance / init_failed logic under test never shells out to xcrun.
|
|
// The real probe makes this pass only on a host with Xcode (dev macOS) and
|
|
// fail on the Linux CI runner with `missing: [xcrun]`. Stub the gate to a
|
|
// no-op so the test exercises the logic it's about, not the host's toolchain.
|
|
// Keep the rest of the module (DependencyMissingError, cache reset) intact.
|
|
vi.mock("../src/utils/check-deps", async (importOriginal) => {
|
|
const actual = await importOriginal<typeof import("../src/utils/check-deps")>();
|
|
return {
|
|
...actual,
|
|
ensureDeps: vi.fn(async () => {}),
|
|
ensureDep: vi.fn(async () => {}),
|
|
};
|
|
});
|
|
|
|
import { flowLaunchGateReason } from "../src/tools/flows/flow-run";
|
|
import { nativeDevtoolsStatusTool } from "../src/tools/native-devtools/native-devtools-status";
|
|
import { nativeDescribeScreenTool } from "../src/tools/native-devtools/native-describe-screen";
|
|
import { nativeFindViewsTool } from "../src/tools/native-devtools/native-find-views";
|
|
import { nativeFullHierarchyTool } from "../src/tools/native-devtools/native-full-hierarchy";
|
|
import { nativeNetworkLogsTool } from "../src/tools/native-devtools/native-network-logs";
|
|
import { nativeViewAtPointTool } from "../src/tools/native-devtools/native-view-at-point";
|
|
import { nativeUserInteractableViewAtPointTool } from "../src/tools/native-devtools/native-user-interactable-view-at-point";
|
|
|
|
function makeNativeApi(options: {
|
|
envSetup?: boolean;
|
|
connected?: boolean;
|
|
appRunning?: boolean;
|
|
initFailure?: NativeDevtoolsInitFailure | null;
|
|
state?: NativeDevtoolsAppState;
|
|
}): {
|
|
api: NativeDevtoolsApi;
|
|
ensureEnvReady: ReturnType<typeof vi.fn>;
|
|
reverifyEnv: ReturnType<typeof vi.fn>;
|
|
isAppRunning: ReturnType<typeof vi.fn>;
|
|
} {
|
|
let envSetup = options.envSetup ?? false;
|
|
const ensureEnvReady = vi.fn(async () => {
|
|
envSetup = true;
|
|
});
|
|
const reverifyEnv = vi.fn(async () => {
|
|
envSetup = true;
|
|
});
|
|
const isAppRunning = vi.fn(async () => options.appRunning ?? false);
|
|
|
|
return {
|
|
api: {
|
|
isEnvSetup: () => envSetup,
|
|
socketPath: "/tmp/mock.sock",
|
|
ensureEnvReady,
|
|
reverifyEnv,
|
|
getInitFailure: () => options.initFailure ?? null,
|
|
isConnected: () => options.connected ?? false,
|
|
isAppRunning,
|
|
listConnectedBundleIds: () => [],
|
|
appConnectionState: async () => {
|
|
if (options.connected) return "connected";
|
|
// Mirrors the real API: the unconnected path re-applies the launchd env
|
|
// before it judges the process, so callers can still pin that repair.
|
|
await reverifyEnv();
|
|
return options.state ?? (options.appRunning ? "stale_process" : "not_running");
|
|
},
|
|
activateNetworkInspection: () => {},
|
|
getNetworkLog: () => [],
|
|
clearNetworkLog: () => {},
|
|
getAppState: async () => {
|
|
throw new Error("not implemented");
|
|
},
|
|
detectFrontmostBundleId: async () => null,
|
|
queryViewHierarchy: async () => ({}),
|
|
},
|
|
ensureEnvReady,
|
|
reverifyEnv,
|
|
isAppRunning,
|
|
};
|
|
}
|
|
|
|
describe("native-devtools-status tool", () => {
|
|
it("reports a running uninjected app as needing restart", async () => {
|
|
const { api, ensureEnvReady } = makeNativeApi({ appRunning: true, connected: false });
|
|
|
|
await expect(
|
|
nativeDevtoolsStatusTool.execute(
|
|
{ nativeDevtools: api },
|
|
{ udid: "11111111-1111-1111-1111-111111111111", bundleId: "com.example.app" }
|
|
)
|
|
).resolves.toEqual({
|
|
envSetup: true,
|
|
appRunning: true,
|
|
connected: false,
|
|
requiresRestart: true,
|
|
state: "stale_process",
|
|
message: expect.stringContaining("restart-app") as string,
|
|
nextLaunchWillBeInjected: true,
|
|
injectable: true,
|
|
});
|
|
|
|
expect(ensureEnvReady).toHaveBeenCalledOnce();
|
|
});
|
|
|
|
it("re-applies the env when the app is not connected (repairs a stale latch after a sim reboot)", async () => {
|
|
// envSetup:false models the cleared launchd state after an out-of-band
|
|
// reboot; reverifyEnv must run and bring it back to true.
|
|
const { api, reverifyEnv } = makeNativeApi({
|
|
appRunning: true,
|
|
connected: false,
|
|
envSetup: false,
|
|
});
|
|
|
|
await expect(
|
|
nativeDevtoolsStatusTool.execute(
|
|
{ nativeDevtools: api },
|
|
{ udid: "11111111-1111-1111-1111-111111111111", bundleId: "com.example.app" }
|
|
)
|
|
).resolves.toEqual({
|
|
envSetup: true,
|
|
appRunning: true,
|
|
connected: false,
|
|
requiresRestart: true,
|
|
state: "stale_process",
|
|
message: expect.stringContaining("restart-app") as string,
|
|
nextLaunchWillBeInjected: true,
|
|
injectable: true,
|
|
});
|
|
|
|
expect(reverifyEnv).toHaveBeenCalledOnce();
|
|
});
|
|
|
|
it("does not re-apply the env when the app is already connected", async () => {
|
|
const { api, reverifyEnv } = makeNativeApi({ appRunning: true, connected: true });
|
|
|
|
await expect(
|
|
nativeDevtoolsStatusTool.execute(
|
|
{ nativeDevtools: api },
|
|
{ udid: "11111111-1111-1111-1111-111111111111", bundleId: "com.example.app" }
|
|
)
|
|
).resolves.toEqual({
|
|
envSetup: true,
|
|
appRunning: true,
|
|
connected: true,
|
|
requiresRestart: false,
|
|
state: "connected",
|
|
nextLaunchWillBeInjected: true,
|
|
injectable: true,
|
|
});
|
|
|
|
expect(reverifyEnv).not.toHaveBeenCalled();
|
|
});
|
|
|
|
it("reports a stopped app as launch-ready without requiring restart", async () => {
|
|
const { api } = makeNativeApi({ appRunning: false, connected: false });
|
|
|
|
await expect(
|
|
nativeDevtoolsStatusTool.execute(
|
|
{ nativeDevtools: api },
|
|
{ udid: "11111111-1111-1111-1111-111111111111", bundleId: "com.example.app" }
|
|
)
|
|
).resolves.toEqual({
|
|
envSetup: true,
|
|
appRunning: false,
|
|
connected: false,
|
|
requiresRestart: false,
|
|
state: "not_running",
|
|
message: expect.stringContaining("launch-app") as string,
|
|
nextLaunchWillBeInjected: true,
|
|
injectable: true,
|
|
});
|
|
});
|
|
|
|
it("reports a com.apple.* system app as a terminal, non-injectable state", async () => {
|
|
// Apple system apps can never load the dylib. Even with the app running and
|
|
// env set up, status must report injectable:false and neither require a
|
|
// restart nor promise the next launch will be injected — otherwise an agent
|
|
// loops restart-app → retry forever.
|
|
const { api, reverifyEnv } = makeNativeApi({
|
|
appRunning: true,
|
|
connected: false,
|
|
envSetup: true,
|
|
});
|
|
|
|
await expect(
|
|
nativeDevtoolsStatusTool.execute(
|
|
{ nativeDevtools: api },
|
|
{ udid: "11111111-1111-1111-1111-111111111111", bundleId: "com.apple.Preferences" }
|
|
)
|
|
).resolves.toEqual({
|
|
envSetup: true,
|
|
appRunning: true,
|
|
connected: false,
|
|
requiresRestart: false,
|
|
nextLaunchWillBeInjected: false,
|
|
injectable: false,
|
|
});
|
|
|
|
// A non-injectable app is terminal — there is nothing to repair, so the
|
|
// stale-latch reverify path must not run.
|
|
expect(reverifyEnv).not.toHaveBeenCalled();
|
|
});
|
|
|
|
it("reports the terminal non-injectable state even when env init has given up", async () => {
|
|
// The precheck's init_failed block must not mask the statically-knowable
|
|
// terminal signal: its "re-boot the simulator" guidance can never make a
|
|
// system app injectable. Mirrors the same ordering inside
|
|
// precheckNativeDevtools (terminal case before the env plumbing).
|
|
const { api, ensureEnvReady } = makeNativeApi({
|
|
appRunning: true,
|
|
initFailure: {
|
|
attempts: MAX_NATIVE_DEVTOOLS_INIT_ATTEMPTS,
|
|
lastError: "ensureEnv timeout",
|
|
givenUp: true,
|
|
},
|
|
});
|
|
|
|
await expect(
|
|
nativeDevtoolsStatusTool.execute(
|
|
{ nativeDevtools: api },
|
|
{ udid: "11111111-1111-1111-1111-111111111111", bundleId: "com.apple.Preferences" }
|
|
)
|
|
).resolves.toEqual({
|
|
envSetup: false,
|
|
appRunning: true,
|
|
connected: false,
|
|
requiresRestart: false,
|
|
nextLaunchWillBeInjected: false,
|
|
injectable: false,
|
|
});
|
|
|
|
// No env work is spent on an app that can never inject.
|
|
expect(ensureEnvReady).not.toHaveBeenCalled();
|
|
});
|
|
|
|
it("falls back to init_failed when the sim cannot even be probed (dead sim, broken env)", async () => {
|
|
// The terminal branch probes isAppRunning (a simctl spawn); on a shut-down
|
|
// or unreachable sim that rejects — exactly the sims where env init fails
|
|
// too. A raw subprocess throw here would be unstructured; the precheck's
|
|
// init_failed guidance (re-boot the simulator) IS corrective for a dead
|
|
// sim, so it must win when the env is broken.
|
|
const { api } = makeNativeApi({
|
|
initFailure: {
|
|
attempts: MAX_NATIVE_DEVTOOLS_INIT_ATTEMPTS,
|
|
lastError: "ensureEnv timeout",
|
|
givenUp: true,
|
|
},
|
|
});
|
|
api.isAppRunning = async () => {
|
|
throw new Error("simctl spawn failed: current state: Shutdown");
|
|
};
|
|
|
|
await expect(
|
|
nativeDevtoolsStatusTool.execute(
|
|
{ nativeDevtools: api },
|
|
{ udid: "11111111-1111-1111-1111-111111111111", bundleId: "com.apple.Preferences" }
|
|
)
|
|
).resolves.toMatchObject({ status: "init_failed" });
|
|
});
|
|
|
|
// The twin of the injectable path's escalation, on the branch that runs no env
|
|
// work of its own: once `ensureEnvReady` latches, a failure recorded since is
|
|
// the only witness left that the sim is gone — and one sim state must not read
|
|
// as a dead sim on one bundle id and a raw `simctl spawn` throw on another.
|
|
it("surfaces a recorded env failure for a non-injectable app on a latched env", async () => {
|
|
const { api } = makeNativeApi({
|
|
envSetup: true,
|
|
initFailure: { attempts: 1, lastError: "Invalid device", givenUp: false },
|
|
});
|
|
api.isAppRunning = async () => {
|
|
throw new Error("simctl spawn failed: Invalid device");
|
|
};
|
|
|
|
await expect(
|
|
nativeDevtoolsStatusTool.execute(
|
|
{ nativeDevtools: api },
|
|
{ udid: "11111111-1111-1111-1111-111111111111", bundleId: "com.apple.Preferences" }
|
|
)
|
|
).resolves.toMatchObject({ status: "init_failed", attempts: 1 });
|
|
});
|
|
|
|
it("rethrows the probe failure for a non-injectable app when the env is healthy", async () => {
|
|
// With a healthy env there is no init_failed to fall back to — a transient
|
|
// isAppRunning failure must surface, not be swallowed into a made-up state.
|
|
const { api } = makeNativeApi({ envSetup: true });
|
|
api.isAppRunning = async () => {
|
|
throw new Error("transient simctl failure");
|
|
};
|
|
|
|
await expect(
|
|
nativeDevtoolsStatusTool.execute(
|
|
{ nativeDevtools: api },
|
|
{ udid: "11111111-1111-1111-1111-111111111111", bundleId: "com.apple.Preferences" }
|
|
)
|
|
).rejects.toThrow("transient simctl failure");
|
|
});
|
|
});
|
|
|
|
describe("isInjectableBundleId", () => {
|
|
it("treats com.apple.* system apps as non-injectable", () => {
|
|
expect(isInjectableBundleId("com.apple.Preferences")).toBe(false);
|
|
expect(isInjectableBundleId("com.apple.mobilesafari")).toBe(false);
|
|
// Matched case-insensitively: iOS treats bundle ids case-insensitively and
|
|
// Apple owns the com.apple namespace in every casing, so a stray mixed-case
|
|
// id must not slip through as injectable and drop the agent into a restart loop.
|
|
expect(isInjectableBundleId("com.Apple.Preferences")).toBe(false);
|
|
expect(isInjectableBundleId("COM.APPLE.PREFERENCES")).toBe(false);
|
|
});
|
|
|
|
it("treats third-party apps as injectable", () => {
|
|
expect(isInjectableBundleId("com.example.MyApp")).toBe(true);
|
|
expect(isInjectableBundleId("com.latekvo.pokemon")).toBe(true);
|
|
// Prefix match is exact — a lookalike that only contains the substring is
|
|
// still injectable.
|
|
expect(isInjectableBundleId("com.appleseed.App")).toBe(true);
|
|
});
|
|
});
|
|
|
|
describe("precheckNativeDevtools — non-injectable terminal error", () => {
|
|
const UDID = "33333333-3333-3333-3333-333333333333";
|
|
|
|
it("throws NATIVE_DEVTOOLS_NOT_INJECTABLE for a com.apple.* bundle (3-arg)", async () => {
|
|
const { api } = makeNativeApi({ appRunning: true, connected: false });
|
|
|
|
await expect(precheckNativeDevtools(api, UDID, "com.apple.Preferences")).rejects.toBeInstanceOf(
|
|
FailureError
|
|
);
|
|
|
|
try {
|
|
await precheckNativeDevtools(api, UDID, "com.apple.Preferences");
|
|
throw new Error("expected precheckNativeDevtools to throw");
|
|
} catch (err) {
|
|
expect(err).toBeInstanceOf(FailureError);
|
|
expect(getFailureSignal(err)?.error_code).toBe(FAILURE_CODES.NATIVE_DEVTOOLS_NOT_INJECTABLE);
|
|
}
|
|
});
|
|
|
|
it("does not throw for the same api via the 2-arg overload (status / launch-app / restart-app path)", async () => {
|
|
const { api } = makeNativeApi({ appRunning: true, connected: false });
|
|
|
|
await expect(precheckNativeDevtools(api, UDID)).resolves.toBeNull();
|
|
});
|
|
|
|
it("throws the terminal error even when env init has given up", async () => {
|
|
// Injectability is a static property of the bundle id — a broken env must
|
|
// not mask the terminal signal behind init_failed's "re-boot the simulator"
|
|
// guidance, which can never make a system app injectable.
|
|
const { api } = makeNativeApi({
|
|
initFailure: {
|
|
attempts: MAX_NATIVE_DEVTOOLS_INIT_ATTEMPTS,
|
|
lastError: "ensureEnv timeout",
|
|
givenUp: true,
|
|
},
|
|
});
|
|
|
|
try {
|
|
await precheckNativeDevtools(api, UDID, "com.apple.Preferences");
|
|
throw new Error("expected precheckNativeDevtools to throw");
|
|
} catch (err) {
|
|
expect(err).toBeInstanceOf(FailureError);
|
|
expect(getFailureSignal(err)?.error_code).toBe(FAILURE_CODES.NATIVE_DEVTOOLS_NOT_INJECTABLE);
|
|
}
|
|
});
|
|
|
|
it("fires before any env work — ensureEnvReady never runs for a non-injectable bundle", async () => {
|
|
// Same ordering guarantee from the other side: no env-setup work is spent
|
|
// on an app that can never load the dylib, and a transiently failing
|
|
// ensureEnvReady cannot swallow the terminal signal.
|
|
const { api } = makeNativeApi({
|
|
initFailure: { attempts: 1, lastError: "first attempt failed", givenUp: false },
|
|
});
|
|
const ensureEnvReady = vi.fn(async () => {
|
|
throw new Error("transient ensureEnv failure");
|
|
});
|
|
api.ensureEnvReady = ensureEnvReady;
|
|
|
|
try {
|
|
await precheckNativeDevtools(api, UDID, "com.apple.Preferences");
|
|
throw new Error("expected precheckNativeDevtools to throw");
|
|
} catch (err) {
|
|
expect(err).toBeInstanceOf(FailureError);
|
|
expect(getFailureSignal(err)?.error_code).toBe(FAILURE_CODES.NATIVE_DEVTOOLS_NOT_INJECTABLE);
|
|
}
|
|
expect(ensureEnvReady).not.toHaveBeenCalled();
|
|
});
|
|
});
|
|
|
|
describe("non-injectable recovery guidance is consistent and points only at working tools", () => {
|
|
const UDID = "55555555-5555-5555-5555-555555555555";
|
|
|
|
it("recommends describe and screenshot but only warns the agent OFF the native-* tools", () => {
|
|
// The recovery must send the agent to tools that actually work on a system
|
|
// app; the view-at-point tools re-run this same precheck and re-throw, so
|
|
// recommending them dead-ends. describe/screenshot are recommended, and the
|
|
// view-at-point tools appear only inside the "do not fall back" warning.
|
|
expect(NON_INJECTABLE_RECOVERY).toMatch(/`describe`/);
|
|
expect(NON_INJECTABLE_RECOVERY).toMatch(/`screenshot`/);
|
|
expect(NON_INJECTABLE_RECOVERY).toContain(NON_INJECTABLE_NATIVE_WARNING);
|
|
expect(NON_INJECTABLE_NATIVE_WARNING).toMatch(
|
|
/Do not fall back to the native-devtools feature tools/
|
|
);
|
|
expect(NON_INJECTABLE_NATIVE_WARNING).toContain("native-view-at-point");
|
|
// The recommendation clause itself never names a native-* tool outside the
|
|
// warning, so nothing points the agent back at a dead-end.
|
|
const recommendationOnly = NON_INJECTABLE_RECOVERY.replace(NON_INJECTABLE_NATIVE_WARNING, "");
|
|
expect(recommendationOnly).not.toContain("native-");
|
|
});
|
|
|
|
it("the precheck throw and the status description share the recovery guidance verbatim", async () => {
|
|
// The precheck throw, the status description, and the describe fallback hint
|
|
// used to recommend different tool sets. They now share the dead-end warning
|
|
// verbatim, so no surface can drift into recommending a native-* tool. This
|
|
// test covers the two pre-describe surfaces, which additionally share the
|
|
// full describe/screenshot recommendation; the third surface (the describe
|
|
// fallback hint) is asserted in describe-tool.test.ts.
|
|
expect(nativeDevtoolsStatusTool.description).toContain(NON_INJECTABLE_RECOVERY);
|
|
|
|
let message = "";
|
|
try {
|
|
await precheckNativeDevtools(makeNativeApi({}).api, UDID, "com.apple.Preferences");
|
|
} catch (err) {
|
|
message = (err as Error).message;
|
|
}
|
|
expect(message).toContain(NON_INJECTABLE_RECOVERY);
|
|
expect(message).toContain(NON_INJECTABLE_NATIVE_WARNING);
|
|
});
|
|
});
|
|
|
|
describe("native-* feature tools — the non-injectable throw propagates out of execute()", () => {
|
|
// The NATIVE_DEVTOOLS_NOT_INJECTABLE guard lives only in the shared precheck;
|
|
// every 3-arg feature tool relies on that throw propagating straight out of
|
|
// its execute() (none wraps the precheck in a try/catch). The precheck-level
|
|
// unit above proves the precheck throws, but not that each tool surfaces it —
|
|
// a later refactor that swallowed the throw inside a tool would leave that
|
|
// unit green while regressing the terminal behavior. Assert it at every tool
|
|
// boundary so that regression can't slip through.
|
|
const U = "44444444-4444-4444-4444-444444444444";
|
|
const SYSTEM_APP = "com.apple.Preferences";
|
|
// The non-injectable throw fires in the precheck before appRunning/connected/
|
|
// appConnectionState are ever consulted, so the mock's device state is inert
|
|
// here — default it so nothing reads as if the restart logic were exercised.
|
|
const mkApi = () => makeNativeApi({}).api;
|
|
|
|
async function expectNotInjectableThrow(run: () => Promise<unknown>): Promise<void> {
|
|
let caught: unknown;
|
|
try {
|
|
await run();
|
|
} catch (err) {
|
|
caught = err;
|
|
}
|
|
expect(caught).toBeInstanceOf(FailureError);
|
|
expect(getFailureSignal(caught)?.error_code).toBe(FAILURE_CODES.NATIVE_DEVTOOLS_NOT_INJECTABLE);
|
|
}
|
|
|
|
it("native-describe-screen surfaces NATIVE_DEVTOOLS_NOT_INJECTABLE", () =>
|
|
expectNotInjectableThrow(() =>
|
|
nativeDescribeScreenTool.execute(
|
|
{ nativeDevtools: mkApi() },
|
|
{ udid: U, bundleId: SYSTEM_APP }
|
|
)
|
|
));
|
|
|
|
it("native-find-views surfaces NATIVE_DEVTOOLS_NOT_INJECTABLE", () =>
|
|
expectNotInjectableThrow(() =>
|
|
nativeFindViewsTool.execute({ nativeDevtools: mkApi() }, { udid: U, bundleId: SYSTEM_APP })
|
|
));
|
|
|
|
it("native-full-hierarchy surfaces NATIVE_DEVTOOLS_NOT_INJECTABLE", () =>
|
|
expectNotInjectableThrow(() =>
|
|
nativeFullHierarchyTool.execute(
|
|
{ nativeDevtools: mkApi() },
|
|
{ udid: U, bundleId: SYSTEM_APP }
|
|
)
|
|
));
|
|
|
|
it("native-network-logs surfaces NATIVE_DEVTOOLS_NOT_INJECTABLE", () =>
|
|
expectNotInjectableThrow(() =>
|
|
nativeNetworkLogsTool.execute(
|
|
{ nativeDevtools: mkApi() },
|
|
{ udid: U, bundleId: SYSTEM_APP, limit: 50, clear: false }
|
|
)
|
|
));
|
|
|
|
it("native-view-at-point surfaces NATIVE_DEVTOOLS_NOT_INJECTABLE", () =>
|
|
expectNotInjectableThrow(() =>
|
|
nativeViewAtPointTool.execute(
|
|
{ nativeDevtools: mkApi() },
|
|
{ udid: U, bundleId: SYSTEM_APP, x: 0, y: 0 }
|
|
)
|
|
));
|
|
|
|
it("native-user-interactable-view-at-point surfaces NATIVE_DEVTOOLS_NOT_INJECTABLE", () =>
|
|
expectNotInjectableThrow(() =>
|
|
nativeUserInteractableViewAtPointTool.execute(
|
|
{ nativeDevtools: mkApi() },
|
|
{ udid: U, bundleId: SYSTEM_APP, x: 0, y: 0 }
|
|
)
|
|
));
|
|
});
|
|
|
|
describe("native-devtools tools — init_failed precheck", () => {
|
|
const FAILED_UDID = "22222222-2222-2222-2222-222222222222";
|
|
|
|
it("native-describe-screen returns init_failed when the api reports givenUp", async () => {
|
|
const { api } = makeNativeApi({
|
|
initFailure: {
|
|
attempts: MAX_NATIVE_DEVTOOLS_INIT_ATTEMPTS,
|
|
lastError: "ensureEnv timeout",
|
|
givenUp: true,
|
|
},
|
|
});
|
|
|
|
const result = await nativeDescribeScreenTool.execute(
|
|
{ nativeDevtools: api },
|
|
{ udid: FAILED_UDID, bundleId: "com.example.app" }
|
|
);
|
|
expect(result).toMatchObject({
|
|
status: "init_failed",
|
|
attempts: MAX_NATIVE_DEVTOOLS_INIT_ATTEMPTS,
|
|
});
|
|
if (result.status === "init_failed") {
|
|
expect(result.message).toContain(FAILED_UDID);
|
|
expect(result.message).toContain("ensureEnv timeout");
|
|
}
|
|
});
|
|
|
|
it("native-describe-screen proceeds normally below the cap", async () => {
|
|
const { api } = makeNativeApi({
|
|
initFailure: {
|
|
attempts: MAX_NATIVE_DEVTOOLS_INIT_ATTEMPTS - 1,
|
|
lastError: "transient",
|
|
givenUp: false,
|
|
},
|
|
});
|
|
api.appConnectionState = async () => "stale_process";
|
|
|
|
const result = await nativeDescribeScreenTool.execute(
|
|
{ nativeDevtools: api },
|
|
{ udid: FAILED_UDID, bundleId: "com.example.app" }
|
|
);
|
|
expect(result).toMatchObject({ status: "restart_required" });
|
|
});
|
|
|
|
it("native-devtools-status returns init_failed when the api reports givenUp", async () => {
|
|
const { api } = makeNativeApi({
|
|
initFailure: {
|
|
attempts: MAX_NATIVE_DEVTOOLS_INIT_ATTEMPTS,
|
|
lastError: "simctl spawn timed out",
|
|
givenUp: true,
|
|
},
|
|
});
|
|
|
|
const result = await nativeDevtoolsStatusTool.execute(
|
|
{ nativeDevtools: api },
|
|
{ udid: FAILED_UDID, bundleId: "com.example.app" }
|
|
);
|
|
expect(result).toMatchObject({ status: "init_failed" });
|
|
});
|
|
|
|
it("converts a transient ensureEnvReady throw into init_failed (fail-fast)", async () => {
|
|
const { api } = makeNativeApi({
|
|
initFailure: {
|
|
attempts: 1,
|
|
lastError: "first attempt failed",
|
|
givenUp: false,
|
|
},
|
|
});
|
|
api.ensureEnvReady = async () => {
|
|
throw new Error("transient ensureEnv failure");
|
|
};
|
|
|
|
const result = await nativeDescribeScreenTool.execute(
|
|
{ nativeDevtools: api },
|
|
{ udid: FAILED_UDID, bundleId: "com.example.app" }
|
|
);
|
|
expect(result).toMatchObject({ status: "init_failed", attempts: 1 });
|
|
if (result.status === "init_failed") {
|
|
expect(result.message).toContain(FAILED_UDID);
|
|
}
|
|
});
|
|
});
|
|
|
|
// A status asserted for every unconnected app would keep "call restart-app"
|
|
// literally true forever, and an agent obeying it restarts indefinitely. Each
|
|
// measured state has to reach the agent as the action that actually applies.
|
|
describe("precheckNativeDevtools maps a measured state to its remedy", () => {
|
|
function precheckWith(state: NativeDevtoolsAppState) {
|
|
const { api } = makeNativeApi({ envSetup: true, appRunning: true, state });
|
|
return precheckNativeDevtools(api, "UDID", "com.example.app");
|
|
}
|
|
|
|
it("leads with restart-app for a process that predates the injection", async () => {
|
|
const result = await precheckWith("stale_process");
|
|
|
|
expect(result).toMatchObject({ status: "restart_required" });
|
|
const message = (result as { message: string }).message;
|
|
expect(message).toContain("restart-app");
|
|
// The escalation belongs to states we could not measure; naming it here puts
|
|
// "restart the tool-server" in front of an app that just needs a relaunch.
|
|
expect(message).not.toContain("argent server stop");
|
|
});
|
|
|
|
it("does not tell a stale_process it is uninjected — one route into it is injected", async () => {
|
|
// `stale_process` is reached both by a process carrying no argent injection
|
|
// and by an injected one older than this service's listener, so "not
|
|
// injected" is false on the second — which is where the `unregistered`
|
|
// remedy lands an agent: a tool-server restart rebinds the same socket path
|
|
// under a fresh `listeningSince`.
|
|
const result = await precheckWith("stale_process");
|
|
|
|
const message = (result as { message: string }).message;
|
|
expect(message).not.toContain("not injected");
|
|
expect(message).toContain("cannot reach");
|
|
expect(message).toContain("restart-app");
|
|
});
|
|
|
|
it("points a stopped app at launch-app rather than a restart", async () => {
|
|
const result = await precheckWith("not_running");
|
|
|
|
expect(result).toMatchObject({ status: "restart_required" });
|
|
expect((result as { message: string }).message).toContain("launch-app");
|
|
});
|
|
|
|
it("refuses to call an injected-but-unregistered process a restart_required", async () => {
|
|
// The loop this whole derivation exists to break: the process already
|
|
// launched under exactly the terms restart-app would recreate.
|
|
const result = await precheckWith("unregistered");
|
|
|
|
expect(result).toMatchObject({ status: "service_stale" });
|
|
const message = (result as { message: string }).message;
|
|
expect(message).toContain("argent server stop && argent server start --detach");
|
|
expect(message).toContain("Restarting the app cannot change that");
|
|
expect(message).not.toContain("restart-app");
|
|
});
|
|
|
|
it("refuses to call a still-connecting process a restart_required", async () => {
|
|
// The other self-perpetuating advice: exec starts the dial, so a relaunch
|
|
// discards the handshake AND resets the age this verdict reads. Not
|
|
// `indeterminate` either — the process WAS inspected.
|
|
const result = await precheckWith("connecting");
|
|
|
|
expect(result).toMatchObject({ status: "connect_pending" });
|
|
const message = (result as { message: string }).message;
|
|
expect(message).toContain("Wait a few seconds");
|
|
// The prohibition itself, not just the absence of the hyphenated tool name:
|
|
// "wait a few seconds, then relaunch the app and retry" satisfies every
|
|
// other assertion here while prescribing the one action that resets the age
|
|
// this verdict reads. The sibling `unregistered` case above pins its
|
|
// prohibition the same way.
|
|
//
|
|
// Matched with its colon, so it is the UNQUALIFIED form: this state is the
|
|
// one where not even a single relaunch is right — it discards a handshake
|
|
// that is already in flight — and "Do NOT restart the app more than once"
|
|
// permits exactly that while still containing the bare phrase.
|
|
expect(message).toContain("Do NOT restart the app:");
|
|
// And the reason, so the prohibition cannot be left standing without the
|
|
// fact that makes it true.
|
|
expect(message).toContain("a relaunch discards the one in progress");
|
|
expect(message).not.toContain("restart-app");
|
|
expect(message).not.toContain("could not be inspected");
|
|
});
|
|
|
|
it("keeps the loop warning for a process it could not inspect", async () => {
|
|
// ios-remote and an unreadable process table land here: restart-app is
|
|
// still the right first move, but nothing measured says it will work, so
|
|
// the way out of the loop has to travel with it.
|
|
const result = await precheckWith("indeterminate");
|
|
|
|
expect(result).toMatchObject({ status: "restart_required" });
|
|
const message = (result as { message: string }).message;
|
|
expect(message).toContain("restart-app");
|
|
expect(message).toContain("argent server stop && argent server start --detach");
|
|
expect(message).toContain("do not keep restarting the app");
|
|
});
|
|
|
|
// `argent server start` defaults to the foreground and never returns, so the
|
|
// bare pair hangs whoever runs it. Every surface prescribing the tool-server
|
|
// restart must pass --detach.
|
|
it("prescribes a tool-server restart that actually returns", async () => {
|
|
for (const state of ["unregistered", "indeterminate"] as const) {
|
|
const message = (await precheckWith(state)) as { message: string };
|
|
expect(message.message, state).toContain("argent server start --detach");
|
|
}
|
|
});
|
|
|
|
// Any expression that admits `unregistered` or `connecting` — including the
|
|
// obvious `appRunning && !connected` — leaves the boolean saying "restart the
|
|
// app" beside a message saying a restart cannot help, and the description
|
|
// tells agents to branch on the boolean.
|
|
it("never asks for a restart of an app a restart provably cannot fix", async () => {
|
|
for (const state of ["unregistered", "connecting"] as const) {
|
|
const { api } = makeNativeApi({ envSetup: true, appRunning: true, state });
|
|
|
|
const result = (await nativeDevtoolsStatusTool.execute(
|
|
{ nativeDevtools: api },
|
|
{ udid: "UDID", bundleId: "com.example.app" }
|
|
)) as { requiresRestart: boolean; state: string };
|
|
|
|
expect(result.requiresRestart, `${state} must not require a restart`).toBe(false);
|
|
expect(result.state).toBe(state);
|
|
}
|
|
});
|
|
|
|
// A sim that dies mid-call makes the measurement reject. Without the guard the
|
|
// raw subprocess error escapes execute() instead of the structured answer.
|
|
it("degrades a rejected measurement instead of letting it escape the tool", async () => {
|
|
const { api } = makeNativeApi({ envSetup: true, appRunning: true });
|
|
api.appConnectionState = async () => {
|
|
throw new Error("Invalid device: UDID");
|
|
};
|
|
|
|
const result = (await nativeDevtoolsStatusTool.execute(
|
|
{ nativeDevtools: api },
|
|
{ udid: "UDID", bundleId: "com.example.app" }
|
|
)) as { state?: string; message?: string };
|
|
|
|
expect(result.state).toBe("indeterminate");
|
|
expect(result.message).toContain("do not keep restarting the app");
|
|
});
|
|
|
|
// `connected` has to come off the same measurement as `state`, not a second
|
|
// `isConnected()` read: a connection landing between the two would pair
|
|
// `connected: true` with a state saying the service never registered it.
|
|
it("reads connected from the one measurement, not a second probe", async () => {
|
|
const { api } = makeNativeApi({ envSetup: true, appRunning: true, state: "unregistered" });
|
|
api.isConnected = () => true; // the map moved on after the measurement began
|
|
|
|
const result = (await nativeDevtoolsStatusTool.execute(
|
|
{ nativeDevtools: api },
|
|
{ udid: "UDID", bundleId: "com.example.app" }
|
|
)) as { connected: boolean; state: string };
|
|
|
|
expect(result.connected).toBe(false);
|
|
expect(result.state).toBe("unregistered");
|
|
});
|
|
|
|
// The precheck is the path all six native-* feature tools take. A rejection
|
|
// here (the env re-apply, on a sim that went away) must not reach the agent as
|
|
// a raw subprocess error — and must never be swallowed as "connected", which
|
|
// would let all six proceed to query a connection that does not exist.
|
|
it("degrades a rejected measurement rather than letting it escape or pass", async () => {
|
|
const { api } = makeNativeApi({ envSetup: true, appRunning: true });
|
|
api.appConnectionState = async () => {
|
|
throw new Error("Invalid device: UDID");
|
|
};
|
|
|
|
const result = await precheckNativeDevtools(api, "UDID", "com.example.app");
|
|
|
|
expect(result).not.toBeNull();
|
|
expect(result).toMatchObject({ status: "restart_required" });
|
|
expect((result as { message: string }).message).toContain("do not keep restarting the app");
|
|
});
|
|
|
|
// When that same rejection came with a recorded env failure, the sim itself is
|
|
// gone — which is init_failed's case, not a connection diagnosis.
|
|
it("reports a dead sim as init_failed rather than an unmeasured connection", async () => {
|
|
const { api } = makeNativeApi({ envSetup: true, appRunning: true });
|
|
api.getInitFailure = () => ({ attempts: 2, lastError: "Invalid device", givenUp: false });
|
|
api.appConnectionState = async () => {
|
|
throw new Error("Invalid device: UDID");
|
|
};
|
|
|
|
const result = await precheckNativeDevtools(api, "UDID", "com.example.app");
|
|
|
|
expect(result).toMatchObject({ status: "init_failed", attempts: 2 });
|
|
});
|
|
|
|
it("passes a connected app straight through", async () => {
|
|
await expect(precheckWith("connected")).resolves.toBeNull();
|
|
});
|
|
|
|
it("reads running-ness out of the state instead of a second device probe", async () => {
|
|
// `appConnectionState` already runs `launchctl list`, and re-verifies the
|
|
// env first — so a separate `isAppRunning` is both an extra round-trip and a
|
|
// snapshot taken seconds apart, which is how `appRunning: true` could land
|
|
// beside `state: "not_running"`. Only `indeterminate` may pay for a probe.
|
|
for (const state of [
|
|
"connected",
|
|
"not_running",
|
|
"stale_process",
|
|
"unregistered",
|
|
"connecting",
|
|
] as const) {
|
|
const { api, isAppRunning } = makeNativeApi({ envSetup: true, connected: false, state });
|
|
|
|
const result = (await nativeDevtoolsStatusTool.execute(
|
|
{ nativeDevtools: api },
|
|
{ udid: "UDID", bundleId: "com.example.app" }
|
|
)) as { appRunning: boolean; state: string };
|
|
|
|
expect(isAppRunning, `${state} must not re-probe the device`).not.toHaveBeenCalled();
|
|
expect(result.state).toBe(state);
|
|
expect(result.appRunning).toBe(state !== "not_running");
|
|
}
|
|
|
|
const { api, isAppRunning } = makeNativeApi({
|
|
envSetup: true,
|
|
connected: false,
|
|
appRunning: true,
|
|
state: "indeterminate",
|
|
});
|
|
const result = (await nativeDevtoolsStatusTool.execute(
|
|
{ nativeDevtools: api },
|
|
{ udid: "UDID", bundleId: "com.example.app" }
|
|
)) as { appRunning: boolean; requiresRestart: boolean };
|
|
|
|
expect(isAppRunning).toHaveBeenCalledTimes(1);
|
|
expect(result.appRunning).toBe(true);
|
|
expect(result.requiresRestart).toBe(true);
|
|
});
|
|
|
|
// Without this pin, collapsing the mapping back to a single status leaves
|
|
// every precheck-level test green while the agent silently gets restart-app
|
|
// guidance again.
|
|
it("carries service_stale out through every native-* tool", async () => {
|
|
const tools = [
|
|
nativeDescribeScreenTool,
|
|
nativeFindViewsTool,
|
|
nativeFullHierarchyTool,
|
|
nativeNetworkLogsTool,
|
|
nativeViewAtPointTool,
|
|
nativeUserInteractableViewAtPointTool,
|
|
];
|
|
|
|
for (const tool of tools) {
|
|
const { api } = makeNativeApi({
|
|
envSetup: true,
|
|
appRunning: true,
|
|
state: "unregistered",
|
|
});
|
|
|
|
const result = (await tool.execute(
|
|
{ nativeDevtools: api },
|
|
{
|
|
udid: "UDID",
|
|
bundleId: "com.example.app",
|
|
x: 1,
|
|
y: 1,
|
|
className: "UIView",
|
|
limit: 1,
|
|
clear: false,
|
|
}
|
|
)) as { status: string; message: string };
|
|
|
|
expect(result.status, `${tool.id} must report service_stale`).toBe("service_stale");
|
|
expect(result.message).toContain("argent server stop && argent server start --detach");
|
|
expect(result.message).not.toContain("restart-app");
|
|
}
|
|
});
|
|
|
|
// The booleans cannot say "one restart, then stop". `requiresRestart: true` is
|
|
// the whole of what an `indeterminate` app reports, and the only state a
|
|
// running app reaches on ios-remote — so without the prose the tool hands back
|
|
// the restart loop and nothing else.
|
|
it("hands back the loop escape for the states the booleans cannot express", async () => {
|
|
for (const state of ["indeterminate", "stale_process", "not_running"] as const) {
|
|
const { api } = makeNativeApi({ envSetup: true, appRunning: true, state });
|
|
|
|
const result = (await nativeDevtoolsStatusTool.execute(
|
|
{ nativeDevtools: api },
|
|
{ udid: "UDID", bundleId: "com.example.app" }
|
|
)) as { message?: string };
|
|
|
|
expect(result.message, `${state} must carry its remedy`).toBeDefined();
|
|
expect(result.message).toContain("com.example.app");
|
|
}
|
|
|
|
const { api: indeterminate } = makeNativeApi({
|
|
envSetup: true,
|
|
appRunning: true,
|
|
state: "indeterminate",
|
|
});
|
|
const escape = (await nativeDevtoolsStatusTool.execute(
|
|
{ nativeDevtools: indeterminate },
|
|
{ udid: "UDID", bundleId: "com.example.app" }
|
|
)) as { message?: string; requiresRestart: boolean };
|
|
expect(escape.requiresRestart).toBe(true);
|
|
expect(escape.message).toContain("do not keep restarting the app");
|
|
expect(escape.message).toContain("argent server stop && argent server start --detach");
|
|
|
|
// A connected app has no remedy to carry, so the field must not appear at
|
|
// all — an empty-string or stale message would read as a live problem.
|
|
const { api: healthy } = makeNativeApi({ envSetup: true, connected: true });
|
|
const ok = (await nativeDevtoolsStatusTool.execute(
|
|
{ nativeDevtools: healthy },
|
|
{ udid: "UDID", bundleId: "com.example.app" }
|
|
)) as Record<string, unknown>;
|
|
expect(ok).not.toHaveProperty("message");
|
|
});
|
|
|
|
// Telling an agent to restart something the same payload says is not running
|
|
// is the contradiction deriving both from one measurement prevents. Only
|
|
// `indeterminate` re-probes, so only it can produce the pair.
|
|
it("does not ask for a restart of an app the probe says is not running", async () => {
|
|
const { api } = makeNativeApi({
|
|
envSetup: true,
|
|
appRunning: false,
|
|
state: "indeterminate",
|
|
});
|
|
|
|
const result = (await nativeDevtoolsStatusTool.execute(
|
|
{ nativeDevtools: api },
|
|
{ udid: "UDID", bundleId: "com.example.app" }
|
|
)) as { appRunning: boolean; requiresRestart: boolean };
|
|
|
|
expect(result.appRunning).toBe(false);
|
|
expect(result.requiresRestart).toBe(false);
|
|
// `message` is the field the description tells agents to prefer, so it has
|
|
// to agree too: the `indeterminate` text opens "Call restart-app then
|
|
// retry", which contradicts `appRunning: false` outright.
|
|
const withMessage = result as unknown as { state: string; message: string };
|
|
expect(withMessage.state).toBe("not_running");
|
|
expect(withMessage.message).toContain("launch-app");
|
|
expect(withMessage.message).not.toContain("could not be inspected");
|
|
});
|
|
|
|
// The dead-sim escalation has to fire on the FIRST failure, not the third.
|
|
// Re-running the precheck would report nothing — `ensureEnvReady` succeeded at
|
|
// the top of the call and latches — so reading the failure `reverifyEnv` just
|
|
// recorded is what turns a shut-down simulator into structured guidance
|
|
// instead of a raw `Command failed: xcrun simctl spawn …`.
|
|
it("surfaces init_failed on a dead sim before the service has given up", async () => {
|
|
const { api } = makeNativeApi({ envSetup: true, state: "indeterminate" });
|
|
api.getInitFailure = () => ({ attempts: 1, lastError: "Invalid device", givenUp: false });
|
|
api.isAppRunning = vi.fn(async () => {
|
|
throw new Error("Invalid device: UDID");
|
|
});
|
|
|
|
const result = (await nativeDevtoolsStatusTool.execute(
|
|
{ nativeDevtools: api },
|
|
{ udid: "UDID", bundleId: "com.example.app" }
|
|
)) as { status?: string; message?: string; attempts?: number };
|
|
|
|
expect(result.status).toBe("init_failed");
|
|
expect(result.attempts).toBe(1);
|
|
expect(result.message).toContain("Invalid device");
|
|
});
|
|
|
|
// These are the two fields an agent reads to decide a launch is worth it, and
|
|
// `isEnvSetup()` is a latch that never clears — so on a simulator whose
|
|
// launchd env was wiped it keeps answering yes, and the agent relaunches into
|
|
// an uninjected process forever. A failure the measurement's re-apply recorded
|
|
// is the one thing that contradicts the latch.
|
|
it("stops promising an injected launch once the env re-apply has failed", async () => {
|
|
const { api } = makeNativeApi({ envSetup: true, appRunning: true, state: "stale_process" });
|
|
api.getInitFailure = () => ({ attempts: 1, lastError: "Invalid device", givenUp: false });
|
|
|
|
const result = (await nativeDevtoolsStatusTool.execute(
|
|
{ nativeDevtools: api },
|
|
{ udid: "UDID", bundleId: "com.example.app" }
|
|
)) as { envSetup: boolean; nextLaunchWillBeInjected: boolean };
|
|
|
|
expect(result.envSetup).toBe(false);
|
|
expect(result.nextLaunchWillBeInjected).toBe(false);
|
|
});
|
|
|
|
// The other half of that reading, and the reason it is conditional. The
|
|
// recorded failure only contradicts the latch where something re-applied the
|
|
// env to record it — `appConnectionState` does that for an app it finds
|
|
// UNCONNECTED. A connected one answers off the connections map first, so the
|
|
// record beside it is whatever an earlier call for another bundle left, and
|
|
// nothing re-tests it while this app stays connected: the reading would be
|
|
// wrong for as long as the app is up, against a process that is demonstrably
|
|
// injected.
|
|
it("keeps reporting the env a live connection proves was in place", async () => {
|
|
const { api, reverifyEnv } = makeNativeApi({ envSetup: true, connected: true });
|
|
api.getInitFailure = () => ({ attempts: 1, lastError: "Invalid device", givenUp: false });
|
|
|
|
const result = (await nativeDevtoolsStatusTool.execute(
|
|
{ nativeDevtools: api },
|
|
{ udid: "UDID", bundleId: "com.example.app" }
|
|
)) as { envSetup: boolean; connected: boolean; nextLaunchWillBeInjected: boolean };
|
|
|
|
// The premise: nothing on this path re-applied the env, so the failure it
|
|
// would have consulted is stale by construction.
|
|
expect(reverifyEnv).not.toHaveBeenCalled();
|
|
expect(result.connected).toBe(true);
|
|
expect(result.envSetup).toBe(true);
|
|
expect(result.nextLaunchWillBeInjected).toBe(true);
|
|
});
|
|
|
|
// Same rule on the terminal branch, which runs no env work at all — and a
|
|
// system app CAN be connected there (#453 saw one runtime refuse the dylib,
|
|
// an E2E run another accept it), so the same contradiction is reachable.
|
|
it("keeps reporting a working env for a connected non-injectable app", async () => {
|
|
const { api } = makeNativeApi({ envSetup: true, connected: true, appRunning: true });
|
|
api.getInitFailure = () => ({ attempts: 1, lastError: "Invalid device", givenUp: false });
|
|
|
|
const result = (await nativeDevtoolsStatusTool.execute(
|
|
{ nativeDevtools: api },
|
|
{ udid: "UDID", bundleId: "com.apple.Preferences" }
|
|
)) as { envSetup: boolean; connected: boolean; injectable: boolean };
|
|
|
|
expect(result.injectable).toBe(false);
|
|
expect(result.connected).toBe(true);
|
|
expect(result.envSetup).toBe(true);
|
|
});
|
|
|
|
// Same reading on the branch that runs no env work of its own: a system app
|
|
// on a sim whose env is broken must not report the env as in place either.
|
|
it("stops reporting a working env for a non-injectable app once it has failed", async () => {
|
|
const { api } = makeNativeApi({ envSetup: true, appRunning: true });
|
|
api.getInitFailure = () => ({ attempts: 1, lastError: "Invalid device", givenUp: false });
|
|
|
|
const result = (await nativeDevtoolsStatusTool.execute(
|
|
{ nativeDevtools: api },
|
|
{ udid: "UDID", bundleId: "com.apple.Preferences" }
|
|
)) as { envSetup: boolean };
|
|
|
|
expect(result.envSetup).toBe(false);
|
|
});
|
|
|
|
// A sim that dies mid-call is healthy at every earlier read: the entry
|
|
// precheck saw a working service and `getInitFailure` was still null on the
|
|
// way in. Hoisting the read out of the catch reports that stale null and lets
|
|
// the raw subprocess error out.
|
|
it("reads the recorded failure after the probe, not before it", async () => {
|
|
const { api } = makeNativeApi({ envSetup: true, state: "indeterminate" });
|
|
let died = false;
|
|
api.getInitFailure = () =>
|
|
died
|
|
? { attempts: MAX_NATIVE_DEVTOOLS_INIT_ATTEMPTS, lastError: "sim gone", givenUp: true }
|
|
: null;
|
|
api.isAppRunning = vi.fn(async () => {
|
|
died = true;
|
|
throw new Error("Invalid device: UDID");
|
|
});
|
|
|
|
const result = (await nativeDevtoolsStatusTool.execute(
|
|
{ nativeDevtools: api },
|
|
{ udid: "UDID", bundleId: "com.example.app" }
|
|
)) as { status?: string; message?: string };
|
|
|
|
expect(result.status).toBe("init_failed");
|
|
expect(result.message).toContain("sim gone");
|
|
});
|
|
|
|
// The other half of that branch: with no recorded failure the env is healthy,
|
|
// so the probe's own error is the honest answer. Degrading it would report a
|
|
// reading the tool never took.
|
|
it("lets the probe's error out when the env recorded no failure", async () => {
|
|
const { api } = makeNativeApi({ envSetup: true, state: "indeterminate" });
|
|
api.getInitFailure = () => null;
|
|
api.isAppRunning = vi.fn(async () => {
|
|
throw new Error("Invalid device: UDID");
|
|
});
|
|
|
|
await expect(
|
|
nativeDevtoolsStatusTool.execute(
|
|
{ nativeDevtools: api },
|
|
{ udid: "UDID", bundleId: "com.example.app" }
|
|
)
|
|
).rejects.toThrow("Invalid device: UDID");
|
|
});
|
|
});
|
|
|
|
// Descriptions are what an agent reads before ever calling, so one that names
|
|
// restart_required while staying silent about service_stale rebuilds the loop in
|
|
// the more prominent place, whatever the runtime message says.
|
|
describe("native-* tool descriptions document every precheck outcome", () => {
|
|
const tools = [
|
|
nativeDescribeScreenTool,
|
|
nativeFindViewsTool,
|
|
nativeFullHierarchyTool,
|
|
nativeNetworkLogsTool,
|
|
nativeViewAtPointTool,
|
|
nativeUserInteractableViewAtPointTool,
|
|
];
|
|
|
|
it("names service_stale and its tool-server remedy alongside restart_required", () => {
|
|
for (const tool of tools) {
|
|
expect(tool.description, `${tool.id} must mention restart_required`).toContain(
|
|
"restart_required"
|
|
);
|
|
expect(tool.description, `${tool.id} must mention service_stale`).toContain("service_stale");
|
|
expect(tool.description, `${tool.id} must mention connect_pending`).toContain(
|
|
"connect_pending"
|
|
);
|
|
// The fourth member of NativeDevtoolsPrecheckBlock, and the only status
|
|
// whose remedy is neither a restart nor a wait — an agent with no arm for
|
|
// it retries a simulator that needs re-booting.
|
|
expect(tool.description, `${tool.id} must mention init_failed`).toContain("init_failed");
|
|
expect(tool.description, `${tool.id} must name the tool-server remedy`).toContain(
|
|
"argent server stop && argent server start --detach"
|
|
);
|
|
}
|
|
});
|
|
|
|
// All six throw the identical terminal NATIVE_DEVTOOLS_NOT_INJECTABLE error
|
|
// for a com.apple.* bundle id, and it is the one outcome an agent can act on
|
|
// BEFORE spending a call, off `tools/list`. Half of them describing a terminal
|
|
// arm and half not is what makes an agent believe the silent ones might work.
|
|
it("names the terminal system-app rejection on every tool that performs it", () => {
|
|
for (const tool of tools) {
|
|
expect(tool.description, `${tool.id} must name the system-app rejection`).toMatch(
|
|
/Apple system app is rejected outright/
|
|
);
|
|
expect(tool.description, `${tool.id} must mark it terminal`).toMatch(/never retry it/);
|
|
}
|
|
});
|
|
|
|
/**
|
|
* The one `If state is <state>:` line out of the description.
|
|
*
|
|
* A whole-description `toContain` is not a pin on any of them: the guidance
|
|
* lines echo each other closely enough that deleting one outright leaves its
|
|
* assertions satisfied by a sibling — "do NOT restart the app again" appears
|
|
* verbatim in the `indeterminate` line too — and the field list and the
|
|
* `injectable: false` paragraph between them supply most of the remaining
|
|
* words, down to "describe" inside `native-describe-screen`.
|
|
*/
|
|
function guidanceLine(state: NativeDevtoolsAppState): string {
|
|
const prefix = `If state is ${state}:`;
|
|
const line = nativeDevtoolsStatusTool
|
|
.description!.split("\n")
|
|
.find((l) => l.startsWith(prefix));
|
|
expect(line, `description has no "${prefix}" line`).toBeTypeOf("string");
|
|
return line!;
|
|
}
|
|
|
|
it("tells native-devtools-status readers to wait out a connecting app", () => {
|
|
expect(guidanceLine("connecting")).toContain("do NOT restart the app");
|
|
});
|
|
|
|
it("tells native-devtools-status readers not to restart an unregistered app", () => {
|
|
expect(guidanceLine("unregistered")).toContain("do NOT restart the app again");
|
|
});
|
|
|
|
// Every remedy in the set is individually correct and together they close into
|
|
// a ring: the tool-server restart this state prescribes rebinds the listener,
|
|
// so the same never-dialing process reads `stale_process` (it now predates the
|
|
// listener) -> restart-app -> `connecting` -> `unregistered` again, unbounded.
|
|
// Nothing measured separates the first landing from the second, so both
|
|
// surfaces have to hand the reader the test — as `not_running` and
|
|
// `indeterminate` already do. Without it this PR lengthens the loop it fixes.
|
|
it("gives an unregistered app a way out of the remedy cycle on its second landing", () => {
|
|
const message = buildAppStateMessage("com.example.app", "unregistered");
|
|
const line = guidanceLine("unregistered");
|
|
|
|
// Pinned as each surface's own conditional clause rather than a shared
|
|
// /already|again/: the description line opens "If state is unregistered: do
|
|
// NOT restart the app again", so a loose pattern matches the wrapper the
|
|
// escape sits in and survives the escape being deleted from it.
|
|
expect(message).toContain("If you have already restarted the tool-server for this app");
|
|
expect(line).toContain("If it reads unregistered again after that restart");
|
|
|
|
for (const [name, text] of [
|
|
["message", message],
|
|
["description", line],
|
|
] as const) {
|
|
// Conditional, so "stop" is not the last word: a reader who has spent both
|
|
// remedies needs somewhere to go.
|
|
expect(text, `${name} names no terminal fallback`).toMatch(
|
|
/treat native devtools as unavailable/i
|
|
);
|
|
// A tool that reads the screen WITHOUT injection. Pointing back at a
|
|
// native-* tool re-enters the same precheck, which is the cycle again with
|
|
// an extra hop — and `native-describe-screen` contains "describe", so the
|
|
// tool name has to be matched on its own.
|
|
expect(text, `${name} names no injection-free reader`).toMatch(/\bscreenshot\b/);
|
|
expect(text, `${name} falls back to a tool behind the same precheck`).not.toMatch(
|
|
/native-(describe-screen|find-views|full-hierarchy|network-logs|view-at-point)/
|
|
);
|
|
}
|
|
});
|
|
|
|
// `indeterminate` gives the agent `requiresRestart: true` and nothing else, so
|
|
// the description is where "restart once, then stop" has to be stated. It is
|
|
// also the only unconnected state a running app reaches on a remote simulator.
|
|
it("bounds the restart advice for a process it could not inspect", () => {
|
|
// Over the whole description neither assertion says anything about this
|
|
// line: the prefix is all the first one needs, and the `unregistered` line
|
|
// supplies the command verbatim. So the bound this test is named for — the
|
|
// one stop-condition on the description side for the only unconnected state
|
|
// a running app reaches on ios-remote — could be deleted outright.
|
|
const line = guidanceLine("indeterminate");
|
|
expect(line).toContain("do NOT restart the app again");
|
|
expect(line).toContain("argent server stop && argent server start --detach");
|
|
});
|
|
|
|
// `indeterminate` sets `requiresRestart: true`, so it matches the boolean
|
|
// rule as well as its own `If state is` line — and an agent that acts on the
|
|
// earlier, more general rule never reaches the later one. That earlier rule
|
|
// is where the bound has to be stated, and on a remote simulator it is the
|
|
// ordinary path rather than a rare one: no app process can be inspected
|
|
// there, so `indeterminate` is the only unconnected state a running app can
|
|
// reach. Verbatim for the reason the stop-conditions below are: the failure
|
|
// is a softened or deleted bound, which any pattern over the opening clause
|
|
// reads straight past.
|
|
it("bounds the requiresRestart rule rather than leaving it open", () => {
|
|
const line = nativeDevtoolsStatusTool
|
|
.description!.split("\n")
|
|
.find((l) => l.startsWith("If requiresRestart is true:"));
|
|
expect(line, "description has no requiresRestart rule").toBeTypeOf("string");
|
|
expect(line!).toBe(
|
|
"If requiresRestart is true: call restart-app once, then proceed with the native feature. " +
|
|
"Read state before acting on a second such reading — indeterminate reaches this rule too, " +
|
|
"and its line below bounds it at that one restart."
|
|
);
|
|
});
|
|
|
|
// Rename a state in one place and the prose describes names the tool never
|
|
// emits — which no runtime assertion catches, because both sides still agree
|
|
// with themselves. Exhaustive by construction: a record fails to compile when
|
|
// a state joins the union unlisted, where a hand-written array silently omits.
|
|
const ALL_STATES: Record<NativeDevtoolsAppState, true> = {
|
|
connected: true,
|
|
not_running: true,
|
|
stale_process: true,
|
|
unregistered: true,
|
|
connecting: true,
|
|
indeterminate: true,
|
|
};
|
|
|
|
it("spells every emitted state exactly as the description names it", () => {
|
|
const states = Object.keys(ALL_STATES) as NativeDevtoolsAppState[];
|
|
for (const state of states) {
|
|
expect(nativeDevtoolsStatusTool.description, `description must name ${state}`).toContain(
|
|
`"${state}"`
|
|
);
|
|
}
|
|
});
|
|
|
|
// `argent server start` defaults to the foreground and never returns, so an
|
|
// agent that runs it as written is gone. Checked per occurrence, not per blob:
|
|
// a surface holding two satisfies a whole-blob `toContain` with one still
|
|
// bare.
|
|
it("never prescribes a tool-server restart the agent cannot return from", () => {
|
|
const BARE_START = /argent server start(?! --detach)/;
|
|
// A surface with no text at all would vacuously satisfy the check below.
|
|
const described = (name: string, text: string | undefined): [string, string] => {
|
|
expect(text, `${name} has no text to check`).toBeTypeOf("string");
|
|
return [name, text!];
|
|
};
|
|
const surfaces: [string, string][] = [
|
|
described("native-devtools-status description", nativeDevtoolsStatusTool.description),
|
|
...tools.map((t) => described(`${t.id} description`, t.description)),
|
|
...(Object.keys(ALL_STATES) as NativeDevtoolsAppState[])
|
|
.filter((s): s is Exclude<NativeDevtoolsAppState, "connected"> => s !== "connected")
|
|
.map((s): [string, string] => [`${s} message`, buildAppStateMessage("com.example.app", s)]),
|
|
];
|
|
|
|
for (const [name, text] of surfaces) {
|
|
expect(text, `${name} prescribes a foreground tool-server start`).not.toMatch(BARE_START);
|
|
}
|
|
// The regex only fires on text that mentions the command at all, so prove
|
|
// the surfaces that must mention it do — otherwise deleting the guidance
|
|
// outright passes as "no bare start".
|
|
expect(buildAppStateMessage("com.example.app", "unregistered")).toContain(
|
|
"argent server start"
|
|
);
|
|
expect(nativeDevtoolsStatusTool.description).toContain("argent server start");
|
|
});
|
|
|
|
// One agent reads the tool description AND the message the same call returns.
|
|
// The descriptions must be plain literals (extract-tools.mjs reads them
|
|
// statically), so the wording is copied eight ways and nothing but this pins
|
|
// the copies together — which is how six of them kept telling the reader to
|
|
// wait "a second or two" after the connect budget outgrew it and the
|
|
// other two were updated.
|
|
it("agrees with the message on how long a connecting app is worth waiting for", () => {
|
|
const surfaces: [string, string][] = [
|
|
["connecting message", buildAppStateMessage("com.example.app", "connecting")],
|
|
["native-devtools-status description", guidanceLine("connecting")],
|
|
...tools.map((t): [string, string] => [`${t.id} description`, t.description!]),
|
|
];
|
|
for (const [name, text] of surfaces) {
|
|
expect(text, `${name} has no wait guidance to check`).toMatch(/wait a few seconds/i);
|
|
// The wording the budget outgrew. Pinned as its own assertion so the
|
|
// failure names the drift rather than a missing phrase.
|
|
expect(text, `${name} still quotes the pre-budget wait`).not.toMatch(/second or two/i);
|
|
}
|
|
// The six carry their own phrasing of the prohibition, so they need their
|
|
// own pin — and it has to reject a qualified one for the same reason the
|
|
// message's does: the first relaunch is already the one that discards the
|
|
// handshake, so "do not restart it more than once" is wrong advice that
|
|
// still contains "do not restart it".
|
|
for (const tool of tools) {
|
|
expect(tool.description!, `${tool.id} qualifies the connecting prohibition`).toContain(
|
|
"do not restart it, wait"
|
|
);
|
|
}
|
|
});
|
|
|
|
// Same split surface, same hazard: round 2 gave `unregistered` a landing that
|
|
// terminates, and the six tool descriptions route `service_stale` — the status
|
|
// that state maps to — with their own copy of the remedy.
|
|
it("stops every service_stale surface from prescribing an unbounded tool-server restart", () => {
|
|
for (const tool of tools) {
|
|
const routing = tool
|
|
.description!.split("\n")
|
|
.find((l) => l.includes("If status is service_stale:"));
|
|
expect(routing, `${tool.id} does not route service_stale`).toBeTypeOf("string");
|
|
expect(
|
|
routing!,
|
|
`${tool.id} prescribes a tool-server restart with no second landing`
|
|
).toMatch(/if the same status comes back|stop restarting/i);
|
|
}
|
|
});
|
|
|
|
/**
|
|
* The remedy prose an agent obeys to decide whether to STOP, pinned verbatim.
|
|
*
|
|
* Every other assertion in this file matches an opening clause, and that has
|
|
* repeatedly proved to constrain nothing: the claim can be negated by a
|
|
* sentence appended after the part that matched ("go ahead and relaunch … keep
|
|
* relaunching until it connects"), qualified inside it ("do NOT restart the app
|
|
* MORE THAN ONCE"), or scoped away ("restart the tool-server AGAIN and keep
|
|
* retrying until it clears") — each of which reopens the unbounded loop this
|
|
* whole change exists to close, with every pattern still green.
|
|
*
|
|
* A pattern cannot fix that, because the defect is always in the words it did
|
|
* not look at. So these strings are duplicated here on purpose, exactly as
|
|
* NON_INJECTABLE_RECOVERY is: rewording one has to fail, be re-read, and be
|
|
* re-approved rather than absorbed. Edit both together.
|
|
*/
|
|
const CONNECTING_PROHIBITION =
|
|
"Do NOT restart the app: launching it is what starts the connection, so a relaunch " +
|
|
"discards the one in progress and returns you to this same state.";
|
|
const UNREGISTERED_ESCAPE =
|
|
"If you have already restarted the tool-server for this app and it reads this way again, " +
|
|
"stop: the process is loading argent's dylib but never dialing, which no further restart on " +
|
|
"either side fixes. Treat native devtools as unavailable — read the screen with describe or " +
|
|
"screenshot and drive it by coordinate.";
|
|
const INDETERMINATE_ESCALATION =
|
|
"If it is still not connected after that restart, the native-devtools service is stale rather " +
|
|
"than the app being uninjected — do not keep restarting the app; restart the tool-server " +
|
|
"(`argent server stop && argent server start --detach`) and retry.";
|
|
|
|
/**
|
|
* An instruction to retry without bound, in any of the ways these surfaces
|
|
* could phrase one.
|
|
*
|
|
* The pins below all constrain a SLICE of a surface — a named line, a clause
|
|
* between two markers, the final sentence — and every round of review has
|
|
* found the same defect in whatever slice was left over: a second guidance
|
|
* line for the same state, a clause appended past the last marker, a
|
|
* permitting sentence before the pinned tail. Pinning more slices only moves
|
|
* the gap, so this asks the one question that does not depend on where the
|
|
* text sits: does this surface, anywhere, tell an agent to keep going?
|
|
*
|
|
* A clause that FORBIDS one is the point of these messages, so it is the
|
|
* un-negated ones that fail.
|
|
*/
|
|
const UNBOUNDED_RETRY =
|
|
/keep [\w-]+ing|more than once|again and again|a couple more|until (it|connected)\b|ignore the stop/i;
|
|
// Deliberately not a bare `stop`: "ignore the stop-conditions above and keep
|
|
// calling restart-app" contains the word while saying the opposite, so the
|
|
// escape hatch would have covered for the negation it exists to allow.
|
|
const FORBIDDEN =
|
|
/\bdo not\b|\bdo NOT\b|\bnever\b|\bcannot\b|\bno further\b|\bat most\b|\bstop (restarting|:)/i;
|
|
|
|
/** Split on sentence and clause boundaries, so polarity is judged locally. */
|
|
function clausesOf(text: string): string[] {
|
|
return text
|
|
.split(/(?<=[.;:])\s+|\s+—\s+|\n/)
|
|
.map((c) => c.trim())
|
|
.filter(Boolean);
|
|
}
|
|
|
|
it("never tells an agent to keep retrying, on any surface that carries a remedy", () => {
|
|
const surfaces: [string, string][] = [
|
|
...(Object.keys(ALL_STATES) as NativeDevtoolsAppState[])
|
|
.filter((s): s is Exclude<NativeDevtoolsAppState, "connected"> => s !== "connected")
|
|
.flatMap((s): [string, string][] => [
|
|
[`${s} message`, buildAppStateMessage("com.example.app", s)],
|
|
// The flow gate rewrites every state for a reader who has just
|
|
// launched, so it is a second copy of the same remedies on a surface
|
|
// none of the verbatim pins reach.
|
|
[`${s} flow-gate reason`, flowLaunchGateReason("com.example.app", s)],
|
|
]),
|
|
["native-devtools-status description", nativeDevtoolsStatusTool.description!],
|
|
...tools.map((t): [string, string] => [`${t.id} description`, t.description!]),
|
|
];
|
|
|
|
for (const [name, text] of surfaces) {
|
|
for (const clause of clausesOf(text)) {
|
|
if (!UNBOUNDED_RETRY.test(clause)) continue;
|
|
expect(FORBIDDEN.test(clause), `${name} permits an unbounded retry: "${clause}"`).toBe(
|
|
true
|
|
);
|
|
}
|
|
}
|
|
});
|
|
|
|
it("gives each state exactly one guidance line to obey", () => {
|
|
// A verbatim `toBe` on the first matching line says nothing about a second
|
|
// one appended later, which an agent reads just as readily.
|
|
const lines = nativeDevtoolsStatusTool.description!.split("\n");
|
|
for (const state of Object.keys(ALL_STATES) as NativeDevtoolsAppState[]) {
|
|
const matches = lines.filter((l) => l.startsWith(`If state is ${state}:`));
|
|
expect(matches.length, `description routes ${state} ${matches.length} times`).toBeLessThan(2);
|
|
}
|
|
});
|
|
|
|
it("routes exactly the four precheck statuses on every native-* tool", () => {
|
|
// The clause pins slice up to the NEXT `If status is `, so the last slice is
|
|
// open-ended: a fifth clause is unconstrained on all six tools at once.
|
|
for (const tool of tools) {
|
|
const count = tool.description!.match(/If status is /g)?.length ?? 0;
|
|
expect(count, `${tool.id} routes ${count} statuses`).toBe(4);
|
|
}
|
|
});
|
|
|
|
it.each([
|
|
["connecting", CONNECTING_PROHIBITION],
|
|
["unregistered", UNREGISTERED_ESCAPE],
|
|
["indeterminate", INDETERMINATE_ESCALATION],
|
|
[
|
|
"not_running",
|
|
"If that launch fails rather than starting the app, the bundle id is not installed on this " +
|
|
"device — this state cannot tell the two apart; install it and no relaunch will be needed.",
|
|
],
|
|
["stale_process", "A fresh process picks up the current one: call restart-app then retry."],
|
|
] as const)("ends the %s message on its stop-condition, verbatim", (state, tail) => {
|
|
// `endsWith`, not `toContain`: the failure mode is a permitting sentence
|
|
// AFTER the stop-condition, which any containment check reads straight past.
|
|
const message = buildAppStateMessage("com.example.app", state);
|
|
expect(message.trimEnd().endsWith(tail), `${state} message does not end on its remedy`).toBe(
|
|
true
|
|
);
|
|
});
|
|
|
|
it.each([
|
|
[
|
|
"connecting",
|
|
"If state is connecting: do NOT restart the app — launching it is what starts the connection, so a relaunch discards the one in progress and returns this same state. Wait a few seconds and repeat this call.",
|
|
],
|
|
[
|
|
"unregistered",
|
|
"If state is unregistered: do NOT restart the app again — it already launched under the terms a restart would recreate. Restart the tool-server (`argent server stop && argent server start --detach`), then retry. If it reads unregistered again after that restart, stop: the process loads argent's dylib but never dials, and no further restart on either side changes it — treat native devtools as unavailable, then use `describe` or `screenshot` and drive by coordinate.",
|
|
],
|
|
[
|
|
"indeterminate",
|
|
"If state is indeterminate: the process could not be inspected, so restart-app is worth one attempt. If this call still reports it after that restart, do NOT restart the app again — the service is stale rather than the app uninjected, so restart the tool-server (`argent server stop && argent server start --detach`) and retry. Remote simulators can never inspect the process, so this is the only unconnected state a running app reaches there.",
|
|
],
|
|
] as const)("routes %s in the description exactly as written", (state, line) => {
|
|
expect(guidanceLine(state)).toBe(line);
|
|
});
|
|
|
|
it.each([
|
|
[
|
|
"connect_pending",
|
|
"If status is connect_pending: the app is injected and still connecting — do not restart it, wait a few seconds and retry.",
|
|
],
|
|
[
|
|
"service_stale",
|
|
"If status is service_stale: the app is already injected, so restarting it cannot help — restart the tool-server (`argent server stop && argent server start --detach`) and retry. If the same status comes back after that restart, stop restarting: follow the message, which names the terminal fallback.",
|
|
],
|
|
])("routes %s identically on all six native-* tools", (status, clause) => {
|
|
for (const tool of tools) {
|
|
const start = tool.description!.indexOf(`If status is ${status}:`);
|
|
expect(start, `${tool.id} does not route ${status}`).toBeGreaterThanOrEqual(0);
|
|
const rest = tool.description!.slice(start);
|
|
const next = rest.slice(1).search(/If status is /);
|
|
expect(rest.slice(0, next === -1 ? undefined : next + 1).trim(), tool.id).toBe(clause);
|
|
}
|
|
});
|
|
|
|
// Whether the dylib loads into an Apple system app is not settled — #453
|
|
// recorded `connected: false` on iOS 26.5, an E2E run `connected: true` on
|
|
// 18.5 — and one agent reads every surface below, so hedging one while its
|
|
// twin says "can never" is what misleads. The DECISION (terminal, do not
|
|
// retry) is unchanged; only the certainty claimed for it has to agree.
|
|
it("does not claim injection is impossible on any agent-facing surface", () => {
|
|
// Fails loudly on an absent description rather than letting the regex below
|
|
// pass over `undefined`, which would satisfy a negative match vacuously.
|
|
const described = (name: string, text: string | undefined): [string, string] => {
|
|
expect(text, `${name} has no text, so this check would be vacuous`).toBeTypeOf("string");
|
|
return [name, text!];
|
|
};
|
|
const surfaces: [string, string][] = [
|
|
described("native-devtools-status description", nativeDevtoolsStatusTool.description),
|
|
...tools.map((t) => described(`${t.id} description`, t.description)),
|
|
described("non-injectable recovery", NON_INJECTABLE_RECOVERY),
|
|
];
|
|
|
|
for (const [name, text] of surfaces) {
|
|
expect(text, `${name} asserts injection is impossible`).not.toMatch(
|
|
/can never (be injected|load|inject)/
|
|
);
|
|
}
|
|
// Softening the claim must not soften the behaviour, or the restart loop
|
|
// #453 reported comes straight back.
|
|
expect(nativeDevtoolsStatusTool.description).toMatch(/Do NOT restart\/retry/);
|
|
});
|
|
|
|
// `not_running` is the absence of a `UIKitApplication:<id>` row, which an
|
|
// uninstalled bundle id lacks too. Prescribing only launch-app makes it a loop
|
|
// of its own there: `simctl launch` refuses, and the state's single remedy is
|
|
// spent. Naming the second reading gives that agent somewhere to go.
|
|
it("admits that not_running cannot see whether the app is installed", () => {
|
|
const message = buildAppStateMessage("com.nonexistent.TotallyFake", "not_running");
|
|
|
|
expect(message).toContain("launch-app");
|
|
expect(message).toMatch(/not installed/);
|
|
// The claim has to be scoped to this state's evidence, not asserted as a
|
|
// measurement it never took.
|
|
expect(message).toMatch(/cannot tell the two apart/);
|
|
});
|
|
});
|