Files
callstack__agent-device/apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+TextEntryFocus.swift
Michał Pierzchała 7db5ad73dd fix(ios): grant the text-entry commit wait time against progress (#2035)
* fix(ios): grant the text-entry commit wait time against progress

The synthesized commit wait used a flat 3s deadline, which cannot tell a
throttled simulator input pipeline (characters keep landing, slowly) from a
wedged one (nothing lands) — it condemned both at the same instant and reported
TEXT_INPUT_COMMIT_NOT_OBSERVED over a `type`/`fill` that was still working, on
branches touching no iOS code.

SynthesizedCommitBudget grants time against progress instead: while the observed
value's expected-prefix grows — the same length-only evidence logCommitCadence
already emits — the wait continues, up to a 10s ceiling. A pipeline making no
progress expires at exactly the 3s the flat deadline used, so a wedge is
condemned no later than before. It is a reference type, and the observe/expire
coupling carries a structural guard, because as a struct that coupling would
rest on Swift boxing one captured var and could revert to the flat deadline
silently.

Text-entry readiness' hardware-keyboard fallback also stops returning a
possibly-unfocused element after 0.35s of "no software keyboard seen"; it now
returns only on confirmed focus of the target and re-arms otherwise. And the
keyboard-hidden precondition of
testBareTypeUsesTappedInputWhenSoftwareKeyboardIsHidden skips rather than fails,
so an environment flip cannot read as a product regression.

The issue's remaining ask — pinning the simulator keyboard preference — is
deliberately not done: measured on a dedicated simulator, per-device
ConnectHardwareKeyboard makes no difference to a headless `simctl boot`, which
always shows the software keyboard. See the PR body for the A/B.

Refs #1874 — not a closing keyword on purpose. This is a mitigation; the
unidentified simulator input-throttle mechanism that issue tracks is untouched
here, so it stays open.

* refactor(ios): move the commit-wait budget into the wait itself

Review follow-up. The budget was a detached object tested in isolation, with a
TypeScript parser asserting that two escaping Swift closures happened to share
it — a guard that only existed because the seam was in the wrong place.

The budget is now a local `var` inside `awaitSynthesizedCommitOutcome` and its
replacement counterpart, advanced from the same observation the progress check
already reads, with the clock injected alongside the existing observation and
pacing seams. Recording progress and asking whether time is up are two
statements in one loop, so there is no coupling left to guard.

The detached tests and the TypeScript wiring guard are deleted. In their place,
four sequence tests drive the shipped waits through a hand-driven clock: a
prefix that keeps growing outlives the flat 3s deadline, a frozen prefix is
condemned at exactly 3s, an indefinitely throttled pipeline stops at the 10s
ceiling, and a value churning between two lengths buys no time. Verified red
first — the two progress tests fail against a no-op `record`, and the two
unchanged-behavior tests stay green.

* fix(ios): grant the text-entry commit wait time against progress

The synthesized commit wait started its clock before reading the field's
placeholder, and that read is an AX round-trip which takes seconds on exactly
the loaded host this budget exists for. Slow setup therefore spent the budget:
with a 3.5s placeholder read the first observation already exceeded the 3s
stall budget, so `type` reported TEXT_INPUT_COMMIT_NOT_OBSERVED after a single
poll — sooner than the flat deadline this replaced, in the one condition it was
written for.

The budget is now two durations, and only the poll loop starts it, from its own
first `now()`. Passing a pre-loop timestamp is no longer expressible. The poll
also takes one clock sample instead of two, so the instant an observation is
recorded at is the instant it is judged against.

testCommitWaitBudgetStartsAtTheLoopRatherThanBeforeIt pins it: 60s of setup
before the wait must still leave the full stall budget. Verified red against a
deadline started outside the loop.

* fix(test-app): stop the form fixture placing its own placeholder in every fill

The `smoke:form-input` half of #1874 is not the commit deadline. This PR's own
iOS lane reproduced it (run 32889322172) and the trace settles it: `wait start
expectedLen=12`, then zero `[DEBUG-1874] poll` lines, then `wait
outcome=notObserved elapsedMs=3608`. The wait never polled — it returned from
the `textMatchesPlaceholder` guard, which refuses before polling because an
empty text field renders its placeholder AS its accessibility value, so a match
cannot prove a commit.

`field-name`'s placeholder was "Ada Lovelace" and every checkout-form suite
fills exactly "Ada Lovelace"; `field-email` had the same collision with
"ada@example.com". Twelve fills across eight files, so `fill` into those fields
is unverifiable by contract. It looked intermittent only because the
synthesized-replacement route is gated on `xCTestChannelPenalized` — it fires
when the host is loaded — which is also why re-running a failed job on the same
commit reproduced it identically.

The collision also made the read-back assertions vacuous: `assertJsonContains(
name, 'Ada Lovelace')` is satisfied by an empty field rendering the placeholder.

Fixed in the fixture rather than in the values, because frozen replay-compat
corpora carry the same fills and must not be edited.
fixture-fill-placeholder-collision.test.ts guards the class: it fails on any
repository fill whose value equals the target field's placeholder.

* refactor(ios): drop the fill/placeholder source guard and flatten the commit deadline

Review: the 83-line guard was a source-reconstruction test, not a fixture
invariant. It regex-parsed JSX and two literal fill spellings and duplicated the
Swift trim/equality rule in TypeScript, so it could stay green while its "every
fill" claim was false — expressions, variables, typed clients and unlisted roots
are all outside what a regex can enumerate. The owning evidence already exists:
the Swift tests prove a placeholder-equal AX value is unobservable, and live
smoke:form-input failed on the prior head for exactly this collision. Deleted;
the two placeholder changes stay.

Same pass over the rest of the change, for the same reason. The commit deadline
was a budget value type, a nested Deadline type and a factory method; it is now
one flat struct the poll loop constructs, with the two durations as defaulted
parameters. Production call sites name no budget at all, tests name one only
when they are asking about time, and SynthesizedCommitBudget.standard and the
tests' unboundedCommitBudget both disappear.

* refactor(ios): split the text-entry readiness and commit-wait seams

Review: the change grew three files past their budgets. Splitting them along the
seams they already had, no behavior change.

RunnerTests+TextEntry.swift (607) keeps the vocabulary, field clearing and value
reading at 259; everything that decides "which element is about to receive text,
and has it taken focus" moves to RunnerTests+TextEntryReadiness.swift at 354.

RunnerTests+SynthesizedTextEntry.swift (503) keeps the private-XCTest synthesis
boundary, the replacement route and the route policies at 356. The commit wait
moves next to the deadline that bounds it: the two waits, the observation and
pacing they poll through, and the value-free cadence line that path may log now
sit together in RunnerTests+SynthesizedCommitDeadline.swift at 206. That also
puts every line touching the polled field value in one file, so
apple-runner-log-redaction.test.ts guards a single surface — its path constant
moves with it.

The deadline's clock and sequence tests leave the policy tests (641 -> 494) for a
sibling RunnerTests+SynthesizedCommitDeadlineTests.swift, which gains the
replacement-route case the review asked for: a growing prefix carries the wait
past the 3s stall budget and the 10s ceiling is what ends it. The injected clock
is now defaulted, so only a test actually asking about time names it.

* refactor(ios): split text-entry target acquisition from readiness

Review residual: the readiness extraction was 354 lines and still owned two
questions. Acquisition — the one-shot tap witness, post-tap stabilization, both
focusTextInputForTextEntry entry points and the refresh point — moves to
RunnerTests+TextEntryFocus.swift (206). Readiness keeps the waits, the keyboard
signals they read and the focus corroboration (158).

The dependency is one-way: acquisition asks readiness, never the reverse, so
waitForTextEntryReadiness and keyboardBecameVisible lose file-private scope and
nothing else does.
2026-08-26 16:50:37 +02:00

207 lines
8.2 KiB
Swift

import XCTest
// Text-entry target acquisition: choosing the element a `type`/`fill` will address and getting
// focus onto it — the one-shot tap witness, the post-tap stabilization, and the two
// `focusTextInputForTextEntry` entry points the command layer calls. Whether that target is ready
// is RunnerTests+TextEntryReadiness.swift's question, and this file asks it rather than answering
// it.
extension RunnerTests {
func rememberTextEntryTap(_ element: XCUIElement?) {
guard let element, isTextEntryElement(element) else {
clearRememberedTextEntryTap()
return
}
textEntryTapWitness = TextEntryTapWitness(
element: element,
bundleId: currentBundleId,
processIdentifier: currentAppProcessIdentifier
)
}
func clearRememberedTextEntryTap() {
textEntryTapWitness = nil
}
private func rememberedTextEntryTarget() -> TextEntryTarget? {
guard let witness = textEntryTapWitness else {
return nil
}
// The tap is proof for one immediately-following bare type only. Consume it before checking
// the element so a failed or interrupted type cannot reuse stale focus evidence.
clearRememberedTextEntryTap()
guard witness.matches(
bundleId: currentBundleId,
processIdentifier: currentAppProcessIdentifier
) else {
return nil
}
let element = witness.element
// XCUIElement is query-backed rather than a stable node identity. A same-identifier field
// introduced by app-side navigation between tap and this immediate type can therefore
// re-resolve here; keep the witness one-shot and fail closed on every observable identity
// boundary instead of using frame equality, which would reject legitimate layout changes.
guard safely("LAST_TAPPED_TEXT_INPUT_EXISTS", false, { element.exists }) else {
return nil
}
// Keep the target scoped to the element that the preceding tap actually selected. Do not
// attach a refresh point: if that element disappeared, bare type must fail closed rather
// than rediscovering a different field or dispatching unscoped app.typeText.
return TextEntryTarget(
element: element,
refreshPoint: nil,
prefersFocusedElement: false,
fromTapWitness: true
)
}
func stabilizeTextInputBeforeTyping(
app: XCUIApplication,
target: XCUIElement?,
keyboardVisibleBeforeTap: Bool? = nil
) -> TextEntryStabilization {
#if os(tvOS)
return TextEntryStabilization(element: target, focusConfirmed: true)
#else
let latest = target
let keyboardVisibleAtEntry = keyboardVisibleBeforeTap ?? isKeyboardVisible(app: app)
let deadline = Date().addingTimeInterval(TextEntryTiming.focusTimeout)
while Date() < deadline {
if let focused = focusedTextInput(app: app) {
return TextEntryStabilization(element: focused, focusConfirmed: true)
}
// focusedTextInput is intentionally nil on iOS; treat the keyboard transitioning to
// visible after our tap as the focus-moved signal. Don't fast-path when it was already up.
if keyboardBecameVisible(app: app, wasVisibleAtEntry: keyboardVisibleAtEntry) {
return TextEntryStabilization(element: latest, focusConfirmed: true)
}
sleepFor(TextEntryTiming.pollInterval)
}
return TextEntryStabilization(element: latest, focusConfirmed: false)
#endif
}
func focusTextInputForTextEntry(app: XCUIApplication, x: Double?, y: Double?) -> TextEntryTarget {
guard let x, let y else {
let softwareKeyboardVisible = isKeyboardVisible(app: app)
if !softwareKeyboardVisible, let rememberedTarget = rememberedTextEntryTarget() {
return rememberedTarget
}
// Bare `type` targets the current first responder. On iOS we intentionally do not trust
// `hasKeyboardFocus`, but an already-visible software keyboard is sufficient evidence that
// app.typeText has a receiver; waiting the full readiness timeout cannot prove a stronger
// target because there is no selector/coordinate focus move to validate.
if softwareKeyboardVisible {
return TextEntryTarget(
element: focusedTextInput(app: app),
refreshPoint: nil,
prefersFocusedElement: true
)
}
let focused = waitForTextEntryReadiness(
app: app,
target: TextEntryTarget(
element: focusedTextInput(app: app),
refreshPoint: nil,
prefersFocusedElement: true
)
)
return TextEntryTarget(element: focused, refreshPoint: nil, prefersFocusedElement: true)
}
let keyboardVisibleBeforeTap = isKeyboardVisible(app: app)
let target = textInputAt(app: app, x: x, y: y)
let requestedPoint = CGPoint(x: x, y: y)
if let target {
let frame = target.frame
if !frame.isEmpty {
_ = tapAt(app: app, x: frame.midX, y: frame.midY)
} else {
_ = tapAt(app: app, x: x, y: y)
}
} else {
_ = tapAt(app: app, x: x, y: y)
}
// A visible keyboard is not enough evidence for app.typeText, because focus may still
// belong to a previous field. With a concrete target we type through XCUIElement.typeText,
// so after tapping it the iOS readiness timeout cannot prove anything stronger.
if keyboardVisibleBeforeTap, let target {
return TextEntryTarget(
element: target,
refreshPoint: textEntryRefreshPoint(for: target) ?? requestedPoint,
prefersFocusedElement: false
)
}
let stabilized = stabilizeTextInputBeforeTyping(
app: app,
target: target,
keyboardVisibleBeforeTap: keyboardVisibleBeforeTap
)
let readyTarget = TextEntryTarget(
element: stabilized.element ?? target,
refreshPoint: requestedPoint,
prefersFocusedElement: false
)
let concreteTargetReady = keyboardVisibleBeforeTap && readyTarget.element != nil
let element = stabilized.focusConfirmed || concreteTargetReady
? (stabilized.element ?? target)
: (waitForTextEntryReadiness(app: app, target: readyTarget) ?? stabilized.element ?? target)
return TextEntryTarget(
element: element,
refreshPoint: textEntryRefreshPoint(for: element) ?? requestedPoint,
prefersFocusedElement: false
)
}
func focusTextInputForTextEntry(app: XCUIApplication, element: XCUIElement) -> TextEntryTarget {
let point = textEntryRefreshPoint(for: element)
let keyboardVisibleBeforeTap = isKeyboardVisible(app: app)
if let point {
_ = tapAt(app: app, x: point.x, y: point.y)
}
// See the coordinate-target path above: direct element typing keeps this scoped to the
// tapped target, while the first-character warmup and final verify still catch dropped input.
if keyboardVisibleBeforeTap {
return TextEntryTarget(
element: element,
refreshPoint: textEntryRefreshPoint(for: element) ?? point,
prefersFocusedElement: false
)
}
let stabilized = stabilizeTextInputBeforeTyping(
app: app,
target: element,
keyboardVisibleBeforeTap: keyboardVisibleBeforeTap
)
let readyTarget = TextEntryTarget(
element: stabilized.element ?? element,
refreshPoint: point,
prefersFocusedElement: false
)
let resolved = stabilized.focusConfirmed
? (stabilized.element ?? element)
: (waitForTextEntryReadiness(app: app, target: readyTarget) ?? stabilized.element ?? element)
return TextEntryTarget(
element: resolved,
refreshPoint: textEntryRefreshPoint(for: resolved) ?? point,
prefersFocusedElement: false
)
}
private func textEntryRefreshPoint(for element: XCUIElement?) -> CGPoint? {
guard let element else {
return nil
}
let frame = element.frame
guard !frame.isEmpty else {
return nil
}
return CGPoint(x: frame.midX, y: frame.midY)
}
/// A focus-moved signal for iOS text entry, where `focusedTextInput` is intentionally nil.
/// The software keyboard TRANSITIONING from hidden (at entry) to visible means the field we
/// just tapped gained first-responder. If the keyboard was ALREADY up (e.g. back-to-back
/// fills into different fields), its visibility is not evidence focus moved to the new field,
/// so callers must keep waiting rather than typing into the previously-focused field.
}