callstack__agent-device/docs/ios-runner-protocol-optimizations.md

iOS runner protocol optimization plan

Issue #656 is now split into protocol infrastructure plus follow-up optimizations. The lifecycle protocol makes commands identifiable, but the performance wins come from changing when the daemon uses uptime, retries, invalidates sessions, and asks the runner for lifecycle status.

Work slices

1. Status-before-invalidate recovery

Status: in progress on codex/ios-runner-status-recovery.

Goal: when a command has been sent and the HTTP response is lost, ask the runner for status(statusCommandId) before invalidating the session or surfacing an ambiguous transport failure.

Acceptance criteria:

• Post-send retryable transport failures issue one bounded status probe with the original commandId before session invalidation.
• completed with retained small response JSON returns the recovered command result without invalidating or resending the command.
• failed returns the runner failure code/message/hint instead of a generic transport failure.
• notAccepted, status timeout, or status transport failure preserves the existing invalidation behavior.
• Read-only commands whose response was not retained keep the existing retry behavior.
• Status recovery probes are short-budget and do not consume the full command timeout.

iOS simulator validation:

• Unit: pnpm exec vitest run packages/platform-apple/src/runner/__tests__/runner-command-retry.test.ts.
• Unit bundle: pnpm exec vitest run packages/platform-apple/src/runner/__tests__/runner-client.test.ts packages/platform-apple/src/runner/__tests__/runner-session.test.ts packages/platform-apple/src/runner/__tests__/runner-command-retry.test.ts packages/platform-apple/src/runner/__tests__/runner-provider.test.ts.
• Build: pnpm build:xcuitest.
• Manual sim smoke after build:
  • pnpm build
  • pnpm clean:daemon
  • run a simple iOS simulator session against Settings with open, snapshot -i, one selector interaction, and close.
  • confirm there is no visible behavior change and diagnostics show no unexpected session invalidation.

2. Adaptive uptime preflight policy

Status: implemented with guardrails (see ADR 0005). The earlier blanket "recent success" cache was shipped and then reverted in #702 because XCTest could return a successful tap response and then fail the runner while re-resolving a navigation-disappeared element, and because sparse AX-fallback snapshots were cached as healthy state. #702's coordinate-first activation removed that teardown trigger, so the skip is reintroduced as a structurally narrower "healthy mutation recency" signal.

Goal: skip the per-command uptime for hot allowlisted interaction loops only when a proven healthy mutating response makes the runner's liveness already known, while every uncertain path keeps preflighting.

Acceptance criteria (as shipped):

• First-command/startup, no-record, stale-record, app-activation-uncertain, and non-allowlisted (conservative) commands still preflight; readiness probes and read-only startup commands keep their existing skips.
• Recency is derived only from healthy (parsed ok, non-runnerFatal) responses of an explicit mutating allowlist (tap, longPress, drag, swipe, scroll, sequence) for the same appBundleId, within a 5s freshness window, and lives only on the session object so it dies with every invalidation/restart. Snapshots and read-only responses never refresh it.
• A transport failure after a skipped preflight clears the recency record and marks the error with the skip context (runnerReadinessPreflightSkipped, distinct from the restart predicate's runnerReadinessPreflightFailed). Connection-shaped failures run status recovery before invalidation — never a replay; timeout-shaped failures propagate with the skip context, matching the existing classification for preflighted sends.
• Diagnostics expose whether a command used, skipped, or recovered from a readiness preflight, including command type, skip reason, and recency age.
• Measured threshold: 1 runner request per hot allowlisted command after the first, with no increase in invalidation or failure rate.

iOS simulator validation:

• Start a fresh simulator session and run one interaction: verify the first mutating command still preflights.
• Run a hot loop of repeated selector interactions against the same visible control: verify the runner remains healthy and diagnostics explain any readiness probe that was skipped.
• Compare median command latency for a hot interaction loop before and after the change. A useful threshold is at least one fewer runner request per hot command and no increase in failure rate.

3. Status-visible transport path

Goal: make accepted and started states practically observable while a command is still running. The Swift journal already records these states, but the runner currently serializes connection handling, so a concurrent status request can be blocked behind the command it is querying.

Acceptance criteria:

• status can be answered while another runner command is waiting on main-thread XCTest work.
• The status path remains journal-only and does not touch app activation, XCTest dispatch, or command retry logic.
• Long-running command status can report accepted or started before the command reaches a terminal state.
• Existing command execution remains serial where mutation ordering matters.

iOS simulator validation:

• Run a deliberately long runner command in one request.
• While it is in flight, query status(statusCommandId) from another request.
• Verify status returns before the long command completes and reports accepted or started.
• Verify normal command ordering is unchanged for back-to-back mutating commands.

4. Session invalidation reduction

Goal: avoid tearing down otherwise healthy runner sessions when lifecycle status proves the command completed or failed cleanly.

Acceptance criteria:

• Completed/failed lifecycle status suppresses invalidation for ambiguous post-send transport errors when the runner remains reachable.
• Unknown status states still invalidate to preserve current safety.
• Diagnostics record why invalidation was skipped or retained.
• No command is replayed after an observed mutating accepted, started, completed, or failed state.

iOS simulator validation:

• Inject or simulate a lost response after a command completes.
• Verify status recovery prevents runner restart.
• Run the next command in the same session and verify it succeeds without re-launching xcodebuild.

5. Response retention tuning

Goal: retain enough small command results for useful recovery without making the runner retain large snapshots or binary-like payloads.

Acceptance criteria:

• Small scalar responses can be recovered from lifecycleResponseJson.
• Snapshot node trees and screenshots are not serialized or retained in the journal.
• The journal memory cap remains bounded by entry count and response JSON size.
• Retention policy is documented in tests or runner fixtures so future commands do not accidentally store large payloads.

iOS simulator validation:

• Run small-result commands and verify status can recover retained JSON.
• Run snapshot-heavy commands and verify status reports terminal state without retained response JSON.
• Confirm the runner remains responsive after repeated snapshots.

Suggested ordering

1. Land status-before-invalidate recovery first. It is the safety net needed before reducing defensive preflights.
2. Add diagnostics/metrics for preflight use, skipped preflights, status recovery, and invalidation reason. This can happen alongside slice 1 or 2.
3. Reduce uptime for hot interaction loops with a conservative command allowlist.
4. Make the status transport path observable during long-running commands.
5. Broaden the preflight policy only after simulator measurements show stable behavior.

Side-by-side work

• Status recovery and diagnostics can be developed together or separately.
• Transport status visibility can proceed independently once the protocol is on main.
• Adaptive uptime should wait for status recovery, because it relies on the same recovery path for ambiguous post-send failures.
• Response retention tuning can proceed independently as long as it preserves the current caps.