364844e01d
feat: add recording quality flag ( #409 )
...
* feat: add recording quality flag
* test: cover quality edge cases
* fix: scale transform translation in resize
* fix: narrow recording quality flag
2026-04-15 13:18:57 +02:00
7df452b3b5
feat: add rotate command for iOS and Android ( #344 )
...
* feat: add device rotation command
* fix: simplify rotate command handling
2026-04-01 20:34:56 +02:00
8df45b69a8
fix: make snapshot refs visible-first ( #337 )
...
* fix: make snapshot refs visible-first
* fix: handle zero-height visible rects
* refactor: simplify scrollintoview success handling
* feat: hint hidden scroll content in snapshots
* refactor: simplify snapshot output helpers
* feat: add hidden scroll-area hints to snapshots
* refactor: keep scroll containers in interactive snapshots
* perf: reuse Android snapshot tree for scroll hints
* refactor: trim hidden scroll hint scope
* refactor: unify mobile visible-first snapshot semantics
* fix: suppress noisy system scroll-container labels
* refactor: unify scroll indicator heuristics
* refactor: trim android hinting overhead
* fix: avoid viewport-sized ancestor promotion for ref taps
* refactor: dedupe viewport helpers and clarify hint naming
2026-04-01 19:00:02 +02:00
f7f7ce531f
chore: add icon for iOS UITest runner ( #328 )
...
* chore: add icon for iOS UITest runner
* chore: tighten uitest runner icon assets
* chore: prefer jpg for uitest runner icon
2026-03-31 21:03:57 +02:00
0fed800eee
fix: avoid focusing iOS simulator window ( #319 )
2026-03-28 19:39:14 +01:00
876f091c00
feat: close macOS parity gaps ( #311 )
...
* feat: close macos parity gaps
* fix: support targeted macOS menubar app snapshots
* chore: clarify menubar fallback heuristics
* chore: address review nits
* fix: complete macOS parity flow
2026-03-28 19:06:52 +01:00
d9d94f0f5a
[codex] close iOS runner responses gracefully ( #306 )
...
* fix: close iOS runner responses gracefully
* fix: close runner connection after send
2026-03-27 17:48:20 +01:00
ac50381270
docs: clarify iOS runner protocol contract ( #300 )
2026-03-27 15:27:56 +01:00
b6592db978
Fix iOS runner fill for tapped secure text fields ( #298 )
...
* fix: target iOS runner fill typing to tapped inputs
* fix: clarify targeted fill error handling
2026-03-27 14:57:56 +01:00
1669c7a101
feat: support pixel-based scroll gestures ( #293 )
...
* feat: add pixel-based scroll gestures
* fix: address remaining scroll review comments
* chore: make scroll direction switch exhaustive
* docs: keep readme informational
* docs: revert readme changes
* refactor: reuse shared apple scroll planning
* refactor: reduce apple scroll runner churn
2026-03-27 12:06:57 +01:00
11b47a25a7
Fix keyboard dismiss semantics across iOS and Android ( #291 )
...
* fix: correct keyboard dismiss semantics
* fix: address PR review feedback
* fix: preserve iOS keyboard dismiss contract
* fix: restore session handler platform import
* docs: update skill guidance for keyboard commands
2026-03-27 11:49:30 +01:00
c9ca550f1e
feat: add --delay-ms support to type and fill ( #292 )
...
* feat: add delayed typing support
* fix: keep literal delay tokens in replay
* chore: tighten delay-ms follow-up polish
2026-03-27 10:43:52 +01:00
605f736f2f
fix: expose collapsed iOS tab items in snapshots ( #287 )
...
* fix: expose collapsed iOS tab items in snapshots
* fix: address snapshot fallback review feedback
* fix: tighten snapshot fallback scope and truncation
2026-03-27 10:25:32 +01:00
dc8f318f76
Split back navigation into explicit in-app and system semantics ( #288 )
...
* feat: split back navigation semantics
* fix: address back review feedback
* fix: make in-app back the default
2026-03-27 10:18:00 +01:00
a39bb6d0fc
fix: improve large text surface discovery and reads ( #261 )
...
* fix: improve large text surface discovery and reads
* chore: trim duplicated skill guidance
* fix: tighten large text read follow-ups
* fix: scope snapshot metadata and log read fallbacks
2026-03-25 17:13:31 +01:00
0cca41f673
feat: add gesture telemetry recording overlays ( #245 )
...
* feat: add gesture telemetry recording overlays
* fix: stabilize recording validation and telemetry artifacts
* fix: tighten recording telemetry helpers
* fix: clean up android recording stop fallback
* refactor: split ios recording handlers
* fix: tighten runner-backed touch telemetry
* refactor: simplify touch recording recovery
* feat: add gesture telemetry recording overlays
* refactor: split touch interaction recording flow
* fix: rebase gesture telemetry onto main
* test: fix recording overlay CI expectations
2026-03-24 13:38:41 +01:00
4b8fc2f4f6
fix: improve macOS runner snapshot and context menus ( #252 )
...
* fix: improve macos runner snapshot and context menus
* fix: address macos runner review feedback
* refactor: simplify xctestrun product resolution
* refactor: future-proof macos runner and click flows
* refactor: remove secondary click alias
* fix: tighten macos runner review follow-ups
* update readme
* fix: discover macos apps from standard bundles
* fix: stabilize macos app listing test on linux
2026-03-23 16:01:52 +01:00
9e1991daf6
feat: add apple-family macOS desktop support ( #246 )
...
* feat: add apple-family macos desktop support
* feat: extend macos desktop command support
* fix: stabilize CI for macos runner build
* fix: harden macos app resolution
* fix: tighten apple-family session routing
* fix: align macos log capabilities
* fix: support macos wait text
* fix: make iOS session refresh test host-agnostic
2026-03-23 10:22:55 +01:00
b79bd86017
fix: remove trailing commas in Swift for Xcode <16.3 compatibility ( #186 )
...
* fix: remove trailing commas in Swift for Xcode <16.3 compatibility
Remove trailing commas from function/initializer argument lists in the
iOS XCUITest runner Swift source files. SE-0439 trailing commas in
comma-separated lists require Swift 6.1+ (Xcode 16.3+), but the runner
is compiled locally by users who may have older Xcode versions.
Users with Xcode 16.2 (Swift 6.0.3) encounter build failures:
error: unexpected ',' separator
This causes the runner to never start, surfacing as:
Error (COMMAND_FAILED): Runner did not accept connection (simctl spawn)
Files fixed (12 trailing commas removed):
- RunnerTests+Snapshot.swift (5)
- RunnerTests+Transport.swift (2)
- RunnerTests+CommandExecution.swift (1)
- RunnerTests+ScreenRecorder.swift (2)
- RunnerTests.swift (2)
Verified: pnpm build:xcuitest passes (TEST BUILD SUCCEEDED).
* ci: guard iOS runner trailing-comma syntax
---------
Co-authored-by: Michał Pierzchała <thymikee@gmail.com >
2026-03-06 15:40:18 +01:00
9b443c526f
refactor: split RunnerTests into focused files ( #149 )
...
* refactor: split RunnerTests into focused files
* chore: remove dead interaction label helper
2026-02-28 13:38:59 +01:00
97f59254bb
fix: make snapshot --raw use target app tree on physical iOS ( #148 )
...
* fix: use snapshots for raw iOS tree traversal
Fixes #147
* refactor: unify snapshot hittable computation across modes
* refactor: account for cousin occlusion in hittable
* refactor: tighten snapshot occlusion heuristic
2026-02-28 12:55:41 +01:00
14de55c9b7
fix: support configurable iOS runner bundle IDs ( #146 )
2026-02-26 16:40:24 +01:00
43aae1d30a
fix(ios): correctly implement runner fallback for physical device screenshots ( #132 )
...
* fix(ios): use XCTest runner for screenshots on physical devices
`xcrun devicectl device screenshot` was removed in Xcode 26.x, causing
`agent-device screenshot` to fail on physical iOS devices with exit code 64.
This fix routes physical device screenshots through the AgentDeviceRunner
XCTest runner (already used for snapshot/interaction commands) instead:
- Add `screenshot` command to the Swift runner's `CommandType` enum
- Implement the handler using `XCUIScreen.main.screenshot()`, writing a
timestamped PNG to the app's temp directory (returned as `tmp/<file>`)
- If `appBundleId` is provided, activate the target app before capturing
so the screenshot shows the app under test rather than the runner itself
- In `screenshotIos()`, pass `appBundleId` to the runner command then pull
the file back to the host via `xcrun devicectl device copy from` (same
mechanism used by the recording feature)
- Thread `appBundleId` through `Interactor.screenshot()` → `dispatch.ts`
→ `screenshotIos()` so the active session's app is always captured
- Extract `IOS_RUNNER_CONTAINER_BUNDLE_IDS` to `runner-client.ts` so it
can be shared between `apps.ts` and `record-trace.ts`
Fixes: screenshots failing with "Failed to capture iOS screenshot" on
devices using Xcode 26.x / devicectl 506.6+
* fix: clean up iOS screenshot fallback follow-ups
---------
Co-authored-by: Michał Pierzchała <thymikee@gmail.com >
2026-02-25 16:11:01 +01:00
8123117e6d
fix: fallback iOS/tvOS screenshot to runner when devicectl screenshot is unavailable ( #130 )
2026-02-25 15:34:14 +01:00
ef20dea6bc
feat: add AndroidTV and tvOS target support ( #125 )
...
* feat: add AndroidTV and tvOS target support
* feat: enable tvOS runner interactions
* refactor: unify apple target handling and tvOS capability gates
* refactor: remove hardcoded tvOS app-switcher sleep
* fix: align tvOS capabilities and docs for simulator-only helpers
* feat: improve AppleTV and AndroidTV physical device detection
* refactor: simplify tv target detection helpers
2026-02-25 14:55:54 +01:00
00ba0be101
fix: speed up and simplify scrollintoview @ref ( #87 )
2026-02-20 19:43:05 +01:00
2fd2bf50fe
chore: remove dead ios runner command paths ( #88 )
2026-02-20 19:38:16 +01:00
8eea063374
feat: add iOS device record support via runner ( #83 )
...
* feat: add iOS device record support via runner
* refactor: simplify record command pathways
* fix: harden iOS record runner failure handling
* chore: emit diagnostic on iOS record stop runner failure
* fix: clear ios device runner cache in build:xcuitest
* chore: drop unused clean-derived env from build script
* fix: stage iOS device recordings via runner temp file
* fix: prefer requested iOS record path with permission fallback
* fix: disable iOS record fallback for explicit output paths
* refactor: remove iOS record fallback staging path flow
* fix: resolve record output paths from request cwd
* chore: log resolved record output path in diagnostics
* fix: avoid app activation when starting iOS recording
* refactor: streamline iOS recording setup paths
* fix: copy iOS device recordings back to host path
* fix: copy iOS recordings from runner test container
* feat: add configurable iOS recording fps
* fix: disable xcode test diagnostics for runner sessions
* fix: use wall-clock timestamps for iOS recording
* feat: default iOS recording to uncapped fps
* fix: recover stale ios runner recording state
* fix: preserve app focus when starting iOS recording
* docs: clarify iOS device recording capture model
* fix: tighten iOS device record start ownership checks
* refactor: collocate iOS record context checks
* refactor: derive uncapped recorder timescale from interval
2026-02-20 18:41:07 +01:00
345b7eee7d
fix: harden ios runner resilience and cancellation ( #85 )
2026-02-20 17:18:37 +01:00
fc10613ecd
fix: harden iOS runner command execution ( #72 )
...
* fix: harden iOS runner command execution
* fix: tighten iOS runner error semantics
* fix: harden runner interactions after snapshot
2026-02-18 16:13:18 +01:00
99cc68463c
fix: align press/click behavior and honor fast press cadence on iOS ( #70 )
...
* fix: align press and click semantics and reduce iOS tap latency
* perf: batch repeated iOS taps in runner
* perf: add opt-in tap batching for repeated iOS presses
* perf: batch repeated iOS swipes in runner
* fix: preserve interaction series semantics in record/replay
* test: cover click/press interaction handler paths
* refactor: dedupe replay and interaction series helpers
* fix: unify press/click double-tap behavior
* refactor: add interactor doubleTap and alias docs
* refactor: normalize click alias to press path
2026-02-18 16:02:45 +01:00
106a5f661d
chore(breaking): remove AX snapshot backend and trim npm artifacts ( #62 )
...
* chore: remove AX snapshot backend and trim npm artifacts
* docs: remove AX backend references from commands page
* chore: update website
2026-02-17 12:00:21 +01:00
ac078905c6
refactor: stabilize iOS app/session and daemon startup flows ( #64 )
...
* refactor: stabilize iOS app/session and daemon startup flows
* fix: wait for android app launch completion
* fix: address review findings for iOS appstate and daemon flows
* fixups
2026-02-16 18:19:17 +01:00
06a8778157
Enable iOS device core parity and align docs/skills ( #55 )
...
* Add iOS device runner support and document platform matrix
* ios: make signing identity override-only
* ios: address PR55 review follow-ups
* docs: remove v1 phrasing from support messages
* ios appstate: prefer xctest before ax fallback
* docs: clarify ios runner derived path override behavior
* ios: harden runner polling and appstate fallback
* ios: remove automatic AX fallback paths
* docs: clarify no automatic AX fallback
* ios: replace ad-hoc runner poll sleep with retry policy
* Harden iOS runner timeouts and cleanup safety
* args: align scrollintoview schema with capabilities
* test: skip iOS integration flows on CI by default
2026-02-15 21:12:10 +01:00
64d27c18d0
Add consolidated gesture controls for press and swipe ( #54 )
...
* Add gesture series controls for press and swipe
* Normalize iOS swipe timing to safe duration
* Harden daemon startup for integration runs
2026-02-14 13:55:58 +01:00
613974ab71
Handle iOS system blocker alerts in snapshots ( #38 )
2026-02-11 19:05:19 +01:00
6a064f229d
fix: update IPHONEOS_DEPLOYMENT_TARGET to 15.6 ( #42 )
2026-02-11 10:07:58 +01:00
4aed3fba72
fix: support TouchableOpacity as [other] on iOS when has content ( #31 )
...
* fix: support TouchableOpacity as [other] on iOS when has content
* update snapshot ios
2026-02-09 19:07:04 +01:00
2e8680518d
fix: iOS snapshot not showing password fields ( #29 )
2026-02-09 13:29:05 +01:00
828284d7e3
fix: set min iOS deployment target to 15.6 ( #27 )
...
* fix: set min iOS deployment target to 15.6
* increase timeout for iOS
2026-02-09 12:39:02 +01:00
d2954f6fd3
fix: missing long-press on iOS simulator ( #26 )
...
* feat: for self-healing e2e tests; assertions
* fix: support long-press on iOS simulators via XCTest runner
The long-press command was failing on iOS simulators because dispatch.ts
called interactor.longPress() which mapped to a stub in ios/index.ts
that always threw UNSUPPORTED_OPERATION. Unlike press, type, fill, etc.,
the long-press case was missing the iOS simulator routing through the
XCTest runner.
Rather than adding yet another platform branch in dispatch.ts, this
refactors the Interactor abstraction to absorb runner routing internally.
getInteractor() now accepts an optional RunnerContext; when the device is
an iOS simulator, it returns an interactor whose tap/longPress/focus/
type/fill/scroll/scrollIntoView methods route through
runIosRunnerCommand. This removes 7 scattered if-ios-simulator branches
from dispatch.ts, making it impossible to forget runner routing for
future commands.
Changes:
- Swift runner: add longPress command type + longPressAt helper using
XCUICoordinate.press(forDuration:)
- runner-client.ts: add longPress to RunnerCommand type + durationMs field
- interactors.ts: add RunnerContext type, createIosSimulatorInteractor()
that routes through the XCTest runner, move invertScrollDirection here
- dispatch.ts: pass RunnerContext to getInteractor, remove all iOS sim
branching for interactor commands (-90 lines)
Co-authored-by: Cursor <cursoragent@cursor.com >
* restore attempts
* refactor: unify iOS interactor, remove dead input stubs
Since runnerContext is always passed and the capability matrix ensures
iOS only runs on simulators in v1, the two iOS interactor code paths
collapse into one: shared methods (open, close, screenshot) plus runner
overrides spread on top. No branch needed.
This deletes 7 dead iOS input stubs from ios/index.ts (pressIos,
longPressIos, focusIos, typeIos, fillIos, scrollIos, scrollIntoViewIos)
that only ever threw UNSUPPORTED_OPERATION errors.
Co-authored-by: Cursor <cursoragent@cursor.com >
* cleanup
* fixup save-script
* update
---------
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-02-09 12:06:26 +01:00
d8440a6645
fix: make fill clear input and progressively fill the input ( #17 )
...
* fix: make fill clear input and progressively fill the input
* fixup ios
* improve text fill reliability
2026-02-06 12:16:20 +01:00
992959c23e
feat(ios): add pinch gesture for zoom on iOS simulators ( #11 )
...
- Add 'pinch' CLI command: pinch <scale> [x y] (scale > 1 zoom in, < 1 zoom out)
- Implement in dispatch, runner-client (RunnerCommand + scale param), Swift runner
- Use double-tap + drag workaround for map zoom (tap center, drag up/down)
instead of XCUITest pinch(withScale:velocity:) which pans on map views
- Update commands.md and agent-device skill with pinch usage and best practices
- Fix skill reference: recording.md -> video-recording.md
Co-authored-by: Jano Detzel <jano.detzel@ewe-go.de >
2026-02-05 17:51:36 +01:00
d8b1800dc0
fix long-running timeouts for session app
2026-02-05 07:59:31 +01:00
1d6994440c
perf: improve xctest backend speed dramatically and make default
2026-02-04 22:35:18 +01:00
701a5ea31e
refactor ios-runner; add skills
2026-02-02 20:33:01 +01:00
24926c0351
improve grouping mechnism for tabbar etc
2026-02-02 17:32:16 +01:00
8845eb2c79
feat: home, app-switcher, wait for
2026-02-02 16:57:50 +01:00
0f90c61e7c
remove rect; adjust error messages
2026-02-02 14:06:11 +01:00
13dc88eb83
setup base publish
2026-01-31 18:23:16 +01:00
Michał Pierzchała