Files
callstack__agent-device/test/integration/android-emulator-e2e/live-lifecycle-scenario.ts
Michał Pierzchała e3f3a2488e test(android): restore full-tier lifecycle and observability scenarios (#1781 A1) (#1793)
* test(android): restore full-tier lifecycle and observability scenarios (#1781 A1)

The nightly Android job has failed on `click id="automation-request-microphone"`
since the full tier landed: `settings permission reset microphone` runs
`pm revoke`, and revoking a *granted* runtime permission kills the app process,
so the round after an accept clicked into an empty launcher surface.

Reproduced on the pinned CI image (android-36 google_apis_playstore, Pixel 7):
pm revoke leaves pid 4259 alive when the permission is denied and kills it when
it is granted, with NexusLauncher resumed afterwards.

Fixing that exposed the rest of a scenario that had never executed end to end:
the post-revoke readback cold-started on the tabs home instead of Automation lab,
the relaunched Automation lab needed its controls revealed, the Form tab does not
exist on the Automation root route, that section needs the system IME back, and
the IME diagnostic sits above the bottom of the form. The observability scenario
then failed the same way (reveal distances tuned for a taller device) plus an
event-timeline walk whose page was smaller than the events each page read appends.

Validated live against a local Pixel_7_CI emulator (API 36, same profile as the
lane): the full tier now runs bootstrap -> inventory -> automation-system ->
form-input -> keyboard-ime -> capture-close -> lifecycle-system ->
observability-artifacts and stops only in full:fixture-replays.

* test(android): repair the drifted fixture replays and pin the catalog canary (#1781 A1)

Review follow-up. The nested batch regression now checks a sibling card instead of the
notice that owns `dismiss-notice`: resolving a child already proves its parent is present,
so the old target could not fail on its own. Confirmed on a Pixel 7 / API 36 emulator that
`gesture-lab-card` and `dismiss-notice` are on screen together at the scenario's existing
0.3 reveal (both present at 0.2-0.4; the card is gone by 0.5).

Getting a full-tier run to complete then required repairing what the lane had never
executed:

- `01-navigation-scroll.ad` clicked `label="Catalog, 0 new notifications"`. #1543 made the
  cart badge conditional, so the live label is `Catalog` — what the iOS twin already used.
- The catalog scroll canary lives inside the scrolling content, and Android accessibility
  snapshots carry on-screen nodes only, so every state except the initial `top` was
  unobservable: `wait "Catalog scroll: down|bottom|up"` could never pass, whatever the
  swipe coordinates were. `stickyHeaderIndices` pins that one line, which makes all four
  states readable at any offset on both platforms rather than tuning the .ad around a
  canary that scrolls away.
- `gesture-lab-android.ad` started its multi-pointer gestures at y=1040, inside the target
  when the file was last repaired but 90px from its top edge after #1567 moved the card
  (targets now span y=949-1525). The second pointer landed outside the view, which reads as
  "the gesture did nothing". Multi-pointer gestures now start at the target centre, and the
  header comment records the geometry they depend on.

Evidence: the lane's own command (`AGENT_DEVICE_ANDROID_E2E_TIER=full` over
smoke-android-emulator.test.ts) passes end to end on a Pixel 7 / API 36 AVD with a
CI-equivalent fixture APK (cached native + head JS through the same repack the workflow
runs): 9/9 scenarios, 153s.
2026-08-18 09:57:24 +02:00

157 lines
6.8 KiB
TypeScript

import assert from 'node:assert/strict';
import { PUBLIC_COMMANDS } from '../../../src/command-catalog.ts';
import { assertElementText, assertWaitText } from './live-assertions.ts';
import { ANDROID_TEST_IME_PACKAGE } from './live-form-scenario.ts';
import { type LiveContext, runStep, verifyBehavior, verifyCommand } from './live-harness.ts';
const C = PUBLIC_COMMANDS;
const AUTOMATION_DEEP_LINK =
'agent-device-test-app:///automation?event=full.lifecycle&payload=%7B%22source%22%3A%22android-nightly%22%7D';
const TABS_DEEP_LINK = 'agent-device-test-app:///';
export const ANDROID_PERMISSION_PROMPT_COMMAND = ['alert', 'wait', '10000'] as const;
export async function assertLifecycleAndSystem(context: LiveContext): Promise<void> {
await openAutomationLab(context, 'open Android fixture lifecycle route');
await assertPermissionRecovery(context);
await assertPushBroadcast(context);
await assertImeRecovery(context);
}
// Leaves the fixture on Automation lab with its permission controls on screen: Android
// snapshots only carry on-screen nodes, and a relaunch always lands above the controls.
async function openAutomationLab(context: LiveContext, step: string): Promise<void> {
await runStep(context, step, ['open', context.appId, '--relaunch', AUTOMATION_DEEP_LINK]);
await assertWaitText(context, 'Automation lab');
await runStep(context, `${step} (reveal automation controls)`, ['scroll', 'bottom']);
}
async function assertPermissionRecovery(context: LiveContext): Promise<void> {
await resetAndRequestMicrophonePermission(context, 'accept');
await assertElementText(context, 'id="automation-microphone-permission"', 'granted');
await resetAndRequestMicrophonePermission(context, 'deny');
await assertElementText(context, 'id="automation-microphone-permission"', 'denied');
await resetAndRequestMicrophonePermission(context, 'accept after denial');
await assertElementText(context, 'id="automation-microphone-permission"', 'granted');
await runStep(context, 'revoke Android microphone permission', [
'settings',
'permission',
'deny',
'microphone',
]);
await openAutomationLab(context, 'restore fixture after microphone revocation');
await assertElementText(context, 'id="automation-microphone-permission"', 'denied');
verifyCommand(
context,
C.settings,
'prompt accept, prompt deny, and adb revoke transitions are fixture-visible',
);
verifyBehavior(
context,
'runtime-permission-recovery',
'fixture observed native prompt accept and deny, then relaunched readback after the pm revoke killed the app',
);
}
// Android kills the app process whenever a *granted* runtime permission is revoked, and
// `settings permission reset` revokes. Every reset therefore relaunches the fixture so the
// following steps always run against a live Automation lab.
async function resetAndRequestMicrophonePermission(
context: LiveContext,
action: 'accept' | 'deny' | 'accept after denial',
): Promise<void> {
await runStep(context, `reset microphone permission before ${action}`, [
'settings',
'permission',
'reset',
'microphone',
]);
await openAutomationLab(context, `restore fixture after reset before ${action}`);
await runStep(context, `request Android microphone permission for ${action}`, [
'click',
'id="automation-request-microphone"',
]);
const prompt = await runStep(context, `inspect Android microphone prompt for ${action}`, [
...ANDROID_PERMISSION_PROMPT_COMMAND,
]);
assert.equal(prompt.json?.data?.alert?.source, 'permission', JSON.stringify(prompt.json));
const alertAction = action === 'deny' ? 'dismiss' : 'accept';
await runStep(context, `${action} Android microphone permission prompt`, ['alert', alertAction]);
}
async function assertPushBroadcast(context: LiveContext): Promise<void> {
const payload = JSON.stringify({ extras: { count: 7, source: 'android-nightly' } });
const pushed = await runStep(context, 'broadcast typed Android fixture push', [
'push',
context.appId,
payload,
]);
assert.equal(pushed.json?.data?.package, context.appId, JSON.stringify(pushed.json));
assert.equal(pushed.json?.data?.extrasCount, 2, JSON.stringify(pushed.json));
await runStep(context, 'scroll to Android push receiver state', ['scroll', 'bottom']);
await runStep(context, 'refresh fixture push receiver state', [
'click',
'id="automation-refresh-push-broadcast"',
]);
await assertElementText(
context,
'id="automation-last-push-broadcast"',
'source=android-nightly,count=7',
);
verifyCommand(context, C.push, 'typed broadcast action and extras reach the fixture receiver');
verifyBehavior(
context,
'push-broadcast-delivery',
'fixture rendered the source and count persisted by its Android broadcast receiver',
);
}
async function assertImeRecovery(context: LiveContext): Promise<void> {
// Automation lab is a root route without the tab bar, so the tabs have to be restored
// before the Form tab can be clicked. This section observes the system IME, and only
// closing the session puts the test IME back, so the reopen follows a close.
await runStep(context, 'release Android test IME before IME recovery', ['close']);
await runStep(context, 'return to Android fixture tabs before IME recovery', [
'open',
context.appId,
'--relaunch',
'--no-test-ime',
TABS_DEEP_LINK,
]);
await assertWaitText(context, 'Agent Device Tester');
await runStep(context, 'open Android fixture form for IME recovery', ['click', 'label="Form"']);
await assertWaitText(context, 'Checkout form');
await runStep(context, 'fill Android IME recovery field', [
'fill',
'id="field-ime-capture-target"',
'agent-device',
]);
const status = await runStep(context, 'inspect Android IME ownership after fill', [
'keyboard',
'status',
]);
assert.equal(status.json?.data?.visible, true, JSON.stringify(status.json));
// Names what `visible: true` only implies: the injecting test IME draws nothing, so reading
// it here would mean the reopen above failed to hand input back to the system keyboard.
assert.notEqual(
status.json?.data?.inputMethodPackage,
ANDROID_TEST_IME_PACKAGE,
JSON.stringify(status.json),
);
await runStep(context, 'dismiss Android IME after fill recovery', ['keyboard', 'dismiss']);
// The diagnostic sits between the IME field and the delivery choices, so scrolling to the
// bottom of the form scrolls past it.
await runStep(context, 'scroll to Android IME diagnostic', ['scroll', 'down', '0.3']);
await assertWaitText(context, 'Android fill input was captured by the active keyboard');
await assertElementText(context, 'id="field-ime-capture-target"', 'agent-device');
verifyBehavior(
context,
'ime-owned-input-recovery',
'fixture input survived IME dismissal and its Android ownership diagnostic remained observable',
);
}