software-mansion__argent

skills/software-mansion__argent

Fork 0

mirror of https://github.com/software-mansion/argent.git synced 2026-09-14 19:27:14 +08:00

Commit Graph

Author	SHA1	Message	Date
filip131311	b232114422	docs: audit every comment in src against the code it describes (#931 ) Audits every comment in `src/` and `scripts/` against the code it describes. 462 files, 57 commits, net −8,194 lines. Comment text only — the whole branch is code-identical to `main`. ## Method One subagent per file, strictly sequential. Scope was `src/` + `scripts/` (459 files); three more files were added at the end because they carried dead references of the same kind — two test headers citing design docs that do not exist, and `publish-npm.yml` citing a retired workflow. Each agent verified every comment — line, block, JSDoc, file header, trailing — against the surrounding code and the rest of the repo, following identifiers, paths, tool ids, config keys, env vars and issue links to see whether they still exist and still behave as described. Rules: - A false or misleading comment is deleted, not reworded. If a claim could not be confirmed by reading the source, it went. That is why the deletion count is so much larger than the rewrite count. - Survivors are cut to the shortest form carrying something the code does not already say. Restatement, preamble, hedging, changelog prose and ASCII banners are gone; the non-obvious why stays. - Preserved byte-identical: license headers, pragmas and directives (`@ts-`, `eslint-disable`, shebangs, `/// <reference>`), JSDoc tag tokens, everything inside a string or template literal, and the sole comment inside an otherwise empty block (ESLint `no-empty` counts a comment-bearing block as non-empty). ## Verification Every file passed two independent gates before being recorded as done: 1. `comments-only` — the required check. 2. A second comment-stripping comparator with a proper mode stack, written for this pass because `comments-only`'s flat scanner desyncs on nested template literals and quote-bearing regex literals and then reports comment lines as code changes. Two files hit that false FAIL (`utils/android-profiler/pipeline/index.ts`, `scripts/extract-tools.mjs`); in both the "changed code" it printed was literally `//` lines, and the second checker confirmed the code was byte-identical. After the last file, all 462 changed files were re-checked against `main` with the same comparator, rather than trusting any agent's self-report. 459 code-identical; 2 are non-code (`.svg`, `.md`); 1 intentional.* The intentional one is `packages/argent/scripts/bundle-tools.cjs`: the changed template literal is the comment header of the file it generates, `packages/native-devtools-android/src/bundled-meta.ts`. Fixing only the generated file would have been reverted by the next build, so the generator changed too — and it has been verified to reproduce the committed generated file byte-for-byte. ## Representative false claims removed Not wording nits — statements a reader would have acted on: - Reversed directions. `proxyStart`'s JSDoc had the tunnel backwards (it is a reverse tunnel: the host binds first and the simulator dials in). A `paste()` doc had the pasteboard copy direction reversed. - Contradicted by the code below it. A timeout budget multiplied by three where the probes run concurrently — the same comment said so six lines later. A "warn once" that warns on every call. A "binary search" that is a linear scan. - Named things that do not exist. A `vega-fast-cli` binary, a `finish-recording.ts`, a `publish-next.yml` workflow, two `profiler-react19-.md` design docs, a `DebuggerTarget.ts`, a commit hash git does not know, two tool ids, an `ensureEnv` cycle. - Wrong by construction.* "Welford accumulators" across four files where the code keeps naive `n`/`sum`/`sumSq`; `sum`/`sumSq` documented over `actualDuration` when reduce sums `selfDuration`; a strict-mode halving written `n/2` where the code ceils; field docs listing enum values the producers never emit. - Guarantees the code does not make. A validation matrix claiming to cover "EVERY tool" that skips flagless ones; a Pareto cutoff that `slice(0, 20)` makes inert; an idempotence claim where the real rule is at-or-ahead; a capability note describing a clean 400 the shape-based device resolver can never produce. - Unverifiable assertions about prebuilt binaries, external CLIs and the cloud SDK — deleted rather than kept as folklore, since nothing in the repo can confirm them. - Stale numbers: invented Android tool versions, hard-coded tool counts and description lengths that had drifted. ## Review A Fable agent reviewed both halves adversarially for over-deletion, misread code, `no-empty` hazards and byte-identity violations. Second-half verdict: SHIP, with two one-line restores, both applied in the final commit — the `npm view ""` rationale behind a blank-token guard, and the note that `argent-mcp` keeps a copy of `SECRET_PLACEHOLDER_MARKER` it cannot import. ## Code issues surfaced but deliberately not fixed This pass changes comments only. Eight genuine findings are logged for a follow-up: 1. `telemetry/src/consent.ts` — a non-ENOENT read error returns null and falls through to the default-on path, so file errors can silently flip telemetry on. 2. `chromium-server/navigation.ts` — `navigate()` is reachable from `POST /api/navigate` with only a `typeof === "string"` check; open-url's schema is a bare `z.string()`, so the "already validated by zod" premise never held. 3. `http.ts` — `constantTimeEqual` returns early on a length mismatch, so the auth token's length is observable. 4. `describe/index.ts:~114` — the ios-remote branch passes `{ isTvOs: false }` unconditionally, so a remote tvOS simulator takes the iOS ax-service path, though `isRemoteTvOsSimulator` exists and shake/paste do use it. 5. `devices/boot-device.ts` — `-crash-report-mode never` is passed unconditionally and appended again by the feature-detecting path, so every emulator spawn passes it twice. 6. `react-profiler/pipeline/04-rank.ts` — `PARETO_THRESHOLD_PCT` is dead: `slice(0, 20)` always wins. 7. `reaped-sessions.ts` — a user-facing hint string tells the agent that `react-profiler-start { force: true }` disposes the debugger and profiler session; it does not. Left byte-identical because it is a string literal, not a comment. 8. `utils/simctl-backend.ts` — `localSimctl` is exported with no importers anywhere. ## Docs No documentation change is needed: this pass touches only source comments, and no user-facing capability, tool, CLI flag, config key or flow-file behaviour changed. --------- Co-authored-by: filip131311 <f.kaminski2000@gmail.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-24 12:16:20 +02:00
Paweł Fornagiel	3ccd89f333	feat: add native android profiling (#275 ) # Android native profiling via in-process Perfetto WASM Connected with: https://github.com/software-mansion/argent-private/pull/16 ## TL;DR Adds Android to the native profiler (previously iOS-only) and does it with a brand-new engine: instead of shipping Perfetto's `trace_processor_shell` native binary and forking it per query, we now run Perfetto's trace-processor compiled to WebAssembly, in-process inside the tool-server. - One ~13 MB `trace_processor.wasm` runs on every OS/arch — no per-platform binary, no subprocess, fully offline. - The native profiler is refactored from an iOS-only implementation into a platform-dispatch facade (`platforms/ios.ts` + `platforms/android.ts`) over a new `profiler-shared/` layer, so iOS and Android share the aggregator, types, lifecycle, and time-alignment code. - New release/pack pipeline fetches + sha256-verifies the WASM bundle from `argent-private-releases`, and bundles it (plus SQL queries and the Perfetto TraceConfig) into the published `@swmansion/argent` package. End-to-end on a 76 MB trace with 1013 jank rows: 6.3 s total analyze (the older per-hang-subprocess approach was projected at ~47 minutes and never finished within the tool-call deadline). --- ## TESTING MANUALLY To test manually run the following packing command checked-out on this branch: ``` npm run pack:mcp -- argent-@pFornagiel-native-android-devtools-perfetto ``` This will bundle the package with the new binaries. After this PR is fixed and the binaries are resolved from the main release, the `argent-@pFornagiel-native-android-devtools-perfetto` release can be deleted. ## Why Android needed native CPU/hang/memory profiling to reach parity with iOS. The natural engine is Perfetto's trace-processor, but the obvious ways to ship it are both bad: - `trace_processor_shell` binary — per-platform (a mac-arm64 pack shipped a mac-arm64-only tarball), pinned an OS/arch matrix, and cost a fresh ~1.3 s subprocess fork per query. With one subprocess forked per detected hang, a large trace projected to ~47 minutes. - `get.perfetto.dev/trace_processor` Python launcher — adds a runtime Python 3 dependency, resolves "latest" at first run (defeats version pinning), and makes the first analyze slow via a network fetch. The WASM engine is one cross-platform artifact, in-process and warm-cached: the trace is parsed once and reused across the whole analyze run plus every drill-down query. --- ## The three big pieces ### 1. In-process Perfetto WASM engine New package surface in `@argent/native-devtools-android`: \| File \| Role \| \| --- \| --- \| \| `src/wasm-trace-processor.ts` (+476) \| The whole engine: boot, glue patching, warm-cache, fault containment, chunked parse, cell decoding \| \| `src/perfetto-engine.d.ts` (+49) \| Hand-written types for the version-agnostic RPC decoder (imported by path, so tsc can't see its own `.d.ts`) \| \| `src/errors.ts` (+68) \| `TraceProcessorUnavailableError` (`wasm_load_failed` / `wasm_path_invalid`) \| \| `src/bundled-meta.ts` (+12) \| `PERFETTO_VERSION` stamp, regenerated at pack time \| \| `src/index.ts` \| Re-exports + package-root accessors (`traceProcessorQueriesDir`, `traceConfigPath`) \| Public surface: `resolveTraceProcessorAssets()`, `queryWarm<Row>(tracePath, sql)`, `ensureTraceProcessorReady(tracePath)`, `disposeWarmEngine(tracePath)`. Highlights of how it works: - Boots Google's prebuilt web/worker wasm under Node. Installs a minimal worker scope (`self`, `WorkerGlobalScope`, `location`), reads the wasm bytes, patches the Emscripten glue (`useMemory64=false`, hand the bytes in directly, expose the bridge), and drives it over a `MessageChannel` + `EngineBase` subclass. Glue patches are applied by unique string anchors that throw if they don't match — the canary for glue-format drift on a Perfetto bump. - Warm-engine cache, one per trace path. LRU with `MAX_WARM_ENGINES = 3` (bounds resident memory at ~26–76 MB trace + wasm heap each), 5-min idle dispose (timer `unref()`'d), no failed-promise caching. - Fault containment (`fatal` promise). `EngineBase.fail()` throws synchronously inside the `MessagePort.onmessage` handler (e.g. an RPC framing error); left unhandled it becomes a Node `uncaughtException` and would kill the entire tool-server, wiping every device + profiler session. Each engine exposes a `fatal: Promise<never>`; the handler try/catches, rejects `fatal`, evicts the engine, and every load/query is `Promise.race`'d against it — so a contained fault is a clean rejection and the server stays up. - Chunked trace parse. Perfetto's RPC ring buffer rejects any frame > 64 MiB, so the trace is fed in 32 MiB chunks before `notifyEof()`. ### 2. Android native profiler + the shared/dispatch refactor Capture (`utils/android-profiler/capture.ts`, `detect-app.ts`): spawns `adb shell perfetto --txt -c - --background-wait -o /data/.../argent-<ts>.pftrace`, piping the TraceConfig on stdin (SELinux denies `shell:s0` writes under `/data/misc/perfetto-traces/`, so the config can't be staged as a file), waits for the PID on stdout, then on stop sends `kill -TERM`, polls `/proc/<pid>`, and `adb pull`s the trace. Pipeline (`utils/android-profiler/pipeline/`): PerfettoSQL does the parsing and aggregation, so the Android pipeline is intentionally two files, not iOS's four. The per-hang annotation pass is batched — `runBatchedHangFolds` inlines all hang windows as a `VALUES` table into one SQL script (two derived views unioned in a single terminal SELECT, because the engine returns only the final statement's rows) and runs it once against the warm engine. The platform-dispatch refactor — `native-profiler-start/stop/analyze` shrank from monolithic iOS files (~160–330 lines each) into thin routers (~60–110 lines) that branch on `api.platform`: ``` native-profiler-start ─┬─ ios? → platforms/ios.ts (xctrace) └─ android → platforms/android.ts (adb perfetto) ``` A new `utils/profiler-shared/` layer holds everything both backends use: \| File \| What was hoisted \| \| --- \| --- \| \| `types.ts` \| Unified `Bottleneck = CpuHotspot \\| UiHang \\| MemoryLeak \\| MemoryRssGrowth`, each now carrying a `platform` field; shared `NativeProfilerAnalyzeResult`; `RECORDING_CAP_MS` \| \| `aggregate.ts` \| Generic `aggregateCpuHotspots(AggregatorInputRow[])` — burst windowing (`BURST_GAP_MS=500`), severity banding (RED >15%, YELLOW 3–15%, <3% dropped). iOS feeds it via a pre-pass; Android feeds it precomputed SQL rows \| \| `lifecycle.ts` \| `shutdownChild` SIGINT→SIGTERM→SIGKILL ladder + `waitForChildExit` (no PID-reuse hazard) \| \| `thread.ts` \| `normalizeThreadName` for both xctrace and Perfetto thread names \| \| `time-align.ts` \| New `buildPerfettoAnchor()` + `windowsOverlap()` alongside the existing iOS/React anchors \| \| `format.ts` \| `formatBytes` (moved out of the combined report) \| iOS was refactored to fit this shape: `ios-profiler/lifecycle.ts` and `types.ts` become thin re-exports; `pipeline/02-aggregate.ts` shrank from ~200 to ~60 lines (keeps its iOS-only dominant-function pre-pass, delegates grouping/bursting/ severity to the shared aggregator). No iOS behavior change — same outputs, shared internals. Session blueprint (`native-profiler-session.ts`): generic `capturePid` / `captureProcess` (was `xctracePid` / `xctraceProcess`), new `platform` and `androidOnDeviceTracePath` fields, and platform-branched dispose (iOS SIGKILLs xctrace; Android `kill -KILL`s perfetto, `rm`s the device trace, and disposes the warm WASM engine). Drill-down tools: `profiler-load.ts` learns to restore Android `.pftrace` sessions (with an optional metadata sidecar / `app_process` override); `profiler-stack-query.ts` splits into iOS and Android branches. Unlike iOS (which caches `parsedData` in memory), Android re-queries the `.pftrace` per drill-down — the warm engine is the cache, so each query costs only SQL execution time. ### 3. Packaging & release pipeline The WASM bundle's three third-party artifacts (`trace_processor.wasm`, `engine_bundle.node.js`, `engine.mjs`) + `LICENSE` are never committed (`.gitignore` lists the generated files but not the dir). They're built and sha256-checksummed in `argent-private` CI, published to `argent-private-releases`, and on our side: - `scripts/download-trace-processor.sh` (+88, new) — `gh release download` → verify tarball sha256 (FATAL on mismatch — the blobs are unsigned) → extract → re-verify via `SHA256SUMS` → assert all four present. - `scripts/pack-mcp.cjs` (+60, new) — single pack entry point: downloads simulator-server, native binaries, and the trace-processor bundle, then builds and `npm pack`s `@swmansion/argent`. `package.json`'s `pack` / `pack:mcp` now call this script. - `packages/argent/scripts/bundle-tools.cjs` (heavily reworked, +574/-...) — copies `assets/trace-processor/`, `assets/queries/`, and `argent.tracecfg.pbtxt` into `packages/argent/assets/`, and regenerates `bundled-meta.ts` to stamp the Perfetto version. - `publish.yml` / `publish-next.yml` — run the download step and then verify: all four WASM files present, the wasm magic-byte check (`head -c4 … \| grep $'\x00asm'`), plus the queries dir, TraceConfig, manifest, and Android helper APK. Asset reorg: `native-devtools-android` now ships `dist/`, `bin/`, and `assets/` (SQL queries, TraceConfig, manifest moved under `assets/`); `.gitignore` swaps the old per-file `packages/argent/manifest.json` entry for the whole `packages/argent/assets/` dir. --- ## Testing 15 new test files under `packages/tool-server/test/android-perfetto/` cover the engine and pipeline, including: - `dispatch.test.ts` — start/stop/analyze route to the right platform backend. - `aggregate-shared.test.ts` — the shared aggregator on both iOS and Android-shaped input. - `hang-folds-batched.test.ts` / `hang-fold.test.ts` — the batched per-hang SQL fold. - `hang-severity.test.ts` — jank-reason → RED/YELLOW classification. - `dispose.test.ts` / `dispose-warm-engine.test.ts` — session dispose kills the daemon, rms the device trace, and tears down the warm engine. - `stop-recovery.test.ts` — `profilingActive` is forced false even if `adb pull` fails (so a retry-start doesn't wedge). - `run-tp.test.ts` — `renderSqlTemplate` token rendering + the "unused substitution" guard. - `manifest-hint.test.ts` — zero-callstack detection emits the manifest hint. - `profiler-load.test.ts` — `.pftrace` session restore with/without the metadata sidecar. Plus `native-profiler-analyze-failure.test.ts` for the engine-unavailable banner, and updated iOS tests for the shared-lifecycle refactor. `@argent/native-devtools-android` gains its own vitest setup (`vitest.config.ts`, `tsconfig.test.json`, `test/errors.test.ts`). --- ## Notes & follow-ups - Manifest requirement. Perfetto's `linux.perf` data source needs `/proc/<pid>/mem` access, granted only to a debuggable or `<profileable android:shell="true"/>` app. A release build without either silently produces zero callstacks; the pipeline detects this and emits a manifest hint instead of an "All clear" report. - Version pinning lives in the `argent-private` submodule (`PERFETTO_VERSION` / `PERFETTO_WASM_TAG` / `LYNX_TRACE_PROCESSOR_VERSION`). Bumping is documented in `argent-private/docs/ANDROID_PERFETTO.md`. JS↔engine version skew is structurally impossible (version-agnostic decoder + vendored wasm). - `disposeWarmEngine` is exported but not wired into session-stop — warm engines are reclaimed only by the idle timer + LRU cap. Acceptable today (bounded memory); wire it into session dispose if explicit teardown is ever needed. - Submodule bump. This PR moves `packages/argent-private` to `22e55f90` — see the companion `argent-private` PR description, including its merge-hygiene note (that branch should pick up latest `argent-private` main before the pointer is finalized). ### Companion docs added in this branch - `utils/android-profiler/ANDROID_PROFILER_REFERENCE.md` — stack, capture, queries, manifest requirement. - `utils/android-profiler/PIPELINE_DESIGN.md` — the "why" (two-file pipeline, shared-aggregator hoist, re-query-don't-cache, batched fold). --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-09 16:06:24 +02:00
Paweł Fornagiel	a96f1f1578	feat: refactor android describe to not rely on uiautomator (#237 ) ## Summary - `formatDescribeTree` was branching on `source === "uiautomator"` to pick nested rendering; the newer `android-devtools` source fell through to flat mode and lost every node below depth 1. Extends the predicate to cover both Android sources so the helper-service path renders with the same hierarchy as the legacy `uiautomator dump` path. - Bumps `packages/argent-private` to pick up the persistent Android describe service that emits `source: "android-devtools"`, plus its build-and-sign CI workflow. - Wires the Android helper APK + `manifest.json` into the published `@swmansion/argent` package via `bundle-tools.cjs`, and adds `pack:mcp:local` / `build:android-binaries` so a local tarball can be produced without going through `download:native-binaries`. - Minor: tightens a couple of comments in `android-helper-install.ts` and `format-tree.ts`. This is connected to argent-private PR: https://github.com/software-mansion/argent-private/pull/8	2026-05-27 11:46:12 +02:00

Author

SHA1

Message

Date

filip131311

b232114422

docs: audit every comment in src against the code it describes (#931 )

Audits every comment in `src/` and `scripts/` against the code it
describes.

**462 files, 57 commits, net −8,194 lines.** Comment text only — the
whole branch is code-identical to `main`.

## Method

One subagent per file, strictly sequential. Scope was `src/` +
`scripts/` (459 files); three more files were added at the end because
they carried dead references of the same kind — two test headers citing
design docs that do not exist, and `publish-npm.yml` citing a retired
workflow. Each agent verified every comment — line, block, JSDoc, file
header, trailing — against the surrounding code and the rest of the
repo, following identifiers, paths, tool ids, config keys, env vars and
issue links to see whether they still exist and still behave as
described. Rules:

- **A false or misleading comment is deleted, not reworded.** If a claim
could not be confirmed by reading the source, it went. That is why the
deletion count is so much larger than the rewrite count.
- Survivors are cut to the shortest form carrying something the code
does not already say. Restatement, preamble, hedging, changelog prose
and ASCII banners are gone; the non-obvious *why* stays.
- Preserved byte-identical: license headers, pragmas and directives
(`@ts-*`, `eslint-disable`, shebangs, `/// <reference>`), JSDoc tag
tokens, everything inside a string or template literal, and the sole
comment inside an otherwise empty block (ESLint `no-empty` counts a
comment-bearing block as non-empty).

## Verification

Every file passed two independent gates before being recorded as done:

1. `comments-only` — the required check.
2. A second comment-stripping comparator with a proper mode stack,
written for this pass because `comments-only`'s flat scanner desyncs on
nested template literals and quote-bearing regex literals and then
reports comment lines as code changes. Two files hit that false FAIL
(`utils/android-profiler/pipeline/index.ts`,
`scripts/extract-tools.mjs`); in both the "changed code" it printed was
literally `//` lines, and the second checker confirmed the code was
byte-identical.

After the last file, all 462 changed files were re-checked against
`main` with the same comparator, rather than trusting any agent's
self-report. **459 code-identical; 2 are non-code (`.svg`, `.md`); 1
intentional.**

The intentional one is `packages/argent/scripts/bundle-tools.cjs`: the
changed template literal *is* the comment header of the file it
generates, `packages/native-devtools-android/src/bundled-meta.ts`.
Fixing only the generated file would have been reverted by the next
build, so the generator changed too — and it has been verified to
reproduce the committed generated file byte-for-byte.

## Representative false claims removed

Not wording nits — statements a reader would have acted on:

- **Reversed directions.** `proxyStart`'s JSDoc had the tunnel backwards
(it is a reverse tunnel: the host binds first and the simulator dials
in). A `paste()` doc had the pasteboard copy direction reversed.
- **Contradicted by the code below it.** A timeout budget multiplied by
three where the probes run concurrently — the same comment said so six
lines later. A "warn once" that warns on every call. A "binary search"
that is a linear scan.
- **Named things that do not exist.** A `vega-fast-cli` binary, a
`finish-recording.ts`, a `publish-next.yml` workflow, two
`profiler-react19-*.md` design docs, a `DebuggerTarget.ts`, a commit
hash git does not know, two tool ids, an `ensureEnv` cycle.
- **Wrong by construction.** "Welford accumulators" across four files
where the code keeps naive `n`/`sum`/`sumSq`; `sum`/`sumSq` documented
over `actualDuration` when reduce sums `selfDuration`; a strict-mode
halving written `n/2` where the code ceils; field docs listing enum
values the producers never emit.
- **Guarantees the code does not make.** A validation matrix claiming to
cover "EVERY tool" that skips flagless ones; a Pareto cutoff that
`slice(0, 20)` makes inert; an idempotence claim where the real rule is
at-or-ahead; a capability note describing a clean 400 the shape-based
device resolver can never produce.
- **Unverifiable assertions** about prebuilt binaries, external CLIs and
the cloud SDK — deleted rather than kept as folklore, since nothing in
the repo can confirm them.
- **Stale numbers**: invented Android tool versions, hard-coded tool
counts and description lengths that had drifted.

## Review

A Fable agent reviewed both halves adversarially for over-deletion,
misread code, `no-empty` hazards and byte-identity violations.
Second-half verdict: **SHIP**, with two one-line restores, both applied
in the final commit — the `npm view ""` rationale behind a blank-token
guard, and the note that `argent-mcp` keeps a copy of
`SECRET_PLACEHOLDER_MARKER` it cannot import.

## Code issues surfaced but deliberately not fixed

This pass changes comments only. Eight genuine findings are logged for a
follow-up:

1. `telemetry/src/consent.ts` — a non-ENOENT read error returns null and
falls through to the default-on path, so file errors *can* silently flip
telemetry on.
2. `chromium-server/navigation.ts` — `navigate()` is reachable from
`POST /api/navigate` with only a `typeof === "string"` check; open-url's
schema is a bare `z.string()`, so the "already validated by zod" premise
never held.
3. `http.ts` — `constantTimeEqual` returns early on a length mismatch,
so the auth token's length is observable.
4. `describe/index.ts:~114` — the ios-remote branch passes `{ isTvOs:
false }` unconditionally, so a remote tvOS simulator takes the iOS
ax-service path, though `isRemoteTvOsSimulator` exists and shake/paste
do use it.
5. `devices/boot-device.ts` — `-crash-report-mode never` is passed
unconditionally *and* appended again by the feature-detecting path, so
every emulator spawn passes it twice.
6. `react-profiler/pipeline/04-rank.ts` — `PARETO_THRESHOLD_PCT` is
dead: `slice(0, 20)` always wins.
7. `reaped-sessions.ts` — a user-facing hint string tells the agent that
`react-profiler-start { force: true }` disposes the debugger and
profiler session; it does not. Left byte-identical because it is a
string literal, not a comment.
8. `utils/simctl-backend.ts` — `localSimctl` is exported with no
importers anywhere.

## Docs

No documentation change is needed: this pass touches only source
comments, and no user-facing capability, tool, CLI flag, config key or
flow-file behaviour changed.

---------

Co-authored-by: filip131311 <f.kaminski2000@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>

2026-08-24 12:16:20 +02:00

Paweł Fornagiel

3ccd89f333

feat: add native android profiling (#275 )

# Android native profiling via in-process Perfetto WASM

Connected with:
https://github.com/software-mansion/argent-private/pull/16

## TL;DR

Adds **Android** to the native profiler (previously iOS-only) and does
it with a
brand-new engine: instead of shipping Perfetto's `trace_processor_shell`
native
binary and forking it per query, we now run **Perfetto's trace-processor
compiled
to WebAssembly, in-process** inside the tool-server.

- **One ~13 MB `trace_processor.wasm`** runs on every OS/arch — no
per-platform
  binary, no subprocess, fully offline.
- The native profiler is refactored from an iOS-only implementation into
a
**platform-dispatch facade** (`platforms/ios.ts` +
`platforms/android.ts`) over
a new **`profiler-shared/`** layer, so iOS and Android share the
aggregator,
  types, lifecycle, and time-alignment code.
- New **release/pack pipeline** fetches + sha256-verifies the WASM
bundle from
`argent-private-releases`, and bundles it (plus SQL queries and the
Perfetto
  TraceConfig) into the published `@swmansion/argent` package.

End-to-end on a 76 MB trace with 1013 jank rows: **6.3 s** total analyze
(the older per-hang-subprocess approach was projected at ~47 minutes and
never
finished within the tool-call deadline).

---

## TESTING MANUALLY

To test manually run the following packing command checked-out on this
branch:
```
npm run pack:mcp -- argent-@pFornagiel-native-android-devtools-perfetto
```

This will bundle the package with the new binaries. After this PR is
fixed and the binaries are resolved from the main release, the
`argent-@pFornagiel-native-android-devtools-perfetto` release can be
deleted.

## Why

Android needed native CPU/hang/memory profiling to reach parity with
iOS. The
natural engine is Perfetto's trace-processor, but the obvious ways to
ship it are
both bad:

- **`trace_processor_shell` binary** — per-platform (a mac-arm64 pack
shipped a
mac-arm64-only tarball), pinned an OS/arch matrix, and cost a fresh ~1.3
s
subprocess fork **per query**. With one subprocess forked per detected
hang, a
  large trace projected to ~47 minutes.
- **`get.perfetto.dev/trace_processor` Python launcher** — adds a
runtime Python 3
dependency, resolves "latest" at first run (defeats version pinning),
and makes
  the first analyze slow via a network fetch.

The WASM engine is one cross-platform artifact, in-process and
warm-cached: the
trace is parsed **once** and reused across the whole analyze run plus
every
drill-down query.

---

## The three big pieces

### 1. In-process Perfetto WASM engine

New package surface in `@argent/native-devtools-android`:

| File | Role |
| --- | --- |
| `src/wasm-trace-processor.ts` (+476) | The whole engine: boot, glue
patching, warm-cache, fault containment, chunked parse, cell decoding |
| `src/perfetto-engine.d.ts` (+49) | Hand-written types for the
version-agnostic RPC decoder (imported by path, so tsc can't see its own
`.d.ts`) |
| `src/errors.ts` (+68) | `TraceProcessorUnavailableError`
(`wasm_load_failed` / `wasm_path_invalid`) |
| `src/bundled-meta.ts` (+12) | `PERFETTO_VERSION` stamp, regenerated at
pack time |
| `src/index.ts` | Re-exports + package-root accessors
(`traceProcessorQueriesDir`, `traceConfigPath`) |

Public surface: `resolveTraceProcessorAssets()`,
`queryWarm<Row>(tracePath, sql)`,
`ensureTraceProcessorReady(tracePath)`, `disposeWarmEngine(tracePath)`.

Highlights of how it works:

- **Boots Google's prebuilt web/worker wasm under Node.** Installs a
minimal
worker scope (`self`, `WorkerGlobalScope`, `location`), reads the wasm
bytes,
patches the Emscripten glue (`useMemory64=false`, hand the bytes in
directly,
expose the bridge), and drives it over a `MessageChannel` + `EngineBase`
subclass. Glue patches are applied **by unique string anchors that throw
if they
  don't match** — the canary for glue-format drift on a Perfetto bump.
- **Warm-engine cache, one per trace path.** LRU with `MAX_WARM_ENGINES
= 3`
(bounds resident memory at ~26–76 MB trace + wasm heap each), 5-min idle
dispose
  (timer `unref()`'d), no failed-promise caching.
- **Fault containment (`fatal` promise).** `EngineBase.fail()` throws
synchronously
inside the `MessagePort.onmessage` handler (e.g. an RPC framing error);
left
unhandled it becomes a Node `uncaughtException` and would **kill the
entire
tool-server**, wiping every device + profiler session. Each engine
exposes a
`fatal: Promise<never>`; the handler try/catches, rejects `fatal`,
evicts the
engine, and every load/query is `Promise.race`'d against it — so a
contained
  fault is a clean rejection and the server stays up.
- **Chunked trace parse.** Perfetto's RPC ring buffer rejects any frame
> 64 MiB,
  so the trace is fed in 32 MiB chunks before `notifyEof()`.

### 2. Android native profiler + the shared/dispatch refactor

**Capture** (`utils/android-profiler/capture.ts`, `detect-app.ts`):
spawns
`adb shell perfetto --txt -c - --background-wait -o
/data/.../argent-<ts>.pftrace`,
piping the TraceConfig on **stdin** (SELinux denies `shell:s0` writes
under
`/data/misc/perfetto-traces/`, so the config can't be staged as a file),
waits for
the PID on stdout, then on stop sends `kill -TERM`, polls `/proc/<pid>`,
and
`adb pull`s the trace.

**Pipeline** (`utils/android-profiler/pipeline/`): PerfettoSQL does the
parsing and
aggregation, so the Android pipeline is intentionally **two files, not
iOS's four**.
The per-hang annotation pass is **batched** — `runBatchedHangFolds`
inlines all
hang windows as a `VALUES` table into one SQL script (two derived views
unioned in
a single terminal SELECT, because the engine returns only the final
statement's
rows) and runs it once against the warm engine.

**The platform-dispatch refactor** —
`native-profiler-start/stop/analyze` shrank
from monolithic iOS files (~160–330 lines each) into **thin routers**
(~60–110
lines) that branch on `api.platform`:

```
native-profiler-start ─┬─ ios?    → platforms/ios.ts      (xctrace)
                        └─ android → platforms/android.ts  (adb perfetto)
```

A new **`utils/profiler-shared/`** layer holds everything both backends
use:

| File | What was hoisted |
| --- | --- |
| `types.ts` | Unified `Bottleneck = CpuHotspot \| UiHang \| MemoryLeak
\| MemoryRssGrowth`, each now carrying a `platform` field; shared
`NativeProfilerAnalyzeResult`; `RECORDING_CAP_MS` |
| `aggregate.ts` | Generic `aggregateCpuHotspots(AggregatorInputRow[])`
— burst windowing (`BURST_GAP_MS=500`), severity banding (RED >15%,
YELLOW 3–15%, <3% dropped). iOS feeds it via a pre-pass; Android feeds
it precomputed SQL rows |
| `lifecycle.ts` | `shutdownChild` SIGINT→SIGTERM→SIGKILL ladder +
`waitForChildExit` (no PID-reuse hazard) |
| `thread.ts` | `normalizeThreadName` for both xctrace and Perfetto
thread names |
| `time-align.ts` | New `buildPerfettoAnchor()` + `windowsOverlap()`
alongside the existing iOS/React anchors |
| `format.ts` | `formatBytes` (moved out of the combined report) |

iOS was refactored to fit this shape: `ios-profiler/lifecycle.ts` and
`types.ts`
become thin re-exports; `pipeline/02-aggregate.ts` shrank from ~200 to
~60 lines
(keeps its iOS-only dominant-function pre-pass, delegates
grouping/bursting/
severity to the shared aggregator). **No iOS behavior change** — same
outputs,
shared internals.

**Session blueprint** (`native-profiler-session.ts`): generic
`capturePid` /
`captureProcess` (was `xctracePid` / `xctraceProcess`), new `platform`
and
`androidOnDeviceTracePath` fields, and **platform-branched dispose**
(iOS SIGKILLs
xctrace; Android `kill -KILL`s perfetto, `rm`s the device trace, and
disposes the
warm WASM engine).

**Drill-down tools**: `profiler-load.ts` learns to restore Android
`.pftrace`
sessions (with an optional metadata sidecar / `app_process` override);
`profiler-stack-query.ts` splits into iOS and Android branches. Unlike
iOS (which
caches `parsedData` in memory), Android **re-queries the `.pftrace`**
per
drill-down — the warm engine is the cache, so each query costs only SQL
execution
time.

### 3. Packaging & release pipeline

The WASM bundle's three third-party artifacts (`trace_processor.wasm`,
`engine_bundle.node.js`, `engine.mjs`) + `LICENSE` are **never
committed**
(`.gitignore` lists the generated files but not the dir). They're built
and
sha256-checksummed in `argent-private` CI, published to
`argent-private-releases`, and on our side:

- **`scripts/download-trace-processor.sh`** (+88, new) — `gh release
download` →
verify tarball sha256 (**FATAL on mismatch** — the blobs are unsigned) →
extract
  → re-verify via `SHA256SUMS` → assert all four present.
- **`scripts/pack-mcp.cjs`** (+60, new) — single pack entry point:
downloads
simulator-server, native binaries, **and** the trace-processor bundle,
then
builds and `npm pack`s `@swmansion/argent`. `package.json`'s `pack` /
`pack:mcp`
  now call this script.
- **`packages/argent/scripts/bundle-tools.cjs`** (heavily reworked,
+574/-...) —
copies `assets/trace-processor/`, `assets/queries/`, and
`argent.tracecfg.pbtxt`
into `packages/argent/assets/`, and regenerates `bundled-meta.ts` to
stamp the
  Perfetto version.
- **`publish.yml` / `publish-next.yml`** — run the download step and
then **verify**:
  all four WASM files present, the wasm magic-byte check
(`head -c4 … | grep $'\x00asm'`), plus the queries dir, TraceConfig,
manifest,
  and Android helper APK.

Asset reorg: `native-devtools-android` now ships `dist/`, `bin/`, and
`assets/`
(SQL queries, TraceConfig, manifest moved under `assets/`); `.gitignore`
swaps the
old per-file `packages/argent/manifest.json` entry for the whole
`packages/argent/assets/` dir.

---

## Testing

15 new test files under `packages/tool-server/test/android-perfetto/`
cover the
engine and pipeline, including:

- `dispatch.test.ts` — start/stop/analyze route to the right platform
backend.
- `aggregate-shared.test.ts` — the shared aggregator on both iOS and
Android-shaped input.
- `hang-folds-batched.test.ts` / `hang-fold.test.ts` — the batched
per-hang SQL fold.
- `hang-severity.test.ts` — jank-reason → RED/YELLOW classification.
- `dispose.test.ts` / `dispose-warm-engine.test.ts` — session dispose
kills the daemon, rms the device trace, and tears down the warm engine.
- `stop-recovery.test.ts` — `profilingActive` is forced false even if
`adb pull` fails (so a retry-start doesn't wedge).
- `run-tp.test.ts` — `renderSqlTemplate` token rendering + the "unused
substitution" guard.
- `manifest-hint.test.ts` — zero-callstack detection emits the manifest
hint.
- `profiler-load.test.ts` — `.pftrace` session restore with/without the
metadata sidecar.

Plus `native-profiler-analyze-failure.test.ts` for the
engine-unavailable banner,
and updated iOS tests for the shared-lifecycle refactor.
`@argent/native-devtools-android` gains its own vitest setup
(`vitest.config.ts`, `tsconfig.test.json`, `test/errors.test.ts`).

---

## Notes & follow-ups

- **Manifest requirement.** Perfetto's `linux.perf` data source needs
`/proc/<pid>/mem` access, granted only to a **debuggable** or
`<profileable
android:shell="true"/>` app. A release build without either silently
produces
zero callstacks; the pipeline detects this and emits a **manifest hint**
instead
  of an "All clear" report.
- **Version pinning lives in the `argent-private` submodule**
(`PERFETTO_VERSION`
/ `PERFETTO_WASM_TAG` / `LYNX_TRACE_PROCESSOR_VERSION`). Bumping is
documented in
`argent-private/docs/ANDROID_PERFETTO.md`. JS↔engine version skew is
structurally
  impossible (version-agnostic decoder + vendored wasm).
- **`disposeWarmEngine` is exported but not wired into session-stop** —
warm
engines are reclaimed only by the idle timer + LRU cap. Acceptable today
(bounded
memory); wire it into session dispose if explicit teardown is ever
needed.
- **Submodule bump.** This PR moves `packages/argent-private` to
`22e55f90` — see the companion `argent-private` PR description,
including its
merge-hygiene note (that branch should pick up latest `argent-private`
main
  before the pointer is finalized).

### Companion docs added in this branch
- `utils/android-profiler/ANDROID_PROFILER_REFERENCE.md` — stack,
capture, queries, manifest requirement.
- `utils/android-profiler/PIPELINE_DESIGN.md` — the "why" (two-file
pipeline, shared-aggregator hoist, re-query-don't-cache, batched fold).

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

2026-06-09 16:06:24 +02:00

Paweł Fornagiel

a96f1f1578

feat: refactor android describe to not rely on uiautomator (#237 )

## Summary

- `formatDescribeTree` was branching on `source === "uiautomator"` to
pick nested rendering; the newer `android-devtools` source fell through
to flat mode and lost every node below depth 1. Extends the predicate to
cover both Android sources so the helper-service path renders with the
same hierarchy as the legacy `uiautomator dump` path.
- Bumps `packages/argent-private` to pick up the persistent Android
describe service that emits `source: "android-devtools"`, plus its
build-and-sign CI workflow.
- Wires the Android helper APK + `manifest.json` into the published
`@swmansion/argent` package via `bundle-tools.cjs`, and adds
`pack:mcp:local` / `build:android-binaries` so a local tarball can be
produced without going through `download:native-binaries`.
- Minor: tightens a couple of comments in `android-helper-install.ts`
and `format-tree.ts`.

This is connected to argent-private PR:
https://github.com/software-mansion/argent-private/pull/8

2026-05-27 11:46:12 +02:00

3 Commits