software-mansion__argent

skills/software-mansion__argent

Fork 0

mirror of https://github.com/software-mansion/argent.git synced 2026-09-14 19:27:14 +08:00

Commit Graph

Author	SHA1	Message	Date
filip131311	5d76ed4d66	feat: tag artifacts with a semantic kind at registration (#564 ) Fresh, from-current-main take on the artifact-kind idea from #398 / #466. Knowing what an artifact represents (not just its MIME type) is what a UI needs to group and name artifacts (e.g. in EAS); this PR delivers exactly that — the tagged handle — without the declared-output-map machinery. ## Summary - Adds a closed `ArtifactKind` union to the artifact wire contract; `kind` is carried on `ArtifactHandle`, `ArtifactEntry`, and the `/artifacts` inventory items. - `ArtifactStore.register` now takes a single `{ hostPath, kind, ... }` options object, so the compiler forces every producer — current and future — to declare what it is emitting. - Tags all producers on main today, including the ones #398/#466 predate: screenshot (all four capture paths), screenshot-diff, flow snapshot baselines/diffs (`screenshot` / `screenshot-diff-context`), screen recordings (new `screen-recording` kind), and the native/React profiler exports and reports. - Types the native-profiler export keys at their source (`IosExportKey` from the `EXPORTS` table, `AndroidExportKey`) and maps them through a kind table that is total over both unions — a newly added export key fails to compile until classified, replacing the previous stringly-keyed loop (and the `?? "native-profile-trace"` silent-default this design would otherwise need). - Client-side (`@argent/tools-client`), the mirrored handle type keeps `kind` optional and widened (`ArtifactKind \| (string & {})`): an older, already-running tool-server sends no kind, and a newer server may send kinds this client build predates. Consumers treat it as advisory metadata; unknown kinds are opaque, never an error. ## Differences from #398 / #466 (deliberate) - No `@argent/artifacts` package, no third `ToolDefinition` generic, no per-tool output-name registrar, no runtime throw for “undeclared output” — the kind is plain data at the registration call site, checked at compile time where it can be, and a wrong label can never fail a tool at runtime. If a pre-invocation `artifacts` declaration in tool-listing metadata is ever needed (an actual EAS consumer), it can be layered on later without changing this wire contract. - The two `ArtifactKind` definitions (registry / tools-client) stay deliberately separate — same as the existing `ARTIFACT_MARKER` must-match convention — because the client-side type is intentionally looser (optional + open) than the server-side one for version skew. - Extending the union stays backward-compatible on the wire. ## 2026-08-24 update Merged current main (0.22.0) into the branch. The flow-visual rewrite that landed since (cropOn crops, per-app snapshot-key dedupe, diff scratch-dir hygiene) is preserved verbatim; its five registration sites now carry kinds (`screenshot` for baseline/current, `screenshot-diff-context` for the annotated diff — the cropped `current` included). No producer added since the PR was opened needed a new kind: `paste`, `shake`, and the other new tools emit no file artifacts, and every register call site in the tree now goes through the typed options object. No docs update is needed: the docs site (added since this PR) does not document the artifact handle wire contract; `kind` is advisory wire metadata with no user-facing surface. ## Validation At the merge with main (0.22.0), 2026-08-24: - `npm run build` (full `tsc --build`) — clean - `npm run typecheck:tests` in registry, tools-client, tool-server, mcp, cli — clean - Full suites green: registry 90, tools-client 191, tool-server 4495 (+1 skipped), mcp 80, cli 472 - `prettier --check .`, `eslint . --max-warnings 0`, and the knip gate — all pass 🤖 Generated with [Claude Code](https://claude.com/claude-code)	2026-08-24 12:09:51 +02:00
filip131311	20cb452fd4	feat(remote): artifact transport for files produced by a remote tool-server (#278 ) ## Why Files produced on the tool-server host (screenshots, profiler exports) were surfaced to the agent as host paths / `127.0.0.1` media URLs, which the MCP client (`argent-mcp`) then fetched a second time to get the bytes. When the tool-server runs remotely, that localhost URL resolves to the client's own machine — nothing there — so screenshots fail with `(Screenshot unavailable: no valid PNG …)`. Profiler exports have the same shape: paths that only exist on the remote host. ## What A reusable artifact boundary layer so any file-producing tool works regardless of where the tool-server runs: Foundation - tool-server (`src/artifacts.ts`, `http.ts`): in-process artifact registry + `GET /artifacts/:id` streaming route. `registerArtifact(hostPath)` returns a wire handle (`{ __argentArtifact, id, filename, mimeType, size }`) instead of a raw path. - argent-mcp (`src/artifacts.ts`, `content.ts`, `mcp-server.ts`): a client materializer deep-walks tool results for handles, downloads each over the remote-aware `TOOLS_URL`, writes it into a structured cache, and rewrites the result to the local path (+ inline image rendering). Cache layout (override root with `ARGENT_ARTIFACTS_DIR`): ``` ~/.argent/artifacts/<project>/<session>/<device>/<filename> ``` - `project` = `basename(cwd)` + short hash of the full path (readable + collision-safe) - `session` = minted once per argent-mcp process (groups a run, easy GC) - `device` = udid/serial when device-scoped, omitted otherwise Consumers wired - screenshot → returns an image artifact; rendered inline from downloaded bytes with no second `127.0.0.1` fetch. Fixes `auto-screenshot` and `flow-execute` screenshots via the same path. - native-profiler-stop → `exportedFiles` (XML) become downloadable artifacts materialized client-side. ## Deferred (follow-ups) - `.trace` bundle — it's an Instruments bundle (a directory); streaming needs archiving (no archiver dep yet). `native-profiler-stop` now returns a `traceFileNote` explaining it stays on the profiling host. - Project source on remote — `react-profiler-component-source` / `react-profiler-analyze` read the user's source tree on the tool-server host. The profiler analysis chains already work remotely (server-internal); tracked separately. ## Compatibility Legacy `{ url, path }` screenshot rendering is retained as a fallback when no artifact context is present (older tool-servers / unit tests). ## Tests - tool-server: artifact registry (infer/override), `GET /artifacts/:id` (200/404/410), updated `stop-recovery` assertions. - argent-mcp: materializer (download/rewrite/nested/failure/device-scoping), artifact-aware `toMcpContent`. - argent-mcp typecheck + full suite green; tool-server artifact/profiler tests green. (Note: a full local tool-server typecheck trips a pre-existing zod-v4-vs-v3 mismatch in `run-sequence` that's unrelated to this change and resolves on this branch's own lockfile.) ## Notes Routes around the `argent-private` native simulator-server binary (which emits the `127.0.0.1` media URL) rather than changing it — no submodule change. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: pFornagiel <pawel.fornagiel2@gmail.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: Paweł Fornagiel <85541429+pFornagiel@users.noreply.github.com>	2026-06-10 12:04:30 +02:00
Paweł Fornagiel	3ccd89f333	feat: add native android profiling (#275 ) # Android native profiling via in-process Perfetto WASM Connected with: https://github.com/software-mansion/argent-private/pull/16 ## TL;DR Adds Android to the native profiler (previously iOS-only) and does it with a brand-new engine: instead of shipping Perfetto's `trace_processor_shell` native binary and forking it per query, we now run Perfetto's trace-processor compiled to WebAssembly, in-process inside the tool-server. - One ~13 MB `trace_processor.wasm` runs on every OS/arch — no per-platform binary, no subprocess, fully offline. - The native profiler is refactored from an iOS-only implementation into a platform-dispatch facade (`platforms/ios.ts` + `platforms/android.ts`) over a new `profiler-shared/` layer, so iOS and Android share the aggregator, types, lifecycle, and time-alignment code. - New release/pack pipeline fetches + sha256-verifies the WASM bundle from `argent-private-releases`, and bundles it (plus SQL queries and the Perfetto TraceConfig) into the published `@swmansion/argent` package. End-to-end on a 76 MB trace with 1013 jank rows: 6.3 s total analyze (the older per-hang-subprocess approach was projected at ~47 minutes and never finished within the tool-call deadline). --- ## TESTING MANUALLY To test manually run the following packing command checked-out on this branch: ``` npm run pack:mcp -- argent-@pFornagiel-native-android-devtools-perfetto ``` This will bundle the package with the new binaries. After this PR is fixed and the binaries are resolved from the main release, the `argent-@pFornagiel-native-android-devtools-perfetto` release can be deleted. ## Why Android needed native CPU/hang/memory profiling to reach parity with iOS. The natural engine is Perfetto's trace-processor, but the obvious ways to ship it are both bad: - `trace_processor_shell` binary — per-platform (a mac-arm64 pack shipped a mac-arm64-only tarball), pinned an OS/arch matrix, and cost a fresh ~1.3 s subprocess fork per query. With one subprocess forked per detected hang, a large trace projected to ~47 minutes. - `get.perfetto.dev/trace_processor` Python launcher — adds a runtime Python 3 dependency, resolves "latest" at first run (defeats version pinning), and makes the first analyze slow via a network fetch. The WASM engine is one cross-platform artifact, in-process and warm-cached: the trace is parsed once and reused across the whole analyze run plus every drill-down query. --- ## The three big pieces ### 1. In-process Perfetto WASM engine New package surface in `@argent/native-devtools-android`: \| File \| Role \| \| --- \| --- \| \| `src/wasm-trace-processor.ts` (+476) \| The whole engine: boot, glue patching, warm-cache, fault containment, chunked parse, cell decoding \| \| `src/perfetto-engine.d.ts` (+49) \| Hand-written types for the version-agnostic RPC decoder (imported by path, so tsc can't see its own `.d.ts`) \| \| `src/errors.ts` (+68) \| `TraceProcessorUnavailableError` (`wasm_load_failed` / `wasm_path_invalid`) \| \| `src/bundled-meta.ts` (+12) \| `PERFETTO_VERSION` stamp, regenerated at pack time \| \| `src/index.ts` \| Re-exports + package-root accessors (`traceProcessorQueriesDir`, `traceConfigPath`) \| Public surface: `resolveTraceProcessorAssets()`, `queryWarm<Row>(tracePath, sql)`, `ensureTraceProcessorReady(tracePath)`, `disposeWarmEngine(tracePath)`. Highlights of how it works: - Boots Google's prebuilt web/worker wasm under Node. Installs a minimal worker scope (`self`, `WorkerGlobalScope`, `location`), reads the wasm bytes, patches the Emscripten glue (`useMemory64=false`, hand the bytes in directly, expose the bridge), and drives it over a `MessageChannel` + `EngineBase` subclass. Glue patches are applied by unique string anchors that throw if they don't match — the canary for glue-format drift on a Perfetto bump. - Warm-engine cache, one per trace path. LRU with `MAX_WARM_ENGINES = 3` (bounds resident memory at ~26–76 MB trace + wasm heap each), 5-min idle dispose (timer `unref()`'d), no failed-promise caching. - Fault containment (`fatal` promise). `EngineBase.fail()` throws synchronously inside the `MessagePort.onmessage` handler (e.g. an RPC framing error); left unhandled it becomes a Node `uncaughtException` and would kill the entire tool-server, wiping every device + profiler session. Each engine exposes a `fatal: Promise<never>`; the handler try/catches, rejects `fatal`, evicts the engine, and every load/query is `Promise.race`'d against it — so a contained fault is a clean rejection and the server stays up. - Chunked trace parse. Perfetto's RPC ring buffer rejects any frame > 64 MiB, so the trace is fed in 32 MiB chunks before `notifyEof()`. ### 2. Android native profiler + the shared/dispatch refactor Capture (`utils/android-profiler/capture.ts`, `detect-app.ts`): spawns `adb shell perfetto --txt -c - --background-wait -o /data/.../argent-<ts>.pftrace`, piping the TraceConfig on stdin (SELinux denies `shell:s0` writes under `/data/misc/perfetto-traces/`, so the config can't be staged as a file), waits for the PID on stdout, then on stop sends `kill -TERM`, polls `/proc/<pid>`, and `adb pull`s the trace. Pipeline (`utils/android-profiler/pipeline/`): PerfettoSQL does the parsing and aggregation, so the Android pipeline is intentionally two files, not iOS's four. The per-hang annotation pass is batched — `runBatchedHangFolds` inlines all hang windows as a `VALUES` table into one SQL script (two derived views unioned in a single terminal SELECT, because the engine returns only the final statement's rows) and runs it once against the warm engine. The platform-dispatch refactor — `native-profiler-start/stop/analyze` shrank from monolithic iOS files (~160–330 lines each) into thin routers (~60–110 lines) that branch on `api.platform`: ``` native-profiler-start ─┬─ ios? → platforms/ios.ts (xctrace) └─ android → platforms/android.ts (adb perfetto) ``` A new `utils/profiler-shared/` layer holds everything both backends use: \| File \| What was hoisted \| \| --- \| --- \| \| `types.ts` \| Unified `Bottleneck = CpuHotspot \\| UiHang \\| MemoryLeak \\| MemoryRssGrowth`, each now carrying a `platform` field; shared `NativeProfilerAnalyzeResult`; `RECORDING_CAP_MS` \| \| `aggregate.ts` \| Generic `aggregateCpuHotspots(AggregatorInputRow[])` — burst windowing (`BURST_GAP_MS=500`), severity banding (RED >15%, YELLOW 3–15%, <3% dropped). iOS feeds it via a pre-pass; Android feeds it precomputed SQL rows \| \| `lifecycle.ts` \| `shutdownChild` SIGINT→SIGTERM→SIGKILL ladder + `waitForChildExit` (no PID-reuse hazard) \| \| `thread.ts` \| `normalizeThreadName` for both xctrace and Perfetto thread names \| \| `time-align.ts` \| New `buildPerfettoAnchor()` + `windowsOverlap()` alongside the existing iOS/React anchors \| \| `format.ts` \| `formatBytes` (moved out of the combined report) \| iOS was refactored to fit this shape: `ios-profiler/lifecycle.ts` and `types.ts` become thin re-exports; `pipeline/02-aggregate.ts` shrank from ~200 to ~60 lines (keeps its iOS-only dominant-function pre-pass, delegates grouping/bursting/ severity to the shared aggregator). No iOS behavior change — same outputs, shared internals. Session blueprint (`native-profiler-session.ts`): generic `capturePid` / `captureProcess` (was `xctracePid` / `xctraceProcess`), new `platform` and `androidOnDeviceTracePath` fields, and platform-branched dispose (iOS SIGKILLs xctrace; Android `kill -KILL`s perfetto, `rm`s the device trace, and disposes the warm WASM engine). Drill-down tools: `profiler-load.ts` learns to restore Android `.pftrace` sessions (with an optional metadata sidecar / `app_process` override); `profiler-stack-query.ts` splits into iOS and Android branches. Unlike iOS (which caches `parsedData` in memory), Android re-queries the `.pftrace` per drill-down — the warm engine is the cache, so each query costs only SQL execution time. ### 3. Packaging & release pipeline The WASM bundle's three third-party artifacts (`trace_processor.wasm`, `engine_bundle.node.js`, `engine.mjs`) + `LICENSE` are never committed (`.gitignore` lists the generated files but not the dir). They're built and sha256-checksummed in `argent-private` CI, published to `argent-private-releases`, and on our side: - `scripts/download-trace-processor.sh` (+88, new) — `gh release download` → verify tarball sha256 (FATAL on mismatch — the blobs are unsigned) → extract → re-verify via `SHA256SUMS` → assert all four present. - `scripts/pack-mcp.cjs` (+60, new) — single pack entry point: downloads simulator-server, native binaries, and the trace-processor bundle, then builds and `npm pack`s `@swmansion/argent`. `package.json`'s `pack` / `pack:mcp` now call this script. - `packages/argent/scripts/bundle-tools.cjs` (heavily reworked, +574/-...) — copies `assets/trace-processor/`, `assets/queries/`, and `argent.tracecfg.pbtxt` into `packages/argent/assets/`, and regenerates `bundled-meta.ts` to stamp the Perfetto version. - `publish.yml` / `publish-next.yml` — run the download step and then verify: all four WASM files present, the wasm magic-byte check (`head -c4 … \| grep $'\x00asm'`), plus the queries dir, TraceConfig, manifest, and Android helper APK. Asset reorg: `native-devtools-android` now ships `dist/`, `bin/`, and `assets/` (SQL queries, TraceConfig, manifest moved under `assets/`); `.gitignore` swaps the old per-file `packages/argent/manifest.json` entry for the whole `packages/argent/assets/` dir. --- ## Testing 15 new test files under `packages/tool-server/test/android-perfetto/` cover the engine and pipeline, including: - `dispatch.test.ts` — start/stop/analyze route to the right platform backend. - `aggregate-shared.test.ts` — the shared aggregator on both iOS and Android-shaped input. - `hang-folds-batched.test.ts` / `hang-fold.test.ts` — the batched per-hang SQL fold. - `hang-severity.test.ts` — jank-reason → RED/YELLOW classification. - `dispose.test.ts` / `dispose-warm-engine.test.ts` — session dispose kills the daemon, rms the device trace, and tears down the warm engine. - `stop-recovery.test.ts` — `profilingActive` is forced false even if `adb pull` fails (so a retry-start doesn't wedge). - `run-tp.test.ts` — `renderSqlTemplate` token rendering + the "unused substitution" guard. - `manifest-hint.test.ts` — zero-callstack detection emits the manifest hint. - `profiler-load.test.ts` — `.pftrace` session restore with/without the metadata sidecar. Plus `native-profiler-analyze-failure.test.ts` for the engine-unavailable banner, and updated iOS tests for the shared-lifecycle refactor. `@argent/native-devtools-android` gains its own vitest setup (`vitest.config.ts`, `tsconfig.test.json`, `test/errors.test.ts`). --- ## Notes & follow-ups - Manifest requirement. Perfetto's `linux.perf` data source needs `/proc/<pid>/mem` access, granted only to a debuggable or `<profileable android:shell="true"/>` app. A release build without either silently produces zero callstacks; the pipeline detects this and emits a manifest hint instead of an "All clear" report. - Version pinning lives in the `argent-private` submodule (`PERFETTO_VERSION` / `PERFETTO_WASM_TAG` / `LYNX_TRACE_PROCESSOR_VERSION`). Bumping is documented in `argent-private/docs/ANDROID_PERFETTO.md`. JS↔engine version skew is structurally impossible (version-agnostic decoder + vendored wasm). - `disposeWarmEngine` is exported but not wired into session-stop — warm engines are reclaimed only by the idle timer + LRU cap. Acceptable today (bounded memory); wire it into session dispose if explicit teardown is ever needed. - Submodule bump. This PR moves `packages/argent-private` to `22e55f90` — see the companion `argent-private` PR description, including its merge-hygiene note (that branch should pick up latest `argent-private` main before the pointer is finalized). ### Companion docs added in this branch - `utils/android-profiler/ANDROID_PROFILER_REFERENCE.md` — stack, capture, queries, manifest requirement. - `utils/android-profiler/PIPELINE_DESIGN.md` — the "why" (two-file pipeline, shared-aggregator hoist, re-query-don't-cache, batched fold). --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-09 16:06:24 +02:00

Author

SHA1

Message

Date

filip131311

5d76ed4d66

feat: tag artifacts with a semantic kind at registration (#564 )

Fresh, from-current-main take on the artifact-kind idea from #398 /
#466. Knowing what an artifact *represents* (not just its MIME type) is
what a UI needs to group and name artifacts (e.g. in EAS); this PR
delivers exactly that — the tagged handle — without the
declared-output-map machinery.

## Summary

- Adds a closed `ArtifactKind` union to the artifact wire contract;
`kind` is carried on `ArtifactHandle`, `ArtifactEntry`, and the
`/artifacts` inventory items.
- `ArtifactStore.register` now takes a single `{ hostPath, kind, ... }`
options object, so the compiler forces every producer — current and
future — to declare what it is emitting.
- Tags **all** producers on main today, including the ones #398/#466
predate: screenshot (all four capture paths), screenshot-diff, **flow
snapshot baselines/diffs** (`screenshot` / `screenshot-diff-context`),
**screen recordings** (new `screen-recording` kind), and the
native/React profiler exports and reports.
- Types the native-profiler export keys at their source (`IosExportKey`
from the `EXPORTS` table, `AndroidExportKey`) and maps them through a
kind table that is **total over both unions** — a newly added export key
fails to compile until classified, replacing the previous stringly-keyed
loop (and the `?? "native-profile-trace"` silent-default this design
would otherwise need).
- Client-side (`@argent/tools-client`), the mirrored handle type keeps
`kind` **optional and widened** (`ArtifactKind | (string & {})`): an
older, already-running tool-server sends no kind, and a newer server may
send kinds this client build predates. Consumers treat it as advisory
metadata; unknown kinds are opaque, never an error.

## Differences from #398 / #466 (deliberate)

- No `@argent/artifacts` package, no third `ToolDefinition` generic, no
per-tool output-name registrar, no runtime throw for “undeclared output”
— the kind is plain data at the registration call site, checked at
compile time where it can be, and a wrong label can never fail a tool at
runtime. If a pre-invocation `artifacts` declaration in tool-listing
metadata is ever needed (an actual EAS consumer), it can be layered on
later without changing this wire contract.
- The two `ArtifactKind` definitions (registry / tools-client) stay
deliberately separate — same as the existing `ARTIFACT_MARKER`
must-match convention — because the client-side type is intentionally
*looser* (optional + open) than the server-side one for version skew.
- Extending the union stays backward-compatible on the wire.

## 2026-08-24 update

Merged current main (0.22.0) into the branch. The flow-visual rewrite
that landed since (cropOn crops, per-app snapshot-key dedupe, diff
scratch-dir hygiene) is preserved verbatim; its five registration sites
now carry kinds (`screenshot` for baseline/current,
`screenshot-diff-context` for the annotated diff — the cropped `current`
included). No producer added since the PR was opened needed a new kind:
`paste`, `shake`, and the other new tools emit no file artifacts, and
every register call site in the tree now goes through the typed options
object.

No docs update is needed: the docs site (added since this PR) does not
document the artifact handle wire contract; `kind` is advisory wire
metadata with no user-facing surface.

## Validation

At the merge with main (0.22.0), 2026-08-24:

- `npm run build` (full `tsc --build`) — clean
- `npm run typecheck:tests` in registry, tools-client, tool-server, mcp,
cli — clean
- Full suites green: registry 90, tools-client 191, tool-server 4495 (+1
skipped), mcp 80, cli 472
- `prettier --check .`, `eslint . --max-warnings 0`, and the knip gate —
all pass

🤖 Generated with [Claude Code](https://claude.com/claude-code)

2026-08-24 12:09:51 +02:00

filip131311

20cb452fd4

feat(remote): artifact transport for files produced by a remote tool-server (#278 )

## Why

Files produced on the **tool-server host** (screenshots, profiler
exports) were surfaced to the agent as host paths / `127.0.0.1` media
URLs, which the MCP client (`argent-mcp`) then fetched a *second* time
to get the bytes. When the tool-server runs **remotely**, that localhost
URL resolves to the client's own machine — nothing there — so
screenshots fail with `(Screenshot unavailable: no valid PNG …)`.
Profiler exports have the same shape: paths that only exist on the
remote host.

## What

A reusable **artifact boundary layer** so any file-producing tool works
regardless of where the tool-server runs:

**Foundation**
- **tool-server** (`src/artifacts.ts`, `http.ts`): in-process artifact
registry + `GET /artifacts/:id` streaming route.
`registerArtifact(hostPath)` returns a wire handle (`{ __argentArtifact,
id, filename, mimeType, size }`) instead of a raw path.
- **argent-mcp** (`src/artifacts.ts`, `content.ts`, `mcp-server.ts`): a
client materializer deep-walks tool results for handles, downloads each
over the remote-aware `TOOLS_URL`, writes it into a **structured
cache**, and rewrites the result to the **local** path (+ inline image
rendering).

**Cache layout** (override root with `ARGENT_ARTIFACTS_DIR`):
```
~/.argent/artifacts/<project>/<session>/<device>/<filename>
```
- `project` = `basename(cwd)` + short hash of the full path (readable +
collision-safe)
- `session` = minted once per argent-mcp process (groups a run, easy GC)
- `device` = udid/serial when device-scoped, omitted otherwise

**Consumers wired**
- **screenshot** → returns an image artifact; rendered inline from
downloaded bytes with **no second `127.0.0.1` fetch**. Fixes
`auto-screenshot` and `flow-execute` screenshots via the same path.
- **native-profiler-stop** → `exportedFiles` (XML) become downloadable
artifacts materialized client-side.

## Deferred (follow-ups)
- **`.trace` bundle** — it's an Instruments bundle (a directory);
streaming needs archiving (no archiver dep yet). `native-profiler-stop`
now returns a `traceFileNote` explaining it stays on the profiling host.
- **Project source on remote** — `react-profiler-component-source` /
`react-profiler-analyze` read the user's source tree on the tool-server
host. The profiler analysis chains already work remotely
(server-internal); tracked separately.

## Compatibility
Legacy `{ url, path }` screenshot rendering is retained as a fallback
when no artifact context is present (older tool-servers / unit tests).

## Tests
- tool-server: artifact registry (infer/override), `GET /artifacts/:id`
(200/404/410), updated `stop-recovery` assertions.
- argent-mcp: materializer
(download/rewrite/nested/failure/device-scoping), artifact-aware
`toMcpContent`.
- argent-mcp typecheck + full suite green; tool-server artifact/profiler
tests green. (Note: a full local tool-server typecheck trips a
pre-existing zod-v4-vs-v3 mismatch in `run-sequence` that's unrelated to
this change and resolves on this branch's own lockfile.)

## Notes
Routes around the `argent-private` native simulator-server binary (which
emits the `127.0.0.1` media URL) rather than changing it — no submodule
change.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: pFornagiel <pawel.fornagiel2@gmail.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Paweł Fornagiel <85541429+pFornagiel@users.noreply.github.com>

2026-06-10 12:04:30 +02:00

Paweł Fornagiel

3ccd89f333

feat: add native android profiling (#275 )

# Android native profiling via in-process Perfetto WASM

Connected with:
https://github.com/software-mansion/argent-private/pull/16

## TL;DR

Adds **Android** to the native profiler (previously iOS-only) and does
it with a
brand-new engine: instead of shipping Perfetto's `trace_processor_shell`
native
binary and forking it per query, we now run **Perfetto's trace-processor
compiled
to WebAssembly, in-process** inside the tool-server.

- **One ~13 MB `trace_processor.wasm`** runs on every OS/arch — no
per-platform
  binary, no subprocess, fully offline.
- The native profiler is refactored from an iOS-only implementation into
a
**platform-dispatch facade** (`platforms/ios.ts` +
`platforms/android.ts`) over
a new **`profiler-shared/`** layer, so iOS and Android share the
aggregator,
  types, lifecycle, and time-alignment code.
- New **release/pack pipeline** fetches + sha256-verifies the WASM
bundle from
`argent-private-releases`, and bundles it (plus SQL queries and the
Perfetto
  TraceConfig) into the published `@swmansion/argent` package.

End-to-end on a 76 MB trace with 1013 jank rows: **6.3 s** total analyze
(the older per-hang-subprocess approach was projected at ~47 minutes and
never
finished within the tool-call deadline).

---

## TESTING MANUALLY

To test manually run the following packing command checked-out on this
branch:
```
npm run pack:mcp -- argent-@pFornagiel-native-android-devtools-perfetto
```

This will bundle the package with the new binaries. After this PR is
fixed and the binaries are resolved from the main release, the
`argent-@pFornagiel-native-android-devtools-perfetto` release can be
deleted.

## Why

Android needed native CPU/hang/memory profiling to reach parity with
iOS. The
natural engine is Perfetto's trace-processor, but the obvious ways to
ship it are
both bad:

- **`trace_processor_shell` binary** — per-platform (a mac-arm64 pack
shipped a
mac-arm64-only tarball), pinned an OS/arch matrix, and cost a fresh ~1.3
s
subprocess fork **per query**. With one subprocess forked per detected
hang, a
  large trace projected to ~47 minutes.
- **`get.perfetto.dev/trace_processor` Python launcher** — adds a
runtime Python 3
dependency, resolves "latest" at first run (defeats version pinning),
and makes
  the first analyze slow via a network fetch.

The WASM engine is one cross-platform artifact, in-process and
warm-cached: the
trace is parsed **once** and reused across the whole analyze run plus
every
drill-down query.

---

## The three big pieces

### 1. In-process Perfetto WASM engine

New package surface in `@argent/native-devtools-android`:

| File | Role |
| --- | --- |
| `src/wasm-trace-processor.ts` (+476) | The whole engine: boot, glue
patching, warm-cache, fault containment, chunked parse, cell decoding |
| `src/perfetto-engine.d.ts` (+49) | Hand-written types for the
version-agnostic RPC decoder (imported by path, so tsc can't see its own
`.d.ts`) |
| `src/errors.ts` (+68) | `TraceProcessorUnavailableError`
(`wasm_load_failed` / `wasm_path_invalid`) |
| `src/bundled-meta.ts` (+12) | `PERFETTO_VERSION` stamp, regenerated at
pack time |
| `src/index.ts` | Re-exports + package-root accessors
(`traceProcessorQueriesDir`, `traceConfigPath`) |

Public surface: `resolveTraceProcessorAssets()`,
`queryWarm<Row>(tracePath, sql)`,
`ensureTraceProcessorReady(tracePath)`, `disposeWarmEngine(tracePath)`.

Highlights of how it works:

- **Boots Google's prebuilt web/worker wasm under Node.** Installs a
minimal
worker scope (`self`, `WorkerGlobalScope`, `location`), reads the wasm
bytes,
patches the Emscripten glue (`useMemory64=false`, hand the bytes in
directly,
expose the bridge), and drives it over a `MessageChannel` + `EngineBase`
subclass. Glue patches are applied **by unique string anchors that throw
if they
  don't match** — the canary for glue-format drift on a Perfetto bump.
- **Warm-engine cache, one per trace path.** LRU with `MAX_WARM_ENGINES
= 3`
(bounds resident memory at ~26–76 MB trace + wasm heap each), 5-min idle
dispose
  (timer `unref()`'d), no failed-promise caching.
- **Fault containment (`fatal` promise).** `EngineBase.fail()` throws
synchronously
inside the `MessagePort.onmessage` handler (e.g. an RPC framing error);
left
unhandled it becomes a Node `uncaughtException` and would **kill the
entire
tool-server**, wiping every device + profiler session. Each engine
exposes a
`fatal: Promise<never>`; the handler try/catches, rejects `fatal`,
evicts the
engine, and every load/query is `Promise.race`'d against it — so a
contained
  fault is a clean rejection and the server stays up.
- **Chunked trace parse.** Perfetto's RPC ring buffer rejects any frame
> 64 MiB,
  so the trace is fed in 32 MiB chunks before `notifyEof()`.

### 2. Android native profiler + the shared/dispatch refactor

**Capture** (`utils/android-profiler/capture.ts`, `detect-app.ts`):
spawns
`adb shell perfetto --txt -c - --background-wait -o
/data/.../argent-<ts>.pftrace`,
piping the TraceConfig on **stdin** (SELinux denies `shell:s0` writes
under
`/data/misc/perfetto-traces/`, so the config can't be staged as a file),
waits for
the PID on stdout, then on stop sends `kill -TERM`, polls `/proc/<pid>`,
and
`adb pull`s the trace.

**Pipeline** (`utils/android-profiler/pipeline/`): PerfettoSQL does the
parsing and
aggregation, so the Android pipeline is intentionally **two files, not
iOS's four**.
The per-hang annotation pass is **batched** — `runBatchedHangFolds`
inlines all
hang windows as a `VALUES` table into one SQL script (two derived views
unioned in
a single terminal SELECT, because the engine returns only the final
statement's
rows) and runs it once against the warm engine.

**The platform-dispatch refactor** —
`native-profiler-start/stop/analyze` shrank
from monolithic iOS files (~160–330 lines each) into **thin routers**
(~60–110
lines) that branch on `api.platform`:

```
native-profiler-start ─┬─ ios?    → platforms/ios.ts      (xctrace)
                        └─ android → platforms/android.ts  (adb perfetto)
```

A new **`utils/profiler-shared/`** layer holds everything both backends
use:

| File | What was hoisted |
| --- | --- |
| `types.ts` | Unified `Bottleneck = CpuHotspot \| UiHang \| MemoryLeak
\| MemoryRssGrowth`, each now carrying a `platform` field; shared
`NativeProfilerAnalyzeResult`; `RECORDING_CAP_MS` |
| `aggregate.ts` | Generic `aggregateCpuHotspots(AggregatorInputRow[])`
— burst windowing (`BURST_GAP_MS=500`), severity banding (RED >15%,
YELLOW 3–15%, <3% dropped). iOS feeds it via a pre-pass; Android feeds
it precomputed SQL rows |
| `lifecycle.ts` | `shutdownChild` SIGINT→SIGTERM→SIGKILL ladder +
`waitForChildExit` (no PID-reuse hazard) |
| `thread.ts` | `normalizeThreadName` for both xctrace and Perfetto
thread names |
| `time-align.ts` | New `buildPerfettoAnchor()` + `windowsOverlap()`
alongside the existing iOS/React anchors |
| `format.ts` | `formatBytes` (moved out of the combined report) |

iOS was refactored to fit this shape: `ios-profiler/lifecycle.ts` and
`types.ts`
become thin re-exports; `pipeline/02-aggregate.ts` shrank from ~200 to
~60 lines
(keeps its iOS-only dominant-function pre-pass, delegates
grouping/bursting/
severity to the shared aggregator). **No iOS behavior change** — same
outputs,
shared internals.

**Session blueprint** (`native-profiler-session.ts`): generic
`capturePid` /
`captureProcess` (was `xctracePid` / `xctraceProcess`), new `platform`
and
`androidOnDeviceTracePath` fields, and **platform-branched dispose**
(iOS SIGKILLs
xctrace; Android `kill -KILL`s perfetto, `rm`s the device trace, and
disposes the
warm WASM engine).

**Drill-down tools**: `profiler-load.ts` learns to restore Android
`.pftrace`
sessions (with an optional metadata sidecar / `app_process` override);
`profiler-stack-query.ts` splits into iOS and Android branches. Unlike
iOS (which
caches `parsedData` in memory), Android **re-queries the `.pftrace`**
per
drill-down — the warm engine is the cache, so each query costs only SQL
execution
time.

### 3. Packaging & release pipeline

The WASM bundle's three third-party artifacts (`trace_processor.wasm`,
`engine_bundle.node.js`, `engine.mjs`) + `LICENSE` are **never
committed**
(`.gitignore` lists the generated files but not the dir). They're built
and
sha256-checksummed in `argent-private` CI, published to
`argent-private-releases`, and on our side:

- **`scripts/download-trace-processor.sh`** (+88, new) — `gh release
download` →
verify tarball sha256 (**FATAL on mismatch** — the blobs are unsigned) →
extract
  → re-verify via `SHA256SUMS` → assert all four present.
- **`scripts/pack-mcp.cjs`** (+60, new) — single pack entry point:
downloads
simulator-server, native binaries, **and** the trace-processor bundle,
then
builds and `npm pack`s `@swmansion/argent`. `package.json`'s `pack` /
`pack:mcp`
  now call this script.
- **`packages/argent/scripts/bundle-tools.cjs`** (heavily reworked,
+574/-...) —
copies `assets/trace-processor/`, `assets/queries/`, and
`argent.tracecfg.pbtxt`
into `packages/argent/assets/`, and regenerates `bundled-meta.ts` to
stamp the
  Perfetto version.
- **`publish.yml` / `publish-next.yml`** — run the download step and
then **verify**:
  all four WASM files present, the wasm magic-byte check
(`head -c4 … | grep $'\x00asm'`), plus the queries dir, TraceConfig,
manifest,
  and Android helper APK.

Asset reorg: `native-devtools-android` now ships `dist/`, `bin/`, and
`assets/`
(SQL queries, TraceConfig, manifest moved under `assets/`); `.gitignore`
swaps the
old per-file `packages/argent/manifest.json` entry for the whole
`packages/argent/assets/` dir.

---

## Testing

15 new test files under `packages/tool-server/test/android-perfetto/`
cover the
engine and pipeline, including:

- `dispatch.test.ts` — start/stop/analyze route to the right platform
backend.
- `aggregate-shared.test.ts` — the shared aggregator on both iOS and
Android-shaped input.
- `hang-folds-batched.test.ts` / `hang-fold.test.ts` — the batched
per-hang SQL fold.
- `hang-severity.test.ts` — jank-reason → RED/YELLOW classification.
- `dispose.test.ts` / `dispose-warm-engine.test.ts` — session dispose
kills the daemon, rms the device trace, and tears down the warm engine.
- `stop-recovery.test.ts` — `profilingActive` is forced false even if
`adb pull` fails (so a retry-start doesn't wedge).
- `run-tp.test.ts` — `renderSqlTemplate` token rendering + the "unused
substitution" guard.
- `manifest-hint.test.ts` — zero-callstack detection emits the manifest
hint.
- `profiler-load.test.ts` — `.pftrace` session restore with/without the
metadata sidecar.

Plus `native-profiler-analyze-failure.test.ts` for the
engine-unavailable banner,
and updated iOS tests for the shared-lifecycle refactor.
`@argent/native-devtools-android` gains its own vitest setup
(`vitest.config.ts`, `tsconfig.test.json`, `test/errors.test.ts`).

---

## Notes & follow-ups

- **Manifest requirement.** Perfetto's `linux.perf` data source needs
`/proc/<pid>/mem` access, granted only to a **debuggable** or
`<profileable
android:shell="true"/>` app. A release build without either silently
produces
zero callstacks; the pipeline detects this and emits a **manifest hint**
instead
  of an "All clear" report.
- **Version pinning lives in the `argent-private` submodule**
(`PERFETTO_VERSION`
/ `PERFETTO_WASM_TAG` / `LYNX_TRACE_PROCESSOR_VERSION`). Bumping is
documented in
`argent-private/docs/ANDROID_PERFETTO.md`. JS↔engine version skew is
structurally
  impossible (version-agnostic decoder + vendored wasm).
- **`disposeWarmEngine` is exported but not wired into session-stop** —
warm
engines are reclaimed only by the idle timer + LRU cap. Acceptable today
(bounded
memory); wire it into session dispose if explicit teardown is ever
needed.
- **Submodule bump.** This PR moves `packages/argent-private` to
`22e55f90` — see the companion `argent-private` PR description,
including its
merge-hygiene note (that branch should pick up latest `argent-private`
main
  before the pointer is finalized).

### Companion docs added in this branch
- `utils/android-profiler/ANDROID_PROFILER_REFERENCE.md` — stack,
capture, queries, manifest requirement.
- `utils/android-profiler/PIPELINE_DESIGN.md` — the "why" (two-file
pipeline, shared-aggregator hoist, re-query-don't-cache, batched fold).

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

2026-06-09 16:06:24 +02:00

3 Commits