mirror of
https://github.com/software-mansion/argent.git
synced 2026-09-14 19:27:14 +08:00
0a6dbec87c
## What
Swaps the `posthog-node` transport for an OpenTelemetry Logs exporter.
Everything above the transport stays as it is - opt-out consent,
per-machine identity and fingerprint, the sanitizer allowlist, the typed
event surface, crash diagnostics. Only the ~3-method client boundary
changes.
One analytics event becomes one OTLP/HTTP log record:
| PostHog | OpenTelemetry |
| --- | --- |
| `capture({ event, distinctId, properties })` | `logger.emit({ body:
event, attributes: { distinct_id, "event.name", ...props } })` |
| event name | log-record `body` |
| `distinctId` | `distinct_id` attribute |
| event/base properties | log-record attributes |
| `client.shutdown()` | `LoggerProvider.shutdown()` (force-flush +
teardown) |
Records go to a hard-coded Software Mansion collector. The endpoint and
the authorization header are passed to the exporter explicitly in code,
and an explicit value beats the corresponding `OTEL_EXPORTER_OTLP_*`
variable - the same anti-exfiltration property the fixed PostHog host
had.
## Changes
- `packages/telemetry/src/posthog.ts` becomes `otel.ts`:
`LoggerProvider` + `BatchLogRecordProcessor` + `OTLPLogExporter`
(HTTP/JSON). Queue size 20 and a 10 s flush interval mirror the old
client; the per-export deadline is 1.5 s, kept at or below `index.ts`'s
drain budget so a stalled export can't hold a short-lived command open.
- Dependencies: drop `posthog-node`, add
`@opentelemetry/{api,api-logs,sdk-logs,exporter-logs-otlp-http,resources}`.
They inline cleanly into the esbuild bundles.
- Build-time ingest token via esbuild `define`
(`ARGENT_OTEL_INGEST_TOKEN`), same mechanism as `ARGENT_CLI_VERSION`.
Unset in a dev build leaves the client unconstructed and telemetry
inert.
- Base props: drop `$process_person_profile` (a PostHog directive,
meaningless to a collector) and rename `$session_id` to `session_id`.
Null-valued attributes are dropped, since OTel rejects them -
`cloud_agent` is null on the common path, and absence means the same
thing.
- `Telemetry.md`: the "who we disclose to" and "international transfers"
sections describe a Software Mansion-operated OTLP collector instead of
PostHog as a third-party processor.
## A hang this uncovered
`timeoutMillis` only bounds a request once its socket is connected,
because the exporter applies it with `req.setTimeout()`. Against a
collector address that drops packets rather than refusing them - a
corporate egress filter, a dead host behind a firewall - the socket sits
in the connecting state where no export deadline reaches it, and the
process stays alive for the OS connect timeout: 75 s past a `shutdown()`
that had already resolved at 1.75 s.
Pairing the export deadline with an agent-level socket timeout, armed at
socket creation so it covers connection establishment too, fixes it:
| collector state | before | after |
| --- | --- | --- |
| healthy | 0.18 s, delivered | 0.18 s, delivered |
| connection refused | 0.94 s | 1.03 s |
| blackholed (http and https) | 75.0 s | 1.6 s |
| connected but never responds | - | 1.6 s |
`posthog-node` exited in 10.6 s under the same blackhole (undici aborts
a connecting socket), so this was a regression as well as a hang. A unit
test that fails without the fix guards it.
## Found in review
Each of these was reproduced before it was changed, and each is pinned
by a test that fails without the fix.
- **The environment's OTLP headers were forwarded to the collector.**
The SDK merges `OTEL_EXPORTER_OTLP_HEADERS` / `_LOGS_HEADERS` into every
request, keeping any key the code does not set itself. Out in the world
that variable holds the developer's *own* observability credential - a
Honeycomb team key, a Dynatrace `Api-Token`, Grafana Cloud basic auth -
so any machine already running OpenTelemetry shipped that third-party
secret here on every batch, without it passing the sanitizer or
appearing in `Telemetry.md`. Reproduced by watching `x-honeycomb-team:
hcaik_…` arrive at a capture server. The variables are now cleared
across the exporter constructor (where the SDK resolves its header set)
and restored after.
- **`httpAgentOptions` silently disabled connection reuse.** Supplying
it replaces the agent the SDK would build, whose default is `keepAlive:
true`. Measured on loopback: three batches shared one socket by default,
and opened three sockets with `httpAgentOptions` - a fresh TCP+TLS
handshake per 10 s batch in the long-lived tool-server. `keepAlive` is
restated alongside the timeout.
- **`markDisabled()`'s drain had no grace period.** It raced flat
against `SHORT_FLUSH_TIMEOUT_MS` while `shutdown()` allowed `timeoutMs +
250`. Since the exporter's own deadline equals that budget, the flat
race fired at the same instant and always abandoned the batch instead of
letting it finish. Both call sites share one `raceDrain()` now.
- **Export failures reached nobody.** The batch processor hands errors
to OpenTelemetry's global handler, so `LoggerProvider.shutdown()`
resolves identically for a delivered batch and a rejected token - making
`shutdown()`'s `catch` unreachable. The SDK's diagnostics are now routed
into the existing debug channel under `ARGENT_TELEMETRY_DEBUG=1`; a run
without the flag stays silent so no host application's diag logger is
disturbed.
- **The build-time token now beats the `__ARGENT_OTEL_TOKEN_TEST` seam**
rather than losing to it, so the seam reaches unbundled source only.
Verified with esbuild across all three build modes.
- **The endpoint guard test mocked the exporter it guards.** It could
only show that the code passes a `url`; an SDK upgrade that changed the
precedence would have kept it green. `otel-endpoint-live.test.ts` drives
the real exporter against loopback servers - dropping the explicit
`url`, the explicit `authorization`, or the header clearing each fails
it.
- **The wire shape had no pins at all.** Bypassing the sanitizer
entirely, sending an explicit `null`, renaming `service.name` and
dropping `event.name` all left the suite green. Four tests now fail on
those mutations. The sanitizer bypass was unpinned on `main` too.
## Before this ships
1. ~~**Collector URL**~~ - done in 8bca962f. `OTLP_LOGS_ENDPOINT` is
`https://argent-otel.swmansion.com/v1/logs`. That host serves OTLP as of
2026-08-12: a `POST` answers the OTLP-shaped `401`
(`{"code":16,"message":"provided authorization does not match expected
scheme or token"}`), where it previously `301`d to `/v1/logs/` and
`404`d there. It is a distinct host from the test collector
(`167.235.245.1` vs `46.224.172.171`, its own certificate).
2. **Ingest token** - `ARGENT_OTEL_INGEST_TOKEN` has to be set in the
release build environment. Nothing under `.github/` sets it today, so
every published artifact resolves an empty token and stays inert -
silently: with no token the client is never constructed, so a run emits
nothing and writes no `telemetry-debug.log` even under
`ARGENT_TELEMETRY_DEBUG=1`. It must be the **production** host's token:
`argent-otel.swmansion.com` runs its own token list and rejects the test
collector's.
3. **IP geolocation** - PostHog's `disableGeoip: true` set
`$geoip_disable` on every event, which is what stopped location being
derived from the source IP. There is no in-repo equivalent for a
collector, so that suppression has to be configured collector-side to
keep `Telemetry.md`'s "what we collect" list accurate.
4. **Legal** - the privacy-notice edits and the version/date bump to
1.03 are drafted to match the new architecture, but Legal should review
and finalize them. Note the notice actually served at
`TELEMETRY_DETAILS_URL` is still Version 1.01 and names PostHog as the
processor.
## Verification
Unit and integration:
- `packages/telemetry`: 293 tests pass, including the rewritten
`index.test.ts`, `otel-endpoint.test.ts` and the new
`otel-endpoint-live.test.ts`.
- Every workspace suite green on the merged branch: argent-cli 276,
argent-installer 535, argent-mcp 78, argent 60, tools-client 178,
configuration-core 83, registry 87, tool-server 3070, update-core 31,
archive 11.
- `bundle-tools.cjs` leaves no `posthog` in any of the four bundles.
OTel is inlined into the three that emit - `cli-cmds.mjs`,
`installer.mjs`, `tool-server.cjs` - each carrying the endpoint and the
token define as string literals. `mcp-server.mjs` carries none: it is a
stdio shim over the tool-server, which is the process that reports.
- `lint`, `prettier --check`, `tsc --build`, the `tsconfig.test.json`
gate and `npm install --package-lock-only` (idempotent on Node 24) all
clean.
821 lines
26 KiB
TypeScript
821 lines
26 KiB
TypeScript
import { describe, expect, it } from "vitest";
|
|
import { FAILURE_CODES } from "@argent/registry";
|
|
import { sanitize, ALLOWED } from "../src/sanitize.js";
|
|
import { DEBUGGER_TOOL_OUTCOMES, EVENT_NAMES, PLATFORMS } from "../src/events.js";
|
|
|
|
describe("sanitize", () => {
|
|
describe("event allowlist", () => {
|
|
it("returns empty object for an unknown event name", () => {
|
|
expect(sanitize("attack:pwned", { foo: 1 })).toEqual({});
|
|
});
|
|
|
|
it("drops every unknown property key", () => {
|
|
expect(
|
|
sanitize("installation:cli_init_start", {
|
|
package_manager: "npm",
|
|
is_non_interactive: true,
|
|
home_path: "/Users/alice", // not in allowlist
|
|
ssn: "123-45-6789",
|
|
})
|
|
).toEqual({ package_manager: "npm", is_non_interactive: true });
|
|
});
|
|
});
|
|
|
|
describe("oneOf validator", () => {
|
|
it("accepts canonical values", () => {
|
|
expect(sanitize("installation:cli_init_start", { package_manager: "npm" })).toEqual({
|
|
package_manager: "npm",
|
|
});
|
|
});
|
|
|
|
it("drops a fork-derived path-leaking value", () => {
|
|
expect(
|
|
sanitize("installation:cli_init_start", {
|
|
package_manager: "npm (/Users/alice/.nvm/versions/node/v20/bin/npm)",
|
|
})
|
|
).toEqual({});
|
|
});
|
|
|
|
it("drops unknown enum values", () => {
|
|
expect(sanitize("installation:cli_init_cancel", { step: "scope_typo" })).toEqual({});
|
|
});
|
|
|
|
it("drops the legacy unknown tool platform value", () => {
|
|
expect(sanitize("tool:invoke", { tool: "list-devices", platform: "unknown" })).toEqual({
|
|
tool: "list-devices",
|
|
});
|
|
});
|
|
|
|
it("accepts every telemetry platform, including the TV variants", () => {
|
|
// The TV variants (`tvos`, `android-tv`) are telemetry-only refinements of
|
|
// `ios` / `android`; the sanitizer must let them through so the split
|
|
// survives to the collector.
|
|
for (const platform of PLATFORMS) {
|
|
expect(sanitize("tool:invoke", { tool: "describe", platform })).toEqual({
|
|
tool: "describe",
|
|
platform,
|
|
});
|
|
}
|
|
expect(PLATFORMS).toContain("tvos");
|
|
expect(PLATFORMS).toContain("android-tv");
|
|
});
|
|
|
|
it("drops the legacy `from_tar` decision (developer-only path is off the books)", () => {
|
|
expect(sanitize("installation:global_install_decision", { decision: "from_tar" })).toEqual(
|
|
{}
|
|
);
|
|
});
|
|
|
|
it("accepts the documented global_install_decision enum values", () => {
|
|
for (const decision of ["install", "cancel", "already_installed"] as const) {
|
|
expect(sanitize("installation:global_install_decision", { decision })).toEqual({
|
|
decision,
|
|
});
|
|
}
|
|
});
|
|
|
|
it("accepts a coarse AI client slug on tool events", () => {
|
|
expect(
|
|
sanitize("tool:invoke", {
|
|
tool: "gesture-tap",
|
|
tool_invocation_id: "11111111-1111-4111-8111-111111111111",
|
|
ai_client: "codex",
|
|
})
|
|
).toEqual({
|
|
tool: "gesture-tap",
|
|
tool_invocation_id: "11111111-1111-4111-8111-111111111111",
|
|
ai_client: "codex",
|
|
});
|
|
});
|
|
|
|
it("never carries the free-form ai_client_name — it is not an allowed key", () => {
|
|
// The free-form name was dropped end-to-end; `other` is recorded as a bare
|
|
// bucket so a client that names itself after the host/user can't leak it.
|
|
expect(
|
|
sanitize("tool:invoke", {
|
|
tool: "gesture-tap",
|
|
tool_invocation_id: "11111111-1111-4111-8111-111111111111",
|
|
ai_client: "other",
|
|
ai_client_name: "some-new-tool",
|
|
})
|
|
).toEqual({
|
|
tool: "gesture-tap",
|
|
tool_invocation_id: "11111111-1111-4111-8111-111111111111",
|
|
ai_client: "other",
|
|
});
|
|
});
|
|
|
|
it("drops unregistered AI client slugs", () => {
|
|
expect(
|
|
sanitize("tool:invoke", {
|
|
tool: "gesture-tap",
|
|
tool_invocation_id: "11111111-1111-4111-8111-111111111111",
|
|
ai_client: "my-private-client",
|
|
ai_client_name: "/Users/alice/secret-client",
|
|
})
|
|
).toEqual({
|
|
tool: "gesture-tap",
|
|
tool_invocation_id: "11111111-1111-4111-8111-111111111111",
|
|
});
|
|
});
|
|
|
|
it("carries the coarse AI client on tool:fail alongside the failure signal", () => {
|
|
const result = sanitize("tool:fail", {
|
|
tool: "screenshot",
|
|
tool_invocation_id: "11111111-1111-4111-8111-111111111111",
|
|
duration_ms: 12,
|
|
error_code: FAILURE_CODES.REGISTRY_TOOL_FAILURE_UNCLASSIFIED,
|
|
ai_client: "other",
|
|
ai_client_name: "some-new-tool",
|
|
});
|
|
expect(result).toMatchObject({
|
|
ai_client: "other",
|
|
error_code: FAILURE_CODES.REGISTRY_TOOL_FAILURE_UNCLASSIFIED,
|
|
});
|
|
expect(result).not.toHaveProperty("ai_client_name");
|
|
});
|
|
});
|
|
|
|
describe("matches validator", () => {
|
|
it("accepts a tool id under 64 chars", () => {
|
|
expect(sanitize("tool:invoke", { tool: "gesture-tap", platform: "ios" })).toEqual({
|
|
tool: "gesture-tap",
|
|
platform: "ios",
|
|
});
|
|
});
|
|
|
|
it("accepts a snake_case tool id (e.g. argent-lens tools)", () => {
|
|
expect(sanitize("tool:invoke", { tool: "await_user_selection", platform: "ios" })).toEqual({
|
|
tool: "await_user_selection",
|
|
platform: "ios",
|
|
});
|
|
});
|
|
|
|
it("rejects a tool id with uppercase / weird chars", () => {
|
|
expect(sanitize("tool:invoke", { tool: "Gesture-Tap!", platform: "ios" })).toEqual({
|
|
platform: "ios",
|
|
});
|
|
});
|
|
|
|
it("rejects an oversize string even when shape matches", () => {
|
|
const longButValid = "a".repeat(200);
|
|
expect(sanitize("tool:invoke", { tool: longButValid, platform: "ios" })).toEqual({
|
|
platform: "ios",
|
|
});
|
|
});
|
|
|
|
it("drops error metadata fields from tool failures", () => {
|
|
expect(
|
|
sanitize("tool:fail", {
|
|
tool: "gesture-tap",
|
|
platform: "ios",
|
|
duration_ms: 1,
|
|
error_message: "ENOENT /Users/alice/.ssh/id_rsa",
|
|
stack: "Error: ENOENT\n at /Users/alice/project/app.ts:1",
|
|
})
|
|
).toEqual({
|
|
tool: "gesture-tap",
|
|
platform: "ios",
|
|
duration_ms: 1,
|
|
});
|
|
});
|
|
|
|
it("allows static failure signal fields on tool failures", () => {
|
|
expect(
|
|
sanitize("tool:fail", {
|
|
tool: "gesture-tap",
|
|
platform: "ios",
|
|
duration_ms: 1,
|
|
error_code: "HTTP_ZOD_VALIDATION_FAILED",
|
|
failure_stage: "http_zod_validation",
|
|
failure_area: "http",
|
|
error_kind: "validation",
|
|
})
|
|
).toEqual({
|
|
tool: "gesture-tap",
|
|
platform: "ios",
|
|
duration_ms: 1,
|
|
error_code: "HTTP_ZOD_VALIDATION_FAILED",
|
|
failure_stage: "http_zod_validation",
|
|
failure_area: "http",
|
|
error_kind: "validation",
|
|
});
|
|
});
|
|
|
|
it("allows coarse subprocess and network failure metadata", () => {
|
|
expect(
|
|
sanitize("tool:fail", {
|
|
tool: "gesture-tap",
|
|
platform: "ios",
|
|
duration_ms: 1,
|
|
error_code: "ANDROID_ADB_COMMAND_FAILED",
|
|
failure_stage: "android_adb_command",
|
|
failure_area: "tool_server",
|
|
error_kind: "subprocess",
|
|
failure_command: "adb",
|
|
failure_exit_code: 1,
|
|
failure_signal: "SIGKILL",
|
|
failure_spawn_code: "ENOENT",
|
|
network_failure: "connection_refused",
|
|
})
|
|
).toEqual({
|
|
tool: "gesture-tap",
|
|
platform: "ios",
|
|
duration_ms: 1,
|
|
error_code: "ANDROID_ADB_COMMAND_FAILED",
|
|
failure_stage: "android_adb_command",
|
|
failure_area: "tool_server",
|
|
error_kind: "subprocess",
|
|
failure_command: "adb",
|
|
failure_exit_code: 1,
|
|
failure_signal: "SIGKILL",
|
|
failure_spawn_code: "ENOENT",
|
|
network_failure: "connection_refused",
|
|
});
|
|
});
|
|
|
|
it("rejects unsafe subprocess and network metadata", () => {
|
|
expect(
|
|
sanitize("tool:fail", {
|
|
tool: "gesture-tap",
|
|
platform: "ios",
|
|
duration_ms: 1,
|
|
failure_command: "adb -s /Users/alice/device shell secret",
|
|
failure_exit_code: 999,
|
|
failure_signal: "SIGUSR1",
|
|
failure_spawn_code: "ESECRET",
|
|
network_failure: "https://internal.example",
|
|
})
|
|
).toEqual({
|
|
tool: "gesture-tap",
|
|
platform: "ios",
|
|
duration_ms: 1,
|
|
});
|
|
});
|
|
|
|
it("accepts every centrally registered failure code", () => {
|
|
for (const errorCode of Object.values(FAILURE_CODES)) {
|
|
expect(
|
|
sanitize("tool:fail", {
|
|
tool: "gesture-tap",
|
|
platform: "ios",
|
|
duration_ms: 1,
|
|
error_code: errorCode,
|
|
})
|
|
).toMatchObject({ error_code: errorCode });
|
|
}
|
|
});
|
|
|
|
it("rejects non-static-looking failure signals", () => {
|
|
expect(
|
|
sanitize("tool:fail", {
|
|
tool: "gesture-tap",
|
|
platform: "ios",
|
|
duration_ms: 1,
|
|
error_code: "ENOENT /Users/alice/.ssh/id_rsa",
|
|
failure_stage: "../secret",
|
|
failure_area: "laptop",
|
|
error_kind: "password",
|
|
})
|
|
).toEqual({
|
|
tool: "gesture-tap",
|
|
platform: "ios",
|
|
duration_ms: 1,
|
|
});
|
|
});
|
|
|
|
it("rejects static-looking but unregistered failure codes", () => {
|
|
expect(
|
|
sanitize("tool:fail", {
|
|
tool: "gesture-tap",
|
|
platform: "ios",
|
|
duration_ms: 1,
|
|
error_code: "SOME_NEW_UNREGISTERED_FAILURE",
|
|
})
|
|
).toEqual({
|
|
tool: "gesture-tap",
|
|
platform: "ios",
|
|
duration_ms: 1,
|
|
});
|
|
});
|
|
|
|
it("allows static failure signal fields on CLI run failures", () => {
|
|
expect(
|
|
sanitize("cli:run_fail", {
|
|
tool: "gesture-tap",
|
|
duration_ms: 1,
|
|
error_code: "CLI_RUN_ARGS_JSON_INVALID",
|
|
failure_stage: "cli_run_parse_raw_args",
|
|
failure_area: "cli",
|
|
error_kind: "validation",
|
|
})
|
|
).toEqual({
|
|
tool: "gesture-tap",
|
|
duration_ms: 1,
|
|
error_code: "CLI_RUN_ARGS_JSON_INVALID",
|
|
failure_stage: "cli_run_parse_raw_args",
|
|
failure_area: "cli",
|
|
error_kind: "validation",
|
|
});
|
|
});
|
|
|
|
it("drops server-only and unsafe fields from CLI run failures", () => {
|
|
expect(
|
|
sanitize("cli:run_fail", {
|
|
tool: "Gesture-Tap!",
|
|
duration_ms: 1,
|
|
tool_invocation_id: "11111111-1111-4111-8111-111111111111",
|
|
platform: "ios",
|
|
error_message: "ENOENT /Users/alice/.ssh/id_rsa",
|
|
stack: "Error: ENOENT\n at /Users/alice/project/app.ts:1",
|
|
error_code: "CLI_RUN_TOOL_CALL_FAILED",
|
|
})
|
|
).toEqual({
|
|
duration_ms: 1,
|
|
error_code: "CLI_RUN_TOOL_CALL_FAILED",
|
|
});
|
|
});
|
|
});
|
|
|
|
describe("number validator", () => {
|
|
it.each([
|
|
["NaN", Number.NaN],
|
|
["+Infinity", Number.POSITIVE_INFINITY],
|
|
["-Infinity", Number.NEGATIVE_INFINITY],
|
|
["negative", -1],
|
|
["over 2^31", 2 ** 31 + 1],
|
|
])("drops poison number %s", (_label, value) => {
|
|
expect(sanitize("tool:complete", { tool: "x", platform: "ios", duration_ms: value })).toEqual(
|
|
{ tool: "x", platform: "ios" }
|
|
);
|
|
});
|
|
|
|
it("accepts a normal duration", () => {
|
|
expect(sanitize("tool:complete", { tool: "x", platform: "ios", duration_ms: 42.5 })).toEqual({
|
|
tool: "x",
|
|
platform: "ios",
|
|
duration_ms: 42.5,
|
|
});
|
|
});
|
|
|
|
it("accepts valid tool invocation ids and drops invalid ones", () => {
|
|
expect(
|
|
sanitize("tool:complete", {
|
|
tool: "x",
|
|
tool_invocation_id: "11111111-1111-4111-8111-111111111111",
|
|
platform: "ios",
|
|
duration_ms: 42.5,
|
|
})
|
|
).toEqual({
|
|
tool: "x",
|
|
tool_invocation_id: "11111111-1111-4111-8111-111111111111",
|
|
platform: "ios",
|
|
duration_ms: 42.5,
|
|
});
|
|
|
|
expect(
|
|
sanitize("tool:complete", {
|
|
tool: "x",
|
|
tool_invocation_id: "/Users/alice/project",
|
|
platform: "ios",
|
|
duration_ms: 42.5,
|
|
})
|
|
).toEqual({
|
|
tool: "x",
|
|
platform: "ios",
|
|
duration_ms: 42.5,
|
|
});
|
|
});
|
|
});
|
|
|
|
it("passes the 'deferred' stop reason through", () => {
|
|
// Emitted by a redundant tool-server instance that lost the port bind to a
|
|
// healthy peer and exited cleanly; must not be scrubbed to undefined.
|
|
expect(
|
|
sanitize("toolserver:stop", {
|
|
reason: "deferred",
|
|
uptime_ms: 950,
|
|
total_tool_calls: 0,
|
|
})
|
|
).toEqual({
|
|
reason: "deferred",
|
|
uptime_ms: 950,
|
|
total_tool_calls: 0,
|
|
});
|
|
});
|
|
|
|
it("allows static failure signal fields on crash stop events", () => {
|
|
expect(
|
|
sanitize("toolserver:stop", {
|
|
reason: "crash",
|
|
uptime_ms: 10,
|
|
total_tool_calls: 2,
|
|
error_code: "TOOLSERVER_UNCAUGHT_EXCEPTION",
|
|
failure_stage: "toolserver_uncaught_exception",
|
|
failure_area: "tool_server",
|
|
error_kind: "crash",
|
|
stack: "Error: secret at /Users/alice/project/app.ts:1",
|
|
})
|
|
).toEqual({
|
|
reason: "crash",
|
|
uptime_ms: 10,
|
|
total_tool_calls: 2,
|
|
error_code: "TOOLSERVER_UNCAUGHT_EXCEPTION",
|
|
failure_stage: "toolserver_uncaught_exception",
|
|
failure_area: "tool_server",
|
|
error_kind: "crash",
|
|
});
|
|
});
|
|
|
|
describe("crash diagnostics on stop events", () => {
|
|
it("passes the coded crash fields through when well-formed", () => {
|
|
expect(
|
|
sanitize("toolserver:stop", {
|
|
reason: "crash",
|
|
uptime_ms: 480,
|
|
total_tool_calls: 0,
|
|
error_code: "TOOLSERVER_UNCAUGHT_EXCEPTION",
|
|
failure_stage: "toolserver_uncaught_exception",
|
|
failure_area: "tool_server",
|
|
error_kind: "crash",
|
|
error_name: "TypeError",
|
|
error_syscall: "EADDRINUSE",
|
|
crash_fingerprint: "0123456789abcdef",
|
|
crash_phase: "startup",
|
|
})
|
|
).toEqual({
|
|
reason: "crash",
|
|
uptime_ms: 480,
|
|
total_tool_calls: 0,
|
|
error_code: "TOOLSERVER_UNCAUGHT_EXCEPTION",
|
|
failure_stage: "toolserver_uncaught_exception",
|
|
failure_area: "tool_server",
|
|
error_kind: "crash",
|
|
error_name: "TypeError",
|
|
error_syscall: "EADDRINUSE",
|
|
crash_fingerprint: "0123456789abcdef",
|
|
crash_phase: "startup",
|
|
});
|
|
});
|
|
|
|
it("accepts ERR_*-style Node error codes as a syscall", () => {
|
|
const out = sanitize("toolserver:stop", {
|
|
reason: "crash",
|
|
uptime_ms: 1,
|
|
total_tool_calls: 0,
|
|
error_syscall: "ERR_MODULE_NOT_FOUND",
|
|
});
|
|
expect(out).toMatchObject({ error_syscall: "ERR_MODULE_NOT_FOUND" });
|
|
});
|
|
|
|
it("drops a message masquerading as error_name (spaces / punctuation / path)", () => {
|
|
const out = sanitize("toolserver:stop", {
|
|
reason: "crash",
|
|
uptime_ms: 1,
|
|
total_tool_calls: 0,
|
|
error_name: "Error: ENOENT /Users/alice/.ssh/id_rsa",
|
|
});
|
|
expect(out).not.toHaveProperty("error_name");
|
|
});
|
|
|
|
it("drops a path-like or lowercase error_syscall", () => {
|
|
for (const bad of ["/Users/alice/x", "eaddrinuse", "listen 127.0.0.1:3001", "PORT_9229"]) {
|
|
const out = sanitize("toolserver:stop", {
|
|
reason: "crash",
|
|
uptime_ms: 1,
|
|
total_tool_calls: 0,
|
|
error_syscall: bad,
|
|
});
|
|
expect(out).not.toHaveProperty("error_syscall");
|
|
}
|
|
});
|
|
|
|
it("drops a fingerprint that is not exactly 16 lowercase hex chars", () => {
|
|
for (const bad of ["0123456789ABCDEF", "0123456789abcde", "0123456789abcdef0", "zzzz", ""]) {
|
|
const out = sanitize("toolserver:stop", {
|
|
reason: "crash",
|
|
uptime_ms: 1,
|
|
total_tool_calls: 0,
|
|
crash_fingerprint: bad,
|
|
});
|
|
expect(out).not.toHaveProperty("crash_fingerprint");
|
|
}
|
|
});
|
|
|
|
it("drops an unknown crash_phase", () => {
|
|
const out = sanitize("toolserver:stop", {
|
|
reason: "crash",
|
|
uptime_ms: 1,
|
|
total_tool_calls: 0,
|
|
crash_phase: "booting",
|
|
});
|
|
expect(out).not.toHaveProperty("crash_phase");
|
|
});
|
|
});
|
|
|
|
describe("package action telemetry", () => {
|
|
it("accepts the requested package-action enum set", () => {
|
|
for (const action of [
|
|
"fresh_install",
|
|
"already_installed",
|
|
"init_triggered_update",
|
|
"no_update",
|
|
"update_skipped",
|
|
"update_failed",
|
|
"standalone_update",
|
|
"standalone_install",
|
|
"mcp_update",
|
|
] as const) {
|
|
expect(
|
|
sanitize("installation:package_action", {
|
|
trigger: action === "mcp_update" ? "mcp_update" : "init",
|
|
action,
|
|
is_success: true,
|
|
duration_ms: 2,
|
|
})
|
|
).toMatchObject({ action, duration_ms: 2 });
|
|
}
|
|
});
|
|
});
|
|
|
|
describe("arrayOf validator", () => {
|
|
it("accepts a valid editors list", () => {
|
|
expect(
|
|
sanitize("installation:editors_select", {
|
|
editors: ["cursor", "claude-code"],
|
|
detected_editor_count: 2,
|
|
scope: "local",
|
|
})
|
|
).toEqual({
|
|
editors: ["cursor", "claude-code"],
|
|
detected_editor_count: 2,
|
|
scope: "local",
|
|
});
|
|
});
|
|
|
|
it("drops an array with a bad element (whole array discarded)", () => {
|
|
expect(
|
|
sanitize("installation:editors_select", {
|
|
editors: ["cursor", "Bad Editor"],
|
|
detected_editor_count: 2,
|
|
scope: "local",
|
|
})
|
|
).toEqual({ detected_editor_count: 2, scope: "local" });
|
|
});
|
|
|
|
it("drops an oversized array (>16 elements)", () => {
|
|
const editors = Array.from({ length: 17 }, (_, i) => `editor-${i}`);
|
|
expect(
|
|
sanitize("installation:editors_select", {
|
|
editors,
|
|
detected_editor_count: 17,
|
|
scope: "local",
|
|
})
|
|
).toEqual({ detected_editor_count: 17, scope: "local" });
|
|
});
|
|
});
|
|
|
|
describe("tool:fail invalid_params validator", () => {
|
|
const base = { tool: "launch-app", duration_ms: 1 };
|
|
|
|
it("accepts camelCase schema names — the /i flag is load-bearing", () => {
|
|
// A third of registered tools declare camelCase zod keys (bundleId,
|
|
// avdName, maxNodes, ...). arrayOf is all-or-nothing, so dropping the
|
|
// case-insensitive flag would silently void the whole property for any
|
|
// call whose failing param is camelCase.
|
|
const out = sanitize("tool:fail", {
|
|
...base,
|
|
invalid_params: ["bundleId", "electronAppPath", "unrecognized_keys"],
|
|
});
|
|
expect(out.invalid_params).toEqual(["bundleId", "electronAppPath", "unrecognized_keys"]);
|
|
});
|
|
|
|
it("accepts exactly 16 names — the emit-side cap must fit through", () => {
|
|
// http.ts caps the derived list at 16 before emitting; the sanitize cap
|
|
// must not be tighter, or every capped emission silently loses the
|
|
// property.
|
|
const names = Array.from({ length: 16 }, (_, i) => `param_${i}`);
|
|
const out = sanitize("tool:fail", { ...base, invalid_params: names });
|
|
expect(out.invalid_params).toEqual(names);
|
|
});
|
|
|
|
it("drops the whole property at 17 names", () => {
|
|
const names = Array.from({ length: 17 }, (_, i) => `param_${i}`);
|
|
const out = sanitize("tool:fail", { ...base, invalid_params: names });
|
|
expect("invalid_params" in out).toBe(false);
|
|
});
|
|
|
|
it("drops the whole property when one element is not an identifier", () => {
|
|
const out = sanitize("tool:fail", {
|
|
...base,
|
|
invalid_params: ["port", "not a name!"],
|
|
});
|
|
expect("invalid_params" in out).toBe(false);
|
|
});
|
|
|
|
it("drops an element longer than 64 characters (whole property voided)", () => {
|
|
const out = sanitize("tool:fail", {
|
|
...base,
|
|
invalid_params: ["a".repeat(65)],
|
|
});
|
|
expect("invalid_params" in out).toBe(false);
|
|
});
|
|
});
|
|
|
|
describe("sensitive-arg drop tests", () => {
|
|
it.each([
|
|
["keyboard.text", { text: "hunter2" }],
|
|
["paste.text", { text: "secret" }],
|
|
["open-url.url", { url: "https://internal.example/admin" }],
|
|
["flow-add-step.args", { args: '{"text":"private"}' }],
|
|
["flow-add-echo.message", { message: "personal data" }],
|
|
["customRoot", { customRoot: "/Users/alice/work" }],
|
|
["fromTar", { from: "/Users/alice/file.tgz" }],
|
|
["bundle_id_hash", { bundle_id_hash: "0123456789ab" }],
|
|
["device_id_hash", { device_id_hash: "0123456789ab" }],
|
|
])("drops %s when accidentally passed to a tool event", (_label, payload) => {
|
|
const out = sanitize("tool:invoke", { tool: "x", platform: "ios", ...payload });
|
|
expect(out).toEqual({ tool: "x", platform: "ios" });
|
|
});
|
|
|
|
it("drops accidental error metadata", () => {
|
|
const out = sanitize("tool:fail", {
|
|
tool: "x",
|
|
platform: "ios",
|
|
duration_ms: 1,
|
|
error_message: "password=hunter2",
|
|
});
|
|
expect(out).toEqual({ tool: "x", platform: "ios", duration_ms: 1 });
|
|
});
|
|
});
|
|
|
|
describe("lens telemetry events", () => {
|
|
it("keeps the round_completed usage flags and drops leaked content", () => {
|
|
const out = sanitize("lens:round_completed", {
|
|
round: 2,
|
|
element_count: 3,
|
|
variant_count: 5,
|
|
annotation_count: 1,
|
|
element_comment_count: 2,
|
|
skipped_comment_count: 1,
|
|
has_global_comment: true,
|
|
inspector_used: true,
|
|
offscreen_revealed: false,
|
|
is_cli_session: true,
|
|
had_parked_await: false,
|
|
round_duration_ms: 1234,
|
|
platform: "ios",
|
|
// Content that must never survive:
|
|
element_name: "Checkout button",
|
|
comment_text: "make it pop",
|
|
});
|
|
expect(out).toEqual({
|
|
round: 2,
|
|
element_count: 3,
|
|
variant_count: 5,
|
|
annotation_count: 1,
|
|
element_comment_count: 2,
|
|
skipped_comment_count: 1,
|
|
has_global_comment: true,
|
|
inspector_used: true,
|
|
offscreen_revealed: false,
|
|
is_cli_session: true,
|
|
had_parked_await: false,
|
|
round_duration_ms: 1234,
|
|
platform: "ios",
|
|
});
|
|
});
|
|
|
|
it("drops a non-boolean inspector_used / offscreen_revealed", () => {
|
|
const out = sanitize("lens:round_completed", {
|
|
round: 1,
|
|
inspector_used: "yes",
|
|
offscreen_revealed: 1,
|
|
});
|
|
// The bool validator rejects non-booleans, so both keys are removed.
|
|
expect(out).toEqual({ round: 1 });
|
|
});
|
|
|
|
it("keeps agent_choice_count on cli_session_started and drops extras", () => {
|
|
const out = sanitize("lens:cli_session_started", {
|
|
agent_choice_count: 2,
|
|
agent_names: ["claude", "cursor"], // must be dropped
|
|
});
|
|
expect(out).toEqual({ agent_choice_count: 2 });
|
|
});
|
|
});
|
|
|
|
describe("tool:fail invalid_params", () => {
|
|
const base = { tool: "debugger-status", platform: "ios", duration_ms: 5 };
|
|
|
|
it("keeps a conforming list of schema-declared names (incl. the unrecognized_keys token)", () => {
|
|
const out = sanitize("tool:fail", {
|
|
...base,
|
|
invalid_params: ["device_id", "port", "unrecognized_keys"],
|
|
});
|
|
expect(out).toEqual({
|
|
...base,
|
|
invalid_params: ["device_id", "port", "unrecognized_keys"],
|
|
});
|
|
});
|
|
|
|
it("voids the whole array when longer than 16 (the emit side must cap first)", () => {
|
|
const out = sanitize("tool:fail", {
|
|
...base,
|
|
invalid_params: Array.from({ length: 17 }, (_, i) => `p${i}`),
|
|
});
|
|
expect(out).toEqual(base);
|
|
});
|
|
|
|
it("voids the whole array when any element is not an identifier-shaped name", () => {
|
|
// A leaked value (spaces, path chars) must take the entire property down,
|
|
// not slip through next to valid names.
|
|
const out = sanitize("tool:fail", {
|
|
...base,
|
|
invalid_params: ["port", "/Users/alice/secret file"],
|
|
});
|
|
expect(out).toEqual(base);
|
|
});
|
|
|
|
it("voids a non-array invalid_params", () => {
|
|
const out = sanitize("tool:fail", { ...base, invalid_params: "device_id" });
|
|
expect(out).toEqual(base);
|
|
});
|
|
});
|
|
|
|
describe("debugger:tool_outcome", () => {
|
|
const UUID_V = "a2d4a13a-6c0f-4e0e-8e14-1c8b8f0a1234";
|
|
|
|
it("accepts every declared outcome — including the gate-branch values", () => {
|
|
// stale_connection / reconnecting come from the socket-state gate, not
|
|
// the catch classifier; the enum must admit them at the SANITIZE level or
|
|
// those emissions silently lose their outcome dimension.
|
|
for (const outcome of DEBUGGER_TOOL_OUTCOMES) {
|
|
expect(
|
|
sanitize("debugger:tool_outcome", {
|
|
tool: "debugger-status",
|
|
outcome,
|
|
platform: "android",
|
|
tool_invocation_id: UUID_V,
|
|
})
|
|
).toEqual({
|
|
tool: "debugger-status",
|
|
outcome,
|
|
platform: "android",
|
|
tool_invocation_id: UUID_V,
|
|
});
|
|
}
|
|
expect(DEBUGGER_TOOL_OUTCOMES).toContain("reconnecting");
|
|
expect(DEBUGGER_TOOL_OUTCOMES).toContain("stale_connection");
|
|
expect(DEBUGGER_TOOL_OUTCOMES).toContain("connected");
|
|
});
|
|
|
|
it("accepts both emitting tools and drops any other tool id", () => {
|
|
for (const tool of ["debugger-status", "debugger-log-registry"] as const) {
|
|
expect(sanitize("debugger:tool_outcome", { tool, outcome: "connected" })).toEqual({
|
|
tool,
|
|
outcome: "connected",
|
|
});
|
|
}
|
|
expect(
|
|
sanitize("debugger:tool_outcome", { tool: "gesture-tap", outcome: "connected" })
|
|
).toEqual({ outcome: "connected" });
|
|
});
|
|
|
|
it("drops an unknown outcome value (free-text error strings must never ride this field)", () => {
|
|
expect(
|
|
sanitize("debugger:tool_outcome", {
|
|
tool: "debugger-status",
|
|
outcome: "ECONNREFUSED 127.0.0.1:8081",
|
|
})
|
|
).toEqual({ tool: "debugger-status" });
|
|
});
|
|
|
|
it("drops a malformed tool_invocation_id and unknown extra properties", () => {
|
|
expect(
|
|
sanitize("debugger:tool_outcome", {
|
|
tool: "debugger-log-registry",
|
|
outcome: "metro_not_running",
|
|
tool_invocation_id: "not-a-uuid",
|
|
detail: "Metro at port 8081 is not running",
|
|
guidance: "Do not retry",
|
|
})
|
|
).toEqual({ tool: "debugger-log-registry", outcome: "metro_not_running" });
|
|
});
|
|
});
|
|
|
|
describe("ALLOWED ↔ EVENT_NAMES sync", () => {
|
|
it("every declared event name has a sanitizer entry", () => {
|
|
for (const name of EVENT_NAMES) {
|
|
expect(ALLOWED[name]).toBeDefined();
|
|
}
|
|
});
|
|
|
|
it("every sanitizer entry is a declared event name", () => {
|
|
for (const name of Object.keys(ALLOWED)) {
|
|
expect(EVENT_NAMES).toContain(name);
|
|
}
|
|
});
|
|
});
|
|
});
|