Files
software-mansion__argent/packages/telemetry/test/base-props.test.ts
filip131311 1aaa94d118 feat(telemetry): detect cloud/remote AI-agent environment and tag every event (#418)
## What

Adds a **`cloud_agent`** field to the base telemetry props — alongside
`is_ci` — naming the **cloud/remote AI coding-agent environment** the
process is running in, or `null`. Since it lives in the base props, it's
attached to **every** telemetry event automatically.

Crucially this is **cloud-only**: it fires for vendor-hosted / remote
autonomous runs, **not** for a developer running an agent CLI locally. A
laptop `cursor-agent`/`claude` session returns `null`; only a genuine
remote environment resolves to a slug. (Generic cloud/CI execution stays
`is_ci`; this names the specific agent.)

## How

- **New `packages/telemetry/src/cloud-agent-detect.ts`** —
`detectCloudAgent(env?)` returns a canonical slug or `null`. Modelled on
`ci-detect.ts`: takes `env` lazily for tests, runs nothing at import
time, and only ever emits a fixed allowlist slug — a raw env value is
never copied into telemetry.
- **`base-props.ts`** — adds `cloud_agent: CloudAgent | null` next to
`is_ci`, computed once in the memoized invariant block (no per-event
cost in the long-lived tool-server).

## Signals (only high-confidence remote markers; local CLI use → `null`)

| Slug | Cloud marker | Basis |
|---|---|---|
| `claude_code` | `CLAUDE_CODE_ENVIRONMENT_KIND ∈ {byoc,
anthropic_cloud}`, a `remote_*`/`claude-in-teams` entrypoint, or
`CLAUDE_CODE_REMOTE_SESSION_ID` | Verified against the Claude Code
binary's own internal cloud check |
| `cursor` | `CURSOR_AGENT_WORKER_ID` / `CURSOR_WORKER_POOL_NAME`
(worker/pool exec) | Verified `cursor-agent` reads it from env; the
local CLI sets `CURSOR_AGENT` + `CURSOR_AGENT_CLI_LOCAL_MODE` instead |
| `copilot` | `GITHUB_ACTIONS` + `copilot-swe-agent` actor/workflow |
Confirmed actor identity (also `is_ci`) |
| `replit` | `REPLIT_AGENT` | Agent-specific (the broad `REPL_ID` fires
in any Repl and is **not** used) |
| `devin` | `/opt/.devin` VM marker | Confirmed (`@vercel/detect-agent`)
|
| `jules` | `/opt/environment_summary.sh` VM marker | Confirmed
(official Jules docs) |

### Deliberately excluded
- **Codex cloud** — no reliable cloud-specific marker exists
(`CODEX_SANDBOX` is the *local* sandbox signal; OpenAI declined an
`AGENT=codex` flag in `openai/codex#13416`). Not detected rather than
guessed.
- **Local agent CLIs** — `CURSOR_AGENT`, `CLAUDECODE` with
`entrypoint=cli`, plain `REPL_ID` workspaces: intentionally `null`.

## Privacy
Reads only a fixed allowlist of env-var **names** plus two filesystem
markers (`/opt/.devin`, `/opt/environment_summary.sh`); emits only a
fixed slug. The filesystem checks are injectable and guarded so a
permission error can never break telemetry.

## Tests
- New `test/cloud-agent-detect.test.ts` — every vendor signal, plus the
key **negatives** (local Claude `entrypoint=cli`, local `cursor-agent`,
plain Repl `REPL_ID`, normal CI run) asserting `null`.
- Updated `test/base-props.test.ts` for the renamed key.
- Full telemetry suite **191 passing**, typecheck + build clean. Live
check confirms this local Claude Code session resolves to `null` while a
simulated `byoc`/remote env resolves to `claude_code`.

---

> Scope note: started as a broad "is an agent in the loop" flag
(`agent_env`); narrowed to cloud-only per review — a human using an
agent locally shouldn't look like a cloud run.
2026-06-26 16:33:32 +02:00

112 lines
3.7 KiB
TypeScript

import { beforeEach, describe, expect, it } from "vitest";
import {
getBaseProps,
getSessionId,
_resetBasePropsCacheForTest,
_resetSessionIdForTest,
} from "../src/base-props.js";
import { snapshotEnv } from "./helpers.js";
const UUID_V4 = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i;
describe("base-props", () => {
// The invariant block (incl. is_ci) is memoized per process; reset it so each
// case re-reads the env it just set up.
beforeEach(() => _resetBasePropsCacheForTest());
it("returns the full base set with coarse CI telemetry", () => {
const restore = snapshotEnv(["CI", "GITHUB_ACTIONS"]);
try {
process.env.CI = "false";
delete process.env.GITHUB_ACTIONS;
const props = getBaseProps("cli");
expect(Object.keys(props).sort()).toEqual(
[
"$process_person_profile",
"$session_id",
"arch",
"cloud_agent",
"cli_version",
"is_ci",
"is_tty",
"node_version_major",
"os",
"runtime",
].sort()
);
expect(props.$process_person_profile).toBe(false);
expect(props.cli_version).toBe("0.0.0");
expect(typeof props.is_tty).toBe("boolean");
expect(props.is_ci).toBe(false);
expect(typeof props.node_version_major).toBe("string");
expect(typeof props.arch).toBe("string");
expect(props.runtime).toBe("cli");
expect(typeof props.$session_id).toBe("string");
expect(props.$session_id).toMatch(UUID_V4);
expect(props).not.toHaveProperty("ci_provider");
expect(props).not.toHaveProperty("is_container");
expect(props).not.toHaveProperty("container_runtime");
} finally {
restore();
}
});
it("sets is_ci when the process is running in CI", () => {
const restore = snapshotEnv(["CI"]);
try {
process.env.CI = "1";
expect(getBaseProps("cli").is_ci).toBe(true);
} finally {
restore();
}
});
it("sets cloud_agent when a cloud/remote agent runtime is detected", () => {
// REPLIT_AGENT is an env-only signal (no filesystem check) and is not the
// ambient env of this test process, so it resolves deterministically.
const restore = snapshotEnv(["REPLIT_AGENT"]);
try {
process.env.REPLIT_AGENT = "1";
expect(getBaseProps("cli").cloud_agent).toBe("replit");
} finally {
restore();
}
});
it("does not carry legacy cli_version_major_minor / full node_version", () => {
const props = getBaseProps("tool_server") as unknown as Record<string, unknown>;
expect(props).toHaveProperty("cli_version");
expect(props).not.toHaveProperty("cli_version_major_minor");
expect(props).not.toHaveProperty("node_version");
});
it("arch matches process.arch verbatim (no transformation)", () => {
const props = getBaseProps("cli");
expect(props.arch).toBe(process.arch);
});
describe("$session_id", () => {
it("is stable within a process across calls and across runtimes", () => {
const a = getBaseProps("cli").$session_id;
const b = getBaseProps("tool_server").$session_id;
const c = getBaseProps("installer").$session_id;
expect(a).toBe(b);
expect(b).toBe(c);
expect(a).toBe(getSessionId());
});
it("is a v4-shaped UUID", () => {
expect(getSessionId()).toMatch(UUID_V4);
});
it("rotates after the test seam runs (asserts fresh-process behaviour)", () => {
const before = getSessionId();
_resetSessionIdForTest();
const after = getSessionId();
expect(after).not.toBe(before);
expect(after).toMatch(UUID_V4);
// Subsequent getBaseProps calls reflect the new id immediately.
expect(getBaseProps("cli").$session_id).toBe(after);
});
});
});