Files
software-mansion__argent/packages/argent-cli/test/flow-script-render.test.ts
Hubert Gancarczyk 9c4bc9ef5b feat(flows): run an external script as a flow step (#865)
Stacked on #864, which adds the executor. This branch adds the `script:`
step
itself: the YAML directive, the runner integration, the CLI and MCP
rendering,
and the authoring reference.

```yaml
- script: { path: ../../scripts/seed-order.mjs }
- script: { path: ../../scripts/seed-order.mjs, timeout: 60000 }
```

A `script:` step runs an external `.mjs` file from the project, in a
fresh Node
process. What the script does is its own business — seeding an order,
creating
a test account, cleaning up what a run left behind are the reasons flows
have
wanted one. It drives no device, so a flow of nothing but scripts runs
with
nothing booted.

## What it adds

- **The directive.** The value is always a map; `path` is required and
obeys
the same name rules a `run:` target does, spelled for `.mjs`. It
resolves
against the directory of the flow file containing the step, so a
fragment
reaches the same script whichever flow composed it. A mis-cased path is
refused rather than run, because macOS and Windows would open it and
Linux CI
  would not.
- **The report.** The script's stdout and stderr come back on the step
and
print under the step line, on a pass as well as a failure — it is the
only
record of what the step did to the backend. Both renderers (CLI, MCP)
carry
  it, live and buffered.
- **The fail/error split.** A `fail` is the script's own answer; an
`error` is
the host's. That is what lets CI tell a regression from the machine it
ran
  on.
- **Two boundaries.** An uploaded flow carrying a `script:` step is
refused
before anything runs — its `.mjs` never left the client. A flow whose
script
  step sits beside a `run:` step still resolves a device.

## Review follow-ups

The last three commits answer a review of the three above them.

- A launch behind a leading `script:` was not read as the flow's leading
launch, so a Chromium flow that seeded a backend first hoisted no boot,
bound
  to whatever browser happened to be up, and passed a launch it never
  performed. The same lead-in walked a fragment past the
`executionPrerequisite` refusal its `echo:`-led twin is given. Both
readers
  now share one exhaustively-switched predicate.
- Coverage: the whole failure-kind table, the mis-cased arm CI never
ran, the
two unreached `scriptFileProblem` arms, symlink resolution for a script
path,
  and the CLI live-progress closure that no test had executed.
- Docs: the reference claimed the log was secret-redacted (it is not),
never
mentioned the environment allowlist, the working directory, the
time-limit
ceiling or the log caps, stated a fail/error rule that was not the
code's,
and was missing from its own table of contents. The skills stated a
closed
list of legal unrecorded insertions that excluded the only way this step
can
  ever be written.

## Verification

`npm run build`, `npx eslint . --max-warnings 0`, `npx prettier --check
.`,
`npm run knip`, `npm run typecheck:scripts` and `npm run test:scripts`
are
green, as are the three `tsconfig.test.json` typechecks. Suites:
tool-server
357 files / 4552 passed, `@argent/cli` 469, `@argent/mcp` 83,
`@argent/registry` 90.

Reproduced end to end against a real tool server and a real Electron
app, not
only in tests: a flow of `script:` then `launch:` now boots its own
Chromium
instance and reports it, where before it attached to a stray browser and
reported a green launch for an app path that did not exist.



<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

- **New Features**
- Added support for local `.mjs` scripts as flow steps, including
configurable timeouts, validation, cancellation, and device-free
execution.
- Script output is displayed in CLI runs and included in MCP flow
results, with indentation and truncation notices.
  - Script failures now provide clearer diagnostics and verdicts.
  - Flow authoring tools recognize and guide creation of script steps.

- **Documentation**
- Added guidance for script syntax, execution limits, output handling,
classification, and remote-run restrictions.
  - Updated flow creation and QA guidance to include script steps.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Hubert Gancarczyk <claude-hubert.gancarczyk@swmansion.com>
2026-09-07 12:08:13 +02:00

157 lines
5.9 KiB
TypeScript

import { describe, it, expect } from "vitest";
import {
renderReport,
renderFailedSteps,
renderScriptLogLines,
type FlowReport,
type StepReport,
} from "../src/flow.js";
function report(steps: StepReport[]): FlowReport {
const counted = steps.filter((s) => s.kind !== "echo");
return {
flow: "seed",
device: "",
ok: counted.every((s) => s.status === "pass" || s.status === "skip"),
passed: counted.filter((s) => s.status === "pass").length,
failed: counted.filter((s) => s.status === "fail").length,
skipped: counted.filter((s) => s.status === "skip").length,
errored: counted.filter((s) => s.status === "error").length,
steps,
};
}
const PASSING: StepReport = {
index: 0,
kind: "script",
status: "pass",
target: "scripts/seed.mjs",
scriptLog: "creating order\norder 4711 created\n",
};
describe("script log rendering", () => {
it("prints one indented line per line the script wrote, under the step line", () => {
const out = renderReport(report([PASSING]));
expect(out).toContain("✓ 1 script scripts/seed.mjs");
expect(out).toContain(" │ creating order");
expect(out).toContain(" │ order 4711 created");
expect(out).not.toMatch(/│ \n/);
});
it("says so when a log limit dropped output, since the text carries no marker", () => {
const lines = renderScriptLogLines({ ...PASSING, scriptLogTruncated: true }, 1);
expect(lines).toHaveLength(3);
expect(lines.at(-1)).toContain("… output truncated");
});
it("prints the truncation notice alone when the whole log was dropped", () => {
const lines = renderScriptLogLines(
{ ...PASSING, scriptLog: undefined, scriptLogTruncated: true },
1
);
expect(lines).toHaveLength(1);
expect(lines[0]).toContain("… output truncated");
});
it("prints nothing for a step with no log, or a non-string one off the wire", () => {
expect(renderScriptLogLines({ ...PASSING, scriptLog: undefined }, 1)).toEqual([]);
expect(renderScriptLogLines({ ...PASSING, scriptLog: "" }, 1)).toEqual([]);
expect(
renderScriptLogLines({ ...PASSING, scriptLog: { evil: true } as unknown as string }, 1)
).toEqual([]);
expect(
renderScriptLogLines(
{ ...PASSING, scriptLog: undefined, scriptLogTruncated: "yes" as unknown as boolean },
1
)
).toEqual([]);
});
it("carries a failed script's log into batch mode's failed-step list", () => {
const failed: StepReport = {
index: 0,
kind: "script",
status: "fail",
target: "scripts/seed.mjs",
reason: "Error: seed API returned 500",
scriptLog: "POST /orders -> 500\n",
};
const lines = renderFailedSteps(report([failed]));
expect(lines[0]).toContain("script scripts/seed.mjs — Error: seed API returned 500");
expect(lines[1]).toContain("│ POST /orders -> 500");
});
it("carries a passing script's log into batch mode, the surface CI reads", () => {
const lines = renderFailedSteps(report([PASSING]));
expect(lines[0]).toContain("✓ 1 script scripts/seed.mjs");
expect(lines[1]).toContain("│ creating order");
expect(lines[2]).toContain("│ order 4711 created");
});
it("keeps a passing seed script's log beside the later step that failed", () => {
const failed: StepReport = {
index: 1,
kind: "tap",
status: "fail",
target: "#checkout",
reason: "not found",
};
const lines = renderFailedSteps(report([PASSING, failed]));
expect(lines[0]).toContain("✓ 1 script scripts/seed.mjs");
expect(lines[1]).toContain("│ creating order");
expect(lines.at(-1)).toContain("✗ 2 tap #checkout — not found");
});
it("prints a passing script's truncation notice in batch mode as well", () => {
const lines = renderFailedSteps(
report([{ ...PASSING, scriptLog: undefined, scriptLogTruncated: true }])
);
expect(lines).toHaveLength(2);
expect(lines[0]).toContain("✓ 1 script scripts/seed.mjs");
expect(lines[1]).toContain("… output truncated");
});
it("carries a passing script's executor note into batch mode, log or no log", () => {
// A silent script under a clamped time limit is the case with nothing else
// to show: no log, no warning, a pass — and a host that quietly lowered the
// bound the flow asked for.
const clamped: StepReport = {
index: 0,
kind: "script",
status: "pass",
target: "scripts/seed.mjs",
reason:
"The requested 10m time limit is above this host's maximum of 5m; the step ran with the maximum.",
};
const lines = renderFailedSteps(report([clamped]));
expect(lines).toHaveLength(1);
expect(lines[0]).toContain("✓ 1 script scripts/seed.mjs");
expect(lines[0]).toContain("above this host's maximum of 5m");
});
it("leaves a passing NON-script step's self-narrating reason out of batch mode", () => {
// A `when` guard, a snapshot and a chromium launch all report a reason on a
// pass. Those narrate a result the summary already counts, so admitting
// every passing reason would print most of the run back.
const guard: StepReport = {
index: 0,
kind: "when",
status: "pass",
reason: "condition met (platform ios)",
};
expect(renderFailedSteps(report([guard]))).toEqual([]);
});
it("leaves a passing step that wrote no log out of batch mode", () => {
const tapped: StepReport = { index: 0, kind: "tap", status: "pass", target: "#checkout" };
const lines = renderFailedSteps(report([tapped, PASSING]));
expect(lines.some((l) => l.includes("tap #checkout"))).toBe(false);
expect(lines[0]).toContain("✓ 2 script scripts/seed.mjs");
});
it("indents a nested script step's log with the step, not against the margin", () => {
const nested = renderScriptLogLines({ ...PASSING, depth: 2 }, 3);
expect(nested[0]).toBe(`${" ".repeat(7)} │ creating order`);
});
});