actionbook__actionbook/playground/actionbook-web-test/references/report-format.md

Test Report Format Reference

How actionbook-web-test generates visual HTML reports using json-ui (@actionbookdev/json-ui).

Overview

After test execution, the agent constructs a json-ui JSON document from collected test results, then renders it to a standalone HTML file. The HTML is self-contained (inline CSS, embedded screenshots) and works offline.

Report Structure

A test report is a json-ui Report node with this layout:

Report
├── BrandHeader              — Report title and branding
├── Section: Summary         — MetricsGrid with pass/fail/skip/duration
├── Section: Environment     — DefinitionList with browser, URL, profile info
├── Section: Results         — Table with per-test status overview
├── Section: <test-name> (×N, one per test, collapsible)
│   ├── Callout              — Error details (FAIL tests only)
│   ├── ContributionList     — Step-by-step execution log with PASS/FAIL/SKIP badges
│   └── Image                — Screenshot at end of test (base64 data URL, ALL tests)
├── Section: Console         — Callout with JS errors (if any)
└── BrandFooter              — Timestamp and attribution

Every test gets its own collapsible section with a ContributionList and Image, regardless of PASS/FAIL/SKIP status. The section icon reflects the outcome: "check" for PASS, "warning" for FAIL, "skip" for SKIP.

Complete Report JSON Template

{
  "type": "Report",
  "props": {
    "title": "Actionbook Test Report"
  },
  "children": [
    {
      "type": "BrandHeader",
      "props": {
        "badge": "Actionbook Test",
        "poweredBy": "actionbook-web-test",
        "showBadge": true
      }
    },

    {
      "type": "Section",
      "props": { "title": "Summary", "icon": "chart" },
      "children": [
        {
          "type": "MetricsGrid",
          "props": {
            "cols": 5,
            "metrics": [
              { "label": "Total", "value": "{{summary.total}}", "icon": "list" },
              { "label": "Passed", "value": "{{summary.passed}}", "trend": "up", "icon": "check" },
              { "label": "Failed", "value": "{{summary.failed}}", "trend": "down", "icon": "warning" },
              { "label": "Skipped", "value": "{{summary.skipped}}", "icon": "skip" },
              { "label": "Duration", "value": "{{summary.duration}}", "icon": "clock" }
            ]
          }
        }
      ]
    },

    {
      "type": "Section",
      "props": { "title": "Environment", "icon": "settings", "collapsible": true },
      "children": [
        {
          "type": "DefinitionList",
          "props": {
            "items": [
              { "term": "Timestamp", "definition": "{{timestamp}}" },
              { "term": "Browser", "definition": "{{browser_info}}" },
              { "term": "Viewport", "definition": "{{viewport}}" },
              { "term": "Profile", "definition": "{{profile}}" },
              { "term": "Mode", "definition": "{{mode}}" }
            ]
          }
        }
      ]
    },

    {
      "type": "Section",
      "props": { "title": "Test Results", "icon": "code" },
      "children": [
        {
          "type": "Table",
          "props": {
            "columns": [
              { "key": "status", "label": "Status" },
              { "key": "name", "label": "Test" },
              { "key": "steps", "label": "Steps" },
              { "key": "assertions", "label": "Assertions" },
              { "key": "duration", "label": "Duration" },
              { "key": "tags", "label": "Tags" }
            ],
            "rows": "{{results_rows}}",
            "striped": true,
            "caption": "{{results_count}} tests executed"
          }
        }
      ]
    },

    "{{per_test_detail_sections}}",

    {
      "type": "BrandFooter",
      "props": {
        "timestamp": "{{timestamp}}",
        "attribution": "Generated by actionbook-web-test"
      }
    }
  ]
}

Note: {{...}} placeholders above are for illustration. The agent replaces them with actual values when constructing the JSON.

---

Component Construction Guide

1. Summary Metrics

Build MetricsGrid from the test summary:

// Pseudocode — agent constructs this mentally
const metricsGrid = {
  type: "MetricsGrid",
  props: {
    cols: 5,
    metrics: [
      {
        label: "Total",
        value: String(summary.total),
        icon: "list"
      },
      {
        label: "Passed",
        value: String(summary.passed),
        trend: summary.passed > 0 ? "up" : undefined,
        icon: "check"
      },
      {
        label: "Failed",
        value: String(summary.failed),
        trend: summary.failed > 0 ? "down" : undefined,
        icon: "warning"
      },
      {
        label: "Skipped",
        value: String(summary.skipped),
        icon: "skip"
      },
      {
        label: "Duration",
        value: formatDuration(summary.duration),  // e.g. "12.4s"
        icon: "clock"
      }
    ]
  }
};

Trend indicators:

• "up" — green arrow, use for passed count
• "down" — red arrow, use for failed count
• omit — neutral, use for total/skipped/duration

2. Results Table

Build Table rows from each test result:

const resultsTable = {
  type: "Table",
  props: {
    columns: [
      { key: "status", label: "Status" },
      { key: "name", label: "Test" },
      { key: "steps", label: "Steps" },
      { key: "assertions", label: "Assertions" },
      { key: "duration", label: "Duration" },
      { key: "tags", label: "Tags" }
    ],
    rows: results.map(r => ({
      status: r.status === "passed" ? "PASS" :
              r.status === "failed" ? "FAIL" : "SKIP",
      name: r.name,
      steps: `${r.steps.passed}/${r.steps.total}`,
      assertions: `${r.assertions.passed}/${r.assertions.total}`,
      duration: formatDuration(r.duration),
      tags: r.tags?.join(", ") ?? ""
    })),
    striped: true
  }
};

3. Per-Test Detail Sections

Generate one collapsible Section for every test (PASS, FAIL, or SKIP). Each section contains a ContributionList of steps and an Image with the test's final screenshot.

const testSection = {
  type: "Section",
  props: {
    title: `${result.name} — Step Details`,
    icon: result.status === "passed" ? "check" :
          result.status === "failed" ? "warning" : "skip",
    collapsible: true
  },
  children: [
    // Error callout — ONLY for failed tests
    ...(result.status === "failed" ? [{
      type: "Callout",
      props: {
        type: "important",
        title: `Step ${failedStep.index}: ${failedStep.name}`,
        content: [
          `**Error:** ${failedStep.error}`,
          failedStep.selector ? `**Selector:** \`${failedStep.selector}\`` : null,
          failedStep.expected ? `**Expected:** ${failedStep.expected}` : null,
          failedStep.actual ? `**Actual:** ${failedStep.actual}` : null
        ].filter(Boolean).join("\n\n")
      }
    }] : []),
    // Step-by-step execution log — ALL tests
    {
      type: "ContributionList",
      props: {
        numbered: true,
        items: result.steps.map(step => ({
          title: step.name,
          badge: step.status === "passed" ? "PASS" :
                 step.status === "failed" ? "FAIL" : "SKIP",
          description: step.cli_command
            ? `\`${step.cli_command}\` (${formatDuration(step.duration)})`
            : `Skipped`
        }))
      }
    },
    // Final screenshot — ALL tests (MUST be base64 data URL)
    {
      type: "Image",
      props: {
        src: result.screenshotBase64,  // "data:image/png;base64,..."
        alt: `${result.name} — ${result.status} state`,
        caption: result.status === "failed"
          ? `Page state at point of failure (${failedStep.name})`
          : `Page state after test completed (${result.status.toUpperCase()})`
      }
    }
  ]
};

4. Console Errors Section

If JS errors were captured during the test run:

const consoleSection = {
  type: "Section",
  props: { title: "Console Errors", icon: "warning", collapsible: true },
  children: consoleErrors.map(error => ({
    type: "Callout",
    props: {
      type: "warning",
      title: error.source ?? "JavaScript Error",
      content: `\`\`\`\n${error.message}\n\`\`\``
    }
  }))
};

Only include this section if there are console errors to report.

5. Environment Info

const envSection = {
  type: "Section",
  props: { title: "Environment", icon: "settings", collapsible: true },
  children: [
    {
      type: "DefinitionList",
      props: {
        items: [
          { term: "Timestamp", definition: new Date().toISOString() },
          { term: "Browser", definition: "Chromium 125.0.6422.0 (headless)" },
          { term: "Viewport", definition: "1280x720" },
          { term: "Profile", definition: setup.profile ?? "default" },
          { term: "Mode", definition: "extension" },  // or "cdp"
          { term: "Target URL", definition: workflow.url },
          { term: "Tags", definition: workflow.tags.join(", ") }
        ]
      }
    }
  ]
};

---

Rendering

The json-ui package is @actionbookdev/json-ui on npm. Use npx — no global install needed:

# Write the report JSON to a file first, then render:
npx @actionbookdev/json-ui render test-report.json -o test-report.html

# Render without auto-opening browser
npx @actionbookdev/json-ui render test-report.json -o test-report.html --no-open

# Pipe from stdin
cat test-report.json | npx @actionbookdev/json-ui render - -o test-report.html

IMPORTANT: Always write the JSON to a file first, then render with npx @actionbookdev/json-ui. Do NOT attempt to generate raw HTML directly — json-ui handles all styling, theming, dark mode, and responsive layout.

---

Example: Minimal Report

For a single test with 3 passing steps:

{
  "type": "Report",
  "props": { "title": "Actionbook Test Report" },
  "children": [
    {
      "type": "BrandHeader",
      "props": { "badge": "Actionbook Test", "poweredBy": "actionbook-web-test" }
    },
    {
      "type": "Section",
      "props": { "title": "Summary", "icon": "chart" },
      "children": [
        {
          "type": "MetricsGrid",
          "props": {
            "cols": 4,
            "metrics": [
              { "label": "Total", "value": "1", "icon": "list" },
              { "label": "Passed", "value": "1", "trend": "up", "icon": "check" },
              { "label": "Failed", "value": "0", "icon": "warning" },
              { "label": "Duration", "value": "3.2s", "icon": "clock" }
            ]
          }
        }
      ]
    },
    {
      "type": "Section",
      "props": { "title": "Test Results", "icon": "code" },
      "children": [
        {
          "type": "Table",
          "props": {
            "columns": [
              { "key": "status", "label": "Status" },
              { "key": "name", "label": "Test" },
              { "key": "steps", "label": "Steps" },
              { "key": "duration", "label": "Duration" }
            ],
            "rows": [
              { "status": "PASS", "name": "homepage-smoke", "steps": "3/3", "duration": "3.2s" }
            ]
          }
        }
      ]
    },
    {
      "type": "Section",
      "props": { "title": "homepage-smoke — Step Details", "icon": "check", "collapsible": true },
      "children": [
        {
          "type": "ContributionList",
          "props": {
            "numbered": true,
            "items": [
              { "title": "Open homepage", "badge": "PASS", "description": "`browser open https://example.com` (0.5s)" },
              { "title": "Verify heading", "badge": "PASS", "description": "assert text-contains \"Welcome\" (0.3s)" },
              { "title": "Verify footer", "badge": "PASS", "description": "`browser wait \"footer\"` (0.1s)" }
            ]
          }
        },
        {
          "type": "Image",
          "props": {
            "src": "data:image/png;base64,...",
            "alt": "homepage-smoke — final state",
            "caption": "Page state after test completed (PASS)"
          }
        }
      ]
    },
    {
      "type": "BrandFooter",
      "props": {
        "timestamp": "2026-03-13T10:00:03Z",
        "attribution": "Generated by actionbook-web-test"
      }
    }
  ]
}

Example: Report with Failures

For 2 tests (1 pass, 1 fail with screenshot):

{
  "type": "Report",
  "props": { "title": "Actionbook Test Report" },
  "children": [
    {
      "type": "BrandHeader",
      "props": { "badge": "Actionbook Test", "poweredBy": "actionbook-web-test" }
    },
    {
      "type": "Section",
      "props": { "title": "Summary", "icon": "chart" },
      "children": [
        {
          "type": "MetricsGrid",
          "props": {
            "cols": 4,
            "metrics": [
              { "label": "Total", "value": "2", "icon": "list" },
              { "label": "Passed", "value": "1", "trend": "up", "icon": "check" },
              { "label": "Failed", "value": "1", "trend": "down", "icon": "warning" },
              { "label": "Duration", "value": "11.3s", "icon": "clock" }
            ]
          }
        }
      ]
    },
    {
      "type": "Section",
      "props": { "title": "Test Results", "icon": "code" },
      "children": [
        {
          "type": "Table",
          "props": {
            "columns": [
              { "key": "status", "label": "Status" },
              { "key": "name", "label": "Test" },
              { "key": "steps", "label": "Steps" },
              { "key": "assertions", "label": "Assertions" },
              { "key": "duration", "label": "Duration" }
            ],
            "rows": [
              { "status": "PASS", "name": "homepage-smoke", "steps": "6/6", "assertions": "3/3", "duration": "3.2s" },
              { "status": "FAIL", "name": "login-flow", "steps": "3/7", "assertions": "1/2", "duration": "8.1s" }
            ],
            "striped": true
          }
        }
      ]
    },
    {
      "type": "Section",
      "props": { "title": "homepage-smoke — Step Details", "icon": "check", "collapsible": true },
      "children": [
        {
          "type": "ContributionList",
          "props": {
            "numbered": true,
            "items": [
              { "title": "Open homepage", "badge": "PASS", "description": "`browser open https://app.example.com` (0.5s)" },
              { "title": "Verify hero section", "badge": "PASS", "description": "`browser wait \".hero\"` (0.3s)" },
              { "title": "Verify nav links", "badge": "PASS", "description": "assert element-count >= 4 (0.2s)" },
              { "title": "Verify CTA button", "badge": "PASS", "description": "`browser wait \"a.cta-button\"` (0.1s)" },
              { "title": "Check no JS errors", "badge": "PASS", "description": "`browser console --level error` (0.5s)" },
              { "title": "Verify footer", "badge": "PASS", "description": "`browser wait \"footer\"` (0.1s)" }
            ]
          }
        },
        {
          "type": "Image",
          "props": {
            "src": "data:image/png;base64,...",
            "alt": "homepage-smoke — final state",
            "caption": "Page state after test completed (PASS)"
          }
        }
      ]
    },
    {
      "type": "Section",
      "props": { "title": "login-flow — Step Details", "icon": "warning", "collapsible": true },
      "children": [
        {
          "type": "Callout",
          "props": {
            "type": "important",
            "title": "Step 4: Click submit",
            "content": "**Error:** Selector not found: `button[type='submit']`\n\n**Recovery attempted:** Snapshot fallback found `button.btn-primary` but element was disabled.\n\n**Possible cause:** Submit button is conditionally rendered after form validation passes."
          }
        },
        {
          "type": "ContributionList",
          "props": {
            "numbered": true,
            "items": [
              { "title": "Open login page", "badge": "PASS", "description": "`browser open https://app.example.com/login` (0.8s)" },
              { "title": "Fill username", "badge": "PASS", "description": "`browser fill \"#email\" \"test@example.com\"` (0.2s)" },
              { "title": "Fill password", "badge": "PASS", "description": "`browser fill \"#password\" \"***\"` (0.1s)" },
              { "title": "Click submit", "badge": "FAIL", "description": "`browser click \"button[type='submit']\"` — Selector not found" },
              { "title": "Verify dashboard redirect", "badge": "SKIP", "description": "Skipped due to previous failure" },
              { "title": "Check welcome message", "badge": "SKIP", "description": "Skipped due to previous failure" },
              { "title": "Close browser", "badge": "SKIP", "description": "Skipped due to previous failure" }
            ]
          }
        },
        {
          "type": "Image",
          "props": {
            "src": "data:image/png;base64,...",
            "alt": "login-flow — failure state",
            "caption": "Page state at point of failure (step 4: Click submit)"
          }
        }
      ]
    },
    {
      "type": "BrandFooter",
      "props": {
        "timestamp": "2026-03-13T10:00:11Z",
        "attribution": "Generated by actionbook-web-test"
      }
    }
  ]
}

---

Duration Formatting

Raw (ms)	Formatted
150	0.2s
3200	3.2s
12400	12.4s
65000	1m 5s
180000	3m 0s

Status Badge Colors

Status	Display	Color Intent
passed	PASS	green (success)
failed	FAIL	red (danger)
skipped	SKIP	gray (muted)
error	ERROR	orange (warning)