Files
software-mansion__argent/scripts/e2e-full/lib/discover-tools.sh
Ignacy Łątka cd4190b80e Full tgz-only E2E harness (scripts/e2e-full) (#495)
Now verified end-to-end on the real Mac:
install/introspection/validation, the android tier (against a booted
emulator), and the chromium tier all pass from a locally packed tgz; the
rn tier auto-skips without `~/dev/bluesky` as designed.

## What this is

A release-gating end-to-end harness under `scripts/e2e-full/` that
starts from **only a `swmansion-argent-*.tgz` bundle** and exercises the
whole product: the install flow, every CLI command, all 70 tools'
argument validation, and a happy-path run of every tool against real
devices. Intended to be run on the real Linux box and the real Mac
before a release.

Everything runs under a throwaway `HOME` + npm prefix, so it never
touches the machine's real `~/.argent`, editor MCP configs, or global
packages (safe on a shared box). It emits a JSONL log + a markdown
coverage report and exits non-zero if any hard assertion fails.

## Phases

| phase | needs | covers |
|---|---|---|
| `install` | npm + network | `npm i -g <tgz>`, bundled binaries, `init`
(global + `--local`), `update`, `uninstall`, telemetry, MCP-config
generation |
| `introspection` | — | `--version/--help`, `tools describe` for **all
70** tools, feature flags, `server start/status/logs/stop`,
`link/unlink` |
| `validation` | — | per tool: missing-required / bad-enum / bad-type
rejection (schema-driven, no hardware) |
| `android` | Android emulator | happy-path of every
touch/gesture/screenshot/app-lifecycle tool |
| `chromium` | Electron (bundled optional dep) + a display | boots a
generated Electron app; drives CDP tools
(scroll/drag/tabs/cookies/storage) |
| `rn` | `~/dev/bluesky` + Android device | debugger + react/native
profiler + network chain against the real Bluesky app |

Tiers auto-skip with a recorded reason when prerequisites are missing.
iOS / tvOS / Vega are intentionally out of scope for now.

## What I actually verified (real runs, Linux)

- **install** 18✓/0✗, **introspection** 85✓/0✗ (all 70 `tools
describe`), **validation** 81✓/0✗, **chromium** 24✓/0✗ (real Electron
app + CDP).
- Holistic run: **190✓ / 0✗ / 34∼, exit 0.** Failure-gate confirmed
(non-zero on fail), `bash -n` clean on all files, all 72 android/rn JSON
arg literals linted.

## What I did NOT verify live (please scrutinize)

- **`android` and `rn` tiers were never run against a live device.** The
Android emulator would not stay up on my box (memory-constrained shared
machine; QEMU threads hung seconds after boot across ~7 attempts). The
code is complete and uses the same machinery the `chromium` tier
exercised live, but it has **not** been proven end-to-end. This is the
main reason for the "not certain how well it works" caveat.
- The RN tier's exact debugger/profiler arg shapes are derived from
`tools describe`, not confirmed against a running Metro.

## Run it

```bash
bash scripts/e2e-full/run-e2e.sh                                   # everything applicable to this OS
bash scripts/e2e-full/run-e2e.sh --skip-install --phase introspection,validation   # fast offline core
bash scripts/e2e-full/run-e2e.sh --phase android --android-serial emulator-5554     # inject a device
```
2026-07-20 16:43:34 +02:00

79 lines
3.1 KiB
Bash

#!/usr/bin/env bash
# Tool discovery + schema parsing.
#
# The whole harness is schema-driven off the CLI's own introspection so it never
# drifts from the shipped tool set:
# - `argent tools` -> the authoritative list of tool names
# - `argent tools describe <t>` -> per-flag model (name / type / required / enum)
#
# The `describe` flag lines look like:
# --udid <value> string (required) Target device id ...
# --x <value> number (required) Normalized horizontal ...
# --button <value> enum: "home" | "back" | ... (required) Hardware button ...
# --scale <value> number Scale factor ...
#
# We parse those into a compact model file per tool under $E2E_WORK/tools/<t>.model,
# one line per flag: <name>\t<kind>\t<required 0|1>\t<enumvals csv>
# kind ∈ string|number|boolean|enum|array|object|unknown
: "${E2E_WORK:?E2E_WORK must be set}"
TOOLS_DIR="$E2E_WORK/tools"
mkdir -p "$TOOLS_DIR"
# List all tool names (cached).
list_tool_names() {
local cache="$E2E_WORK/tool-names.txt"
if [ ! -s "$cache" ]; then
argent_cli tools || true
printf '%s\n' "$CLI_OUT" \
| grep -oE '^ [a-z][a-z0-9-]+' \
| tr -d ' ' \
| sort -u > "$cache"
fi
cat "$cache"
}
# Parse one tool's describe output into a model file; echoes the model path.
parse_tool_model() { # <tool>
local tool="$1"
local model="$TOOLS_DIR/$tool.model"
if [ -s "$model" ]; then printf '%s\n' "$model"; return 0; fi
argent_cli tools describe "$tool" || true
# Isolate the "Flags:" section and walk each "--flag" line.
printf '%s\n' "$CLI_OUT" | awk '
/^Flags:/ { inflags=1; next }
inflags && /^[[:space:]]*--/ {
line=$0
# flag name
match(line, /--[a-zA-Z0-9-]+/); name=substr(line, RSTART+2, RLENGTH-2)
req = (line ~ /\(required\)/) ? 1 : 0
kind="unknown"; enums=""
if (line ~ /enum:/) {
kind="enum"
# capture the quoted enum members
s=line
while (match(s, /"[^"]+"/)) {
v=substr(s, RSTART+1, RLENGTH-2)
enums = (enums=="") ? v : enums "," v
s=substr(s, RSTART+RLENGTH)
}
} else if (line ~ /\bnumber\b/) { kind="number" }
else if (line ~ /\bboolean\b/) { kind="boolean" }
else if (line ~ /\bstring\b/) { kind="string" }
else if (line ~ /\barray\b/ || line ~ /\[\]/) { kind="array" }
else if (line ~ /\bobject\b/) { kind="object" }
printf "%s\t%s\t%s\t%s\n", name, kind, req, enums
}
inflags && /^[[:space:]]*\(no parameters\)/ { }
' > "$model"
printf '%s\n' "$model"
}
# Helpers over a parsed model file.
model_required_flags() { awk -F'\t' '$3==1 {print $1}' "$1"; } # names of required flags
model_flag_kind() { awk -F'\t' -v n="$2" '$1==n {print $2}' "$1"; } # kind of flag $2
model_enum_flags() { awk -F'\t' '$2=="enum" {print $1}' "$1"; } # names of enum flags
model_enum_values() { awk -F'\t' -v n="$2" '$1==n {print $4}' "$1"; } # csv enum values of flag $2
model_number_flags() { awk -F'\t' '$2=="number" {print $1}' "$1"; } # names of number flags
model_flag_count() { wc -l < "$1" | tr -d ' '; }