mirror of
https://github.com/software-mansion/argent.git
synced 2026-09-14 19:27:14 +08:00
cd4190b80e
Now verified end-to-end on the real Mac: install/introspection/validation, the android tier (against a booted emulator), and the chromium tier all pass from a locally packed tgz; the rn tier auto-skips without `~/dev/bluesky` as designed. ## What this is A release-gating end-to-end harness under `scripts/e2e-full/` that starts from **only a `swmansion-argent-*.tgz` bundle** and exercises the whole product: the install flow, every CLI command, all 70 tools' argument validation, and a happy-path run of every tool against real devices. Intended to be run on the real Linux box and the real Mac before a release. Everything runs under a throwaway `HOME` + npm prefix, so it never touches the machine's real `~/.argent`, editor MCP configs, or global packages (safe on a shared box). It emits a JSONL log + a markdown coverage report and exits non-zero if any hard assertion fails. ## Phases | phase | needs | covers | |---|---|---| | `install` | npm + network | `npm i -g <tgz>`, bundled binaries, `init` (global + `--local`), `update`, `uninstall`, telemetry, MCP-config generation | | `introspection` | — | `--version/--help`, `tools describe` for **all 70** tools, feature flags, `server start/status/logs/stop`, `link/unlink` | | `validation` | — | per tool: missing-required / bad-enum / bad-type rejection (schema-driven, no hardware) | | `android` | Android emulator | happy-path of every touch/gesture/screenshot/app-lifecycle tool | | `chromium` | Electron (bundled optional dep) + a display | boots a generated Electron app; drives CDP tools (scroll/drag/tabs/cookies/storage) | | `rn` | `~/dev/bluesky` + Android device | debugger + react/native profiler + network chain against the real Bluesky app | Tiers auto-skip with a recorded reason when prerequisites are missing. iOS / tvOS / Vega are intentionally out of scope for now. ## What I actually verified (real runs, Linux) - **install** 18✓/0✗, **introspection** 85✓/0✗ (all 70 `tools describe`), **validation** 81✓/0✗, **chromium** 24✓/0✗ (real Electron app + CDP). - Holistic run: **190✓ / 0✗ / 34∼, exit 0.** Failure-gate confirmed (non-zero on fail), `bash -n` clean on all files, all 72 android/rn JSON arg literals linted. ## What I did NOT verify live (please scrutinize) - **`android` and `rn` tiers were never run against a live device.** The Android emulator would not stay up on my box (memory-constrained shared machine; QEMU threads hung seconds after boot across ~7 attempts). The code is complete and uses the same machinery the `chromium` tier exercised live, but it has **not** been proven end-to-end. This is the main reason for the "not certain how well it works" caveat. - The RN tier's exact debugger/profiler arg shapes are derived from `tools describe`, not confirmed against a running Metro. ## Run it ```bash bash scripts/e2e-full/run-e2e.sh # everything applicable to this OS bash scripts/e2e-full/run-e2e.sh --skip-install --phase introspection,validation # fast offline core bash scripts/e2e-full/run-e2e.sh --phase android --android-serial emulator-5554 # inject a device ```
79 lines
3.1 KiB
Bash
79 lines
3.1 KiB
Bash
#!/usr/bin/env bash
|
|
# Tool discovery + schema parsing.
|
|
#
|
|
# The whole harness is schema-driven off the CLI's own introspection so it never
|
|
# drifts from the shipped tool set:
|
|
# - `argent tools` -> the authoritative list of tool names
|
|
# - `argent tools describe <t>` -> per-flag model (name / type / required / enum)
|
|
#
|
|
# The `describe` flag lines look like:
|
|
# --udid <value> string (required) Target device id ...
|
|
# --x <value> number (required) Normalized horizontal ...
|
|
# --button <value> enum: "home" | "back" | ... (required) Hardware button ...
|
|
# --scale <value> number Scale factor ...
|
|
#
|
|
# We parse those into a compact model file per tool under $E2E_WORK/tools/<t>.model,
|
|
# one line per flag: <name>\t<kind>\t<required 0|1>\t<enumvals csv>
|
|
# kind ∈ string|number|boolean|enum|array|object|unknown
|
|
|
|
: "${E2E_WORK:?E2E_WORK must be set}"
|
|
TOOLS_DIR="$E2E_WORK/tools"
|
|
mkdir -p "$TOOLS_DIR"
|
|
|
|
# List all tool names (cached).
|
|
list_tool_names() {
|
|
local cache="$E2E_WORK/tool-names.txt"
|
|
if [ ! -s "$cache" ]; then
|
|
argent_cli tools || true
|
|
printf '%s\n' "$CLI_OUT" \
|
|
| grep -oE '^ [a-z][a-z0-9-]+' \
|
|
| tr -d ' ' \
|
|
| sort -u > "$cache"
|
|
fi
|
|
cat "$cache"
|
|
}
|
|
|
|
# Parse one tool's describe output into a model file; echoes the model path.
|
|
parse_tool_model() { # <tool>
|
|
local tool="$1"
|
|
local model="$TOOLS_DIR/$tool.model"
|
|
if [ -s "$model" ]; then printf '%s\n' "$model"; return 0; fi
|
|
argent_cli tools describe "$tool" || true
|
|
# Isolate the "Flags:" section and walk each "--flag" line.
|
|
printf '%s\n' "$CLI_OUT" | awk '
|
|
/^Flags:/ { inflags=1; next }
|
|
inflags && /^[[:space:]]*--/ {
|
|
line=$0
|
|
# flag name
|
|
match(line, /--[a-zA-Z0-9-]+/); name=substr(line, RSTART+2, RLENGTH-2)
|
|
req = (line ~ /\(required\)/) ? 1 : 0
|
|
kind="unknown"; enums=""
|
|
if (line ~ /enum:/) {
|
|
kind="enum"
|
|
# capture the quoted enum members
|
|
s=line
|
|
while (match(s, /"[^"]+"/)) {
|
|
v=substr(s, RSTART+1, RLENGTH-2)
|
|
enums = (enums=="") ? v : enums "," v
|
|
s=substr(s, RSTART+RLENGTH)
|
|
}
|
|
} else if (line ~ /\bnumber\b/) { kind="number" }
|
|
else if (line ~ /\bboolean\b/) { kind="boolean" }
|
|
else if (line ~ /\bstring\b/) { kind="string" }
|
|
else if (line ~ /\barray\b/ || line ~ /\[\]/) { kind="array" }
|
|
else if (line ~ /\bobject\b/) { kind="object" }
|
|
printf "%s\t%s\t%s\t%s\n", name, kind, req, enums
|
|
}
|
|
inflags && /^[[:space:]]*\(no parameters\)/ { }
|
|
' > "$model"
|
|
printf '%s\n' "$model"
|
|
}
|
|
|
|
# Helpers over a parsed model file.
|
|
model_required_flags() { awk -F'\t' '$3==1 {print $1}' "$1"; } # names of required flags
|
|
model_flag_kind() { awk -F'\t' -v n="$2" '$1==n {print $2}' "$1"; } # kind of flag $2
|
|
model_enum_flags() { awk -F'\t' '$2=="enum" {print $1}' "$1"; } # names of enum flags
|
|
model_enum_values() { awk -F'\t' -v n="$2" '$1==n {print $4}' "$1"; } # csv enum values of flag $2
|
|
model_number_flags() { awk -F'\t' '$2=="number" {print $1}' "$1"; } # names of number flags
|
|
model_flag_count() { wc -l < "$1" | tr -d ' '; }
|