mirror of
https://github.com/software-mansion/argent.git
synced 2026-09-14 19:27:14 +08:00
Ignacy Łątka
2cc1f5281c
Audit of every assertion in the repo that pins shipped agent-facing text, looking for more of the `/restart/i` shape that reddened main in #871 — an assertion satisfiable only by making the product string restate itself. **The class is essentially absent. One instance found.** ## The one `tool-server/test/android-perfetto/hang-stacks-offcpu.test.ts` ```ts expect(out).toMatch(/No usable on-CPU stack samples/); // L156 // The thread was on-CPU/executing — this is genuine CPU work, not a wait. expect(out).toMatch(/on-CPU/); // L158 ← removed expect(out).toMatch(/executing/); ``` `/on-CPU/` is a substring of the literal asserted two lines above, so `L156 ∧ L158 ≡ L156`. It cannot fail while L156 passes. Mutation-checked rather than argued: changing the pipeline's sentence from `_No usable on-CPU stack samples were captured…` to `_No usable stack samples were captured…` still fails the test — ``` × explains an empty sample set as un-unwound CPU work when the thread was executing AssertionError: expected ... to match /No usable on-CPU stack samples/ ``` — so nothing this line was catching goes uncaught. `/executing/` is **not** implied and stays; the comment above it still reads correctly. ## What the audit covered, and what it found instead | stage | count | |---|---| | assertions pinning shipped agent-facing text | 707 | | …in the `/restart/i` shape (short generic literal beside stronger siblings on the same value) | 44 | | …surviving hand-verification | **1** | The 43 rejected are doing real work. A sample, so the reasoning is auditable: - `"844x390"` / `"390x844"` — dimension-mismatch test; the point is argument order. - `"(2×)"` / `"(1×)"` — different dedup counts. - `/wait/` beside `/No on-CPU stack samples/` — the sibling literal contains no "wait"; it pins the wait-vs-CPU explanation. - `"localhost:8081"` / `"127.0.0.1:8081"` — two hosts the interceptor must filter. - metro `discovery.test.ts` `"Do not retry in a loop"` — a *different* test from the one asserting the full sentence. More importantly, the 707 is not a pile of redundancy. It is mostly the only way to test a function whose output is a string — `formatTraceFreshness(NOW - 5*60*MIN, NOW)` can only be checked by asserting `/5 hours ago/` — plus assertions that are load-bearing in their own right, e.g. `expect(description).not.toContain("hunter2")` in `configuration-core/tests/secrets.test.ts`, the sole guarantee a secret value never reaches a source description. Stripping the category wholesale would empty 74 tests into unconditional passes. ## Two adjacent findings, not changed here - `tool-server/test/native-profiler-missing-trace.test.ts:90` — `expect(result.report).toMatch(/missing|not found|unreadable/i)` is meant (per its comment) to check the wording tells the user the file is missing rather than the export empty. It passes vacuously: the fixture is named `missing_cpu.xml`, so `/missing/i` matches the filename the assertion two lines up already pins. It enforces nothing. The fix is a discriminating regex or a renamed fixture, not deletion. - `tool-server/test/capability.test.ts:58-59` — `toContain("demo-tool")` and `toContain("android")` are both substrings of `"tools/demo-tool/platforms/android.ts"` asserted at L60, so they are provably implied. Left in place deliberately: the test is named *"composes a uniform message with toolId, platform, and the file"*, and deleting them would make the name over-claim relative to what the body checks. ## Verification Branched off `366f17c61`. | gate | result | |---|---| | `vitest run` (tool-server) | **358 files, 4401 passed, 1 skipped**, exit 0 | | `tsc --build` | clean | | `typecheck:tests --workspaces` | clean | | `eslint . --max-warnings 0` | exit 0 (with `npm ci` in `packages/docs`, as CI does) | | `prettier --check` | clean |