mirror of
https://github.com/software-mansion/argent.git
synced 2026-09-14 19:27:14 +08:00
fix/docs-high-severity
3 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 filip131311
|
b232114422 |
docs: audit every comment in src against the code it describes (#931)
Audits every comment in `src/` and `scripts/` against the code it
describes.
**462 files, 57 commits, net −8,194 lines.** Comment text only — the
whole branch is code-identical to `main`.
## Method
One subagent per file, strictly sequential. Scope was `src/` +
`scripts/` (459 files); three more files were added at the end because
they carried dead references of the same kind — two test headers citing
design docs that do not exist, and `publish-npm.yml` citing a retired
workflow. Each agent verified every comment — line, block, JSDoc, file
header, trailing — against the surrounding code and the rest of the
repo, following identifiers, paths, tool ids, config keys, env vars and
issue links to see whether they still exist and still behave as
described. Rules:
- **A false or misleading comment is deleted, not reworded.** If a claim
could not be confirmed by reading the source, it went. That is why the
deletion count is so much larger than the rewrite count.
- Survivors are cut to the shortest form carrying something the code
does not already say. Restatement, preamble, hedging, changelog prose
and ASCII banners are gone; the non-obvious *why* stays.
- Preserved byte-identical: license headers, pragmas and directives
(`@ts-*`, `eslint-disable`, shebangs, `/// <reference>`), JSDoc tag
tokens, everything inside a string or template literal, and the sole
comment inside an otherwise empty block (ESLint `no-empty` counts a
comment-bearing block as non-empty).
## Verification
Every file passed two independent gates before being recorded as done:
1. `comments-only` — the required check.
2. A second comment-stripping comparator with a proper mode stack,
written for this pass because `comments-only`'s flat scanner desyncs on
nested template literals and quote-bearing regex literals and then
reports comment lines as code changes. Two files hit that false FAIL
(`utils/android-profiler/pipeline/index.ts`,
`scripts/extract-tools.mjs`); in both the "changed code" it printed was
literally `//` lines, and the second checker confirmed the code was
byte-identical.
After the last file, all 462 changed files were re-checked against
`main` with the same comparator, rather than trusting any agent's
self-report. **459 code-identical; 2 are non-code (`.svg`, `.md`); 1
intentional.**
The intentional one is `packages/argent/scripts/bundle-tools.cjs`: the
changed template literal *is* the comment header of the file it
generates, `packages/native-devtools-android/src/bundled-meta.ts`.
Fixing only the generated file would have been reverted by the next
build, so the generator changed too — and it has been verified to
reproduce the committed generated file byte-for-byte.
## Representative false claims removed
Not wording nits — statements a reader would have acted on:
- **Reversed directions.** `proxyStart`'s JSDoc had the tunnel backwards
(it is a reverse tunnel: the host binds first and the simulator dials
in). A `paste()` doc had the pasteboard copy direction reversed.
- **Contradicted by the code below it.** A timeout budget multiplied by
three where the probes run concurrently — the same comment said so six
lines later. A "warn once" that warns on every call. A "binary search"
that is a linear scan.
- **Named things that do not exist.** A `vega-fast-cli` binary, a
`finish-recording.ts`, a `publish-next.yml` workflow, two
`profiler-react19-*.md` design docs, a `DebuggerTarget.ts`, a commit
hash git does not know, two tool ids, an `ensureEnv` cycle.
- **Wrong by construction.** "Welford accumulators" across four files
where the code keeps naive `n`/`sum`/`sumSq`; `sum`/`sumSq` documented
over `actualDuration` when reduce sums `selfDuration`; a strict-mode
halving written `n/2` where the code ceils; field docs listing enum
values the producers never emit.
- **Guarantees the code does not make.** A validation matrix claiming to
cover "EVERY tool" that skips flagless ones; a Pareto cutoff that
`slice(0, 20)` makes inert; an idempotence claim where the real rule is
at-or-ahead; a capability note describing a clean 400 the shape-based
device resolver can never produce.
- **Unverifiable assertions** about prebuilt binaries, external CLIs and
the cloud SDK — deleted rather than kept as folklore, since nothing in
the repo can confirm them.
- **Stale numbers**: invented Android tool versions, hard-coded tool
counts and description lengths that had drifted.
## Review
A Fable agent reviewed both halves adversarially for over-deletion,
misread code, `no-empty` hazards and byte-identity violations.
Second-half verdict: **SHIP**, with two one-line restores, both applied
in the final commit — the `npm view ""` rationale behind a blank-token
guard, and the note that `argent-mcp` keeps a copy of
`SECRET_PLACEHOLDER_MARKER` it cannot import.
## Code issues surfaced but deliberately not fixed
This pass changes comments only. Eight genuine findings are logged for a
follow-up:
1. `telemetry/src/consent.ts` — a non-ENOENT read error returns null and
falls through to the default-on path, so file errors *can* silently flip
telemetry on.
2. `chromium-server/navigation.ts` — `navigate()` is reachable from
`POST /api/navigate` with only a `typeof === "string"` check; open-url's
schema is a bare `z.string()`, so the "already validated by zod" premise
never held.
3. `http.ts` — `constantTimeEqual` returns early on a length mismatch,
so the auth token's length is observable.
4. `describe/index.ts:~114` — the ios-remote branch passes `{ isTvOs:
false }` unconditionally, so a remote tvOS simulator takes the iOS
ax-service path, though `isRemoteTvOsSimulator` exists and shake/paste
do use it.
5. `devices/boot-device.ts` — `-crash-report-mode never` is passed
unconditionally *and* appended again by the feature-detecting path, so
every emulator spawn passes it twice.
6. `react-profiler/pipeline/04-rank.ts` — `PARETO_THRESHOLD_PCT` is
dead: `slice(0, 20)` always wins.
7. `reaped-sessions.ts` — a user-facing hint string tells the agent that
`react-profiler-start { force: true }` disposes the debugger and
profiler session; it does not. Left byte-identical because it is a
string literal, not a comment.
8. `utils/simctl-backend.ts` — `localSimctl` is exported with no
importers anywhere.
## Docs
No documentation change is needed: this pass touches only source
comments, and no user-facing capability, tool, CLI flag, config key or
flow-file behaviour changed.
---------
Co-authored-by: filip131311 <f.kaminski2000@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Ignacy Łątka
|
19995985d3 |
docs(scripts): stop citing flow-add-step's since-reworded description in a fixture comment (#527)
Follow-up to #436. The escape-cooking fixture's comment cited flow-add-step.ts as a currently-shipping example of a `\n` description; main's flow rewrites reworded that description, so the citation is stale. The comment now leans on the fixture alone instead of any live tool's wording. No code change - `npm run test:scripts` still 59/59. |
||
|
Ignacy Łątka
|
78d3184ba8 |
fix(scripts): stop extract-tools dropping tools with long descriptions (#436)
## Defect
`scripts/extract-tools.mjs` extracts MCP tool definitions by regex to
feed the downstream `spidershield scan --tools-json` security scan. For
each `id:` match it scanned only the next 2000 chars (`const descWindow
= afterId.slice(0, 2000)`) for the `description:` value, and the
description regexes require the string's closing delimiter to fall
inside that window.
Any tool whose description's closing delimiter sits past ~2000 chars is
therefore matched as `null` and **silently dropped** - never emitted,
never security-scanned. Two shipping tools hit this:
- `run-sequence`
(`packages/tool-server/src/tools/run-sequence/index.ts`) - a ~4285-char
multi-line template-literal description whose closing backtick sits
~4324 chars past its `id:`.
- `describe` (`packages/tool-server/src/tools/describe/index.ts`) - a
~2195-char description whose closing backtick sits ~2252 chars past its
`id:`.
**Security impact:** `describe` is the accessibility / DOM tree tool,
exactly the kind of tool a `spidershield` scan should cover, and it was
being silently excluded alongside `run-sequence`. A dropped tool is not
a scan finding, so the gap was invisible.
### Repro (before)
```
$ node scripts/extract-tools.mjs | grep -c '"run-sequence"'
0
$ node scripts/extract-tools.mjs | grep -c '"describe"'
0
$ node scripts/extract-tools.mjs | grep -c '"gesture-tap"'
1
# total tools emitted: 70 (run-sequence AND describe both missing)
```
### After
```
$ node scripts/extract-tools.mjs | grep -c '"run-sequence"'
1
$ node scripts/extract-tools.mjs | grep -c '"describe"'
1
# total tools emitted: 72 (both captured, run-sequence's full 4285-char description intact)
```
## Fix
- Drop the fixed 2000-char lookahead. Each tool's `description:` is
resolved at the **same object-literal brace level** as its `id:` by a
forward scan that tracks brace depth over real code only: comments,
string literals, template literals (with `${...}` interpolations skipped
opaquely, recursively), and regex literals (prev-token heuristic incl.
regex-permitting keywords like `return`) are skipped whole, so nothing
inside them can open a fake string, shift the depth, or match as a
`description:`/`id:` token. The value is then parsed to its actual
closing delimiter, so a description of any length is captured in full.
- Because matching is by brace scope rather than raw position, a stray
`id:`/`description:` token can't corrupt extraction: a description-less
tool can't borrow the next tool's description, an `id:` appearing inside
description text or a comment can't truncate or overwrite anything, and
an `id:` key nested inside a value object (e.g. `defaultPayload: { id:
"example" }`) is neither emitted as a spurious tool nor allowed to drop
the real one.
- Captured text follows full JS cooked-string semantics: named escapes,
`\xNN`, `\uNNNN`, `\u{...}`, identity escapes, line continuations, and
template `\r\n`/`\r` -> `\n` normalization - the emitted (and
security-scanned) text is the string the runtime renders.
- A description value that is not a single plain literal - a
concatenation, a `${...}` interpolation, a method/operator suffix
(`"...".slice(0, 3)`), `as const` - is never emitted as a
truncated/wrong literal: it warn-skips on **stderr** (stdout stays valid
JSON for the scanner). Non-interpolated template-literal ids (`` id: `x`
``) are extracted; interpolated ids stay out of scope like
const-reference ids.
- Duplicate ids keep the first occurrence and warn (exported as a pure,
unit-tested function).
- The extractor stays dependency-free (node: builtins only): the
tool-description-quality workflow runs it on a bare checkout without
`npm ci`.
- Output JSON shape is unchanged.
## CI guard
Adds `scripts/extract-tools.test.mjs` (node's built-in test runner,
wired into `npm run test:scripts`; 36 tests). Ground truth: the
real-tree tests parse every tool source file with the actual
**TypeScript parser** (a root devDependency, test-only) and require the
extractor's output to equal the AST **exactly - names and descriptions
byte-for-byte**. Any drop, corruption, or fabrication on a shape that
enters the tree fails CI loudly, with no reliance on stderr warning
formats and no false red from `id:`-like text in comments or strings.
Two policy gates keep un-extractable shapes out of the tree instead of
letting them silently leave the scan: a `description` that cannot be
read statically (non-literal value, shorthand/getter/method/computed
member, or a spread sibling that may carry/override it) and a
plain-literal description written **before** its `id:` (the scan is
forward-only) each fail with an actionable, file-naming message.
Fixture tests pin the parsing rules independent of any real tool's
wording - long descriptions (>4000-char value, >2000 chars of siblings
between `id:` and `description:`), embedded `id:`/`description:` tokens,
comments, keyword-position and sibling regex literals, division, nested
templates and interpolation content, template/`as const` ids, the full
escape suite, CRLF cooking, single-quoted forms, trim, warn-skip
classes, and the duplicate-id warning. Each new fixture was verified to
fail under a targeted mutation of the extractor.
Note: tools registered with a const-reference / dynamic id (e.g. `id:
AWAIT_UI_ELEMENT_TOOL_ID`) rather than a static literal stay out of
scope for this extractor - a separate, pre-existing limitation unrelated
to the description window, and one this static/lexical extractor cannot
close soundly (it has no tool-scope knowledge, so a blanket warning on
every `id: <identifier>` would also fire on legitimate nested non-tool
keys like `{ id: someVar }` in a payload/example object). The full set
currently taken out of the scan this way is `app-map-crawl`,
`app-map-export`, `app-map-navigate`, `app-map-query`,
`await-screen-idle`, and `await-ui-element`. A proper fix is a follow-up
covering const-ref id resolution plus a real-tree emit-count and
empty-stderr assertion so this silent-omission class trips CI if it ever
regresses.
## Verification
- `node scripts/extract-tools.mjs` -> 72 tools, `run-sequence` and
`describe` both present (total 70 -> 72), byte-identical to the
TypeScript-AST parse of the tree, no stderr warnings
- `npm run test:scripts` -> all pass (`extract-tools.test.mjs`: 36
tests)
- mutation checks: window-bounding, depth-anchor removal, trim removal,
dedup neutralization, single-quote-branch neutralization each fail the
suite
- `npm run typecheck:scripts` -> clean
- `prettier` + `eslint` on changed files -> 0 problems
- lockfile: `npm install --package-lock-only --ignore-scripts` -> no
diff (idempotent)
|