mirror of
https://github.com/trailofbits/skills.git
synced 2026-09-14 14:28:48 +08:00
agentic-actions-cli-detection
8 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 kz-tob
|
4b1b74b181 |
Give differential-review a trigger, and name every component in its README (#278)
* Give differential-review a trigger, and name every component in its README differential-review's description listed what it does and never named a situation, so it competed on capability wording alone. It now closes with the triggers its own README already documents — reviewing a PR, commit, or diff; checking whether a change re-introduces a fixed bug; asking what else a change could break; finding modified code with no test. The same plugin's README never mentioned adversarial-modeler, which is what Phase 5 dispatches for HIGH RISK changes. Checking whether that was isolated turned up more of it, and the sweep found three kinds of gap: zeroize-audit's agent table was missing three of its eleven agents — 0-preflight, which gates the entire run, plus 5b-poc-validator and 5c-poc-verifier. All three appear in the phase diagram directly above the table, which is why they read as present. constant-time-analysis documents the ct-analyzer CLI end to end and never says the plugin also ships a skill and a command. entry-point-analyzer lists phrases that trigger its skill but never names the skill or its command. Three more READMEs describe their skill without naming it. That matters most where the skill name is not the plugin name and a user cannot guess it: chrome-mcp-troubleshooting and interpreting-culture-index. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Fix review findings and make the README sweep a gate The two PoC rows I added to zeroize-audit said Phase 4. The diagram three lines above them, SKILL.md, and workflows/phase-5-poc-validation.md all say Phase 5, steps 5a and 5b. "Wave 5a" is a label that exists nowhere. A debugger consulting the table — the artifact this branch designates as what runs when — would have opened phase-4-poc-generation.md and found no validation in it. Also corrected the sentence introducing that table, which still said 10 agents across 8 phases against 11 across 9, and the Phase 0 diagram line, which still credited the orchestrator for a gate the new row credits to 0-preflight. differential-review's README claimed the agent is "dispatched", and named it bare in a column whose other rows are namespaced. Nothing dispatches it: the only instruction is prose in SKILL.md, and a bare subagent_type fails at runtime. Namespaced both, and corrected the five stale line counts in the same file — reporting.md is 369 lines, not the ~120 the token-efficiency section budgets for. Drop the dead `name: trailofbits:<cmd>` key from five command files. The three newest command files carry no name: at all, #275 namespaced 22 bare invocations, and this branch documents the `/<plugin>:<cmd>` form — so the key contradicts the docs it sits next to. Then make the sweep repeatable. Doing this by hand three times found eight gaps and missed two more, both of the same shape: a workflow ships under meta.name, not its filename, so a README citing the filename never writes the name a reader types. The validator now checks that a README names every skill, agent, command, and workflow its plugin ships, reading meta.name for workflows. It refuses a run that inspected zero components, and six self-test assertions hold it to known-bad fixtures. It found git-cleanup on its first run: ships as /git-cleanup:git-cleanup-analysis, README cites workflows/analyze-branches.js four times and that name never. static-analysis had the same gap for codeql-build. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Fix both P2s: the gate was a substring test, and the dispatch was still bare The README gate ran `name not in text`. That reads as thorough and could not fail for a large share of what it counted: `draw` was satisfied by "(draw cards instead)", `semgrep-rule` by the plugin's own name in the install line, `burp-search` by a `scripts/burp-search.sh` path that is a different thing, and `audit` by the prose "shared-state struct audit". Match by kind instead. Commands and workflows are reachable only as `/<plugin>:<name>`, so require that literal — it is the only string a user can type. Agents are dispatched by identifier and never typed as prose, so require an identifier-shaped mention. Skills are genuinely referred to by bare name, so require only a delimited occurrence, which is what stops "draws" counting as `draw`. That surfaced seven real gaps, the four above plus insecure-defaults' audit-pipeline workflow, mutation-testing's skill, and trailmark's code-slice-worker. All seven fixed. adversarial-modeler was still bare at SKILL.md:96. Line 77 was the decision-tree mention; line 96 is the "Delegate to this agent" instruction a model actually acts on, so the runtime failure the last commit claimed to fix survived it. Namespaced, and it now says why. Also from the review: a per-kind floor, since a single total stays healthy while skill_files() — 63% of coverage — silently stops matching; workflow_names anchored to the meta block, because a bare search takes any earlier `name:` in a comment, and .mjs was invisible; and AGENTS.md documents the new hard failure. Self-test 88 -> 96, each new rule with a negative control. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Dan Guido
|
1e0cc133f4 |
trailmark: pass --language to graph-evolution's diff calls (#266)
* trailmark: pass --language to graph-evolution's diff calls `trailmark diff` defaults `--language` to `python`. On a non-Python target it exits 0 and writes well-formed JSON with empty `nodes`, `edges`, and `entrypoints` arrays, which the skill's own "stop if empty" guard read as "no changes" rather than "wrong language". Both call sites now pass `--language auto`, mirroring the `language="auto"` default that Phase 2's `build_and_export` already uses, and the guard text says to confirm the language before treating an empty diff as an unchanged codebase. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * trailmark: fix the two remaining bare diff calls and the stale empty-diff guard The core skill's Quick Start (SKILL.md:147) and query-patterns.md:223 still ran `trailmark diff` without `--language`, hitting the same silent python default this PR fixes in graph-evolution — worse, the entrypoints line directly above each passes `--language auto`, so the omission read as deliberate. Pass `--language auto` at both. With `auto` in the documented command, "empty most often means the language was wrong" no longer names the likely cause, and "compare the two outputs" gives no decision rule. Key the empty-diff guard on Phase 2's graph summaries instead: empty diff plus healthy node counts on both snapshots is genuine stability, empty diff with a (near-)zero count means the parse missed the code. The checklist item now states that checkable condition rather than a self-attested "language was confirmed". Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014UFqJu1ada7gXjD9peo1rX --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Scott Arciszewski <147527775+tob-scott-a@users.noreply.github.com> |
||
|
Dan Guido
|
9e06dc67a3 |
Make every documented command runnable under our own python shims (#258)
* Make every documented command runnable under our own python shims
The modern-python plugin ships PATH shims that refuse `python <script>`,
`pip install`, `python -m pip` and `uv pip install`. Twelve other plugins
in this marketplace issued exactly those forms, so installing our own
plugin broke our own skills — and CI was green throughout.
The worst case was not theoretical. c-review and rust-review both call
their Phase 4 planner as `python3 "${PLUGIN_ROOT}/scripts/build_run_plan.py"`,
so with the shim installed every run died before spawning a worker.
Verified both directions: the new form exits 0 with the shim on PATH, the
old form exits 1.
Phase 1's reading pass named 16 skills. A mechanical sweep found 96
candidate lines across 44 files, and scanning shell scripts as well as
markdown found 10 more the docs sweep had missed. That gap is the reason
the check below exists.
The fix is not one substitution. Four classes needed different treatment:
- Our own scripts become `uv run --no-project <script>`. Not bare `uv run`,
because these execute inside the *target* repo, which may be a Python
project that cannot resolve; verified against a broken pyproject.toml and
against validate_artifacts.py's sibling import of generate_sarif.
- Package installs become `uv add` for a dependency, `uv tool install` for a
CLI, `uv sync` for a project's own editable install.
- Third-party CLIs we merely document — OSS-Fuzz's infra/helper.py, yarGen —
become `uv run --no-project python <script>`, which keeps upstream's exact
semantics rather than handing their script an environment we manage.
- atheris's instrumented build keeps its source build, as
`uv add --no-binary-package cbor2`. Dropping that flag would silently
produce an uninstrumented fuzzer, which is worse than a visible failure.
Its prose was updated to name the flag it now uses.
Two factual corrections fell out. `pip install caracal` was wrong twice
over: caracal is a Rust tool (Cargo.toml at its root), so it is now
upstream's own `cargo install --git`, not a uv equivalent that would fetch
an unrelated PyPI package. And `pip install uv` cannot bootstrap uv under
a shim that intercepts pip, so culture-index now points at the official
installer.
Thirteen lines stay as they are, each deliberately: Dockerfile `RUN` lines
and oss-fuzz's build.sh run in containers where our shims are absent;
codeql's pip calls install the *analysed* project's dependencies, and that
project is arbitrary; trailmark's dispatch skills must keep saying "Do NOT
run `pip install`"; and modern-python documents what it intercepts.
`make shell-suites` passes again as a result — exit 0 with the 1.6.0 shim,
where AGENTS.md previously recorded it as broken by variant-analysis.
The guardrail: check_python_invocations scans 698 markdown and shell files
and fails on the four refused forms, with structural exemptions for
dockerfile fences and an `allow-legacy-python: <reason>` marker that scopes
to its code block. Eleven self-test fixtures cover it, four asserting it
fires and seven asserting it stays quiet on the compliant forms. It was
mutation-tested in both languages, and it caught its own worst bug during
development: unanchored patterns first flagged `uv run --no-project python
fuzz.py`, the very form the advice recommends. Self-test goes 45 -> 56.
* Review pass: fix the atheris flow, drop a stray exemption, trim comments
Three corrections from reviewing the branch diff:
- atheris's install now opens with `uv init --bare`, without which the
documented `uv add atheris` errors in a bare harness directory. The old
pip form assumed an activated venv, so setup was always implicit; now
it is one explicit line.
- ossfuzz carried an allow-legacy-python marker on a C++ build block that
contains no python at all — yesterday's insertion matched the first of
three "Build in build.sh" headings instead of the python one. The
exemption now sits only on the block that needs it.
- The anti-vacuity message said "read no markdown" for a scan that also
covers shell scripts.
The rest is weight: the new check's comment blocks, the hardcoded-path
constants' commentary, the AGENTS.md bullets and the three exemption
markers all said the same things at two to three times the length. Each
keeps its one-line why; the narratives are gone. No behavioural change —
self-test still passes 56 assertions and the full scan is unchanged.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Address the review: fix where packages land, widen the check to match the shim
The review's core insight was right twice over. Several substitutions had
changed WHERE a package lands, breaking the documented next step, and the
checker enforced a narrower invariant than the shim it exists to mirror.
Where packages land:
- trailmark is imported as a library from five skills, and a `uv tool
install` environment is not importable — the retry loop at
trailmark/SKILL.md:47-51 would have spun forever on the exact error it
names. The CLI install stays `uv tool install`; the import snippets now
run under `uv run --with trailmark python -`.
- `uv add` writes to the manifest of whatever project you are standing
in, which for sarif-parsing is the audited repo. Its scripting rows,
ijson comment and jsonschema example now use `uv run --with <pkg>`,
which leaves no trace. atheris keeps `uv add` deliberately: the fuzzing
harness is the user's own project, made explicit by `uv init --bare`.
- `uv sync` leaves ct-analyzer in .venv/bin, so the README's very next
line failed with command not found. Now `uv tool install .`, verified
end to end: the console script lands on PATH and --help runs.
- yarGen needs pefile/lxml/yara-python, which `--no-project` had detached;
now `uv run --with-requirements requirements.txt`.
- The cbor2 source-build preference now persists via
`no-binary-package = ["cbor2"]` under [tool.uv] (field verified against
uv's accepted-settings list), so a later `uv sync` cannot silently swap
in an uninstrumented wheel.
The checker, widened to the shim's actual behaviour:
- `python3 --version` and `python3 -u foo.py` are refused by the shim but
passed the old patterns; one live instance (constant-time-analysis
README) proved it. Both forms are now caught.
- Every `uv pip` subcommand is refused, not just install; `-t` joins the
allowed tool-managed flags.
- .py files are scanned too: usage strings and error messages told users
to run refused commands from ten scripts, including the --help of the
very planner this PR fixed. All rewritten.
- The evals/tests exemption now tests path parts relative to plugins/, so
a checkout under a directory named tests no longer exempts every file.
- An allow-marker's scope ends at a blank line as well as a fence, so one
marker cannot blanket a whole file; quality-assessment.md gains the
second marker that scoping made necessary.
Also from the review: zeroize's preflight gets `which python3` back (a
helper script still needs the binary; the shim never required removing
it), the Makefile's shell-suites note no longer describes an interception
that is gone, and the cairo CI example warns that it rebuilds caracal
from source each run.
Self-test 56 -> 63; every new pattern and exemption is fixture-covered
and was mutation-probed against the real tree. Full scan: 0 findings over
773 files.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Address the second review: prerequisite probe, checker parity, package placement
The review's P2 was a regression this PR introduced for a population the
first fix ignored: c-review and rust-review now require uv, and a box
with python3 but no uv would die at Phase 4 exactly the way shimmed boxes
died before. Phase 1 (Prerequisites) in both skills now probes
`command -v uv` and aborts with install guidance. zeroize-audit's
preflight already checked uv. The four converted shell suites gain the
same guard with a clear message instead of a bare 127 mid-run.
Checker parity with the shims, second pass:
- pipx and the non-install pip subcommands are refused by catch-all shim
arms and passed the checker; both get named-subcommand patterns.
- A script named by variable or path (`python3 "$MERGE"`) has no `.py`
token; a new pattern covers it and immediately caught one live
instance — a codeql test stub that fakes uv itself, now carrying an
allow-marker with its reason.
- finditer everywhere: a compliant `uv run` earlier on a line no longer
masks a refused command later on it, which was exactly the table-cell
case the unanchored design exists for.
- Prohibition phrases now test the text BEFORE the match, so
"Use `pip install semgrep` instead of the tarball" is flagged while
"Do NOT run `pip install`" stays exempt.
- The uv-pip allowance matches whole flags after the command, so
`--target-dir` no longer counts as `--target` and a trailing `-t /tmp`
does; `uv pip` precedes `pip` in the pattern order so its lines get
the right advice; a pip match directly after `uv ` defers to the
uv-pip verdict instead of double-reporting.
Package placement, continued from the same insight as round one:
- yarGen regains --no-project alongside --with-requirements, plus a cd
into the checkout so requirements.txt resolves where it lives.
- sarif-parsing's jsonschema example no longer names a script that does
not exist, and the table's run-forms show a concrete script.py.
- culture-index's two messages now agree and name the actual remedy
(`uv run --project` on the scripts directory) instead of re-adding a
dependency its pyproject already declares.
- merge_sarif's usage line gains --no-project; the generator plugin's
install section stops prescribing a venv its own runner never uses.
- generate_poc declared requires-python >=3.9 while using `str | None`
in a signature, a TypeError on 3.9 that uv's interpreter selection
made reachable; now >=3.10.
- The GitLab CI example exports ~/.local/bin onto PATH, without which
`uv tool install` warns and the next line dies command-not-found.
Self-test 63 -> 71; the masking, prohibition-direction, flag-position
and pipx cases are all fixtures, and each new pattern was probed live
against the tree (plant, error, remove, clean — 0 findings over 773
files).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Address the third review: importable trailmark, honest probes, sturdier scan
The P2 was the residue of round two's own fix, applied to the siblings
but not the flagship: trailmark/SKILL.md told the model to cure an import
error with `uv tool install`, which cannot cure it — a tool env is not
importable — while forbidding every fallback. The install block now says
what each remedy is for: `uv tool install` for the CLI, `uv run --with
trailmark python -` for the snippets, and the other five library-first
docs carry the same one-line annotation next to their install command.
Empirically settled rather than taken from the review: `uv run python3
<script>` works fine under the shims — uv prepends its environment's bin
directory, so python3 resolves to a real interpreter, not the shim. The
review's claim to the contrary would have meant rewriting the Makefile
and a bats suite; a two-minute transcript said no. Also declined: a
zeroize uv-prerequisite (its preflight already lists uv and uvx; the
C/C++ `which` line now names uv too).
Real and fixed:
- ct-analyzer's availability probe ran `python3 --version` by subprocess
— the one refused form — so under the shims it reported "Python is not
available" on machines where it plainly is. It now probes
sys.executable, the interpreter the analyzer itself runs under.
Verified under the shim: probe returns True.
- The flag step-over in both script patterns handles long and
value-taking flags (`python3 -W ignore harness.py`, `--verbose
tool.py`), matching the shim's two-slot consumption.
- A bare `allow-legacy-python:` with no reason no longer exempts
anything; the reason the docs demand is now enforced.
- `uv run {baseDir}/...` gets --no-project at the ten semgrep and
culture-index call sites that round two missed, and the culture-index
remediation strings now name that same runnable command instead of a
--project mechanism nothing uses.
- pip gains cache/config; the pattern comment now says the subcommand
list is deliberately a subset.
- Both filesystem scans skip .venv/node_modules-style directories, after
a stray local .venv (left by this session's own uv probe, and invisible
to CI) turned the path scan red.
Smaller review items: the uv-probe prose says "Phase 4 onward" rather
than a wrong phase range, run_fixtures' comment stops claiming PEP 723
headers its stdlib-only helpers do not have, the yarGen one-liners say
to run from the checkout, `uv tool install` sites note or export the
tool bin dir the way a fresh container needs, and sarif-parsing's table
column says Install / run and stops naming a file that does not exist.
Self-test 71 -> 74. Full scan: 0 findings over 773 files.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Daniel McCarney
|
4db56eff30 |
trailmark: update vector-forge skill w/ wycheproof tooling (#251)
* trailmark: update vector-forge skill w/ wycheproof tooling We landed some helpful tooling for adding/editing/replacing vector data upstream in Wycheproof that is helpful for LLMs to know about. This commit updates stale references to the Python based tooling and points to the new `vectorgen` tool/docs. * version 0.10.0 -> 0.10.1 --------- Co-authored-by: kz-tob <kara.zaffarano@trailofbits.com> Co-authored-by: Dan Guido <dan@trailofbits.com> |
||
|
Scott Arciszewski
|
841bffed0d |
trailmark: add v0.4/v0.5 support and graph-assisted security workflows (#183)
* trailmark: update skills to use v0.4.0 features * fix: resolve code review findings for PR #183 Redraws the version boundary to match the actual Trailmark release history, verified against the v0.2.0/v0.2.1/v0.2.2/v0.3.1/v0.4.0 tags of trailofbits/trailmark: P1 (misclassified APIs, fixed): - ancestors_of(), reachable_from(), entrypoint_paths_to(), nodes_with_annotation(), clear_annotations() and the diff/entrypoints CLI exist since v0.2.0 — moved from the v0.4+ list to the v0.2-safe baseline; removed needless hasattr() gates and degraded fallbacks - supported_languages()/detect_languages() (trailmark.parse) are 0.3+ modules, not v0.2-safe — annotated as such - CLI --version/version were added in 0.2.2, not 0.4 — documented as 0.2.2+ and version-probe failure semantics clarified - graph-evolution/SKILL.md reverted to main: native diff has existed since v0.2.0 with identical args, so the 0.2.x-fallback rewrite was built on a false premise (also resolves the quality-checklist contradiction flagged in review) P3 (fixed): lexical version comparison hazard noted; stale diagramming-code checklist label; README baseline list now defers to the SKILL.md Version Gate instead of keeping a second divergent copy Dismissed: 'diff CLI signatures contradict' (both forms valid — before/ after positionals accept paths or git refs, --repo/--json exist since 0.2.0); 'subgraph_edges has no edge_kinds param' (v0.4.0 signature has edge_kinds keyword); 'diagram.py fallback broken on 0.2.x' (trailmark.diagram module exists in 0.2.x; only the CLI subcommand is new); 'phantom diff_against()' (real, v0.2-safe, now in baseline) Verified: check_claude_loadability.py, check_codex_loadability.py, pre-commit hooks pass; all SKILL.md files under 500 lines Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * trailmark: add assurance workflow skills (#187) * trailmark: add assurance workflow skills * fix: drop hasattr gates on v0.2-baseline query APIs entrypoint_paths_to() and reachable_from() are in the v0.2-safe baseline, so the hasattr fallbacks were dead code — and the entrypoint_paths_to fallback indexed attack_surface() entries with entry["name"] instead of node_id, which would raise KeyError. Addresses PR #187 review feedback. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * fix(trailmark): make detect_languages import v0.2-safe in operational skills The trailmark-summary and trailmark-structural skills claimed v0.2-safe workflows but unconditionally imported trailmark.parse, a 0.3+ module. detect_languages() has existed in trailmark.query.api since v0.2.0 (kept as a deprecated alias in 0.3+), so gate the import with a fallback instead of relabeling the skills as 0.3+. Version Gate docs updated to document the v0.2-safe import path; supported_languages() remains 0.3+ with no 0.2.x equivalent. Verified against trailmark v0.2.0 source (fallback branch) and the current 0.4 line (canonical branch). Addresses review feedback on PR #183. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * trailmark: document and enable v0.5.0 features Trailmark 0.5.0 adds a PostgreSQL-oriented sql parser (node kinds schema/table/view/procedure), the stable .trailmark/links.toml configuration for cross-language/FFI/RPC/external links, repository links/proxies/type_uses edges on single-language parses, Solidity entrypoints from parser metadata (visibility/mutability/overridden-by attributes), node attributes in attack_surface() entries, TypeScript constructed-receiver resolution, and C# file-scoped namespaces. Updates the Version Gate with a v0.5+ section and a structural probe ('SCHEMA' in NodeKind.__members__ — 0.5.0 adds no new QueryEngine methods, so hasattr() cannot detect it), adds a Repository Links section and cross-boundary query recipe, extends the parser list and graph model docs, notes the reachability-vs-taint limitation, and threads the 0.5 output additions through trailmark-structural and audit-augmentation. Plugin version 0.9.0 -> 0.10.0. All version claims verified against trailofbits/trailmark v0.2.0 and v0.5.0 builds, including a live links.toml materialization test. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * trailmark: add slicing-code-context skill (#203) Adds a skill for delegating focused code tasks to constrained or locally hosted models using bounded, graph-informed Trailmark source packets: - scripts/build_slice_packet.py: deterministic PEP 723 packet builder (Trailmark 0.5.x) with five selection modes, whole-unit budget admission, explicit omission accounting, path-traversal rejection, an embedded untrusted-source notice, and structured JSON errors (26 tests, including a real Trailmark integration test) - agents/code-slice-worker.md: repository-tool-free Haiku worker returning a source-cited JSON contract - SKILL.md + references/slice-packet.md: coordinator workflow, packet and worker response contracts, and validation rules Reviewed with a multi-agent Claude pass and two Codex passes; fixes from those reviews are included (doc/selection-order reconciliation, relationship deduplication, line-range anchors no longer expand to full nodes in path/entrypoint modes, background-safe worker toolset, structured io_error handling, replacement-packet expansion semantics). Co-authored-by: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Dan Guido <dan@trailofbits.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Lixin2026
|
d5fe2e6a78 |
feat(codex): add UI metadata for skills (#175)
* feat(codex): add skill UI metadata * Use official Trail of Bits logo * fix: resolve code review findings for PR #175 Codex silently drops the icons as authored: its loader (codex-rs/core-skills resolve_asset_path) requires icon paths containing '..' to resolve under <plugin_root>/assets/, and the repo-root .codex/assets location fails that containment check. Verified empirically via codex app-server plugin/read: every iconSmall/iconLarge came back null; only brand_color applied. P1 fixed: - Vendor trail-of-bits-mark.svg into plugins/<name>/assets/ for all 38 plugins with skills and point every openai.yaml at ../../assets/trail-of-bits-mark.svg (the supported plugin-level shared asset pattern). Icons now resolve for marketplace installs too, since nothing escapes the plugin root. - Drop the .codex/ additions: .codex/skills/gh-cli/agents/ openai.yaml resolved nowhere (.codex/skills is not a Codex discovery root) and PR #173 removes the whole .codex/ tree P2 fixed: - Patch-bump all 38 touched plugins in plugin.json and marketplace.json so installed clients pick up the metadata Verified: - Static check replicating Codex's resolution algorithm: all 73 yaml files resolve under their plugin assets/ and exist - Live codex app-server probe: 71/72 loadable skills report resolved iconSmall/iconLarge and brand_color #D83A34 (claude-in-chrome-troubleshooting fails to load on main due to a pre-existing 64-char qualified-name limit, fixed by #173's rename; zeroize-audit's manifest mcpServers object is likewise a pre-existing Codex incompatibility fixed by #173) - validate_codex_skills.py, validate_plugin_metadata.py, prek all pass Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(codex): use skill-local icon assets --------- Co-authored-by: Dan Guido <dan@trailofbits.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Scott Arciszewski
|
48bd2626a1 |
Refresh trailmark skills for public Trailmark 0.2.x (#153)
* Refresh trailmark skills for public Trailmark 0.2.x Aligns all skills with the now-public trailmark package (pypi.org/project/trailmark, github.com/trailofbits/trailmark): - Replace hardcoded language tables with runtime detection via trailmark.parse.detect_languages() and --language auto, so the skill never goes stale as Trailmark adds languages (21 supported as of 0.2.x: Python, JS/TS, PHP, Ruby, C/C++, C#, Java, Go, Rust, Solidity, Cairo, Circom, Haskell, Erlang, Miden Assembly, Swift, Objective-C, Kotlin, Dart) - Remove phantom --passes CLI flag from trailmark-structural; pre-analysis is a QueryEngine.preanalysis() method, not a flag - Use native trailmark diff --json in graph-evolution alongside the subgraph-diff helper script - Add `finding` and `audit_note` annotation kinds to the documented list (set by augment_sarif / augment_weaudit) - Document trailmark entrypoints and trailmark augment subcommands that exist in the public package - Bump plugin to 0.8.1 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * Address Trailmark skill review nits --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: Dan Guido <dan@trailofbits.com> |
||
|
Scott Arciszewski
|
6b79b5e879 |
feat(trailmark): skills that reason about code as graphs (#133)
* feat(trailmark): skills that reason about code as graphs * Add Codex skill symlinks for trailmark plugin The trailmark plugin's 10 skills were missing .codex/skills/ mappings, which caused the validate_codex_skills CI check to fail. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Address PR #133 review feedback - Add Rationalizations (Do Not Skip) sections to 5 security skills: trailmark, audit-augmentation, crypto-protocol-diagram, mermaid-to-proverif, graph-evolution - Fix requires-python: diagram.py >= 3.12 (was 3.13), protocol.py >= 3.12 (was 3.11) to match trailmark's actual requirement - Rename diagram/ to diagramming-code/ to match SKILL.md frontmatter name and all cross-skill references; update Codex symlink Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Fix diagram skill to use uv run instead of plain python The diagram.py script carries PEP 723 inline metadata declaring trailmark as a dependency. Plain python ignores this metadata, causing ImportError for users who haven't pre-installed trailmark. uv run processes the metadata and handles dependency resolution. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Address second round of PR #133 review feedback - Fix README directory tree: diagram/ -> diagramming-code/ - Fix diagram-types.md: python -> uv run for all script invocations - Fix graph-evolution Phase 3: replace undefined shell variables ($BEFORE_JSON etc) with template substitutions ({before_json} etc) - Fix vector-forge mutation-frameworks.md: replace cross-skill file link with prose reference to genotoxic skill (avoids reference chain) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Local skill-improver review pass across all 10 trailmark skills diagramming-code: - Fix arrow syntax inconsistency: uncertain edges use ..-> not -.-> - Fix extra closing paren in diagram-types.md - Fix diagram.py docstring to match uv run invocation crypto-protocol-diagram: - Remove reference chain: spec-parsing-patterns.md no longer links to mermaid-sequence-syntax.md, inlines the arrow syntax instead - Fix ProVerif example note: "Tamarin/ProVerif" -> "ProVerif" trailmark: - Replace "path/to/project" with {targetDir} in query-patterns.md - Add uv run prefix to CLI examples in query-patterns.md - Add circom to supported language list - Add pre-analysis annotation kinds to annotation docs genotoxic: - Remove reference chains: triage-methodology.md and mutation-frameworks.md no longer link to graph-analysis.md vector-forge: - Add trailmark to Prerequisites section - Fix bare trailmark commands to use uv run with {targetDir} Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Address third round of PR #133 review feedback mermaid-to-proverif: - Fix ProVerif type error: verify(...) = true is a type mismatch since verify returns bitstring. Use let _ = verify(...) in instead, which aborts on destructor failure (correct ProVerif pattern) trailmark-summary, trailmark-structural: - Add 8 missing language extensions to find command (.rb, .php, .cs, .java, .hs, .erl, .cairo, .circom) - Remove unsupported .lean extension - Split .c -> --language c and .cpp -> --language cpp (separate parsers) All 7 security skills: - Rename "Rationalizations (Do Not Skip)" to "Rationalizations to Reject" per CLAUDE.md convention Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Address fourth round of PR #133 review feedback mermaid-to-proverif: - Fix ProVerif type errors in process template: pkey values cannot appear in bitstring positions. Add pkey2bs() and concat() to the function declarations and rewrite the template to use them, matching the sample-output.pv example trailmark-summary: - Split .js/.ts mapping: .js -> --language javascript, .ts -> --language typescript (separate parsers) graph-evolution: - Replace bare python with python3 in graph_diff.py invocations (python does not exist on modern Ubuntu/Debian/macOS) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Address fifth round of PR #133 review feedback graph-evolution: - Change python3 to uv run for graph_diff.py invocations to match ecosystem convention trailmark-structural, trailmark-summary: - Add Rationalizations to Reject sections (both are security skills running blast radius, taint, and privilege boundary analysis) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Fix ProVerif type consistency and graph-evolution template vars mermaid-to-proverif: - Rename senc/sdec to aead_enc/aead_dec in Step 3 preamble to match the process template and sample-output.pv - Fix hkdf signature: hkdf(key, bitstring): key (first arg is DH shared secret which has type key, not bitstring) crypto-to-proverif-mapping.md: - Fix hkdf declaration and summary table to match corrected signature - Fix example to use concat/pkey2bs for type-correct HKDF input graph-evolution: - Replace $BEFORE_DIR/$AFTER_DIR shell vars in Phase 5 with {before_dir}/{after_dir} template substitutions Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Comprehensive vivisect-style review of all trailmark skills ProVerif correctness (mermaid-to-proverif): - Fix broken ForwardSecrecyTest pattern in security-properties.md: process waited on c_fs but nothing sent on it, past_session_key was never bound to any session. Replaced with working pattern that leaks long-term keys and checks session key secrecy. - Fix hkdf(bitstring,bitstring) -> hkdf(key,bitstring) in proverif-syntax.md to match SKILL.md and sample-output.pv - Fix type-incorrect example in proverif-syntax.md: tuple of (key,pkey,pkey) passed where bitstring expected. Now uses concat2/pkey2bs for type-correct serialization. - Align senc/sdec -> aead_enc/aead_dec in proverif-syntax.md and crypto-to-proverif-mapping.md to match SKILL.md and example - Fix auth query parameter count in security-properties.md: beginI fires before session key is known, so has fewer params Cross-skill consistency: - Fix 3 stale "diagram skill" references -> "diagramming-code" in trailmark/SKILL.md and preanalysis-passes.md - Add PEP 723 header to graph_diff.py for convention consistency README and helper skills: - Add trailmark-summary and trailmark-structural to README skills table and directory tree - Add secondary file extensions (.jsx, .tsx, .h, .hpp, .cc, .cxx) to language detection in summary and structural skills - Inline language mapping in trailmark-structural (was deferred to trailmark-summary, violating one-level-deep rule) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Fix ProVerif type consistency and graph-evolution template vars - Fix endInitiator -> endI in mermaid-to-proverif Step 6 template (endInitiator was never declared as an event) - Add missing msg2_label constant to Step 3 constants block - Add .hh/.hxx C++ header extensions to language detection in trailmark-summary and trailmark-structural Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Fix mermaid-to-proverif template: missing beginI event and secrecy witness Step 6 Initiator template: - Add missing event beginI(pk(sk_I), pk_R) before first out — without it, authentication queries always report false attacks - Replace local new secret_I with free private_I [private] to match sample-output.pv's secrecy witness pattern security-properties.md: - Fix beginI/beginR from 3 args to 2 args in mutual auth section and query checklist (begin events fire before session key is known, so they only take the two public keys) - Update "Placing Events" table to match 2-param form Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Address sixth round of PR #133 review feedback proverif-syntax.md Two-Party Process example: - Fix type errors: pkey values passed directly to bitstring params in sign() and verify(). Now uses concat2(pkey2bs(...)) pattern. - Add missing pkey2bs declaration to function list - Add missing info_session constant declaration - Fix msg2_label -> msg2 in verification check example to match the file's own constant declarations trailmark-structural: - Fix contradiction: Rationalizations table said "Install trailmark first" but Execution section forbids install commands. Changed to "Report not installed and return" to match execution policy. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |