mirror of
https://github.com/trailofbits/skills.git
synced 2026-09-14 14:28:48 +08:00
ec2450d05e
* Trim dwarf-expert: single-file skill, effort level, contract test - Consolidate SKILL.md from four files into one (115 lines); delete reference/. Cut generic behavior rules, duplicate When-to-Use sections, a 9-line readelf reference hop, and an ASCII decision tree; fix five typos. - Fix --lookup=<offset> misuse carried over from the original: DIE at a section offset is --debug-info=<offset>; --lookup takes a program address. Verified live against llvm-dwarfdump. - Drop the invented v3-v5 version scoping from the skill, plugin.json, marketplace.json, and both READMEs. Version-specific judgment that earned its place (the -gdwarf-N pin inference, v2 surface forms) lives in the Verifying section. - Frontmatter: add effort: medium, rewrite the trigger description, extend allowed-tools with Write/Edit/WebFetch. - Add tests/test_skill_contract.py (stdlib + pytest): every dwarfdump flag documented in SKILL.md must exist in a live llvm-dwarfdump (>= 19, the verified floor for --error-display/--verify-json), plus a frontmatter contract check. Zero extracted flags or a missing tool fails rather than skips. - CI: python-tests installs llvm-19, falling back to the default llvm on future runner images. - Bump version 1.0.1 -> 1.1.0. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Address review: fence-aware extraction, capability-gated resolver - body_sections() no longer strips fenced blocks from section text: fences are ignored only for heading detection, so flags documented inside examples are extracted and verified. New unit test covers both properties; a fenced --bogus-flag mutation now goes red. - Replace the parsed-version LLVM floor with a capability gate: the resolver requires --error-display/--verify-json in --help, because Apple's LLVM numbering does not track upstream and a numeric major is not comparable across toolchains. Failure lists rejected tools with their version banners. README wording matches. - Raise the extraction floor from 10 to 15 (21 flags documented). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: kz-tob <kara.zaffarano@trailofbits.com>
145 lines
5.7 KiB
YAML
145 lines
5.7 KiB
YAML
name: Lint
|
|
|
|
on:
|
|
push:
|
|
branches: [main]
|
|
pull_request:
|
|
branches: [main]
|
|
|
|
permissions:
|
|
contents: read
|
|
|
|
concurrency:
|
|
group: ${{ github.workflow }}-${{ github.ref }}
|
|
cancel-in-progress: true
|
|
|
|
jobs:
|
|
pre-commit:
|
|
name: Pre-commit
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
|
with:
|
|
persist-credentials: false
|
|
- uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
|
|
with:
|
|
python-version: "3.13"
|
|
- uses: pre-commit/action@2c7b3805fd2a0fd8c1884dcaebf91fc102a13ecd # v3.0.1
|
|
|
|
bats:
|
|
name: Shell (bats)
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
|
with:
|
|
persist-credentials: false
|
|
- name: Install bats and uv
|
|
run: |
|
|
sudo apt-get update
|
|
sudo apt-get install -y bats
|
|
curl -LsSf https://astral.sh/uv/install.sh | sh
|
|
echo "$HOME/.local/bin" >> "$GITHUB_PATH"
|
|
- name: Run bats tests
|
|
# --no-run-if-empty would turn "discovery broke" into a silent pass. This repo
|
|
# ships bats suites, so finding none is a failure, not a clean run.
|
|
run: |
|
|
set -euo pipefail
|
|
mapfile -d '' files < <(find plugins -name '*.bats' -type f -print0)
|
|
if [ "${#files[@]}" -eq 0 ]; then
|
|
echo "ERROR: no .bats files found — discovery is broken"
|
|
exit 1
|
|
fi
|
|
echo "Running ${#files[@]} bats file(s)"
|
|
bats "${files[@]}"
|
|
|
|
- name: Run shell regression suites
|
|
# Matched by no glob in the previous discovery, so these never ran in CI.
|
|
run: |
|
|
# find, not a glob: `**` needs globstar and silently degrades to `*`
|
|
# without it, so a suite one directory deeper stops running with no signal.
|
|
# And an empty result is a failure, not a pass — this repo ships suites.
|
|
set -euo pipefail
|
|
mapfile -d '' suites < <(
|
|
find plugins -type f -path '*/tests/*' -name 'run_*.sh' -print0
|
|
)
|
|
if [ "${#suites[@]}" -eq 0 ]; then
|
|
echo "ERROR: no shell regression suites found — discovery is broken"
|
|
exit 1
|
|
fi
|
|
echo "Running ${#suites[@]} shell regression suite(s)"
|
|
for s in "${suites[@]}"; do
|
|
echo "::group::$s"
|
|
bash "$s"
|
|
echo "::endgroup::"
|
|
done
|
|
|
|
python-tests:
|
|
name: Python tests
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
|
with:
|
|
persist-credentials: false
|
|
- uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
|
|
with:
|
|
python-version: "3.13"
|
|
- name: Install pytest
|
|
# The rust-review / c-review test files are pytest-based
|
|
run: python3 -m pip install pytest
|
|
- name: Install toolchain dependencies
|
|
# constant-time-analysis tests cross-compile C to aarch64 via
|
|
# `clang --target=aarch64-unknown-linux-gnu`. Without the cross
|
|
# sysroot, clang fails with:
|
|
# fatal error: 'bits/libc-header-start.h' file not found
|
|
# libc6-dev-arm64-cross supplies the aarch64 headers and crt files;
|
|
# gcc-aarch64-linux-gnu pulls in binutils-aarch64-linux-gnu (the
|
|
# cross linker clang invokes). --no-install-recommends suppresses
|
|
# Recommends, so each package must be listed explicitly.
|
|
#
|
|
# llvm-19 supplies llvm-dwarfdump for the dwarf-expert contract test,
|
|
# which fails rather than skips when the tool is missing. 19 is the
|
|
# floor: --error-display and --verify-json landed in LLVM 19, and
|
|
# ubuntu-24.04's default llvm is 18. The fallback covers future
|
|
# runner images whose default llvm is >= 19 but that no longer
|
|
# package llvm-19 itself.
|
|
run: |
|
|
sudo apt-get update
|
|
sudo apt-get install -y --no-install-recommends \
|
|
clang \
|
|
gcc-aarch64-linux-gnu \
|
|
libc6-dev-arm64-cross
|
|
sudo apt-get install -y --no-install-recommends llvm-19 \
|
|
|| sudo apt-get install -y --no-install-recommends llvm
|
|
- name: Discover and run plugin Python tests
|
|
shell: bash
|
|
run: |
|
|
# Intentionally omit -e: we want every directory to run even if one
|
|
# fails, then exit with a combined failure code via `failed`.
|
|
#
|
|
# pytest per directory, not `python3 <file>` per file. A test file with no
|
|
# `if __name__ == "__main__"` block exits 0 under the old loop having run
|
|
# nothing at all, which is indistinguishable from a pass. Running pytest
|
|
# from inside each directory (via `python3 -m`, which puts the CWD on
|
|
# sys.path) preserves the sibling imports these suites rely on, and
|
|
# --import-mode=importlib keeps c-review and rust-review from colliding on
|
|
# their identically-named test_split.py.
|
|
set -uo pipefail
|
|
mapfile -t dirs < <(
|
|
find plugins -type f \( -name 'test_*.py' -o -name '*_test.py' \) \
|
|
-exec dirname {} \; | sort -u
|
|
)
|
|
if [ "${#dirs[@]}" -eq 0 ]; then
|
|
echo "ERROR: no Python test files found — discovery is broken"
|
|
exit 1
|
|
fi
|
|
failed=0
|
|
for d in "${dirs[@]}"; do
|
|
echo "::group::$d"
|
|
if ! ( cd "$d" && python3 -m pytest -q --import-mode=importlib . ); then
|
|
failed=1
|
|
fi
|
|
echo "::endgroup::"
|
|
done
|
|
echo "Ran ${#dirs[@]} test directory/ies"
|
|
exit "$failed"
|