mirror of
https://github.com/auth0/agent-skills.git
synced 2026-09-19 02:38:28 +08:00
Frederik Prijck
0d42696025
* fix(auth0): harden against security-scan false positives The v2.0 re-architecture (#137) triggered a critical finding on the Agent Trust Hub / Socket / Snyk scans. This addresses the actionable, scan-scoped surface without removing legitimate behavior. Placeholder URLs - Replace registrable placeholder hostnames (your-api.com, your-production-domain.com, your-app.com, your-spa-domain.com, your-web-app.com, your-domain.com) with RFC-2606 example.com forms across references/*.md. Kills the "malicious URL" finding (your-production-domain.com) and pre-empts future URL-reputation hits. Matches the skill's already-dominant `api.example.com` convention. .auth0.com domain placeholders and client-id/secret tokens are left as-is: not registrable / not URLs, so not a scanner surface. Move the behavioral eval harness out of the skill dir - Socket flagged tests/behavioral/graders.mjs (reads .env, pipes source into the local `claude` CLI via execa). That's a dev-only test grader, but it sits inside per-skill scan scope. Move tests/ -> repo-root evals/ so the executable harness is no longer bundled with the skill consumers install. Fix run-evals.mjs path resolution, the routing checker's routing-cases.json lookup, the README, and the AGENTS.md layout doc. skillsaw: exempt template-routed reference families - framework-*.md / tooling-*.md are routed only via the `references/framework-{framework}.md` / `tooling-{tooling}.md` placeholder templates, which skillsaw's literal-scan agentskill-unreferenced-files rule can't resolve. Co-located tests were previously (accidentally) satisfying that reachability; moving them out exposed it. Their reachability is properly enforced by scripts/check_router_reachability.py, so exempt the two template-routed families in .skillsaw.yaml. feature-*/pattern-* orphan detection stays. Verified: skillsaw --strict (0/0, grade A), check_router_reachability, check_routing_evals, and run-evals --dry-run (17 case files) all pass. * fix(auth0): keep load_cases skill-dir-relative for unit tests The previous commit hardcoded routing-cases.json to repo-root evals/, which broke scripts/test_check_routing_evals.py — the unit tests build a self-contained temp skill with its own tests/routing-cases.json and rely on load_cases(skill_dir) reading from that dir. Prefer a skill-local tests/routing-cases.json when present (unit tests), fall back to repo-root evals/routing-cases.json otherwise (production, after the harness move). Both the pytest suite (10/10) and the real check pass. * chore(auth0): bump plugin + skill version to 2.0.1 Patch bump for the security-scan hardening: placeholder-URL cleanup, eval harness move out of the skill dir, and the skillsaw template-route exemption. No new routes or breaking changes. Updates all six plugin/ marketplace manifests and SKILL.md frontmatter in lockstep.
15 lines
363 B
JSON
15 lines
363 B
JSON
{
|
|
"name": "auth0-behavioral-evals",
|
|
"version": "1.0.0",
|
|
"description": "Consolidated behavioral eval runner for the unified auth0 skill",
|
|
"type": "module",
|
|
"scripts": {
|
|
"eval": "node run-evals.mjs",
|
|
"eval:list": "node run-evals.mjs --list",
|
|
"eval:dry-run": "node run-evals.mjs --dry-run"
|
|
},
|
|
"dependencies": {
|
|
"execa": "^9.0.0"
|
|
}
|
|
}
|