skills/efremidze__swift-architecture-skill
1
0
Fork 0
mirror of https://github.com/efremidze/swift-architecture-skill.git synced 2026-09-14 20:49:06 +08:00
Code Issues Packages Projects Releases Wiki Activity
Files
improvement/declutter
efremidze__swift-architectu…/tooling/evals
T
History
Lasha Efremidze f4971d93ba Fix misleading test name and non-deterministic stub in coordinator benchmark 
- Rename test_pop_cancellation_onEmptyPath_doesNotCrash to test_pop_onEmptyPath_doesNotCrash
  to accurately reflect empty-path guard behaviour (no actual cancellation involved)
- Rename test_addChild_retainsChildCoordinatorUntilRemoved to
  test_cancelChildFlow_removesChildCoordinator, which genuinely exercises
  sub-flow cancellation and satisfies the contract cancellation-pattern requirement
- Fix StubUserRepository to return a fixed UUID literal instead of UUID() per call,
  making the stub deterministic as stated in the section description

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-05 21:01:43 -08:00
..
benchmarks
Fix misleading test name and non-deterministic stub in coordinator benchmark
2026-03-05 21:01:43 -08:00
corpus
Relocate evals and scripts into tooling/
2026-02-26 00:46:38 -08:00
cases.json
Relocate evals and scripts into tooling/
2026-02-26 00:46:38 -08:00
contract.json
Add MVP and Coordinator architecture playbooks
2026-03-05 20:45:07 +00:00
README.md
Relocate evals and scripts into tooling/
2026-02-26 00:46:38 -08:00
rubric.md
Relocate evals and scripts into tooling/
2026-02-26 00:46:38 -08:00

README.md

Skill Evaluation Harness

This folder provides scenario-based evaluations for swift-architecture-skill.

Goal

Check whether the skill catches architecture/code smells and proposes concrete, pattern-consistent fixes.

Layout

  • tooling/evals/cases.json: all eval cases in one file
  • tooling/evals/rubric.md: manual scoring rubric
  • tooling/scripts/run/evals.sh: runner

Case Format (cases.json)

Each case includes:

  • id
  • notes
  • prompt
  • required_keywords (legacy all-of keywords)
  • required_any (optional grouped any-of checks; preferred)
  • forbidden_keywords
  • forbidden_any (optional grouped any-of checks)

Run a Batch

  1. Prepare a run package (manual mode):
./tooling/scripts/run/evals.sh

This creates tooling/evals/runs/<timestamp>/ with:

  • packet.md (all prompts)
  • responses/<id>.md
  • scorecards/<id>.md
  • cases.json snapshot
  1. Optional: run with a command that reads stdin and writes stdout:
./tooling/scripts/run/evals.sh --cmd "your-command-here"

The command is executed per case as:

bash -lc "$CMD" < prompt_text > responses/<id>.md
  1. Grade a completed run:
./tooling/scripts/run/evals.sh --grade-only --run-dir tooling/evals/runs/<timestamp>

Scoring Guidance

Keyword scoring is a quick heuristic. Use manual scorecards as source of truth.

Manual scorecard dimensions:

  • Smell detection
  • Architecture fit
  • Fix quality
  • Safety (error/cancellation guidance)
  • Clarity
Reference in New Issue View Git Blame Copy Permalink