mirror of
https://github.com/juxt/allium.git
synced 2026-09-14 13:56:44 +08:00
Matt Ford
44d5801f3a
When validating after edits, the distill/weed/tend skills previously
said "Fix any reported issues before presenting the result" but
relied on `allium check`'s exit code as the validated/done signal.
The CLI exits 0 even when warnings and info diagnostics are
reported — only errors fail. Concrete observation: a tended spec
under tests/.tier4-runs/distill-stability-2026-05-13T12-58-38-134Z
exited 0 from `allium check` but had three warnings (unreachable
status value, two missing-source-hint flags) and three info
diagnostics (unreachable trigger). The model treated exit 0 as
"validated" and shipped.
Coordinated edit across all three skills + the two agent variants
(weed and tend; distill is interactive only):
- Verification text now says, plainly: "Do not rely on the CLI's
exit code as the validated signal — `allium check` exits 0 even
when warnings and info diagnostics are reported. Read the
diagnostics list and address every error, warning, and info
entry: either fix the spec, or — if a diagnostic is genuinely
acceptable — note in your response which diagnostics you
considered and why each is acceptable."
- distill skill also gains a checklist item in "Checklist: Code
literalism": "allium check produces zero diagnostics — OR every
warning/info diagnostic is explicitly listed in your response
with a one-line rationale for why it is acceptable. The CLI's
exit code is *not* the validation signal; the diagnostics list
is." Following the meta-finding from the variance experiment
(closing-checklist rules convert reliably).
.github/agents/{weed,tend}.agent.md regenerated from agents/*.md
via scripts/generate-multi-editor.mjs.
Skip-path stays green: 51 passed, 0 failed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>