Eight scripts (banned_phrase_scan, structure_scan, silhouette_scan,
readability_metrics, extract_constraints, suggest, check_suggestions,
voice_score) raised a raw UnicodeDecodeError traceback on non-UTF-8
stdin or file input. Switch stdin reads to
sys.stdin.buffer.read().decode("utf-8", errors="replace") and CLI
file reads to errors="replace", matching the idiom already used by
voice_profile.py, voice_card.py, and run_mimic_refine.py.
harvest_samples.py aborted an entire batch when one sibling file had
bad encoding or was unreadable: apply errors="replace" at its three
read sites and wrap per-file dispatch in collect_sources with a
try/except (OSError, UnicodeDecodeError) that records a new
"unreadable" drop-stat and warning (reusing the existing
stats/warnings plumbing that already tracks "instruction-injection")
and continues with the rest of the batch.
evals/run_local.py tracebacked when the claude CLI binary was
missing; catch (FileNotFoundError, OSError) alongside the existing
TimeoutExpired handling and degrade that one task instead of crashing
the whole run.
Add six ENC-01..06 eval rows (evals/adversarial-evals.json) covering
each fix, plus a small harvest fixture
(evals/fixtures/harvest/fixture_bad_encoding/good.md) whose marker
sentence must survive a sibling bad-encoding file generated inline by
the eval row (not committed as binary). Confirmed red against
unfixed code via git stash before restoring the fixes.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K6CYksdLbXbTAxcAQjvHz5
Insert `from __future__ import annotations` in the 17 files that use
PEP 604/585 annotations in module-level positions evaluated at import
time, so scripts/banned_phrase_scan.py and friends no longer raise
TypeError on Python 3.8/3.9. Add a 3.8 leg to the CI matrix so the
floor claim in README.md is actually gated, and correct the two
imprecise "439 deterministic cases" references to "440 deterministic
script cases (439 pass, 1 documented xfail)".
New scripts must carry the future-import until the floor is raised;
if the maintainer later chooses 3.10+, delete the CI 3.8 leg and
README claim together.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K6CYksdLbXbTAxcAQjvHz5
Add agent-invoked co-writer mode and a graceful non-English decline, both
under the existing scanner constitution (eval-first; all gates green).
scripts/suggest.py
Emit LSP-style structured suggestions {span, severity, category, rationale,
suggested_replacement, phrased_as_question} from banned_phrase_scan +
structure_scan. Detection is deterministic; replacement generation is
DELEGATED (suggested_replacement null) with a --apply-replacements FILE mode
that merges externally-produced replacements and light-validates them. Soft
findings are phrased as questions. Deterministic order, non-overlapping spans.
scripts/check_suggestions.py
Blocking contract gates, each a named failure: span-minimality (edit touches
only its span; whole-sentence rewrites fail), replacement-scanner (each
replacement passes both scanners in isolation and in context), accept-all
(applying every suggestion yields a doc passing both scanners with
validate_preservation exit 0), span-overlap.
English-only decline
Cheap function-word heuristic (english_function_share / is_probably_english)
added identically to banned_phrase_scan.py and structure_scan.py. Below the
conservative threshold -> {"non_english": true, "violations": []}, exit 0 with
a stderr note. Threshold tuned so ESL English still scans.
Eval rows (RED before these scripts existed): SUGG-01..05 (suggestion emission,
soft-as-question, oversized-replacement rejection, accept-all safety, overlap
rejection) and LANG-01a/01b + LANG-02a/02b (Spanish declined by both scanners,
ESL English scanned by both). Fixtures under evals/fixtures/suggest/.
SKILL.md gains a compact Co-writer Mode section (agent-invoked; cheap detection,
delegated replacements, blocking contract gates, suggestions surfaced never
silently applied).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K6CYksdLbXbTAxcAQjvHz5
Clayton Kim