Three AI-ism patterns wild-caught by the owner on his own marketing page,
added eval-first (rows appended red, then made green):
- slogan_fragment (banned_phrase_scan STRUCTURAL_PATTERNS, soft): standalone
"N X, one Y." slogan-cadence line/header ("Four presets, one input.").
- spec_fragment (banned_phrase_scan STRUCTURAL_PATTERNS, soft): standalone
"N noun-phrase, past-participle ..." spec fragment ("Eight criteria,
scored 1 to 5.").
- every_template_openers (structure_scan, soft, min 2 paragraphs): repeated
paragraph-initial "Every <noun> <verb>" template opener; a lone
"Every child deserves a good school." stays clean.
Both banned-phrase patterns fire only on whole-line/header contexts, so
prose-embedded counts ("The unit has two bedrooms, one bath, and a den.",
"We rated eight criteria, scored 1 to 5, before deciding.") stay clean.
Regexes are linear (bounded interior runs, no adjacent unbounded overlaps).
Rows OWNER-01/02/03 (FN specimens) went red first, then green after the
scanner changes; OWNER-04 + FP-86/FP-87 guard the false-positive twins.
New categories mapped into pack-voice and documented in taboo-phrases.md.
Suite: 462 pass / 1 xfail / 0 fail. Coverage, schema, taboo parity, packs,
kata, seeded-docs, pairs, silhouette, voice, benchmark, model-parity green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K6CYksdLbXbTAxcAQjvHz5
structure_scan.py and silhouette_scan.py each carried a private copy of
words(), a markdown-to-prose stripper, and a paragraph splitter. The
strippers had drifted: structure blanked blockquote lines and
silhouette didn't; silhouette stripped **bold** and structure didn't.
Move words(), strip_markdown_for_prose(), and paragraphs() into
_lang.py as the UNION of both original code paths, gated behind
blank_blockquotes / strip_bold flags so each caller keeps its exact
prior behavior on purpose instead of silently converging. structure_scan
now calls paragraphs(text, blank_blockquotes=True); silhouette_scan calls
paragraphs(text, strip_bold=True). silhouette's STOPWORDS import from
structure_scan is untouched (out of scope).
Verified byte-identical: snapshotted `structure_scan.py` and
`silhouette_scan.py` stdout for all 12 AI fixtures, all 8 human
fixtures, and README.md before and after the swap (42 outputs) --
`diff -r` empty. check_silhouette.py --reference still reports
"reference ok: 5 metrics over 15 human sources"; --separation still
12/12 AI flagged / 0/8 human flagged with identical per-file penalties
(e.g. 05_essay.txt 14.90, 05_readme.txt human 0.62, unchanged). Full
adversarial suite: 439 PASS, 1 XFAIL (known), 0 FAIL. Dual-mode import
(package import and direct script invocation) verified for both
scanners.
Open maintainer question, deliberately unresolved: converge both
scanners on one prose view? Doing so changes silhouette's committed
reference and needs deliberate regeneration + separation re-verification
-- its own eval-first plan.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K6CYksdLbXbTAxcAQjvHz5
Eight scripts (banned_phrase_scan, structure_scan, silhouette_scan,
readability_metrics, extract_constraints, suggest, check_suggestions,
voice_score) raised a raw UnicodeDecodeError traceback on non-UTF-8
stdin or file input. Switch stdin reads to
sys.stdin.buffer.read().decode("utf-8", errors="replace") and CLI
file reads to errors="replace", matching the idiom already used by
voice_profile.py, voice_card.py, and run_mimic_refine.py.
harvest_samples.py aborted an entire batch when one sibling file had
bad encoding or was unreadable: apply errors="replace" at its three
read sites and wrap per-file dispatch in collect_sources with a
try/except (OSError, UnicodeDecodeError) that records a new
"unreadable" drop-stat and warning (reusing the existing
stats/warnings plumbing that already tracks "instruction-injection")
and continues with the rest of the batch.
evals/run_local.py tracebacked when the claude CLI binary was
missing; catch (FileNotFoundError, OSError) alongside the existing
TimeoutExpired handling and degrade that one task instead of crashing
the whole run.
Add six ENC-01..06 eval rows (evals/adversarial-evals.json) covering
each fix, plus a small harvest fixture
(evals/fixtures/harvest/fixture_bad_encoding/good.md) whose marker
sentence must survive a sibling bad-encoding file generated inline by
the eval row (not committed as binary). Confirmed red against
unfixed code via git stash before restoring the fixes.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K6CYksdLbXbTAxcAQjvHz5
Insert `from __future__ import annotations` in the 17 files that use
PEP 604/585 annotations in module-level positions evaluated at import
time, so scripts/banned_phrase_scan.py and friends no longer raise
TypeError on Python 3.8/3.9. Add a 3.8 leg to the CI matrix so the
floor claim in README.md is actually gated, and correct the two
imprecise "439 deterministic cases" references to "440 deterministic
script cases (439 pass, 1 documented xfail)".
New scripts must carry the future-import until the floor is raised;
if the maintainer later chooses 3.10+, delete the CI 3.8 leg and
README claim together.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K6CYksdLbXbTAxcAQjvHz5
Applies a reviewed cleanup list without changing any script's CLI flags, JSON
keys, exit codes, or stderr messages (all pinned by eval rows).
Shared modules (dedup):
- scripts/_lang.py: ENGLISH_FUNCTION_WORDS/english_function_share/is_probably_english,
previously byte-identical copies in banned_phrase_scan.py and structure_scan.py.
- structure_scan.py now imports split_sentences from readability_metrics instead
of keeping its own copy.
- silhouette_scan.py's stopword set is a verified pure superset of
structure_scan's; SILHOUETTE_STOPWORDS = structure's set | the extras.
- harvest_classify.py imports recency_value/DATE_FLOOR from harvest_samples.
- evals/_check_support.py: ROOT, run(cmd, timeout=60), load_evals() -- the
timeout=60 safety net that only check_contrib.py had now covers check_pairs,
check_seeded_docs, check_mimic, check_contrib, check_voice, and
check_pattern_coverage too.
Small cleanups: contribute.py's row_fn drops its unused category param;
run_model_parity.py's resolve_models param renamed responses->payload;
harvest_classify.py's heuristic() attaches suspect_ai/dictated so
rank_enriched needs no reconstruction; check_pattern_coverage.py's paired flag
formulas become a plain "if neither: both = True"; check_contrib.py drops the
__import__("scripts.contribute", ...) spelling for a normal import;
run_mimic_refine.py computes docs_a/matrix_a once and passes it to both
make_live_source and write_outputs; calibrate_pairs.py factors its four
near-identical contraction-replacement closures into one _contraction_repl
helper used by both directions.
Altitude items: GENRE_SUPPRESSIONS lookup tables replace the inline
`genre != "..."` conditionals in structure_scan.py and silhouette_scan.py;
check_gates_doc.py additively verifies every *.py token in a gate command
exists under ROOT (behavioral-tune and rubric-judge are exempt -- neither
command has a .py token).
Efficiency (Phase 2), each verified against the same eval rows / diffed
outputs before landing:
- run_model_parity.py replaces its subprocess-per-scanner-call helpers with
in-process imports of banned_phrase_scan/structure_scan/validate_preservation
(mirrors run_mimic_refine's import pattern). PARITY slice: 14.2s -> 0.33s.
- voice_score.py's lcs_len (O(n*m) DP) is replaced by
has_common_substring_over(), an O(n+m) rolling-hash check for "any shared
substring longer than the 120-char threshold" (hash matches are verified
against the source text, so no false positives). Nothing pins the exact
longest_common_substring value (checked); it now reports the matched
threshold window length on a hit, 0 otherwise -- documented in the
docstring. The violation boolean is unchanged.
- gi_score() precomputes per-key distances once per candidate/impostor
instead of recomputing distances() from scratch every trial; trials do a
subset-weighted sum over the precomputed values. Arithmetically exact
(same RNG draw order, same float sums) -- verified the VOICE-08
determinism value and the full check_voice --separation/--gi/--gaming
output are byte-identical before/after.
- check_voice.py and check_pairs.py convert their subprocess-per-cell/row
scanner calls to in-process imports (voice_score/voice_profile,
banned_phrase_scan/structure_scan), mirroring the CLI's own decline/exit
logic so output stays byte-compatible.
Deferred (out of scope for a contract-safe pass): a protects-grain redesign,
giving silhouette_scan.py its own English-decline gate, and decomposing
run_mimic_refine.py's build_report().
python3 evals/run_adversarial.py: 434 PASS / 1 XFAIL / 0 FAIL, unchanged
throughout; wall time 54.8s -> 25.3s.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K6CYksdLbXbTAxcAQjvHz5
- Both scanners now decline as non-English only when the function-word
heuristic fails AND zero patterns fired: imperative headline stacks and
buzzword noun-lists are English slop with few function words, and the old
order silently defeated the shipped headline-cadence detection
(LANG-3a/3b pin both genres; Spanish still declines)
- 'tells a story' flat hard ban replaced with an inanimate-subject structural
pattern (data/numbers/charts/... tells a story): grandmothers and clinical
case notes stay clean (FP-85), the genuine false-agency form still flags
(REC-50)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Add agent-invoked co-writer mode and a graceful non-English decline, both
under the existing scanner constitution (eval-first; all gates green).
scripts/suggest.py
Emit LSP-style structured suggestions {span, severity, category, rationale,
suggested_replacement, phrased_as_question} from banned_phrase_scan +
structure_scan. Detection is deterministic; replacement generation is
DELEGATED (suggested_replacement null) with a --apply-replacements FILE mode
that merges externally-produced replacements and light-validates them. Soft
findings are phrased as questions. Deterministic order, non-overlapping spans.
scripts/check_suggestions.py
Blocking contract gates, each a named failure: span-minimality (edit touches
only its span; whole-sentence rewrites fail), replacement-scanner (each
replacement passes both scanners in isolation and in context), accept-all
(applying every suggestion yields a doc passing both scanners with
validate_preservation exit 0), span-overlap.
English-only decline
Cheap function-word heuristic (english_function_share / is_probably_english)
added identically to banned_phrase_scan.py and structure_scan.py. Below the
conservative threshold -> {"non_english": true, "violations": []}, exit 0 with
a stderr note. Threshold tuned so ESL English still scans.
Eval rows (RED before these scripts existed): SUGG-01..05 (suggestion emission,
soft-as-question, oversized-replacement rejection, accept-all safety, overlap
rejection) and LANG-01a/01b + LANG-02a/02b (Spanish declined by both scanners,
ESL English scanned by both). Fixtures under evals/fixtures/suggest/.
SKILL.md gains a compact Co-writer Mode section (agent-invoked; cheap detection,
delegated replacements, blocking contract gates, suggestions surfaced never
silently applied).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K6CYksdLbXbTAxcAQjvHz5
- whether-you're pattern requires a role noun phrase (a/an/just starting) and
matches sentence-initially: 'Whether you're right or wrong' and activity
gerunds are clean, canonical audience flattery still flags (FP-39/40, REC-19)
- Bare 'bolstered' ban dropped in favor of the tense-aware collocation gate;
literal reinforcement clean, rhetorical support flags (FP-41, REC-20);
catalog row updated
- double-down collocation inflected (doubled/doubling) with its/their/a
determiners (FN-22)
- and/or extracted as a preservation constraint: collapse to plain 'and'
fails, rewording to 'or (or both)' passes (PRES-28/29)
- structure_scan opener-repetition ignores enumeration openers
(Section/Chapter/Step/Figure...); academic roadmap abstracts clean
(STRUCT-17)
- run_local.py fails a case on CLI error output (Not logged in etc.) instead
of grading the error page; stale DONOHARM-01 tune artifact regenerated with
a real answer (similarity 1.0)
- SKILL.md preservation-gate wording aligned with --strict semantics
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Clayton Kim