mirror of
https://github.com/theclaymethod/unslop.git
synced 2026-09-19 01:45:09 +08:00
Clayton Kim
d6f632117f
Applies a reviewed cleanup list without changing any script's CLI flags, JSON
keys, exit codes, or stderr messages (all pinned by eval rows).
Shared modules (dedup):
- scripts/_lang.py: ENGLISH_FUNCTION_WORDS/english_function_share/is_probably_english,
previously byte-identical copies in banned_phrase_scan.py and structure_scan.py.
- structure_scan.py now imports split_sentences from readability_metrics instead
of keeping its own copy.
- silhouette_scan.py's stopword set is a verified pure superset of
structure_scan's; SILHOUETTE_STOPWORDS = structure's set | the extras.
- harvest_classify.py imports recency_value/DATE_FLOOR from harvest_samples.
- evals/_check_support.py: ROOT, run(cmd, timeout=60), load_evals() -- the
timeout=60 safety net that only check_contrib.py had now covers check_pairs,
check_seeded_docs, check_mimic, check_contrib, check_voice, and
check_pattern_coverage too.
Small cleanups: contribute.py's row_fn drops its unused category param;
run_model_parity.py's resolve_models param renamed responses->payload;
harvest_classify.py's heuristic() attaches suspect_ai/dictated so
rank_enriched needs no reconstruction; check_pattern_coverage.py's paired flag
formulas become a plain "if neither: both = True"; check_contrib.py drops the
__import__("scripts.contribute", ...) spelling for a normal import;
run_mimic_refine.py computes docs_a/matrix_a once and passes it to both
make_live_source and write_outputs; calibrate_pairs.py factors its four
near-identical contraction-replacement closures into one _contraction_repl
helper used by both directions.
Altitude items: GENRE_SUPPRESSIONS lookup tables replace the inline
`genre != "..."` conditionals in structure_scan.py and silhouette_scan.py;
check_gates_doc.py additively verifies every *.py token in a gate command
exists under ROOT (behavioral-tune and rubric-judge are exempt -- neither
command has a .py token).
Efficiency (Phase 2), each verified against the same eval rows / diffed
outputs before landing:
- run_model_parity.py replaces its subprocess-per-scanner-call helpers with
in-process imports of banned_phrase_scan/structure_scan/validate_preservation
(mirrors run_mimic_refine's import pattern). PARITY slice: 14.2s -> 0.33s.
- voice_score.py's lcs_len (O(n*m) DP) is replaced by
has_common_substring_over(), an O(n+m) rolling-hash check for "any shared
substring longer than the 120-char threshold" (hash matches are verified
against the source text, so no false positives). Nothing pins the exact
longest_common_substring value (checked); it now reports the matched
threshold window length on a hit, 0 otherwise -- documented in the
docstring. The violation boolean is unchanged.
- gi_score() precomputes per-key distances once per candidate/impostor
instead of recomputing distances() from scratch every trial; trials do a
subset-weighted sum over the precomputed values. Arithmetically exact
(same RNG draw order, same float sums) -- verified the VOICE-08
determinism value and the full check_voice --separation/--gi/--gaming
output are byte-identical before/after.
- check_voice.py and check_pairs.py convert their subprocess-per-cell/row
scanner calls to in-process imports (voice_score/voice_profile,
banned_phrase_scan/structure_scan), mirroring the CLI's own decline/exit
logic so output stays byte-compatible.
Deferred (out of scope for a contract-safe pass): a protects-grain redesign,
giving silhouette_scan.py its own English-decline gate, and decomposing
run_mimic_refine.py's build_report().
python3 evals/run_adversarial.py: 434 PASS / 1 XFAIL / 0 FAIL, unchanged
throughout; wall time 54.8s -> 25.3s.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K6CYksdLbXbTAxcAQjvHz5
34 lines
1.5 KiB
Python
Executable File
34 lines
1.5 KiB
Python
Executable File
#!/usr/bin/env python3
|
|
import json, sys
|
|
from _check_support import ROOT, run
|
|
DOC_DIR=ROOT/'evals/fixtures/docs'
|
|
|
|
def scan_phrase(path):
|
|
p=run(['python3','scripts/banned_phrase_scan.py',str(path.relative_to(ROOT))])
|
|
try: data=json.loads(p.stdout)
|
|
except Exception: data={'violations':[],'total_violations':999}
|
|
return p,data
|
|
def scan_struct(path):
|
|
p=run(['python3','scripts/structure_scan.py',str(path.relative_to(ROOT))])
|
|
try: data=json.loads(p.stdout)
|
|
except Exception: data={'flags':[],'flagged':{}}
|
|
return p,data
|
|
|
|
def main():
|
|
ok=True
|
|
for manifest in sorted(DOC_DIR.glob('*_manifest.json')):
|
|
slug=manifest.name[:-14]
|
|
seeded=DOC_DIR/f'{slug}_seeded.md'; clean=DOC_DIR/f'{slug}_clean.md'
|
|
expected=json.loads(manifest.read_text())['expected']
|
|
_,pdata=scan_phrase(seeded); _,sdata=scan_struct(seeded)
|
|
cats={v.get('category') for v in pdata.get('violations',[])} | set(sdata.get('flagged',{}).keys())
|
|
missing=[e['category'] for e in expected if e['category'] not in cats]
|
|
if missing:
|
|
ok=False; print(f'{slug}: missing {missing}; saw {sorted(cats)}')
|
|
cp,cpdata=scan_phrase(clean); cs,_=scan_struct(clean)
|
|
if cp.returncode!=0 or cpdata.get('total_violations')!=0 or cs.returncode!=0:
|
|
ok=False; print(f'{slug}: clean twin failed phrase={cpdata.get("total_violations")} structure_exit={cs.returncode}')
|
|
if ok: print('seeded docs ok')
|
|
return 0 if ok else 1
|
|
if __name__=='__main__': sys.exit(main())
|