67 Commits

Author SHA1 Message Date
Clayton Kim 6161e92bae Add macro-structure layer: deterministic structure scanner + evals
- scripts/structure_scan.py: document-level tells with research-validated
  thresholds — sentence burstiness (<0.55, min 8 sentences), conclusion coda
  (trigger words + first-paragraph bigram echo), bold-colon listicles,
  one-line staccato, connective paragraph openers, signpost density, opener
  repetition, participial closers; info-only metrics (summary sandwich,
  paragraph CV, triads, em-dash rate); --genre docs|social carve-outs;
  markdown-aware prose extraction
- Opener repetition tuned against the research corpus: 3-run parallel
  enumerations (In Rails / In Django / In raw SQL) are human — standalone
  triggers now need a 4-run or 4+ uses; corpus separates 10/12 AI docs
  flagged, 8/8 human docs clean
- Rows STRUCT-01..16 (FN/FP pairs per metric incl. genre carve-outs),
  ROB-12..14; SKILL-MACRO-01 (tune) / SKILL-MACRO-02 (holdout) with
  phrase-clean but structurally-AI multi-paragraph essays and a
  structure_scan backstop on the rewrite
- SKILL.md diagnosis + blocking validation integration; taboo-phrases.md
  Macro Structure section (judge-only tells marked agent-judgment);
  structure-scan gate in --list-gates and CHECKS.md

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-04 10:18:02 -07:00
Clayton Kim 86e188a6c8 Merge branch 'wp3-scripts' into eval-hardening
# Conflicts:
#	evals/adversarial-evals.json
2026-07-04 09:28:47 -07:00
Clayton Kim 4b1c64fa7c Expand scanner coverage and enforce catalog parity
- Collocation-gate seven literal-collision hard words (unpack, lean into,
  double down, showcase, bolster, garner, stakeholder) with word-boundary
  phrase matching; notwithstanding and 'the landscape of' literal uses spared
- New detection: wh-opener self-Q&A, negative listing, isn't-just parallelism
  variants, noting-phrase contraction/expansion pairs, cliffhanger fragments,
  modal+potentially hedge stacks, whether-you're audience openers, emoji
  section headers, 'underscores the need', substrate metaphor gate (biology
  spared), catalog families (analysts predict, treasure trove, ever-evolving,
  mosaic of, let's explore/examine)
- Em-dash and exclamation overuse no longer match across paragraph breaks
- game-changer containment dedup (one span, one violation)
- check_taboo_parity.py + DOC-01: scanner keys must be documented in
  references/taboo-phrases.md (two-way parity, drift fails the suite)
- Rows FP-15..38, FN-07..21, REC-08..18, DET-08..17, DOC-01

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-04 09:28:22 -07:00
Clayton Kim 87990359b6 Harden support scripts: unit families, magnitudes, curly quotes, faithful-rewrite fixes
- validate_preservation: unit-swap detection (km/mi, °C/°F, GB/TB, hr) with
  full alias families both directions; spelled magnitude words beyond currency;
  curly-quote tracking; range/quarter/time faithful rewordings pass while true
  swaps fail; --strict promotes semantic warnings (negation/scope/modality) to
  exit 1; modality strengthening (may->will) warned
- extract_constraints: proper-noun spans no longer glue across phrase
  boundaries; and/or no longer extracted as an /or endpoint
- readability_metrics: staccato detection sees em-dash/semicolon-joined
  fragments; sentence splitter handles closing quotes
- diff_check: pure reorder no longer excessive_change; docstring documents the
  exemption
- Rows PRES-10..27, SEM-03..07, ROB-09..11, AS-05 with paired fail guards

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-04 09:27:58 -07:00
Clayton Kim 10aa31ae16 Add "X is real" emphasis-crutch crime + eval
The meme-derived "the struggle is real" / "the stakes are real" punch line
asserts significance instead of showing it. Catalog it as a soft emphasis
crutch, restricted to abstract-emphasis subjects so the literal "is it
genuine?" sense (a painting that is real, not a forgery) is spared.

- banned_phrase_scan.py: new STRUCTURAL_PATTERNS regex (soft, emphasis_crutch)
- taboo-phrases.md: Emphasis Crutches table rows + detection-regex entry
- adversarial-evals.json: FN-06 (detection) + FP-14 (literal-sense guard)
- SKILL.md: 2.1.0 -> 2.2.0

Adversarial harness: PASS 54, FAIL 0; benchmark in sync.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 16:13:07 -07:00
Clayton Kim 6aec4e443e Shape eval-first pattern workflow 2026-06-16 10:26:01 -07:00
Claude 718550da8f Revise gold examples toward varied rhythm; align em-dash rule
Addresses the root cause behind the adversarial writing audit's B1: the skill
TAUGHT the anti-slop register. Its gold examples modeled bare fragment contrasts
and staccato triplets, so faithful operators reproduced them.

- edit-library.md: rewrote the three worst After examples (Patterns 1, 3, 12)
  from fragment/staccato cadence into single varied sentences; tightened Pattern
  2; added a "Vary the rhythm" transformation principle.
- crisp-human.md (the engine): sentence-length guidance now requires variety
  ("never three sub-six-word sentences in a row; avoid 'Not X. Y.'"); "one idea
  per sentence / no compound thoughts" relaxed to allow flowing clauses; the two
  antithesis gold examples rewritten; checklist gains rhythm + load-bearing-hedge
  items.
- warm-human.md: rewrote the gold example that tripped binary_contrast.
- personality-guide.md: added two guardrails (don't invent a first-person voice
  for impersonal copy; don't mistake de-hedging for voice in regulated content);
  scoped the "show uncertainty" example.
- banned_phrase_scan.py: dropped the single-em-dash soft flag, keeping only the
  2+ overuse flag, to match the revised "one appositive dash is fine" rule.

Result: all 0 gold After examples trip the scanner (was 8). Harness unchanged
(50 PASS / 1 XFAIL / 0 FAIL).
2026-06-14 22:41:03 +00:00
Claude 6bf34a1445 Adversarial writing audit: close the anti-slop register + semantic blind spots
A red-team of the writing the skill PRODUCES (not just detection) confirmed the
core thesis: unslop can swap corporate slop for an equally-detectable "anti-slop"
register, and its validation is blind to the semantic facts most at risk. An
incentivized adversary attacked faithful skill outputs; the top findings were
re-verified by hand before fixing.

Scanner (B1, B7): the gold fragment-contrast form ("Not the technology. The
people.") and staccato triplets scored 0 — the scanner was blind to the skill's
own house style. Added an anti_slop_register category (fragment contrast +
staccato run, soft) and a readability staccato flag with no sentence-count gate.

Validation (B2): extract_constraints now captures cited references (Section 12(b),
Figure 2, Eq. 3) as hard must-preserve; validate_preservation warns when
negations, scope words, or conditionals are dropped ("does not support" ->
"supports"). SKILL.md no longer claims validation covers meaning.

SKILL.md guards (B3/B4/B6/B8): added "Register & genre guards" — don't de-hedge
regulated/technical text, don't invent first-person voice, avoid your own
staccato/fragment tell, keep warm text warm. Em-dash rule rewritten from "default
to zero" (which 2/4 presets violate) to "sparing appositive dash, never a splice".

Evals: +8 deterministic (AS-01/02/03, FP-12, PRES-08/09, SEM-01) and +8 behavioral
(SKILL-FRAGMENT/HEDGE/LEGAL/STACCATO/WARMTH/NOINVENT/OVEREDIT/EMDASH). Full
write-up in evals/ADVERSARIAL-WRITING-ANALYSIS.md.

Suite: 77 cases (51 script: 50 PASS / 1 XFAIL / 0 FAIL, 26 behavioral).
2026-06-14 22:34:30 +00:00
Claude ddc3a49876 Expand pattern catalog from Wikipedia + peer-reviewed sources
Adds research-sourced AI-writing tells the scanner was missing, applied
conservatively to keep false positives low, plus detection/FP-guard evals.

New patterns (scanner + references/taboo-phrases.md, with sources):
- Wikipedia "Signs of AI writing": plays a {vital,crucial,significant} role,
  enduring/lasting legacy, leaves a lasting impact, watershed moment, in the
  realm of, rich cultural heritage, stunning natural beauty, has been described
  as, is widely regarded as, some critics argue, observers have noted, would be
  complete without, in this article we will explore, as an AI language model
- Berens & Kobak (Science Advances 2024, excess-vocab study of 14M PubMed
  abstracts): elucidate, delineate, underpin, unveil, seamless, invaluable,
  noteworthy, revolutionize, holds (great) promise, new avenues, shed new light,
  warrants further, it should be noted that, in recent years, with the advent
  of, taken together, garnered significant attention, a deeper understanding of

Restraint to avoid false positives:
- academic single-words are soft (flag when clustered, not on one use)
- gated "serves as a"/"stands as a" behind significance nouns, so "serves as a
  kitchen" is clean but "serves as a testament" still flags
- skipped cluster-only words (notably, particularly, demonstrate, exhibit)

Evals: DET-01..07 detection cases, FP-10/11 false-positive guards.
Suite now 62 cases (44 script: 43 PASS / 1 XFAIL / 0 FAIL, 18 behavioral skill).
2026-06-14 21:43:20 +00:00
Claude 93c79fd579 Harden scripts/runner, add CI, expand eval coverage to 35 script cases
Robustness:
- every script now exits cleanly (code 2) on a missing/unreadable input file
  instead of raising a traceback (was only fixed for validate_preservation)
- runner adds a per-case timeout and survives a missing command/crash instead
  of aborting the whole run
- gated two more context-sensitive single words behind their jargon
  collocations: "harness" (a horse harness vs "harness the power of") and
  "foster" (a foster family vs "foster a culture of")

CI:
- .github/workflows/evals.yml compiles the scripts and runs the regression
  harness on every push and PR; only an undocumented FAIL breaks the build

New eval cases (script target, all passing):
- FP-07/08 literal harness/foster; FP-09 smart-quote exemption
- REC-01/02/03 recall guards proving the gating still flags real jargon and
  stacked slop (so de-noising can't silently gut detection)
- PRES-05 comma-format equality, PRES-06 $2.4M == $2.4 million, PRES-07 dropped
  quarter is a lost fact
- ROB-04..07 missing-file robustness for each script, ROB-08 empty stdin

Suite: 53 cases (35 script: 34 PASS / 1 XFAIL / 0 FAIL, 18 behavioral skill).
2026-06-14 21:33:42 +00:00
Claude 2fb6375a23 Fix script-level defects exposed by the adversarial suite
Flips 19 of the 20 deterministic xfail cases to PASS (FP-06, literal
"delve into a place", stays xfail as an accepted regex sense-ambiguity limit).
Final: 20 PASS, 1 XFAIL, 0 FAIL.

Fact preservation (validate_preservation.py, extract_constraints.py):
- currency compares absolute magnitude, so $47.3M no longer equals $47.3 billion
- percentages require an exact token match (12% no longer matches 120%)
- dates require the month, not just the year (March 3 2020 != December 2020)
- bare integers and whole phone numbers are now tracked as constraints
- faithful rewordings ($47.3M -> $47.3 million) still pass, no false missing
- missing input files exit cleanly instead of raising a traceback

Scanner (banned_phrase_scan.py):
- removed over-broad entries: "the real" (hit "the real estate"), bare
  "period."/"full stop." (hit any sentence ending in "period")
- gated context-sensitive words (leverage, navigate, tapestry, boasts) behind
  jargon collocations so literal/financial senses aren't flagged
- added missing tells: in conclusion, firstly/secondly, underscore the
  importance, treasure trove, ever-evolving, rich mosaic, plus stop-slop's
  false-agency family (numbers speak for themselves, data tells a story)
- quote masking: dropped the 500-char cap, allowed multi-line quotes, and
  stopped masking single-quoted prose that was hiding real slop

Robustness (readability_metrics.py, diff_check.py):
- numeric-only text counts as words instead of reading as empty
- punctuation-only edits register as change instead of 0%

Kept references/taboo-phrases.md in sync with the new scanner patterns.
2026-06-14 21:25:00 +00:00
Clayton Kim f461b866a3 Ban dramatic-fragment loop, isn't/it's contrast, and X things/one thing
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-19 19:47:21 -07:00
Clayton Kim 07162227f9 Add audit-only mode and quote-aware AI pattern detection 2026-04-18 09:13:37 -07:00
Clayton Kim fee780e4c7 Sync AI-isms from Wikipedia rev 1349290390 (2026-04-16)
Adds chatbot-deference phrases, stacked-negation parallelisms,
and AI vocabulary (meticulous, bolstered, vibrant, etc.).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-16 15:48:17 -07:00
Clayton Kim d54ddbb4ce Aggressive enforcement of em-dashes, punctuation abuse, and missing AI tells
- Add em-dash detection to scanner (soft for single, hard for multiple per paragraph)
- Add ~40 new banned phrases: resonates with, speaks volumes, at the intersection of, etc.
- Add structural patterns: colon reveals, staccato fragmentation, So-openers, exclamation overuse
- Promote Punctuation Abuse to top-level section in taboo-phrases.md
- Fix contradictory examples that used em-dashes in "after" text
- Update SKILL.md workflow to explicitly require em-dash elimination in Pass 2

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-12 06:49:33 -08:00
Clayton Kim ad7fcd584f Integrate humanizer patterns + Wikipedia auto-update
Add 13 new AI-tell categories from Wikipedia's Signs of AI writing guide:
significance inflation, promotional language, superficial -ing analyses,
vague attributions, copula avoidance, formulaic challenges, negative
parallelisms, communication artifacts, knowledge-cutoff disclaimers,
generic conclusions, false ranges, elegant variation, AI vocabulary.

- taboo-phrases.md: ~150 phrases across 22 categories (up from ~87/9)
- banned_phrase_scan.py: 185 scanner entries with severity field (hard/soft)
- edit-library.md: 9 new before/after patterns (#16-24)
- personality-guide.md: new file for voice/personality guidance
- rubric.md: expanded Human Authenticity criterion
- wiki_sync.py: Wikipedia source page sync script (check/diff/prompt)
- SKILL.md: /unslop --wiki-sync maintenance command

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 12:59:54 -08:00
Clayton Kim 2ae9d63743 Initial commit: unslop skill for humanizing AI content
Two-pass system (diagnosis → reconstruction) with:
- Comprehensive taboo phrases reference (throat-clearing, jargon, etc.)
- 8-criteria scoring rubric (40 points max)
- 4 voice presets (crisp, warm, expert, story)
- Python validation scripts (constraints, banned phrases, readability)
- Before/after examples for articles, LinkedIn, sales content

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-08 06:38:37 -08:00