7 Commits

Author SHA1 Message Date
daymade 776b760c10 fix(github-sensitive-data-cleanup): 关闭 git 输出解码崩溃类 + 文档精确化(审阅轮 2) (#330)
代码(审阅 HIGH,已实跑复现):
- 全部 4 个脚本的 13 处 subprocess text=True 解码统一补 errors="replace"
  ——blob 通道(git grep, grep_all_commits)至今 strict 解码,GBK 编码
  源文件含命中行时 verify/scan 照样 UnicodeDecodeError 崩掉无报告;
  Lesson 9 的处方此前只落在 message 通道,现对该类整体闭环
- 验证:GBK 源文件含泄漏 → FAILED 且 blob+message 双通道各自定位
  commit hash(此前崩溃点);scan_repo 同仓 exit 0;GBK message 不崩

文档(审阅 LOW×2 + INFO×1):
- tooling_notes/SKILL.md/Lesson 7 的 git log 字面命令与实际
  --format=%H%x1f%B%x1e 不符,改为行为描述(hash 标注记录格式)
- tooling_notes/Lesson 9/CHANGELOG 补 commit_message_commits 前 10 截断说明
- CHANGELOG 修正 #328 的 --yes 修复面(Step 4 两块 + reference 节一块)
- Lesson 9 补记 blob 通道同类缺陷的发现与闭环 + 跨编码检测边界
  (errors=replace 防崩不让 UTF-8 pattern 命中 GBK 字节,归 Layer 4)

Co-authored-by: Claude <noreply@anthropic.com>
2026-08-23 12:21:35 +08:00
daymade 43d0d845a3 fix(github-sensitive-data-cleanup): 审阅 4 项 — GBK 解码崩溃/--yes 缺失/bundle verify -C/message 定位 (#328)
- verify_cleanup: check_pattern_in_messages 改 errors=replace(GBK/旧编码
  commit message 不再把 verify 崩成 UnicodeDecodeError traceback),并返回
  commit hash 列表(报告新增 commit_message_commits,FAILED 时可直接定位)
- rewrite_history: git bundle verify 补 -C <repo>(非 git cwd 调用不再
  traceback),create_backup 的 RuntimeError 纳入捕获
- SKILL.md Step 4 与脚本说明的 rewrite 命令块补 --yes(照文档原样跑此前
  exit=1 连 backup 都不建)

审阅驱动的端到端标定:GBK 仓含实体 → FAILED(commit_message_hits+hash);
GBK 干净仓 → PASSED;非 git cwd 跑 rewrite → 正常产出 backup。

Co-authored-by: Claude <noreply@anthropic.com>
2026-08-23 09:53:12 +08:00
daymade 9f56bcf74d fix(github-sensitive-data-cleanup): commit-message 通道重写与校验补全 (#326)
- rewrite_history.py 新增 --message-replacements:同一 pass 跑
  git filter-repo --replace-message(blob-only 重写会让实体继续活在
  commit message 里,含 main 上的 squash merge message)
- verify_cleanup.py 校验补 message 层(git log --all --format=%B),
  blob-only 验证无法发现 message-only 泄漏;输出新增 commit_message_hits
- SKILL.md 两处用法同步;incident-lessons.md 补 Lesson 7(message 通道
  双盲区)与 Lesson 8(共享 checkout 外人 untracked 文件阻塞重写)

端到端标定:scratch 仓 message 藏实体 → 重写前 FAILED(commit_message_hits=1)、
重写后 PASSED;rewrite_history 双 flag 内容与 message 同步改写。

Co-authored-by: Claude <noreply@anthropic.com>
2026-08-23 09:24:27 +08:00
daymade ab2c014374 fix(github-sensitive-data-cleanup): address code-review findings in scan/verify scripts
- Search the full commit history by batching git grep across all commits
  instead of truncating to the newest 1000.
- Propagate git grep errors from verify_cleanup instead of swallowing them.
- Parse gitleaks.toml with tomllib when available to handle TOML escapes
  correctly; keep a minimal fallback parser for older Python.
- Replace tempfile.mktemp with NamedTemporaryFile(delete=False).
- Strip whitespace from literal:/regex: prefixes in replacements.
- Fix docstring path for the identities file.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-06-26 02:17:13 +08:00
daymade f441a13faf fix(github-sensitive-data-cleanup): implement Layer 3/4 scanning with git grep --perl-regexp
- Replace git log --pickaxe-regex -S with git grep --perl-regexp across all
  commits so Layer 2/3 patterns with \b word boundaries match correctly.
- Parse private-domain-context and private-ip-context from user's gitleaks.toml
  plus optional identities file for Layer 3.
- Add Layer 4 AI semantic review prompt and eval.
- Update verify_cleanup.py to treat literal: and regex: search sides correctly.

Co-authored-by: Claude <noreply@anthropic.com>
2026-06-25 21:38:29 +08:00
daymade ef2a5d7775 fix(github-cleanup): address ultracode audit findings
- safe_push.py: let gh repo view infer repo from cwd; remove fallback
  defaults on forkCount/stargazerCount; abort on incomplete metadata
- rewrite_history.py: verify backup bundle with git bundle verify;
  check clean working tree before rewrite; verify git-filter-repo runs
- verify_cleanup.py: skip empty left-hand patterns from replacements
- evals.json: add refuses-no-verify-bypass eval
- SKILL.md: document .pii-patterns in .gitignore and backup verification

Co-Authored-By: Claude <noreply@anthropic.com>
2026-06-25 21:26:01 +08:00
daymade bfe3d19e5b feat(security): add github-sensitive-data-cleanup skill
Add a skill for scanning and removing sensitive data from GitHub repo
history. Includes:

- scan_repo.py: gitleaks + custom pattern scan
- rewrite_history.py: backup + git-filter-repo wrapper
- verify_cleanup.py: post-rewrite verification from replacements file
- safe_push.py: visibility check + safe force push
- references/incident-lessons.md: hard-won rules from real cleanups
- references/tooling_notes.md: git-filter-repo/BFG guidance
- evals/evals.json: test prompts

Also register the skill in .claude-plugin/marketplace.json.

All examples use safe placeholders; real private domains go in an
untracked .pii-patterns file.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-06-25 18:54:21 +08:00