mirror of
https://github.com/max-sixty/worktrunk.git
synced 2026-09-14 20:00:38 +08:00
e4238e8369
Six guards read `src/`, or the snapshot corpus, as text rather than
compiling it — no stray `println!` outside the allowlist, no `eprintln!`
resolving to std's macro instead of anstream's, no bare
`env!("VERGEN_…")`, no `include_str!` of a path the package won't ship,
no host path in a `.snap`, and nothing but `src/testing/mod.rs` spawning
`wt` through `CARGO_BIN_EXE_wt`. Each asserts *absence* over what its
walk handed it, and that polarity is what makes a swallowed read
dangerous here in a way it isn't for code that acts on what it finds: a
file that drops out of the walk is indistinguishable from a file with
nothing wrong in it, so `violations` stays empty and the test passes
green over less than it claims.
Each carried its own copy of the same recursion — read the directory,
iterate entries, recurse into subdirectories, read and check files with
a matching extension. The copies drifted, and #3911 is the evidence:
closing a swallowed read in one left the identical swallow in the
others, and the review that caught it had to list remaining sites by
hand. `tests/common/source_scan.rs` owns the walk now and all six call
it. The guards keep what is actually theirs: which extension, what
counts as a violation, and their own coverage assertion.
Six swallowed reads go with it. `output_system_guard.rs` had
`entries.flatten()` in both its walks and `Err(_) => return` in both its
file readers; `version_build.rs` had `entries.flatten()` and `let Ok(..)
else { return }`. `flatten()` is the one that hides, dropping a
per-entry `io::Error` without looking like a `return` at all — a
`println!` in a file whose `stat` fails went unreported by
`check_no_unexpected_stdout_writes`, and a bare `env!("VERGEN_…")` in a
file that failed to read went unreported by
`vergen_env_vars_are_read_optionally`, the guard standing between that
and another #3123.
`visit_files` returns how many files it visited, because an absence
assertion also passes over an empty walk. Three guards had nothing that
would fail in that case and now assert the count. The other three
already assert something an empty walk cannot satisfy — a non-empty
asset set, a per-root snapshot count, an offender list equal to one
named file — so a count beside those would be a second mechanism for one
guarantee.
The return is `#[must_use]`, which matters more than it looks. Left
advisory, the classification of who needs the count would live only in
prose, and a seventh caller would inherit none of it — the same shape as
the bug being fixed, a per-caller obligation that four of five copies
quietly skipped. As a `must_use` it is a compile error under `-D
warnings`, and "my proof is elsewhere" becomes a `let _ =` a reviewer
can see and check.
The `scan` label names the calling guard. It previously claimed to and
didn't: three call sites passed the same `"src/ scan"`, so a permissions
failure read identically whichever guard was running, and only libtest's
thread name told them apart. The file-contents panic was likewise worded
exactly like the directory-listing one, separated only by whether the
path happened to be a file.
One behaviour change worth naming: `output_system_guard`'s allowlist
checks now run against a file the walk has already read, rather than
short-circuiting before the read. An allowlisted file that can't be read
is a file in the tree that can't be read, and now says so instead of
being skipped.
`tests/CLAUDE.md` records the rule, beside "a test's setup is part of
its proof" — the same failure in different dress. Nothing wrote the
hazard down, which is why each guard was free to rediscover it.
The rule holds outside the source scans too, so two walks in
`config_show.rs`'s `test_plugin_layout_is_consolidated` now count what
they assert absence over. One walks `plugins/worktrunk/skills` asserting
no entry is a symlink — if the mirror never generated, it visits nothing
and passes; `is_dir()` runs first, but an existing empty directory is
exactly what a failed mirror leaves. The `skills/` loop below it has the
same shape for `SKILL.md`. Neither reads file contents, so `visit_files`
isn't a drop-in and they carry their own counts.
## Test plan
`cargo fmt --check` and `clippy --all-targets` clean; the 15 tests
across the guard files pass.
Green proves least here, since a walker that silently visits nothing
passes every one of those tests. So each behaviour was reached for
directly:
<details><summary>Reach checks</summary>
**The walk recurses and reads.** A `println!` injected three levels deep
in `src/trace/profile.rs`, a file on no allowlist:
```
src/trace/profile.rs:1306: println!("probe");
```
**The walk covers roots outside `src/`.** The `CARGO_BIN_EXE_wt` needle
injected into a file under `tests/`:
```
left: ["src/testing/mod.rs", "tests/integration_tests/list_layout.rs"]
```
**The coverage assert fires.** A scan pointed at a directory that
exists, reads fine, and holds no `.rs` files:
```
scanned no files under …/.github
```
**The per-root snapshot count fires on a partial move.** `tests` alone —
1,088 of the 1,156 snapshots — swapped for a directory holding none,
leaving `src`'s 64 and `docs`' 4 in place:
```
.github/ holds no .snap files — the corpus has moved, so these assertions are now passing over whatever is left of it
```
An aggregate non-empty check survives that case, which is why the count
is per root. Every edit was reverted after its run.
</details>
> _This was written by Claude Code on behalf of max-sixty_
318 lines
12 KiB
Rust
318 lines
12 KiB
Rust
//! Guard tests over the committed snapshot corpus.
|
|
//!
|
|
//! Snapshots capture real output and are approved by reviewers. Subtle
|
|
//! problems slip through review: formatting issues (stray blank lines,
|
|
//! detached hints), and host-specific paths that insta never compares.
|
|
//! These tests scan every committed `.snap` file to enforce the rules.
|
|
//!
|
|
//! Rules enforced:
|
|
//! - **Hints attach to their subject** — no blank line before `↳`
|
|
//! - **No double blank lines** — one blank line maximum between elements
|
|
//! - **No host-specific paths** — insta compares only snapshot *content*
|
|
//! (stdout/stderr/exit code); the `info:` block insta-cmd records
|
|
//! (`args:`, `env:`) is metadata that is written but never compared, and
|
|
//! `add_filter` doesn't apply to it. A test missing a redaction therefore
|
|
//! bakes the generating machine's paths into the committed file while
|
|
//! passing everywhere — surfacing only as churn when someone regenerates
|
|
//! the snapshot on another machine (and under libtest, the `repo`
|
|
//! fixture's leaked settings binding can mask the omission; see `repo()`
|
|
//! in `tests/common`). This rule makes the leak fail deterministically.
|
|
//!
|
|
//! When a formatting rule fails:
|
|
//! 1. Fix the source code producing the bad output
|
|
//! 2. Re-run the failing test to regenerate the snapshot
|
|
//! 3. If the blank line is intentional (e.g., phase boundary before a status
|
|
//! item that uses `↳`), add the snapshot to the allowlist with a comment
|
|
//!
|
|
//! When the host-path rule fails: add a redaction — env values go in
|
|
//! `add_standard_env_redactions`, path arguments are covered by the
|
|
//! `.args[]` redaction in `add_repo_and_worktree_path_filters` (both in
|
|
//! `tests/common/mod.rs`); see tests/CLAUDE.md "Snapshot env drift".
|
|
|
|
use crate::common::source_scan::visit_files;
|
|
use ansi_str::AnsiStr;
|
|
use regex::Regex;
|
|
use std::path::Path;
|
|
use std::sync::LazyLock;
|
|
|
|
/// Snapshots where a blank line before `↳` is intentional (phase boundary, not
|
|
/// a detached hint). Each entry needs a comment explaining why.
|
|
const BLANK_BEFORE_HINT_ALLOWED: &[&str] = &[
|
|
// config show: blank line separates "Shell integration not active" diagnostic
|
|
// from per-shell status section. The `↳` here is a status item, not a hint.
|
|
"config_show__config_show_partial_shell_config_shows_hint",
|
|
"config_show__config_show_unmatched_candidate_warning",
|
|
"config_show__config_show_unmatched_candidate_not_suppressed_by_wrapper",
|
|
];
|
|
|
|
#[test]
|
|
fn test_no_blank_line_before_hint_in_snapshots() {
|
|
let project_root = Path::new(env!("CARGO_MANIFEST_DIR"));
|
|
|
|
let mut violations = Vec::new();
|
|
|
|
for_each_snapshot(project_root, |path, content| {
|
|
let stem = path.file_stem().and_then(|s| s.to_str()).unwrap_or("");
|
|
if BLANK_BEFORE_HINT_ALLOWED
|
|
.iter()
|
|
.any(|allowed| stem.contains(allowed))
|
|
{
|
|
return;
|
|
}
|
|
|
|
let clean = content.ansi_strip();
|
|
let lines: Vec<&str> = clean.lines().collect();
|
|
|
|
for (i, line) in lines.iter().enumerate() {
|
|
if line.trim().starts_with('↳') && i > 0 && lines[i - 1].trim().is_empty() {
|
|
let relative = path.strip_prefix(project_root).unwrap_or(path);
|
|
violations.push(format!(
|
|
"{}:{}: blank line before hint\n {}: {:?}\n {}: {:?}",
|
|
relative.display(),
|
|
i + 1,
|
|
i,
|
|
lines[i - 1],
|
|
i + 1,
|
|
line,
|
|
));
|
|
}
|
|
}
|
|
});
|
|
|
|
if !violations.is_empty() {
|
|
panic!(
|
|
"Blank line before hint (↳) in {} snapshot(s):\n\n{}\n\n\
|
|
Hints attach to their subject — no blank line between a message and its hint.\n\
|
|
Fix the source code, not the snapshot. If the blank line is intentional,\n\
|
|
add the snapshot to BLANK_BEFORE_HINT_ALLOWED with a comment.",
|
|
violations.len(),
|
|
violations.join("\n\n"),
|
|
);
|
|
}
|
|
}
|
|
|
|
#[test]
|
|
fn test_no_double_blank_lines_in_snapshot_output() {
|
|
let project_root = Path::new(env!("CARGO_MANIFEST_DIR"));
|
|
|
|
let mut violations = Vec::new();
|
|
|
|
for_each_snapshot(project_root, |path, content| {
|
|
// Extract output content, skipping YAML header.
|
|
// Two formats: stdout/stderr sections (insta_cmd) or expression
|
|
// content after closing `---`.
|
|
for (label, text) in extract_output_sections(content) {
|
|
let clean = text.ansi_strip();
|
|
if clean.contains("\n\n\n") {
|
|
let relative = path.strip_prefix(project_root).unwrap_or(path);
|
|
violations.push(format!("{} ({label})", relative.display()));
|
|
}
|
|
}
|
|
});
|
|
|
|
if !violations.is_empty() {
|
|
panic!(
|
|
"Double blank lines in {} snapshot output section(s):\n\n{}\n\n\
|
|
One blank line maximum between output elements.",
|
|
violations.len(),
|
|
violations.join("\n"),
|
|
);
|
|
}
|
|
}
|
|
|
|
/// Markers that only ever come from the generating machine: macOS per-user
|
|
/// temp dirs, the LLVM profile fallback dir (always under the host temp
|
|
/// dir), and tempfile's random `.tmpXXXXXX` directories.
|
|
static HOST_MARKERS: LazyLock<[Regex; 3]> = LazyLock::new(|| {
|
|
[
|
|
Regex::new(r"/var/folders/").unwrap(),
|
|
Regex::new(r"wt-test-profraw").unwrap(),
|
|
Regex::new(r"\.tmp[A-Za-z0-9]{6}").unwrap(),
|
|
]
|
|
});
|
|
|
|
/// Home-style paths are allowed only for the deliberate fake users in docs
|
|
/// examples and mocked output; a real username means an unredacted $HOME
|
|
/// leaked. `{1,2}` separators: YAML double-quoted scalars escape
|
|
/// backslashes, so a Windows path arrives as `C:\\Users\\name`.
|
|
static HOME_PATH: LazyLock<Regex> =
|
|
LazyLock::new(|| Regex::new(r"[/\\]{1,2}(?:Users|home)[/\\]{1,2}([A-Za-z0-9._-]+)").unwrap());
|
|
|
|
const FAKE_USERS: &[&str] = &["me", "user"];
|
|
|
|
fn line_has_host_specific_path(line: &str) -> bool {
|
|
HOST_MARKERS.iter().any(|re| re.is_match(line))
|
|
|| HOME_PATH
|
|
.captures_iter(line)
|
|
.any(|c| !FAKE_USERS.contains(&&c[1]))
|
|
}
|
|
|
|
#[test]
|
|
fn test_no_host_specific_paths_in_snapshots() {
|
|
let project_root = Path::new(env!("CARGO_MANIFEST_DIR"));
|
|
let mut violations = Vec::new();
|
|
|
|
for_each_snapshot(project_root, |path, content| {
|
|
for (i, line) in content.lines().enumerate() {
|
|
if line_has_host_specific_path(line) {
|
|
let relative = path.strip_prefix(project_root).unwrap_or(path);
|
|
violations.push(format!("{}:{}: {}", relative.display(), i + 1, line.trim()));
|
|
}
|
|
}
|
|
});
|
|
|
|
if !violations.is_empty() {
|
|
panic!(
|
|
"Host-specific paths in {} committed snapshot line(s):\n\n{}\n\n\
|
|
These churn whenever the snapshot is regenerated on another machine.\n\
|
|
Add a redaction: env values in `add_standard_env_redactions`, path\n\
|
|
arguments via the `.args[]` redaction in `add_repo_and_worktree_path_filters`\n\
|
|
(tests/common/mod.rs); see tests/CLAUDE.md \"Snapshot env drift\".\n\
|
|
Deliberate example paths use the fake users in FAKE_USERS.",
|
|
violations.len(),
|
|
violations.join("\n"),
|
|
);
|
|
}
|
|
}
|
|
|
|
/// Visit every committed `.snap` file. Snapshot dirs live under `src`
|
|
/// (unit tests), `tests` (integration tests), and `docs` (demo fixtures).
|
|
fn for_each_snapshot(project_root: &Path, mut f: impl FnMut(&Path, &str)) {
|
|
// Per root rather than in aggregate: `tests` alone holds the large majority
|
|
// of the corpus, so a total count survives that root moving away. Each root
|
|
// is load-bearing as a result — one that legitimately stops holding
|
|
// snapshots belongs out of this list, not exempted from it.
|
|
for root in ["src", "tests", "docs"] {
|
|
let seen = visit_files(&project_root.join(root), "snap", "snapshot scan", &mut f);
|
|
assert!(
|
|
seen > 0,
|
|
"{root}/ holds no .snap files — the corpus has moved, so these \
|
|
assertions are now passing over whatever is left of it"
|
|
);
|
|
}
|
|
}
|
|
|
|
/// Extract labeled output sections from a snapshot file.
|
|
///
|
|
/// Handles two formats:
|
|
/// - **insta_cmd**: sections delimited by `----- stdout -----` / `----- stderr -----`
|
|
/// - **expression**: content after the closing `---` YAML delimiter
|
|
fn extract_output_sections(content: &str) -> Vec<(&str, &str)> {
|
|
let mut sections = Vec::new();
|
|
|
|
// Try stdout/stderr sections (insta_cmd format)
|
|
for section in ["stdout", "stderr"] {
|
|
let marker = format!("----- {section} -----");
|
|
let Some(start) = content.find(&marker) else {
|
|
continue;
|
|
};
|
|
let after_marker = &content[start + marker.len()..];
|
|
let end = after_marker
|
|
.find("----- stdout -----")
|
|
.or_else(|| after_marker.find("----- stderr -----"))
|
|
.unwrap_or(after_marker.len());
|
|
sections.push((section, &after_marker[..end]));
|
|
}
|
|
|
|
// If no stdout/stderr sections, try expression format (content after closing ---)
|
|
// Match `---` only at line boundaries to avoid false matches on content like
|
|
// `--- a/file.txt` in git diffs.
|
|
if sections.is_empty() {
|
|
let first_delim = if content.starts_with("---\n") {
|
|
Some(0)
|
|
} else {
|
|
content.find("\n---\n").map(|pos| pos + 1)
|
|
};
|
|
if let Some(pos) = first_delim {
|
|
let after_first = &content[pos + 3..];
|
|
if let Some(nl_pos) = after_first.find("\n---\n") {
|
|
let output = &after_first[nl_pos + 4..];
|
|
sections.push(("expression", output));
|
|
}
|
|
}
|
|
}
|
|
|
|
sections
|
|
}
|
|
|
|
#[cfg(test)]
|
|
mod tests {
|
|
use super::*;
|
|
|
|
/// The corpus test passes vacuously if the detector rots — pin its
|
|
/// sensitivity on both sides. Leak lines are the shapes actually found
|
|
/// in committed snapshots (#3009 and the `args:` leak this fixed).
|
|
#[test]
|
|
fn test_host_specific_path_detection() {
|
|
let leaks = [
|
|
// macOS per-user temp dir (env value, the #3009 class)
|
|
"LLVM_PROFILE_FILE: /var/folders/v3/8k5q0_6j3/T/wt-test-profraw/cov.profraw",
|
|
// tempfile-crate dir in the args: block
|
|
"- /private/var/folders/wf/s6ycxvvs/T/.tmpSeGxzx/repo",
|
|
// tempfile-crate dir under /tmp (Linux)
|
|
"path: /tmp/.tmpAbC123/repo",
|
|
// real home dirs, Unix and YAML-escaped Windows
|
|
"HOME: /Users/maximilian/workspace",
|
|
"HOME: /home/runner/work",
|
|
r#"USERPROFILE: "C:\\Users\\runneradmin\\AppData""#,
|
|
];
|
|
for line in leaks {
|
|
assert!(line_has_host_specific_path(line), "should flag: {line}");
|
|
}
|
|
|
|
let deterministic = [
|
|
// deliberate fake users in docs examples and mocked output
|
|
"e.g., /Users/me/code/myproject",
|
|
"To /Users/user/workspace/repo/.git",
|
|
r#"PSModulePath: "C:\\Users\\user\\Documents""#,
|
|
// machine-independent literals and placeholders
|
|
"WORKTRUNK_CONFIG_PATH: /nonexistent/wt/config.toml",
|
|
"LLVM_PROFILE_FILE: \"[LLVM_PROFILE_FILE]\"",
|
|
"WORKTRUNK_SYSTEM_CONFIG_PATH: /etc/xdg/worktrunk/config.toml",
|
|
];
|
|
for line in deterministic {
|
|
assert!(!line_has_host_specific_path(line), "should allow: {line}");
|
|
}
|
|
}
|
|
|
|
#[test]
|
|
fn test_extract_output_sections_ignores_mid_line_dashes() {
|
|
// Simulate a snapshot whose output body contains `--- a/file.txt` (git diff).
|
|
// The old code matched `---` as a bare substring and would split incorrectly.
|
|
let content = "\
|
|
---
|
|
source: tests/some_test.rs
|
|
expression: output
|
|
---
|
|
diff --git a/file.txt b/file.txt
|
|
--- a/file.txt
|
|
+++ b/file.txt
|
|
@@ -1 +1 @@
|
|
-old
|
|
+new
|
|
";
|
|
let sections = extract_output_sections(content);
|
|
assert_eq!(sections.len(), 1);
|
|
assert_eq!(sections[0].0, "expression");
|
|
assert!(
|
|
sections[0].1.contains("--- a/file.txt"),
|
|
"mid-line `---` should be part of the output, not a delimiter"
|
|
);
|
|
}
|
|
|
|
#[test]
|
|
fn test_extract_output_sections_expression_format() {
|
|
let content = "\
|
|
---
|
|
source: tests/some_test.rs
|
|
expression: output
|
|
---
|
|
hello world
|
|
";
|
|
let sections = extract_output_sections(content);
|
|
assert_eq!(sections.len(), 1);
|
|
assert_eq!(sections[0].0, "expression");
|
|
assert_eq!(sections[0].1.trim(), "hello world");
|
|
}
|
|
}
|