Implement sources_sync.sh from scratch using e2e_log.sh (was empty).
Add e2e_log.sh sourcing and standard event emission to cli_flow.sh,
semantic_index.sh, and cass_daemon_e2e.sh while preserving their
existing custom JSONL logging (Option A compatibility layer).
All scripts now emit structured JSONL to test-results/e2e/ in addition
to their custom output locations. Remove sources_sync.sh from .gitignore
since it is no longer a stub.
Closes br-20bz.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Improve Rust E2E test discovery robustness:
1. Check if git command is available before using git ls-files
2. Fallback to find command if git unavailable or returns empty
3. Properly strip path prefixes (./, tests/) and .rs extension
4. Use while-read loop for safer handling of filenames
This fixes test discovery in environments where:
- Git is not installed
- Running from a non-git directory
- Sparse checkout excludes tests
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Enhance run_all.sh to dynamically discover and run Rust E2E tests:
1. Use git ls-files to find tests/e2e_*.rs test files
2. Skip suite gracefully if no E2E tests found (yellow warning)
3. Build cargo test command with explicit --test flags per file
4. Enable E2E_LOG=1 for structured test output
5. Add --all-features and --verbose for comprehensive testing
6. Keep --nocapture for visible test output
Previous behavior used a glob pattern that might miss tests or include
non-E2E tests. New approach is explicit and handles empty test sets.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Two bugs were preventing the daemon E2E test from working:
1. Semantic indexing needed --full flag - incremental scan found no
messages because the test data timestamps were considered "old"
2. Test data filename pattern was wrong - Codex connector expects
rollout-*.jsonl, not daemon-e2e.jsonl
Also add scripts/e2e/daemon_fallback.sh wrapper so the daemon test
is included in the orchestrated E2E runner (run_all.sh picks up
scripts/e2e/*.sh).
Part of T7.3: E2E daemon fallback + health script
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Major improvements to the benchmark comparison script:
- Added statistical analysis with confidence intervals
- Improved baseline comparison logic
- Better outlier detection using IQR method
- Added support for multiple benchmark formats
- Enhanced human-readable output with color coding
- Added JSON output mode for CI integration
- Improved threshold configuration
- Added memory regression detection
- Better handling of missing baselines
- Added trend analysis for multi-run comparisons
The script now provides more accurate regression detection for CI
pipelines by using statistical methods rather than simple threshold
comparisons.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit enhances the daemon fallback E2E script (bead
coding_agent_session_search-yx9h) to emit structured JSONL logs following
the unified E2E logging schema defined in test-results/e2e/logging-audit.md.
New JSONL Event Functions:
- emit_run_start(): Captures environment metadata (git sha, branch, OS, arch)
- emit_phase_start(name, description): Marks logical phase boundaries
- emit_phase_end(name): Records phase duration in milliseconds
- emit_test_start(name, suite): Individual test start marker
- emit_test_end(name, status, duration_ms, error?): Test completion with pass/fail
- emit_metrics(name, key=value...): Performance metrics capture
- emit_run_end(exit_code): Final summary with test counts
New Test Coverage:
- Added test_status_command: Validates cass status --json health check
- Added test_daemon_fallback_unavailable: Tests fallback when daemon unavailable
- Both tests emit proper test_start/test_end events with timing
Phase Markers Added:
- build: Compile cass binary (when --no-build not specified)
- setup_sandbox: Prepare test fixtures and codex session data
- indexing: Build full-text and semantic indexes
- health_check: Validate cass status command
- daemon_fallback_test: Exercise daemon unavailable fallback path
Output Files:
- daemon_e2e.jsonl: Structured event log compatible with E2E aggregation
- report.json: Enhanced with test_summary (total/passed/failed) and jsonl path
CLI Options:
- Added --skip-health-check flag to bypass status validation
- Enhanced --help with full option descriptions
The JSONL events follow the schema:
{"ts":"ISO8601","event":"phase_start|phase_end|test_start|test_end|metrics|run_start|run_end","run_id":"...","runner":"bash",...}
This enables unified E2E log aggregation across Rust, Playwright, and bash
test runners for CI observability.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Create .github/workflows/install-test.yml for real install testing
- Builds actual release binary on Linux and macOS
- Runs install.sh with real rustc/cargo/sha256sum
- Verifies installed binary works
- Uploads structured logs as artifacts
- Outputs repro command on failure
- Fix scripts/validate_ci.sh node_modules exclusion
- Change glob from '!node_modules/**' to '!**/node_modules/**'
- Now properly excludes tests/node_modules/ and other nested dirs
- Update test-results/no_mock_allowlist.json
- Mark e2e_install_easy.rs:fake_bin as permanent (CI covers real testing)
- Mark install.rs:mock_system_info as permanent (test fixture, not mock)
- Mark install.rs:mock_resources as permanent (test fixture, not mock)
- Add documentation to tests/e2e_install_easy.rs
- Explain relationship between local test and CI workflow
- Reference allowlist status and rationale
Closes: bd-30qg (P6.14e)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Introduces comprehensive documentation for the bakeoff validation process
and adds integration testing for the daemon client. Includes shell scripts
for automated E2E validation of the cass bakeoff system.
- Add cass_bakeoff_validation.md documenting validation methodology
- Create bakeoff E2E validation shell script
- Add daemon directory with startup/management scripts
- Implement daemon client integration tests
Co-Authored-By: Claude <noreply@anthropic.com>
Major improvements to the semantic search subsystem:
- Add fastembed-based reranker for improved result relevance
- Implement model manager for dynamic embedding model selection
- Expand indexer semantic capabilities with batch processing
- Enhance TUI with reranker-aware result presentation
- Add e2e test script for semantic indexing validation
The reranker module provides a secondary ranking pass that
significantly improves search result quality for ambiguous queries.
Co-Authored-By: Claude <noreply@anthropic.com>
- tests/docs/readme.rs: Tests for README.md generation accuracy
- tests/docs/help.rs: CLI help and generated HTML validation tests
- scripts/validate_docs.sh: Shell script for link and content validation
- docs/DOCUMENTATION_STYLE.md: Documentation style guide
- Cargo.toml: Added [[test]] entry for docs test crate
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The XPath //testsuite/@tests only returns the first testsuite's
stats, not the aggregate totals. Changed to /testsuites/@tests
which correctly reads from the root element where nextest puts
aggregate counts.
Before: showed 422 tests (first suite only)
After: shows 513 tests (actual total)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Change `test(e2e_filters)` to `binary(e2e_filters)` - the test()
filter matches test function names, not test binary names
- Fix E2E filter in CI workflow
- Fix --quick and --e2e filters in test-report.sh
- Add binary(~e2e) to exclude E2E binaries in quick mode
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Update CI workflow to use cargo-nextest with JUnit XML output
- Add test-reporter action for PR/commit test summaries
- Add scripts/test-report.sh for local test report generation
- Configure separate profiles for unit tests (ci) and E2E (e2e)
The existing .config/nextest.toml already had JUnit configuration;
this change integrates it into the CI pipeline and provides local tooling.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Fix coverage-uncovered.sh to run tests once (was running 2x)
- Add jq dependency check with warning
- Replace bc with portable awk for floating-point math
- Add concurrent search benchmarks to runtime_perf.rs
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Optimization to coverage.sh:
- Run tests ONCE with --no-report flag
- Use 'cargo llvm-cov report' to generate LCOV, JSON, HTML from collected data
- Add jq dependency check with helpful install message
Previously the script would re-run tests for each report format.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add two test runner scripts for development workflow:
## test-quick.sh (129 lines)
Fast feedback loop for development - runs only essential tests.
Usage:
./scripts/test-quick.sh # Run quick tests
./scripts/test-quick.sh --lib # Only library unit tests
./scripts/test-quick.sh --cli # Only CLI tests
Features:
- Auto-detects cargo-nextest for parallel execution
- Color output when running in terminal
- Focused test selection for rapid iteration
## test-all.sh (537 lines)
Comprehensive test runner with detailed logging.
Usage:
./scripts/test-all.sh # Run all standard tests
./scripts/test-all.sh --all # Include SSH and slow tests
./scripts/test-all.sh --help # Show all options
Features:
- Timestamped log files in test-logs/
- JSONL structured output for CI parsing
- Separate test phases: unit, integration, doc, clippy
- Optional SSH and slow test inclusion
- Summary report with pass/fail counts
- Exit codes for CI integration
- Parallel execution support via nextest
Both scripts are self-contained with no external dependencies
beyond standard cargo tooling.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Dicklesworthstone