mirror of
https://github.com/conorluddy/ios-simulator-skill.git
synced 2026-09-14 20:07:12 +08:00
Conor
c6c0ef8105
* feat: HangBuster session mode for hang_watcher (#77) Evolves hang_watcher.py from a passive `tail -f`-style log stream into an agent-native session recorder with progressive disclosure. Default --stop output is ~80-120 tokens (vs ~3-5k for v1 raw streaming) with explicit drill paths to per-cluster detail. Architecture: pure-function filter pipeline in common/hang_pipeline.py (parse → normalise → threshold → bucket → cluster → aggregate → rank → format). Session storage in common/hang_sessions.py with its own dir layout (~/.ios-simulator-skill/sessions/<id>/{meta.json, events.jsonl, summary.json}); 4-hex session-id suffix avoids same-second collisions. New CLI surface (HangBuster): - --start: detached worker via Popen(start_new_session=True); worker writes its own pid into meta.json (no pidfile race), opens events.jsonl line- buffered, applies normalise+threshold+bucket inline, select-based readline timeout with stream_ended sentinel. - --stop / --get-details [--cluster N | --raw] / --list-sessions / --clear-sessions [--older-than DURATION] / --diff A B - --budget-tokens picks the densest level (L0/L1/L2) that fits via char/4 estimator; --terse forces L0. - --auto-sample soft-imports main_thread_sampler (#62) with debounce (one sample per cluster per session); graceful no-op if missing. - --diff guards against fingerprint_version mismatch and emits a verdict line (regression / improvement / drift / no change) for CI. Backward compat: HangWatcher class + --watch / --since unchanged. Stateless parse helpers extracted to common/hang_pipeline.py; HangWatcher delegates. New env vars (all via env_int): IOS_SIM_HANG_MIN_MS (default 250), IOS_SIM_HANG_SESSION_TTL_HOURS (24), IOS_SIM_HANG_DEFAULT_TOP_N (3), IOS_SIM_HANG_BUDGET_TOKENS (unset). Tests: scaffolds pytest infrastructure (pyproject.toml + tests/conftest.py + pythonpath config) since the repo had none. 88 tests across pipeline (51), sessions (23), token budgets (8), diff (6). Token-budget contract asserted: 50-event session emits ≤200 tokens at L1, ≤2000 tokens at L2/L3. Docs: SKILL.md and README updated with HangBuster section + env vars. Closes #77 * docs: update CLAUDE.md with HangBuster + new shared utilities - Add hang_watcher.py + common/hang_pipeline.py + common/hang_sessions.py to the project structure tree - Add hang_watcher.py to Testing & Analysis category (now 6 scripts) - Document hang_pipeline.py + hang_sessions.py under Shared Utilities - Update Key Statistics: 7 shared utilities (was 6), reference 88 tests * test: manual smoke script for HangBuster lifecycle Walks through help / empty session / real session / drill (L2 + L3) / budget collapse / JSON output / diff / list+clear / legacy --watch in one script. Skips simulator-dependent steps when none is booted. Verified on macOS 26 with a booted iPhone simulator: 21 checks pass. Usage: tests/manual/hangbuster_smoke.sh # use booted simulator tests/manual/hangbuster_smoke.sh "iPhone 15" # boots a device first IOS_SIM_HANG_MIN_MS=100 tests/manual/hangbuster_smoke.sh * fix: address PR #78 review — blockers, atomicity, and code hygiene Blockers: - hang_watcher.py: `--top N` ternary was inverted; `--top 5` silently returned all clusters. Drop the ternary and always slice. - pyproject.toml: add "." to pytest pythonpath so bare `pytest tests/` resolves `tests.fixtures.*` imports (previously only worked via `python -m pytest`). Atomicity / boundary fixes: - hang_sessions.py: fsync before os.replace in _write_meta and _write_summary — atomic rename does not imply durability. - hang_sessions.py: new SessionStore.persist_worker_counters() public method; HangBuster.run_worker now calls it instead of reaching into the private _write_meta. The new method re-reads meta so a concurrent stop() that already set status=stopped is not clobbered to running. - hang_sessions.py: parse JSON for stream_ended sentinel detection instead of substring match (avoids false-positives on any message containing the literal 'stream_ended'). Code hygiene: - hang_pipeline.py: cluster_to_json / event_to_jsonl drop redundant Severity.value overwrites — asdict() leaves StrEnum members in place and json.dumps serialises them as strings (StrEnum is a str subclass). - hang_pipeline.py: distinct CRITICAL icon (‼) so format_l1 no longer uses the same ⚠ for both WARN and CRITICAL. - hang_watcher.py: delete duplicated HangWatcher._resolve_predicate method; route the two call sites to the module-level function that the worker already uses. Verification: - ruff check, black --check: clean - pytest tests/ and python -m pytest tests/: 88 passed * chore: simplify marketplace manifest - Move description to top-level field (was nested under metadata) - Drop pinned version from plugin.json so the git commit SHA becomes the version, giving users updates on every push
134 lines
4.0 KiB
Python
134 lines
4.0 KiB
Python
"""Diff intelligence: regressions, improvements, drift, version mismatch."""
|
|
|
|
from __future__ import annotations
|
|
|
|
from common.hang_pipeline import (
|
|
SummaryBuilder,
|
|
bucket_severity,
|
|
compute_fingerprint,
|
|
diff_sessions,
|
|
format_diff,
|
|
)
|
|
from tests.fixtures.sample_session import make_events, make_summary
|
|
|
|
|
|
# === diff_sessions ===
|
|
|
|
|
|
def test_diff_no_change_when_identical():
|
|
a = make_summary(session_id="hang-a")
|
|
b = make_summary(session_id="hang-b")
|
|
result = diff_sessions(a, b)
|
|
assert result["verdict"] == "no change"
|
|
assert not result["new_clusters"]
|
|
assert not result["resolved_clusters"]
|
|
|
|
|
|
def test_diff_detects_new_critical_cluster():
|
|
a = make_summary(session_id="hang-a")
|
|
b_events = make_events()
|
|
# Add a new high-severity event with a fresh fingerprint.
|
|
from common.hang_pipeline import NormalisedEvent
|
|
|
|
b_events.append(
|
|
NormalisedEvent(
|
|
delta_ms=29_000,
|
|
process="MyApp",
|
|
pid=42,
|
|
duration_ms=1800,
|
|
severity=bucket_severity(1800),
|
|
symbol="[BrandNew explode:]",
|
|
message_prefix="brand-new hang",
|
|
fingerprint=compute_fingerprint("[BrandNew explode:]", "brand-new hang"),
|
|
)
|
|
)
|
|
b = SummaryBuilder(
|
|
session_id="hang-b",
|
|
started_at="2026-05-22T14:35:00",
|
|
duration_ms=30_000,
|
|
).build(b_events)
|
|
result = diff_sessions(a, b)
|
|
assert any(
|
|
"[BrandNew explode:]" == c["symbol_or_prefix"] for c in result["new_clusters"]
|
|
)
|
|
assert "regression" in result["verdict"]
|
|
assert "critical" in result["verdict"]
|
|
|
|
|
|
def test_diff_detects_resolved_cluster():
|
|
a = make_summary(session_id="hang-a")
|
|
# B has fewer fingerprints than A — at least one resolved.
|
|
b_events = [
|
|
e for e in make_events() if e.symbol != "[ImageDecoder decode:]"
|
|
]
|
|
b = SummaryBuilder(
|
|
session_id="hang-b",
|
|
started_at="2026-05-22T14:35:00",
|
|
duration_ms=30_000,
|
|
).build(b_events)
|
|
result = diff_sessions(a, b)
|
|
assert any(
|
|
c["symbol_or_prefix"] == "[ImageDecoder decode:]" for c in result["resolved_clusters"]
|
|
)
|
|
|
|
|
|
def test_diff_flags_drift_when_max_duration_grows():
|
|
a_events = make_events()
|
|
a = SummaryBuilder(
|
|
session_id="hang-a", started_at="t", duration_ms=30_000
|
|
).build(a_events)
|
|
# B with the same fingerprints but 40% longer durations.
|
|
from common.hang_pipeline import NormalisedEvent
|
|
|
|
b_events = [
|
|
NormalisedEvent(
|
|
delta_ms=e.delta_ms,
|
|
process=e.process,
|
|
pid=e.pid,
|
|
duration_ms=e.duration_ms * 1.4,
|
|
severity=bucket_severity(e.duration_ms * 1.4),
|
|
symbol=e.symbol,
|
|
message_prefix=e.message_prefix,
|
|
fingerprint=e.fingerprint,
|
|
)
|
|
for e in a_events
|
|
]
|
|
b = SummaryBuilder(
|
|
session_id="hang-b", started_at="t", duration_ms=30_000
|
|
).build(b_events)
|
|
result = diff_sessions(a, b)
|
|
assert len(result["drift"]) > 0
|
|
# 40% bump should flag as worsened, not improved.
|
|
assert all(d["delta_pct"] > 0 for d in result["drift"])
|
|
|
|
|
|
def test_diff_version_mismatch_skips_structural():
|
|
a = make_summary(session_id="hang-a")
|
|
b = make_summary(session_id="hang-b")
|
|
b.fingerprint_version = 99
|
|
result = diff_sessions(a, b)
|
|
assert result["version_mismatch"] is True
|
|
assert "skipped" in result["verdict"]
|
|
# No structural keys when mismatched.
|
|
assert "new_clusters" not in result or not result.get("new_clusters")
|
|
|
|
|
|
# === format_diff ===
|
|
|
|
|
|
def test_format_diff_no_change():
|
|
a = make_summary(session_id="hang-a")
|
|
b = make_summary(session_id="hang-b")
|
|
out = format_diff(diff_sessions(a, b))
|
|
assert "no change" in out
|
|
assert "hang-a" in out
|
|
assert "hang-b" in out
|
|
|
|
|
|
def test_format_diff_version_mismatch_carries_warning():
|
|
a = make_summary(session_id="hang-a")
|
|
b = make_summary(session_id="hang-b")
|
|
b.fingerprint_version = 99
|
|
out = format_diff(diff_sessions(a, b))
|
|
assert "mismatch" in out.lower()
|