The trigger menu's "Run specific..." entry was unusable for idle probes
because slugs are sourced from inflight.services, which is null in the
typical manual-trigger case. Hide the entry entirely when serviceSlugs
is empty (Option C); operators can still trigger all targets via "Run
all". A richer ProbeScheduleEntry contract that exposes discovered
targets for idle probes is tracked separately.
Also wraps onTrigger awaits in try/catch so async rejections surface
through a local error state instead of bubbling up as unhandled promise
rejections, and resets the selected slug set on completion so the next
trigger session starts clean.
runTone returned "gray" (unknown) when finishedAt was set but summary
was null, while runStateLabel returned "completed" for the same case —
producing a gray badge labeled "completed", exactly the misleading
combination the tone comment said it was avoiding. runStateLabel now
returns "unknown" so label and tone agree.
When all probes are idle, findNextScheduled previously returned the
entry with the smallest nextRunAt regardless of whether that timestamp
was already in the past, producing nonsensical "Next: smoke 5m ago"
banners. Filter out entries whose nextRunAt is < now and fall back to
the existing "No upcoming runs scheduled." sentinel.
Also typed STATE_ICON as Record<ServiceState, string> so missing service
states fail to compile rather than silently rendering undefined icons.
The local interface in status-tab.tsx was marked TODO and had drifted
from the canonical lib/ops-api definition: it lacked error? and
finishedAt? on service progress entries, and used a bare 'string' for
'kind' instead of the ProbeKind union.
Delete the local interface and re-export the canonical type from
lib/ops-api so existing 'import type { ProbeScheduleEntry } from
"./status-tab"' call sites continue to compile against the single
source of truth (CR-B1.1).
The previous useProbes/useProbeDetail hooks reset aliveRef.current to
true at the top of every effect run. A fetch from the prior cycle that
resolved between cleanup and re-init could pass the new effect's alive
check and clobber state with stale data.
- useProbes: per-effect 'cancelledRef' (mirrored so the imperative
refetch callback can honor it). Cleanup flips cancelledRef true.
- useProbeDetail: per-effect 'let cancelled = false' closure; cleanup
flips it. Each effect run gets its own closure, so a stale fetch from
the prior cycle cannot land setData (CR-B1.3).
- useProbeDetail also clears data immediately on id change so the
panel header (driven by prop) is never out of sync with the body
while the new fetch is still in flight (CR-B1.4).
- useTriggerProbe gains a per-call AbortController whose signal is
forwarded to triggerProbe; unmount aborts any in-flight POST so we
don't fire duplicate runs after the user navigates away (CR-B1.5).
Tests cover: rapid-deps stale-data path, rapid-id stale-data path,
data-clear on id change, and unmount-during-trigger abort.
- ProbeLastRun.summary is now nullable to match ProbeRun.summary; the
server can complete a run with no summary (showcase-ops CR-A1.5).
- ensureOk surfaces body-read failures with a 'body read failed: ...'
marker instead of swallowing them silently (CR-B1.6 / CR-C15).
- New parseJson helper attributes JSON parse failures to the requesting
URL so call sites can tell which endpoint produced bad bytes.
- triggerProbe gains a TriggerProbeOptions interface with an optional
AbortSignal; passed straight through to fetch() so callers can cancel
in-flight POSTs (CR-B1.5).
A6 (subject scope): probe-config-parity.test.ts resolved
`path.resolve(process.cwd(), "config/probes")`, which broke when
vitest was launched from any directory other than `showcase/ops`.
Switch to `import.meta.url`-based resolution: derive `__dirname` via
`fileURLToPath`, then `path.resolve(__dirname, "../../../config/probes")`
walks up from `src/probes/loader/` to the configs dir regardless of
cwd. Updated the comment to drop the "mirrors probe-loader.test.ts"
claim and explain why cwd was wrong.
B1 (test polish): the shortest-service-first integration test's
per-goto sleep was `demoCount × 1ms` (5/20/38ms total). With
concurrency=2 and `Date.now()` resolution at the millisecond, the
`tiny.lastGotoAt < huge.firstGotoAt` assertion could tie at the
millisecond boundary on a fast host. Bump the multiplier to 5×
(25/100/190ms per goto, total 125/2000/7220ms) so the assertion is
well-separated. Bumped the per-test timeout to 15s to accommodate
the wider wall-clock.
Test counts after this round: full-suite 845 passing (up from 842),
2 pre-existing failures unchanged (probe-loader watch + aimock-
fixture-coverage).
A5 (subject scope): the cost-budget paragraph said "17 framework
packages and ~32 demos each, total ~544 page loads" but referenced a
"38-demo service" elsewhere — internal contradiction. Rewrite to:
- explicit ~32-demo average vs ~38-demo largest service
- explicit absolute worst case ~38 × 30s ≈ ~19 min (against the
20-min outer cap)
- explicit "wall-clock bounded by slowest service under bounded
concurrency, NOT aggregate-divided-by-concurrency"
- typical 3-5s per goto stays as the realistic case
No semantic config changes — comment-only.
A2 (subject scope): `loadDemosMap` called `fs.readFile(path, "utf-8")`
which ignored `ctx.abortSignal`. A stalled volume mount could orphan
past the probe's `timeout_ms`. Pass `{ encoding: "utf-8", signal:
ctx.abortSignal }` so an AbortError lands in the same catch as other
fs failures and degrades to an empty demos map (sibling probes still
get the service list, just without demo enrichment).
Added regression test that pre-aborts the discovery context's signal
before calling `enumerate()`, then asserts `registry-read-failed`
warned exactly once and (when GraphQL didn't also abort) every record
has `demos: []`.
A1 (subject scope): the `ProbeInvokerErrorClass` union declared
`"discovery-error"` but the enumerate-throw path returned `[]` — operators
saw zero writes, indistinguishable from "no services matched the
filter". Mirror the misconfigured-discovery sentinel pattern: when
`source.enumerate()` throws, return a single sentinel `ResolvedInput`
keyed `${probeId}:enumerate-failed`. The invoker's existing top-level
sentinel branch writes a synthetic `state:"error"` ProbeResult with
`errorClass: "discovery-error"`. Sibling probes are unaffected.
A3 (subject scope): drop the unconditional `discoveryAbort.abort()` in
the discovery `finally`. Aborting after a successful enumerate signals
timeout-on-success to any source listener that snapshots
`signal.reason` — the inverse of what the signal represents. Listener
cleanup is not a real concern: the controller is local to the function
call and listeners die with it on GC. Updated comment to capture the
rationale.
A4 (subject scope): rewrite the timeout-bound paragraph in
`buildProbeInvoker` to remove the contradiction between "bounded by the
slowest single phase" and the formula `timeout_ms × (1 + ⌈n/c⌉)`. The
formula IS roughly the sum (discovery + ⌈n/c⌉ rounds of executors), so
the framing is now consistent.
A7 (subject scope): the late-rejection unhandled-handler test now waits
on a deterministic `whenRejected` promise resolved by the driver right
after scheduling its late reject(), then waits an additional 250ms (was
50ms) for any `unhandledRejection` to land. Removes the "passes
vacuously on slow CI" risk.
E1 test updated: hanging discovery source now emits ONE synthetic tick
with `errorClass: "discovery-error"` (was zero writes).
B2 polish: tighten `peak >= 2` to `peak === 3` in the max_concurrency
test so a half-broken pool can't pass.
New tests:
- enumerate-throw → one synthetic discovery-error tick
- sibling probes' invocations unaffected by a throwing source
Call-site enumeration:
- `discoveryAbort.signal.aborted` post-success: grep showed only smoke.ts
and e2e-demos.ts use `.signal.aborted` on their OWN per-driver
AbortController instances, not on `discoveryAbort` — A3 is safe.
- `MISCONFIGURED_DISCOVERY_INPUT` identity equality: only the top-level
sentinel branch checks identity; the new `ENUMERATE_FAILED_INPUT`
sentinel is checked in the same branch via `||` and discriminated
by the same identity check.
Threads runtime deps the prior slots flagged so the Status tab works end-to-end:
- buildProbeInvoker now receives `scheduler` + `runWriter` so B7's
ProbeRunTracker registration and probe_runs row start/finish actually fire
in production. Pre-fix both deps were optional and unset, leaving tracker
+ run-history dead code.
- Mounts the /api/probes router (B3) on the live server when
OPS_TRIGGER_TOKEN is set. Skipped with an info log otherwise so existing
tests / dev boots without the env var still come up.
- Threads `getProbeConfig(id)` from a per-probe config map populated in
`diffProbeSchedules`, so the router renders timeout_ms / max_concurrency /
discovery instead of nulls.
- Adds an integration test that boots the orchestrator with a minimal probe
YAML and verifies GET /api/probes returns the probe with non-null config
and the resolved kind.
- Patches FakeScheduler in http/probes.test.ts with a `setEntryTracker`
no-op (B7 widened the Scheduler interface; B3's test fake was integrated
before that change).
lastRun.state plumbing left for follow-up — the scheduler's lastRun fields
don't carry per-run state today, and B3's router hard-codes 'completed'.
Surfacing a real state requires extending EntryStatus + scheduler bookkeeping
through a path that touches several test surfaces; deferred.
The CatalogCell type gained three required fields (integration_name,
feature_name, category_name) before this branch landed; the existing
test fixtures predate that change and surface as 32 typecheck errors
in CI. Add plausible Title-Case names to each fixture so the type
checker is happy. Test logic and assertions are unchanged.
StatusTable gains an optional onSelect(probeId) prop. When provided, rows
become clickable (cursor-pointer) and the action cell stops propagation so
that clicking the trigger button does not also open the drilldown.
StatusTab owns the selection state (useState) and renders StatusDetailPanel
beneath the running panel. The panel returns null when no probe is
selected, so this is a zero-cost addition to the default render.
Adds three new components for the Status tab drilldown:
- StatusRunsList: recent-runs table with relative-time, duration, state badge
(running/completed/failed tones), manual chip when triggered=true, and
pass/total summary with em-dash fallback.
- StatusDurationSparkline: pure-SVG inline sparkline (no charting deps),
oldest-to-newest convention, dash placeholder for <2 points, configurable
width/height.
- StatusDetailPanel: renders nothing when probeId is null, shows loading,
error, and data states from useProbeDetail; composes the runs list and the
sparkline; close button calls onClose.
All three components ship with red-green vitest + RTL coverage (21 new tests).
B5. New test asserts smoke.yml / e2e-smoke.yml / e2e-demos.yml share
an identical `discovery.filter.nameExcludes` list. The three YAMLs'
header comments claim "kept in sync" but no automated check
enforced it — a new infra service added to one would silently flap
red on the others.
Reads the YAMLs directly (not through createProbeLoader) to avoid a
registry-mock bootstrap; the parity invariant is independent of
probe registration.
Compares as sorted arrays so list order doesn't trip the assertion;
only set membership matters for exclusion semantics.
Extends `resolveCell` to read the `d5:<slug>/<featureType>` and
`d6:<slug>/<featureType>` PB row keys produced by the e2e-deep (D5)
and e2e-parity (D6) drivers. The new badges are informational and do
NOT contribute to the rollup — same model as the post-Phase-3 smoke
slot — because the alert engine routes those dimensions independently.
`useLiveStatus` already pulls every dimension via the page-level
unfiltered subscription, so no subscription-shape change is required.
The `d5` / `d6` literals already live in the closed `DIMENSIONS` enum
in showcase/ops/src/types/index.ts.
Adds D5 / D6 chips to `CellStatus` next to the existing E2E badge —
two ~4-line additions that reuse the existing LiveBadge wrapper, no
structural changes to feature-grid / cells-view / parity-view.
B1. Framework count drift: 15 → 17 (matches MEMORY/platform reality).
Aggregate page loads per tick recomputed: 17 × ~32 = ~544.
B2. Wall-clock arithmetic: rewritten to express the actual bound. Under
bounded `max_concurrency`, wall-clock is dominated by the SLOWEST
single service's serial per-demo fan-out (~3 minutes for a 38-demo
service at ~3-5s/goto), NOT aggregate-divided-by-concurrency.
Smaller services interleave in parallel slots and don't extend
wall-clock past the slowest one.
B3. `key_template` comment block: separated from the preceding
`nameExcludes` list with a `#`-only blank comment line so the
visual hierarchy makes clear `key_template` is a sibling of
`source` / `filter` (not under `filter`). Schema confirms this
placement: `loader/schema.ts:DiscoveryBlockSchema` declares
`key_template` directly on `discovery`, not under `filter`.
B4. Outer-timeout side-emit claim: clarified that the per-demo abort
rows on outer timeout are the DRIVER's responsibility, not the
invoker's. Verified against `e2e-demos.ts` lines 452-473: the
driver's per-demo loop checks `abort.signal.aborted` at the top
of every iteration and writes a red side-row before continuing,
so the YAML's existing claim is accurate. Comment now explicitly
distinguishes driver vs invoker behaviour to prevent future
confusion.
Plus a TS typing fix on the new A1 regression test (the rejecting
driver.run now has an unreachable return-block to satisfy the
ProbeDriver signature).
D1. Concurrency-peak test: per-tick sleep bumped from 15ms → 50ms so
"peak >= 2" holds under heavy CI load. Wall-clock impact ≤ ~170ms.
D2. Invoker-timeout-while-driver-ignores-abort test: ceiling raised
from 70ms → 100ms. The assertion still catches the regression
where the invoker awaited the full driver.run() promise (would
cross the 80ms driver-sleep), but stops flaking on slow workers.
D3. driverResolved assertion tightened from `void` to
`expect(driverResolved).toBe(false)`. Given timeout_ms=20 <
driver-sleep=80, the assertion is deterministic and the previous
"non-deterministic" comment was overcautious.
C1. mkCtx() now stubs `fetchImpl` + `abortSignal` so ProbeContext is
fully populated. Both fields are optional today but a complete stub
catches future tightening without rewriting every test.
C2. starter-skip test: replaces `state.closed === false` (weak proxy —
could pass even on a launch+leak regression) with an explicit
`launched` flag captured inside the launcher closure. Asserts the
launcher was never invoked AND no contexts opened.
C3. failure-on-goto test: adds `state.contextsClosed === contextsOpened`
assertion mirroring the happy-path test. Catches browser-context
leaks where a throw inside runDemo() bypasses context.close().
C4. emits-green-without-goto test: documents the load-bearing coupling
between pageScripts.length and the driver's routeless-cell skip
behaviour. The gotoCalls assertion is the actual regression catch;
the array length is documentary.
Adds regression tests for the bucket (c) probe-invoker fixes landed in
the prior commit. Each test asserts a single behavioural contract so a
future refactor that breaks one item produces a localised failure.
E1. Discovery enumerate timeout — when source.enumerate() observes
abortSignal and the per-tick timeout_ms fires, the invoker returns
promptly with zero writes (discovery failed → nothing to fan out).
E2. Writer.write failure isolation — a writer hiccup on one key does
NOT short-circuit sibling targets; all 3 targets attempt their
write, only the failing key throws.
A1 regression test — late driver rejections (after the timeout has
already settled the race) must not surface as
process.on('unhandledRejection'). The driver promise is now caught
via a sacrificial .catch(() => {}) so a well-behaved driver that
rejects on abort doesn't leak.
A3 collision test — two records with missing template paths receive
distinct unique keys (`__unresolved_<idx>_<n>` suffix) so the writer
doesn't collapse them into one row.
A3 non-primitive test — interpolating an object-typed path emits the
unresolved-suffix instead of producing `[object Object]` keys.
A6 array-typed record — falls back to the `{ key }` branch with a
structured warning, rather than spreading the array shape.
A11 errorClass discriminator — four call sites all flow the correct
class through the synthetic ProbeResult signal:
- timeout
- input-rejected
- driver-error
- discovery-source-missing
Wires B10 (SSE interceptor), B11 (parity-compare), and B12 (reference-
capture) into an executable D6 e2e-parity driver. Each tick picks ONE
integration via the scoping helper (weekly-rotation by default; on-
demand via env), runs every D5 featureType for that integration,
captures a ParitySnapshot, and compares against the reference snapshot
on disk under fixtures/d6-reference/.
Files:
- src/probes/helpers/d6-scoping.ts/test.ts — rotation math (ISO week
modulo sorted slug list), on-demand env selector, and the reference-
snapshot loader (ENOENT->missing, JSON/shape errors->invalid).
- src/probes/drivers/e2e-parity.ts/test.ts — driver. Aggregate state
(red/degraded/green) over per-feature severity. Severity: 0 axis
failures->green, 1-2->amber (ProbeState "degraded"), 3-4->red. Skip
semantics: missing script or missing reference -> green note;
malformed reference -> red. Per-turn SSE-interceptor attach pattern,
starter short-circuit, scoping/launcher/goto error mapping.
- config/probes/e2e-parity.yml — Mondays 04:00 UTC, timeout 600_000,
max_concurrency 2, same discovery filter as e2e-deep. Documents on-
demand mode (D6_MODE/D6_TARGET_INTEGRATION).
- scripts/d6-capture-references.ts — CLI for refreshing reference
snapshots; defaults to langgraph-python; --feature targets one type;
exits 1 on any failure.
- src/types/index.ts — adds "e2e_parity" + "d6" to DIMENSIONS.
- fixtures/d6-reference/README.md — points operators at the CLI.
Tests: 37/37 D6-specific (20 scoping + 17 driver). Full suite 1024
passing; 2 pre-existing failures unrelated to D6 (aimock-fixture
coverage missing byoc-json-render fixtures, probe-loader chokidar
flake). YAML config validates against ProbeConfigSchema.
Bucket (c) sweep — pre-existing defects in probe-invoker.ts surfaced during
CR R1. Each item below is independently testable; existing tests already
cover the timeout/race contract (probe-invoker.test.ts) and discovery flow
(e2e-demos.test.ts integration), so no behavioural regressions.
A1. Late driver-rejection unhandled-rejection: attach a sacrificial
`.catch(() => {})` to the driver promise before racing it against the
timeout promise. A well-behaved driver that rejects on abort AFTER the
invoker has settled the race no longer surfaces as
`process.on('unhandledRejection')`.
A2. timeoutPromise dead-code path: resolve the synthetic timeout
ProbeResult directly inside the setTimeout callback rather than
routing through an abort-event listener. Anything else that aborts
the controller in the future cannot starve the timeout promise.
abortCtrl.abort() is still called separately so well-behaved drivers
receive the cancellation signal.
A3. interpolateTemplate empty-key/non-primitive/empty-path collisions:
widen the regex to `[^}]*`, route empty-path/missing-path/non-
primitive-value cases through a structured `probe.template-path-
unresolvable` warning AND a unique `__unresolved_<recordIndex>_<n>`
suffix so colliding paths surface as distinct keys instead of silently
overwriting each other.
A4. discovery-source-missing produces a synthetic dashboard tick: when
the YAML names a source the registry doesn't know, emit a synthetic
`state: "error"` ProbeResult with stable key
`${cfg.id}:misconfigured` and `errorClass: "discovery-source-missing"`.
Operators previously had to check the log stream to notice.
A5. Defensive `Math.max(concurrency, 1)` worker-pool floor. Schema
already requires `>= 1` but the floor protects against a future
schema relaxation silently dropping every input.
A6. Discovery records that are arrays no longer spread into bare `key`
objects. Arrays go through the `{ key }` fallback branch with a
`probe.discovery-record-non-object` warning.
A7. Discovery AbortController gets `.abort()` in the success-path
`finally` so listeners attached by the source clear out promptly.
AbortController is idempotent so the call is safe.
A8. Wall-clock comment updated to truthful bound:
`timeout_ms × (1 + ⌈n/c⌉)` instead of "roughly 2×timeout".
A9. logger.error call sites enriched with `errName` + first-5-line stack
via a new `logErrorWithStack` helper. Truncation matches Slack /
PocketBase render budgets.
A10. inputSchema rejections now log `parsed.error.issues` (structured
array) so nested zod errors are debuggable from the log stream
without re-running the probe.
A11. `errorClass` discriminator added to synthetic-error ProbeResults:
`"timeout" | "input-rejected" | "driver-error" | "discovery-error"
| "discovery-source-missing"`. Type exported as
`ProbeInvokerErrorClass` + `ProbeInvokerSyntheticSignal`. Alert
rules now have a stable taxonomy to branch on without parsing
`errorDesc` prose.
A12. `inputs[idx]!` non-null assertion replaced with explicit
`if (!item) break;` guard in the worker-pool loop.
A13. `writer` field removed from `ExecuteOneOpts` (was unused locally —
only consumed via `ctx.writer` which now reads `parentWriter`).
Call-site enumeration: `ExecuteOneOpts` is local to this file (no
external consumers); the synthetic-result type was previously inline so
no consumer was checking the old shape; `interpolateTemplate` has one
caller in this file.
The "Until PD1 lands the sort, this block is RED" paragraph predates
the merge of the invoker sort. The tests are green and have been since
PD1 landed. The surrounding context comment that describes what the
test actually catches (regressions where the sort is bypassed) stays.
Update the invoker's sort comment block to point to where demos
actually arrive on the input - the railway-services discovery source's
loadDemosMap, which reads registry.json once per enumerate() call.
Previously the comment was correct in shape but silent on the source
of input.demos, so a future maintainer reading the gate would not know
the sort depends on the discovery enrichment landing.
Also refresh demoCount's docstring to drop the misleading "zod-validated
by e2e-demos.ts:inputSchema" claim - the inputSchema runs inside
executeOne AFTER the sort has already used the count, so it cannot
gatekeep the sort path.
YAML: e2e-demos.yml's max_concurrency block previously said the sort
read from "the registry/discovery record's demos array, zod-validated
by the driver schema". Both halves were misleading: the registry vs.
discovery distinction mattered (now: registry.json joined onto the
discovery record at enumerate time), and the zod-validated claim was
wrong (zod runs in executeOne, which is post-sort). Rewritten to
describe the real plumbing.
The probe-invoker's shortest-service-first sort for e2e_demos reads
input.demos to compute demoCount(input). Until now, the railway-services
discovery source emitted records with no demos field, so demoCount was
always 0 and the sort degenerated to key-only ordering. The lazy
demosResolver in the e2e-demos driver runs INSIDE driver.run, AFTER the
invoker has dispatched, so it cannot feed the pre-dispatch sort.
Discovery now reads registry.json once per enumerate() call, parses
integrations[].slug -> demos[].id, derives the slug from the Railway
service name (showcase-<slug>), and attaches demos: string[] to every
emitted record. Read failures degrade gracefully: a structured warn on
discovery.railway-services.registry-read-failed and demos: [] for every
service. A missing/unreadable registry must not abort the tick - sibling
probes (smoke, image-drift) need their service list regardless.
Path resolution honours REGISTRY_JSON_PATH for tests/dev, falling back
to /app/data/registry.json (matches the e2e-demos driver's default and
the production Dockerfile mount).
Call-site enumeration of the RailwayServiceInfo interface change: the
new readonly demos: readonly string[] field is non-breaking. Existing
consumers (smoke, image-drift, aimock-wiring) read by named fields only
and ignore unknown keys, so adding demos has zero behavioural impact on
them. New consumer is the probe-invoker's e2e_demos sort, which now
reads a real count instead of always 0.
Tests: 4 new cases in railway-services.test.ts covering happy-path slug
join, missing-from-registry default, registry-read-failure warn path,
and REGISTRY_JSON_PATH override. All red before this commit, green
after. Full discovery suite: 42/42 passing.
Three Hono routes mounted at /api/probes:
- GET /api/probes : full schedule + execution state
- GET /api/probes/:id : probe + last 10 runs from probe_runs
- POST /api/probes/:id/trigger : bearer-auth gated, 5-min rate-limited
The list/detail handlers iterate scheduler.list() and synthesize the
EntryStatus + ProbeConfig + nextRunAt into the spec's JSON envelope,
including duck-typed tracker.snapshot() for inflight rendering. The
trigger handler enforces a per-probe 5-minute window held in-process
(sufficient for showcase-ops's single-instance deploy), passes the
optional filter.slugs through to scheduler.trigger() so when B7's
invoker honours it the wiring is already in place, and translates
InflightConflictError into 409 {error:"inflight"}.
Cache-Control: no-cache on every response per spec — the dashboard
client polls these endpoints and any proxy caching would mask
in-flight state transitions.
Auth construction is fail-loud at boot (MissingAuthTokenError when
neither triggerToken nor OPS_TRIGGER_TOKEN is set) so a misconfigured
deploy can't silently expose an unauthenticated trigger.
server.ts gains a single optional `probes` dep on ServerDeps; the
existing /health and /metrics paths are untouched. Older test setups
that only need /health continue to work without supplying scheduler
plumbing.
Closes B3 / blitz-status part 1 (API endpoints) + part 6 (auth).
Probe-invoker now registers a ProbeRunTracker on the scheduler entry for
the duration of each run, calls enqueue/start/complete/fail per service,
and clears the tracker on completion. Returns a RunSummary so the
scheduler populates lastRunSummary, and writes a row to the probe_runs
collection (start at run start, finish on completion). PB writer
failures are logged but never thrown — observability stays best-effort.
Also: scheduler now imports the real ProbeRunTracker class from
probes/run-tracker.ts (was a structural placeholder pre-B2) and exposes
setEntryTracker(id, tracker) so the invoker can mutate the slot's
tracker field through the public API instead of the read-only snapshot.
Provides the data layer for the dashboard's Status tab against the
showcase-ops HTTP contract being built in parallel (B3).
- lib/ops-api.ts: types + fetchProbes / fetchProbeDetail / triggerProbe.
baseUrl resolves explicit param → NEXT_PUBLIC_OPS_BASE_URL → /api/ops.
Trigger sends Bearer token from caller; URL-encodes ids; uniform
status-line errors on non-2xx.
- hooks/use-probes.ts: useProbes / useProbeDetail / useTriggerProbe.
10s default polling, AbortController cancels in-flight on unmount or
interval rollover, AbortError suppressed from user-visible state.
33 new tests (red→green), full shell-dashboard suite green (163/164,
1 pre-existing skip).
Implements the D6 reference-capture helper that drives D5 conversation
scripts against the LangGraph-Python (LGP) showcase to produce one
ParitySnapshot file per featureType under fixtures/d6-reference/.
The helper is dependency-injected end-to-end (browser launcher, SSE
interceptor, conversation runner, DOM serializer, snapshot writer) so
unit tests cover the orchestration without spinning up chromium or
touching the disk.
Per-turn SSE interceptor attach/stop lets multi-turn conversations
aggregate captures cleanly: tool calls concatenate in arrival order,
TTFT averages across non-empty-stream turns, and p50 chunk takes the
median-of-medians. Contract fields union across turns with first-seen
type winning on collision. JSON output is diff-stable: object keys
sorted, domElements sorted by (testId, tag, classes); toolCalls order
preserved (semantic).
Failure-mode contract is fail-closed: any failure (browser launch,
navigation, conversation failure_turn, DOM serialization, write)
returns failed without writing a partial snapshot. Browser is always
closed in finally.
Includes README documenting refresh cadence (weekly cron + manual
trigger) and the validation checklist for captured snapshots.
Adds a 5th "Status" tab to shell-dashboard that visualizes the probe
schedule (humanized cron, relative next/last run, duration, pass/fail
result), a "currently running" panel with progress bar and per-service
state grid, and a per-row Trigger control with "Run all" / "Run
specific..." menu.
Components live in src/components: status-tab, status-table,
status-running-panel, status-trigger-button. Local ProbeScheduleEntry
interface is a placeholder; B4b's lib/ops-api types replace it at
merge time. page.tsx wires the tab with an empty placeholder list and
no-op trigger; B4b's useProbes() hook lands in a follow-up.
19 new vitest cases cover humanized cron, relative-time tick, last-run
color tone (green/red/gray), trigger menu actions, and idle vs inflight
panel modes. Full dashboard suite green (150 tests, 1 skipped).
Per-invocation probe run history. One row per probe tick, distinct from
status/status_history (per-result state machine) — captures run-level
metadata (duration, triggered-vs-scheduled, pass/fail counts) for the
dashboard's "last N runs" widget consumed by B3 (status route) and B7
(probe-invoker hook).
PB migration creates the `probe_runs` collection with public-read /
superuser-write rules, mirroring `status` / `status_history`. Two
indexes: composite (probe_id, started_at DESC) for last-N lookups and
standalone (started_at DESC) for retention sweeps. Up/down migrations
are idempotent (no-op on re-apply, best-effort drop on rollback).
run-history.ts exports PROBE_RUNS_COLLECTION constant + ProbeRunWriter
with start/finish/recent. duration_ms is computed off the persisted
started_at (not a caller-supplied param) so the contract holds even if
clocks drift between start and finish. Filter values are JSON.stringify'd
to defend against probe ids containing PB filter metacharacters.
Tests use the same fakePb pattern as status-writer.test.ts; coverage
includes start state shape, finish state transitions (completed/failed),
duration computation off persisted row, recent() filter+sort+limit, and
empty-result handling.
Surfaces per-entry run metadata (started/finished/duration/summary), exposes
a typed getEntry snapshot, adds a manual trigger() that throws
InflightConflictError on overlap, and surfaces croner's nextRun() so the
forthcoming /api/probes routes can render schedule + last-run + next-run.
Handler return shape widened to RunSummary | void; existing void-returning
invokers keep working unchanged. ProbeRunTracker is a structural placeholder
local to scheduler.ts so this slot does not depend on B2's not-yet-existing
file.
Tests added (red-green): finished/duration populate after a scheduled run,
getEntry returns live state and undefined for unknown ids, trigger() runs
off-schedule and returns a runId, trigger() rejects when inflight,
nextRunAt returns a Date, triggeredRun toggles correctly, lastRunSummary
surfaces a handler-returned summary.
Hono middleware gating probe trigger endpoint. Reads OPS_TRIGGER_TOKEN at
construction (or accepts explicit expectedToken for tests). Fails loud if
no token configured — defaulting to "always allow" is the failure mode
this gate exists to prevent.
- 401 + JSON {error: "unauthorized"} on missing/malformed/wrong token
- Bearer scheme is case-insensitive per RFC 6750 §2.1
- Constant-time compare via crypto.timingSafeEqual; length-mismatch
returns false instead of throwing (raw timingSafeEqual leaks length)
- Empty tokens always rejected so a construction bug producing "" cannot
masquerade as a valid match
constantTimeEqual is exported for direct unit-test coverage.
Pure data class that tracks per-service progress during a probe
invocation. The probe-invoker registers services via enqueue/start
and finishes them via complete/fail; the HTTP API (GET /api/probes)
calls snapshot() to surface in-flight progress.
No I/O, no logging, no scheduler awareness. now() is injectable for
deterministic testing. Illegal transitions (e.g. complete on a
non-running service) are tolerated and overwrite state without
throwing — the invoker is the only writer, so out-of-order calls
are defensive cases, not real bugs.
Consumed at integration time by the probe-invoker (B7) and the
HTTP probes endpoint (B3).
Jordan Ritter