The orchestrator's inline `gql` in `createRailwayAdapter` diverged from
the discovery source's `makeGql` on two axes:
1. Non-JSON response bodies (HTML edge-proxy 5xx pages) surfaced as a
raw `SyntaxError` from `res.json()` instead of a schema-class
classification — operators reading the log stream couldn't tell
schema drift from a bad gateway.
2. Any non-empty `errors[]` threw even when `data` was present;
`makeGql` returns the partial payload and logs the errors. Two
callers running the same query saw different exception shapes
depending on whether they hit the inline gql or the discovery one.
Fix: export `makeGql` and `RAILWAY_GRAPHQL_ENDPOINT` from
`probes/discovery/railway-services.ts` and route `createRailwayAdapter`
through it, so error taxonomy (Auth / Backend / Schema / Transport
class hierarchy) is uniform across both consumers.
While here, drop the in-adapter `cachedServices` cache (D6). The
aimock-wiring probe always calls `listServices` first, and per-tick
refetch costs one round-trip but ensures renamed/added services
surface on the next cron tick without orchestrator restart.
`createRailwayAdapter` is also exported now and accepts an optional
`{ fetchImpl }` dep so unit tests can drive arbitrary HTTP behaviour.
R5-G4 D2/D3/D6.
Discovery returning zero records (everything filtered out, registry
empty, etc.) was silent — no signal to operators that the tick existed
but produced zero writes. Add a `probe.no-inputs` info log carrying
`probeId` + `kind` so log-aggregation can correlate "no signal"
against "discovery returned empty". Behaviour is unchanged — zero
inputs still mean zero writes — this is observability only.
Also drop the dead `if (!item) break;` defensive guard in the worker
loop. The loop precondition `cursor < inputs.length` guarantees
`inputs[idx]` is defined; the guard was a TypeScript-strictness
artifact, not a real invariant. Replace with `inputs[idx]!` plus a
comment documenting the loop-precondition invariant.
TDD red-green test: discovery source returns `[]`, asserts zero
writes plus exactly one structured `probe.no-inputs` info log.
Reorganises the cost-budget paragraph in config/probes/e2e-demos.yml
to cleanly distinguish:
- per-service average (~32 demos)
- the largest single service (~38 demos, wall-clock dominator)
- aggregate page loads per tick (~544)
and explicitly calls out the per-demo wall-clock bound enforced by
runDemo()'s single deadline across goto + fallback selectors. The
20-min outer cap and the worst-case ~19 min math now follow from
that bound directly, matching the driver's actual behaviour.
Driver throws can carry multi-MB Playwright stack traces or browser
console dumps. Without truncation those propagate untouched into PB
rows and Slack alerts, blowing past render budgets and making the
dashboard unreadable.
Add a 1200-char `SYNTHETIC_ERROR_MSG_BUDGET` (matching the budget the
e2e drivers themselves self-truncate to) and pipe the synthetic
errorDesc through `truncateUtf8` from `render/filters.js`. Same UTF-8-
safe truncation the renderer uses, so the synthetic path matches what
drivers produce.
TDD red-green test throws a 50KB error and asserts errorDesc <= 1200.
Adds 10 new tests in src/probes/drivers/e2e-demos.test.ts covering the
behaviors hardened in the driver:
- registry-read-failed at info on ENOENT (downgraded from warn)
- registry-parse-failed at warn on corrupt JSON
- registry-shape-invalid at warn when root is not a plain object
- non-abort error classified as driver-error even if abort races
- abort mid-selector-loop short-circuits without walking all 5 selectors
- empty-string route emits red config-invalid side row
- slow writer does not block abort fan-out (fire-and-forget side emit)
- writer-missing warn deduped to at-most-once per run
- per-demo wall-clock bounded to pageTimeoutMs across goto + selectors
- synthetic resolver-error side row when custom resolver throws
Also introduces mkSpyLogger() / mkCtxWithLogger() helpers and an
imported Logger type to keep the new structured-logging assertions
type-safe.
When the railway-services discovery source can't load registry.json (or
every record otherwise has zero demos), the existing sort still runs
its tie-break on `key` ascending — silently re-ordering discovery's
natural emission order without operator signal. Skip the sort entirely
in that case and emit a structured `probe.e2e-demos.sort-no-demos`
warn with the probe id, input count, and a hint pointing operators at
the most likely cause (registry.json missing/corrupt).
TDD red-green test exercises three records in non-alphabetic order
(zulu/alpha/mike) and asserts dispatch preserves enumeration order
plus exactly one structured warn fires.
Pre-fix `diffProbeSchedules` wrote `process.env.E2E_DEMOS_TIMEOUT_MS = String(cfg.timeout_ms)`
directly. Three problems with that approach:
1. Stale across YAML reloads — when a reload dropped timeout_ms, the
env var stayed set and leaked into every subsequent probe tick.
2. Last-write-wins silently across multiple e2e_demos configs.
3. Mutating shared global state from a diff function broke parallel
test isolation.
The fix: a pure `envForCfg(cfg, baseEnv)` overlay function. The
orchestrator now snapshots process.env once at the top of the diff
loop, and each probe gets a per-cfg overlay handed to its invoker.
process.env is never mutated.
R5-G4 D1.
Two related observability fixes:
A2: stamp `errName: "TimeoutError"` on the synthetic signal returned
by both timeout-path branches (setTimeout callback and the post-race
`if (timedOut)` branch). Operators triaging "timeout vs. driver-error"
no longer have to parse free-form errorDesc — the discriminator is on
the signal directly.
A8: tighten the object-shape gate in resolveInputs to reject `null`
explicitly (previous `record &&` truthiness check let null records
fall through silently). Else-branch now emits a single
`probe.discovery-record-non-object` warn for every non-object record
(array/null/number/string/boolean) with a `recordKind` discriminator.
Previously only arrays warned; null and primitives slipped through
silently.
TDD red-green tests for both: timeout signal carries
errName='TimeoutError'; non-object discovery records (null, number,
string, boolean) each emit a structured warn with the right kind.
Several CR R5 G3 fixes to drivers/e2e-demos.ts, all in the same file
because the changes touch overlapping regions:
C1: drop redundant path.resolve("/app/data/registry.json") wrap. No-op
for an absolute path; mirror railway-services.ts's plain string fallback
so the constant stays greppable across the two readers.
C3: replace empty catch {} in defaultDemosResolver with structured
read/parse/shape error logging. Mirrors loadDemosMap in
railway-services.ts: ENOENT → info (steady-state in dev), other read
errors → warn, JSON parse errors → warn, non-object root → warn. A
corrupt or missing registry now emits a distinct log key per failure
mode instead of silently flipping every e2e_demos cell green forever.
C4: document the per-run cache lifetime. The factory is invoked inside
run() so the cache survives within one tick (avoids 17 redundant reads)
but is discarded across ticks (so a registry update mid-day takes
effect on the next tick without driver restart). Hoisting to module
scope would make the registry stale forever.
C5: classify run errors via err.name === "AbortError" instead of
reading abortSignal.aborted after the catch settles. The prior approach
let a concurrent abort flip the flag mid-throw and misclassify a real
TypeError / TargetClosedError as "abort".
C6: log page/context teardown failures at debug instead of swallowing
silently. Stable log keys (page-close-failed, context-close-failed) so
operators can distinguish the two cases.
C7: check abortSignal.aborted at the top of every selector-loop
iteration so a cap that fires mid-loop bails promptly rather than
walking all 5 selectors at pageTimeoutMs each.
C8: treat empty-string route as a config mistake. Surface a red row
with errorClass="config-invalid" and errorDesc="route is empty
string" instead of falling into the informational-cell green path.
C9: void sideEmit on the abort branch instead of awaiting. A slow /
hung writer was defeating the prompt-shutdown contract for remaining
demos.
C10: dedup writer-missing warn via a per-run closure. N demos with no
writer wired used to produce N warns; now exactly one per run.
C11: bound per-demo wall-clock to pageTimeoutMs across goto + all
selector waits via a shared deadline. Worst case was 6*pageTimeoutMs
per demo; now strictly bounded.
C12: emit a synthetic e2e:<slug>/__resolver red row with
errorClass="resolver-error" when a custom resolver throws. Prior
behaviour fell through to demos: [] which masquerades as a green
aggregate; the synthetic row surfaces the configuration mistake on
the dashboard distinctly from an empty registry.
C2: align the abort-branch comment with the actual continue (not
break) semantics — every remaining demo gets a red side row.
A driver rejection BEFORE the timeout fires reaches both the detached
`.catch` observer and the outer `try/catch` which logs probe.run-failed.
Without this guard we double-log on every normal rejection. Add an
`if (!timedOut) return;` so only the true late-rejection path (timeout
won the race, driver rejected afterwards) emits the debug log.
Adds a TDD red-green test that asserts exactly one probe.run-failed log
and zero probe.driver-late-rejection logs on a normal driver throw.
The docstring for `selectorCascade` says "Race a list of selectors and
return the first one to resolve", but the implementation iterated
selectors sequentially with `await page.waitForSelector` — on a full
miss with 4 selectors at 3s each, total wait was 12s instead of 3s.
Convert to `Promise.any` over concurrent `page.waitForSelector` probes;
all selectors are dispatched in the same tick and whichever resolves
first wins. On full miss, throw with a consolidated reason listing
each tried selector's failure detail (preserves the existing label-
based error contract for callers).
Add a covering test that asserts the helper dispatches all
waitForSelector calls within ~50ms of each other (would fail at >50ms
spread under the previous sequential implementation, which would
serialize 50ms-rejecting probes). Plus a fast-vs-slow probe test
that asserts the FAST resolution wins regardless of cascade order
(the race semantic the docstring promises).
The existing `pickTimeSlot` and `approveOrDeny` callers continue to
work unchanged — `Promise.any` returns the SAME selector value the
sequential loop returned (the first to resolve), and the caller-side
selector composition (`${cardSelector} ${slotEntry}`) is unaffected.
The fenced ts block had `launchBrowser: /* see scripts/d6-capture-references.ts for the wiring */,`
which is invalid TS — a comment cannot be a property value. Replace the
broken example with a `yourLauncher` placeholder and a prose pointer to
the canonical wiring in scripts/d6-capture-references.ts.
Also drop the duplicate exit-code paragraph that lived under the
programmatic helper section — exit codes are a CLI concern only, and
the CLI section already states them on line 59.
R3-A.1: filter.slugs path now retains every input with a preError set so
discovery-time key_template failures still surface as synthetic-error
results when a `--slugs foo` manual trigger runs. Pre-fix the resolver's
synthetic preError keys (e.g. `<probeId>:invalid-key-template:N`) never
matched operator-supplied slugs and were silently dropped, hiding the
exact misconfigurations operators were investigating.
R3-A.2: invoker now reads `deps.schedulerId` (defaulting to `cfg.id` for
test fixtures) and uses it for `scheduler.getEntry` and
`setEntryTracker`. Orchestrator passes the prefixed `probe:<cfg.id>` it
registers under. Pre-fix the invoker passed the bare `cfg.id` while the
orchestrator registered with the `probe:` prefix, so `getEntry` returned
undefined and `setEntryTracker` was a silent no-op against the live
scheduler — `/api/probes` never surfaced inflight tracker data for
cron-tick runs and the cleanup setEntryTracker(null) was a no-op too.
R3-A.3: emit a structured `probe.run-row-orphan-risk` warn alongside the
existing `probe.run-writer-start-failed` error when runWriter.start
throws. The PB-side create may have succeeded before the response was
lost, leaving an orphan `running` row with no caller holding the id.
The warn is the observability surface operators grep for; the run
continues normally because probe_runs is best-effort observability.
Defense-in-depth pair to R3-A.4. Orchestrator now trims the token at
boot so the value handed to the bearer-auth middleware is the same one
the comparison runs against. The fail-loud empty-string guard already
handled the all-whitespace case at the env-var level; this hunk only
normalises non-empty padded values so a " abc " env var boots cleanly
and accepts "Bearer abc" without forcing operators to know about the
auth layer's internal trimming.
Replace `String(err)` in `logger.error` call sites across orchestrator.ts
with the shared `logErrorWithStack` helper from probe-invoker.ts. The
helper attaches `errName`, the message, a 5-line truncated stack at
error level, and emits the full stack in a parallel `<msg>.full-stack`
debug log. Pre-fix, error logs lost the stack entirely — operators
post-mortem-reading the orchestrator log saw only the message and had
no way to correlate errors back to the throwing call site.
Export `logErrorWithStack` from probe-invoker.ts so it can be reused
without introducing module-level duplication. Bus emit payloads and
warn-level logs that carry an `err` field keep `String(err)` (their
contracts and audience are different — bus payloads are stringified
for downstream consumers, warn logs are non-fatal).
Pre-fix the middleware trimmed the presented Bearer token but compared
against the verbatim env value. A copy-pasted OPS_TRIGGER_TOKEN with
surrounding whitespace booted clean but silently 401'd every well-formed
request. Trim BOTH sides at construction so the comparison is symmetric;
an all-whitespace expected token collapses to empty and trips the
existing fail-loud MissingAuthTokenError guard.
The per-service variables-loop catch used to rethrow on
`ctx.abortSignal?.aborted === true` OR a real AbortError from
fetch. The first half of that disjunction over-rethrew unrelated
mid-loop failures: once an external tick-timer fires the
controller, every subsequent iteration sees signal.aborted === true,
so a plain Railway 500 on the next service iteration also escaped
the catch and killed the whole tick — operators saw a discovery
abort with no useful breadcrumb on which service actually 500'd.
Narrow the rethrow to a real AbortError only. We walk up to two
`cause` links because gql() wraps fetch rejections in
DiscoverySourceTransportError; the underlying AbortError lives on
err.cause. Walking the chain catches that case without
re-tripping on unrelated transport errors that happened to fire
while the signal was aborted. AuthError rethrow remains unchanged.
While here, propagate B2 partial-data correlation: gql() now
returns { data, partialData } so downstream Zod-failure logs can
tag `cause: "partial-data"` when the same request already fired
the partial-errors warn. Operators reading the log stream now see
one correlated pair instead of two unrelated keys for the same
root cause. Project-level schema-failure throw includes
"(cause: partial-data)" in the message; the per-service
variables-schema warn carries cause: "partial-data" as a meta
field.
Regression test: 3-service project, abort fires between service-1
and service-2, service-2's variables call returns 500. Asserts the
500 records as env: {} + a single variables-failed warn instead
of escaping the loop, and that signal.aborted === true at the
end (confidence the abort actually fired).
The widened package-name regex in classifyShape() accepts
multi-segment names like `showcase-langgraph-python` without
warning — but it can't distinguish a legitimate multi-segment
package name from a 1-edit-distance typo of `showcase-starter-*`.
`showcase-strater-ag2` (a/r transposition) currently classifies
as package silently, so a typo-renamed Railway service flips
deployment archetype on the next tick without any audit signal.
Add a Levenshtein-distance-1 nudge (with adjacent-transposition
detection — `strater` is distance-2 by pure Levenshtein but
distance-1 with transpositions): when classifying as package, if
the first segment after `showcase-` is exactly one edit away from
`starter`, emit a structured warn
(`discovery.railway-services.classify-typo-suspected`) with the
service name and a suggested correction. Classification is NOT
changed — that would break the documented "anything-not-starter is
package" fallback. The warn is the audit trail; operators decide
whether to act.
Tests cover: (a) the typo case fires the warn with name +
suggested, (b) legitimate multi-segment package names do not
fire, (c) well-formed starter names do not fire.
ENOENT on registry.json is steady-state for non-demos consumers
(image-drift, smoke, aimock-wiring) — they don't mount the registry
volume — so in dev/test it correctly logs at info to avoid pulsing
the alert stream every tick. In production however a missing
registry is genuinely an operational concern: the volume mount may
have failed, or the image was built without the registry payload
copied in. Promote ENOENT back to warn only when
NODE_ENV === "production" so production operators see the missing
registry as a pageable signal while local/CI runs stay quiet.
Other read errors (EACCES, EIO, AbortError) already logged at warn
regardless of environment and continue to do so — those signal an
active fault, not steady-state.
Reorders the out.push({...}) properties to match the
RailwayServiceInfo interface declaration order (name, imageRef,
publicUrl, env, shape, deployedDigest, demos). The previous order
interleaved deployedDigest between imageRef and publicUrl, which
matched neither the interface nor any logical grouping. Pure
ergonomic refactor — no behaviour change. Reviewers can now grep
the interface and the emit site side by side without re-shuffling
fields.
R3-B.1: status-table lastRunTone() returned green for partial-pass runs
(failed=0, passed<total), reintroducing the misread that R2-D.4 fixed in
status-runs-list. Schedule table is the most-viewed surface, so this is
the highest-impact regression. Mirror runs-list semantics: amber tone +
"P/T (S skipped)" result text. Also explicit "P/T pass (F fail)" for
failed runs so the column reads consistently with the tone.
R3-B.2: StatusDetailPanel durations filter used \`typeof d === "number"\`
which admits NaN/Infinity (typeof NaN === "number"). The sparkline then
computed Math.min(...durations) = NaN and emitted "NaN,NaN" coords in
the polyline. Switch to Number.isFinite() so non-finite durations are
dropped.
Tests added (red-green TDD):
- status-table: amber tone + "skipped" text for partial pass; explicit
"pass (N fail)" text for failed; "N/N pass" for fully-passed
- status-detail-panel: NaN/Infinity durations don't appear in sparkline
Adds explicit unit coverage for two paths the suite was previously
silent on:
B5 — deployedDigest extraction. The fixture builder used to discard
latestDeployment entirely, leaving the production code path that
pulls imageDigest from latestDeployment.meta unit-test uncovered.
The fixture now optionally accepts a latestDeployment block; new
tests assert: happy path string digest flows through; null
latestDeployment yields ''; missing meta yields ''; non-string
imageDigest is rejected defensively.
B6 — resolveShape mismatch-throw path. The classifier-vs-explicit
disagreement throw is the load-bearing assertion preventing silent
shape drift at driver boundaries; previously it had zero tests.
Two new tests cover both directions (starter classifier + package
input, package classifier + starter input) and pin the error
message to include both values for operator triage.
R3-C.1: useTriggerProbe.trigger now returns Promise<TriggerResponse | null>.
The supersession path resolves to `null` (typed) instead of the previous
`undefined as unknown as TriggerResponse` cast, which lied about the
return shape and would crash any caller that read fields off the result.
R3-C.2: useProbeDetail clears `error` on id change so the detail panel no
longer renders a stale error message under a freshly-selected probe header.
R3-C.3: useProbes clears `error` on dep change (e.g. baseUrl swap) so a
prior dep tuple's failure does not persist under the new one.
R3-C.4: useTriggerProbe clears `error` at the start of each trigger call
so a successful retry leaves no sticky error from the prior failure.
Bonus: parseJson now attaches the original parse error as `cause` on the
wrapper so debuggers can walk back to the SyntaxError name/stack without
re-parsing the wrapped message.
Tests: red-green tests added for each behavior; full file suite 49/49.
R3-D.2: the toggle-close branch on the Trigger button forgot to reset
pickerOpen and selected, while outside-click and Escape both did. So
toggling the menu closed and reopening it showed ghost checks from a
prior session. Extract a closeMenu() helper used by all three close
paths so the invariant is structural, not duplicated.
Bonus: add aliveRef guard so async rejections from onTrigger that
resolve after unmount don't trigger setState-on-unmounted warnings.
R3-D.1: fetchProbes and fetchProbeDetail are polled every ~10s for live
status data. Without an explicit cache directive the browser and Next.js
App Router fetch wrapper may serve stale responses, defeating the poll.
Add cache: 'no-store' to both GET call sites and cover with tests.
E1: railway-services floor scan now matches both `.yml` and `.yaml`
extensions so a future probe authored as `<name>.yaml` cannot silently
bypass the floor invariant.
E2: every entry in `RAILWAY_PROBES_EXEMPT_FROM_EXCLUDES_FLOOR` must
either carry a `namePrefix` strictly narrower than the broad
`showcase-` floor, or its rationale string must include the grep-able
phrase `all services`. Today qa.yml passes via the narrow-prefix path
(`showcase-langgraph-python`); image-drift.yml passes via the
rationale-phrase path. If qa.yml is later widened to `showcase-`
without dropping the exemption (or restating an explicit
all-services rationale), this guard fires instead of letting infra
services flap red on a stale exemption.
E3: shorten the duplicated cwd-resolution rationale at the top of the
file; the load-bearing explanation now lives once on `configsDir()`.
The watch-callback test for probe-loader was flaking because chokidar's
native FSEvents backend on macOS does not reliably emit events for files
written into a freshly mkdtemp'd directory under Node 22+: the parent path
isn't pre-watched by the kernel, so the first add silently drops.
Add a test-only watcherOptionsOverride field on ProbeLoaderDeps so tests
can opt into chokidar's polling backend (the documented chokidar workaround
for this exact case), and switch the flaky test to use it. Also replace the
fixed setTimeout(500ms) wait with vi.waitFor polling so the assertion no
longer races chokidar's debounce on slow CI.
Production callers leave watcherOptionsOverride undefined, so prod still
gets FSEvents/inotify with no perf change.
Verified stable across 7 consecutive runs of the watch test.
The aimock-fixture-coverage probe scans byoc-json-render.spec.ts and was
flagging two prompts without matching feature-parity.json fixtures:
- "Break down revenue by category as a pie chart"
- "Show me monthly expenses as a bar chart"
Existing pie/bar fixtures used substrings ("revenue distribution by
category", "expenses by category") that don't appear in these prompts,
so the substring matcher returned no hits.
Added two new fixtures with substrings that DO appear in the prompts
("revenue by category as a pie chart", "monthly expenses as a bar
chart") so the coverage probe stays green and aimock returns
deterministic chart data when the BYOC json-render specs run against
the sidecar.
The degraded-state tooltip read "stale — last pass @ <observed_at>", but
observed_at is the most recent producer tick (when the row was last
*seen* in any state) — for a degraded row that is when degradation was
last observed, NOT when it last went green. Operators reading "last
pass @ ..." would mis-triage by treating that timestamp as the last
green tick.
Change copy to "stale — last seen @ <observed_at>" and add a covering
test asserting the new wording (and the absence of "last pass").
R11: The 'How to invoke' example referenced a nonexistent
defaultLaunchBrowser symbol. Reorganized to lead with the CLI wrapper
(scripts/d6-capture-references.ts) which already wires the default
deps; the programmatic example below is annotated as test/internal-
only with a comment pointing at the script for the canonical wiring.
R7 (e2e-parity half): Same per-turn slicing pattern as reference-
capture — runFeatureCapture's inner runConversation always reports
failure_turn === 1. Translated to outer 1-based index in the
'conversation-error' errorDesc so dashboards show which turn actually
failed. Added covering test asserting turn 3 (not turn 1) is reported
when the third turn fails.
R8: createDefaultFleetResolver cached successful reads forever, so
operators editing registry.json needed a process restart for D6
rotation to see the new fleet membership. Added mtime-based
invalidation plus a 60s wall-clock TTL safety rail. Resolver factory
now accepts injected now/stat for deterministic testing. Added two
covering tests: mtime-change invalidation and TTL elapsed even when
mtime is unchanged.
R10: When selectedThisTick === false (rotation skips a slug), the
driver returned aggregate green but emitted no per-feature side rows,
leaving stale red on the dashboard for the entire rotation cycle (up
to a week). Now emits a green per-feature side row with a 'not
selected this tick' note for every requested-known feature so cells
refresh while clearly signalling we didn't actually verify parity.
Updated the existing rotation-skip test to assert one side row per
requested feature and tightened slug expectations.
R9: pickToolCallName checked the bare 'name' field before the
structured 'tool_call.name' / 'tool_use.name' fields. Some transports
include a top-level 'name' that's a step/event label alongside a
structured tool descriptor — that ordering harvested the label and
poisoned the parity-compare engine's tool-call list.
Reordered to: toolCallName -> tool_call.name -> tool_use.name -> name.
Added 3 covering tests asserting structured fields win over a top-level
step-label and the canonical toolCallName still wins over everything.
R7 (reference-capture half): The capture helper runs each turn as
runConversation(page, [turn]) so the SSE interceptor only sees that
turn's request. The inner runConversation always reports
failure_turn === 1 (its slice-local index), which would mislabel
every per-turn capture failure as 'turn 1' regardless of which
outer turn actually failed. Translated to outer 1-based turn index
when constructing the failure reason. Updated the existing covering
test to assert turn 2 (not turn 1) when the second turn fails.
R6: Four sibling files duplicated the assistant-message selector
cascade with subtle drift; some used a bare '[role="article"]'
fallback that picked up user-input bubbles into the transcript and
caused turn-1 user-message substrings to spuriously match the user's
own input.
Hoisted ASSISTANT_MESSAGE_PRIMARY_SELECTOR (canonical testid) and
ASSISTANT_MESSAGE_FALLBACK_SELECTOR ([role="article"] excluding
data-message-role="user") to conversation-runner.ts. Updated
_hitl-shared, _gen-ui-shared, d5-agentic-chat, and d5-shared-state to
import the shared constants and apply them via a string-templated
new Function so closures don't have to cross the page.evaluate
boundary while DOM-lib remains excluded from the helper module.
R5: TIME_PICKER_SLOT_SELECTORS entries themselves embedded the
'[data-testid="time-picker-card"] ...' prefix, and the helper then
composed cardSelector + entry — producing nonsense like
'[data-testid="time-picker-card"] [data-testid="time-picker-card"] button >> nth=0'
that never matched. Simplified the cascade to its bare slot variants
('[data-testid="time-picker-slot"]', 'button >> nth=0') and rely on
the helper's prefixing for card anchoring.
Added _hitl-shared.test.ts with 3 covering tests: no doubled card
prefix anywhere in probed selectors, click composes card + slot, and
fallthrough cascade still anchors under the resolved card selector.
R4: The CLI script's package-root resolution was inlined and not unit
tested. Hoisted to src/probes/helpers/package-root.ts with dependency-
injected exists predicate so it picks up vitest's src/**/*.test.ts
glob. Added 5 covering tests (source-tree, dist-tree, startDir match,
missing throws, deepest stop).
R1: classesEqual was a strict length-equality check that rejected
captured-superset cases. Replaced with capturedClassesIncludeReference
(multiset superset) so a captured DOM with extra classes still passes
while a missing reference class fails the comparison.
R2-R3: When captured.total_chunks === 0 but reference.total_chunks > 0
the timing math degenerates and the stream axis silently passed. Added
an early-return guard with a clear failure reason so the parity verdict
reflects reality.
R2-B.1 — diffCronSchedules now awaits scheduler.unregister and emits
orchestrator.cron-unregister-failed on rejection; orphan rule stays
visible in scheduler.list() (mirrors CR-A2.2 probe-path fix). Watch
callback uses .then/.catch to surface reload failures via the bus.
R2-B.2 — boot wraps serve() in try/catch; on throw, awaits
scheduler.stop() and resets schedulerRunning before rethrowing, so a
late bind failure (e.g. EADDRINUSE) cannot leave cron jobs ticking
without an owner.
Tests: 2 new red-green specs added under orchestrator.test.ts
covering both rejection paths (vi.spyOn(logger.error) + vi.doMock for
the @hono/node-server serve() throw).
R2-B.3: distinguish unset from set-but-empty. An operator who mistypes
`OPS_TRIGGER_TOKEN=` ships an empty value AND sees the silent-skip
log instead of a clear error. Trim and reject whitespace-only too.
- unset → boot succeeds, info log (intentional disable)
- set-empty → boot throws (misconfig — refuse to mount probes router)
- set-value → boot succeeds, router mounted
Adds 4 covering tests under "OPS_TRIGGER_TOKEN empty-string handling".
Jordan Ritter