CB-1 (Slot 2 #22): convert B2 deploy-webhook tests to vi.stubEnv with
`vi.unstubAllEnvs()` in afterEach instead of manual process.env mutation
+ try/finally restoration. Test runner now guarantees restoration and the
test bodies are shorter / less error-prone (CR Slot 2 #22).
CB-2 (Slot 2 #28): when `loadWebhookSecrets`' escape hatch fires with a
real-looking NODE_ENV (anything except "test"), log at `warn` instead of
`info` so a production typo (NODE_ENV=staging + HARNESS_ALLOW_NO_SECRET=1)
is visible in dashboards / log alerting. Pure local-dev (NODE_ENV=test)
stays at info level so a normal unit-test boot doesn't spam warnings.
CB-3 (Slot 4 #17): clarify `loadWebhookSecrets`' docstring + bypass-log
message — "set" is ambiguous (empty string would qualify pre-fix); use
"non-empty" to match the actual predicate.
R1-F2 (bucket b, defensive ordering): hoist fail-loud config validation
to the TOP of boot() and runControlPlane() — BEFORE any pb client, queue,
bus, scheduler, writer, S3 uploader, fleet-health, or aggregator
allocations. Pre-fix `loadWebhookSecrets` lived AFTER the entire
scheduler+writer (and CP queue+aggregator) setup, so a misconfigured boot
allocated expensive resources and mounted file watchers before throwing.
R1-F3 (bucket b, predicate symmetry): broaden the POCKETBASE_URL fail-loud
predicate to match SHARED_SECRET semantics. Pre-fix the POCKETBASE_URL
guard only fired on `NODE_ENV === "production"`, while `loadWebhookSecrets`
fired unless NODE_ENV=test or an explicit escape hatch — so staging /
unset / "development" deploys silently bound to http://localhost:8090.
Extracted `loadPocketbaseUrl(logger)` with the same test-or-escape-hatch
predicate as `loadWebhookSecrets`. A new `HARNESS_ALLOW_NO_PB_URL=1` env
flag mirrors `HARNESS_ALLOW_NO_SECRET=1` for local dev. Worker
`boot()` and the CP's `resolveFleetPbConfig` both call the helper.
Tests:
- HF13-A2 production-only assertion broadened (SHARED_SECRET set so the
test specifically exercises the POCKETBASE_URL guard).
- New R1-F3 tests cover NODE_ENV=development without POCKETBASE_URL
(throws) and the HARNESS_ALLOW_NO_PB_URL=1 escape hatch (succeeds).
R1-F1 (bucket a, load-bearing): the CP boot path now subscribes to
`deploy.result` events through its status writer. Pre-fix, B2 mounted POST
/webhooks/deploy on the CP role but only the worker boot path had a
`bus.on("deploy.result", ...)` listener — signed POSTs to the CP host
returned 202, the bus event fired with no subscriber, and the deploy-overall
dashboard row never landed.
Extracted the handler into a shared `subscribeDeployResults(bus, writer)`
helper exported from orchestrator.ts so both boot paths share the IDENTICAL
logic. Worker boot keeps its inline pattern (busUnsubs.push) so teardown is
unchanged; CP keeps the subscription alive for the lifetime of the bus.
Red-green tests in src/orchestrator.test.ts under "R1-F1" assert that
emitting deploy.result on the returned handle's bus drives one write keyed
"deploy:overall" through a stubbed status writer — once for runControlPlane
(the new path) and once for boot() (regression guard for the helper
extraction).
## Summary
PR #5453 renamed stale catchall userMessage `"check Tokyo weather
forecast"` → `"forecast for Tokyo"` to match the D5 probe input. But
some integrations' catchall fixtures used **pill-aligned userMessages**
(e.g. LG-TS, LG-FastAPI used `"Chain a few tools in this single turn"`,
`"What's the weather in San Francisco?"`) — they had no `"forecast for
Tokyo"` matcher to rename. Result: D5 catchall probe still misses → live
LLM fallback → cells stay red on staging.
## Fix
Add the canonical emit+narrate fixture pairs for `"forecast for Tokyo"`
(default + custom catchall) and `"What's the current price of AAPL?"`
(custom catchall only) — adapting mastra's working template — to each
integration that was missing them.
## Scope
4 integrations × 1-2 files each (6 files total):
- LG-TS, LG-FastAPI: both default + custom catchall
- google-adk, ms-agent-dotnet: custom catchall only (their default is
already green)
Existing pill-aligned fixtures preserved (pure prepend at start of
array). First-match-wins means new fixtures match the D5 probe inputs
("forecast for Tokyo", "What's the current price of AAPL?") without
colliding with existing pill prompts. `toolCallId` values are unique per
integration to avoid cross-integration shadowing.
Note: the originally-flagged langroid, llamaindex, agno,
claude-sdk-python, and strands files already have `forecast for Tokyo` +
AAPL matchers in their catchall fixtures (likely from a prior pass) —
they did not need changes and are not in this PR. If those staging cells
are still red, the root cause is elsewhere (control-plane staleness,
deploy lag, or different probe variant).
## Verification
Post-merge, fleet-cp's e2e-deep probe will re-run within ≤6h staleness.
Cells should flip d6:<slug>/tool-rendering-{default,custom}-catchall =
green.
The catchall userMessage rename (#5453) only renamed STALE strings to 'forecast for Tokyo'. Integrations whose catchall fixtures used PILL-ALIGNED userMessages ('What's the weather in San Francisco?', 'Find flights from SFO to JFK.', 'Chain a few tools in this single turn', etc.) had no 'forecast for Tokyo' matcher to begin with — so the D5 catchall probe still fell through to live LLM and the cells stayed red on staging.
This PR ADDS the canonical 'forecast for Tokyo' (default+custom catchall) and 'What's the current price of AAPL?' (custom catchall only) emit+narrate fixture pairs to each integration that was missing them. Existing pill-aligned fixtures are preserved (pure prepend at the start of the fixtures array; first-match-wins means new fixtures match the D5 probe inputs without colliding with existing pill prompts).
Integrations touched:
- langgraph-typescript: both default + custom
- langgraph-fastapi: both default + custom
- google-adk: custom only (default already green)
- ms-agent-dotnet: custom only (default already green)
After merge, fleet-cp's e2e-deep probe will re-run within ≤6h staleness window and flip cells GREEN.
The 'Showcase: Verify Deploy' workflow's notify-harness step has been
POSTing to /webhooks/deploy and getting 404 on every main deploy since
at least 2026-06-12 (5+ consecutive failures). Root cause:
1. POST /webhooks/deploy is only registered when
webhookSecrets.length > 0 (gate at src/http/server.ts:119).
2. The CP buildServer call in runControlPlane omitted BOTH
webhookSecrets and metrics — so the public Railway host running
the CP role never mounted the route. The worker boot path had
them, but it isn't the host receiving notify-harness POSTs.
3. The FATAL-CONFIG guard on missing SHARED_SECRET was gated on
NODE_ENV === 'production' — any deploy with NODE_ENV unset or set
to something other than the literal 'production' silently shipped
with webhookSecrets=[] and no fatal error.
Part 1 — wire webhookSecrets + metrics into the CP buildServer call.
Lifted the env→secrets loader into a new exported helper
'loadWebhookSecrets()' (~orchestrator.ts:175-209) so BOTH boot paths
consume the same predicate. The worker 'boot()' call (~line 836) and
the CP 'runControlPlane' call (~line 2421) both invoke it, and the CP
buildServer call (~line 3047) now passes webhookSecrets+metrics
alongside the existing bus/probes/fleetRuns wiring.
Part 2 — tighten the FATAL-CONFIG predicate.
The new predicate throws unless EITHER a secret is set OR NODE_ENV ===
'test' OR HARNESS_ALLOW_NO_SECRET === '1' (narrow escape hatch for
local dev). The thrown error message names the env-vars, the gate
location ('src/http/server.ts:119'), the silent-404 symptom, and the
escape hatches, so an operator reading the deploy log can recover
without spelunking.
Updated the existing R5-G4 D5 regex (orchestrator.test.ts:1592) since
the error message changed; covered by the new B2 Tests A/C/D below.
Tests: 4 new red-green tests in 'B2: /webhooks/deploy registered on CP
+ fail-loud on missing SHARED_SECRET':
- Test A (CP /webhooks/deploy registration): RED 404 → GREEN 401.
- Test B (worker /webhooks/deploy regression): GREEN 401.
- Test C (FATAL-CONFIG when NODE_ENV=development): RED resolves →
GREEN rejects with /FATAL-CONFIG.*SHARED_SECRET/.
- Test D (NODE_ENV=test escape hatch): GREEN, no throw.
Full harness suite: 2719/2719 green. typecheck: clean.
Requires a CP-role redeploy on the public Railway host for the new
route registration to take effect.
Adds four red-green tests for the notify-harness 404 follow-up:
A. CP boot registers POST /webhooks/deploy when SHARED_SECRET is set
— RED pre-fix (404, route not mounted on CP); GREEN post-fix (401,
the route's own HMAC reject path fires on an unsigned POST).
B. Worker boot still registers POST /webhooks/deploy (regression
guard) — already green; pins the unchanged behavior.
C. boot throws FATAL-CONFIG when SHARED_SECRET unset AND NODE_ENV is
non-test — RED pre-fix (current guard fires only on
NODE_ENV='production'); GREEN post-fix once the predicate is
tightened.
D. boot succeeds when SHARED_SECRET unset AND NODE_ENV='test'
(test-mode escape hatch) — already green; pins the escape hatch.
Red capture (pre-fix run):
- Test A: AssertionError: expected 404 to be 401
- Test C: AssertionError: promise resolved instead of rejecting
Bumps the dojo e2e matrix from `depot-ubuntu-24.04` (2 vCPU) to
`depot-ubuntu-24.04-4` (4 vCPU) and `NX_PARALLEL: 4` so the build uses
the extra cores. This is the non-serializing way to cut dojo wall-clock
(the build-once dedup tried in #5450 regressed wall-clock and was
reverted).
## Result: −40% wall-clock (measured on CI)
Dojo wall-clock = the single slowest suite (the 15 run in parallel).
Comparison vs the 2-vCPU baseline:
| metric | 2-vCPU baseline | 4-vCPU | Δ |
|---|---|---|---|
| **wall-clock** (long pole `langgraph-python`) | 623s (10.4m) | **373s
(6.2m)** | **−40%** |
| runner-minutes (wall summed, 15 suites) | 105m | 76m | −28% |
| **billed compute** (vCPU-min; 4-vCPU ≈ 2× rate) | ~210 | ~304 |
**+45%** |
Every suite got faster; the long-pole suites benefited most:
| suite | 2-vCPU | 4-vCPU |
|---|---|---|
| langgraph-python | 623s | 373s |
| langgraph-typescript | 547s | 362s |
| langgraph-fastapi | 500s | 337s |
| adk-middleware | 414s | 286s |
| (… all 15 faster …) | | |
Long-pole `langgraph-python` step breakdown:
| phase | 2-vCPU | 4-vCPU |
|---|---|---|
| Build cpk | 82s | 48s |
| Prep dojo | 94s | 52s |
| **Run tests (Playwright)** | **271s** | **117s** |
| total | 623s | 373s |
**Key finding:** the Playwright phase more than halved → the e2e suites
are **CPU/worker-bound, not LLM-latency-bound**. A bigger runner is the
right lever; test sharding is not needed to reach ~6 min.
## Trade-off
−40% wall-clock for **~+45% billed compute** (4-vCPU costs ~2×/min,
partly offset by finishing 28% sooner). If the cost bump isn't worth it
across all 15 suites, a follow-up can scope `-4` to just the slow suites
via a per-matrix `runner` field (wall ~6.5m, smaller cost increase).
Companion to #5450 (unit-test `nx affected`).
useCreditCards() kept per-instance React state and was called independently by
the dashboard page and by copilot-context (where the chat's approve / finalize /
open-exception tools live). A mutation made through one instance refetched only
itself — so when the agent approved an over-limit charge in chat, the dashboard
pending table kept showing it as pending until a manual reload.
Add a module-level revalidation bus: each useCreditCards() instance registers a
refetch callback, and every mutation calls notifyDataChanged() to fan a refetch
out to all live instances. The dashboard now reflects agent-driven approvals
immediately (verified: recall-approve in chat drops the charge from the pending
table with no reload).
Adds a "What each mode actually recalls" subsection to the demo README: in OSS
mode the taught workflow is recalled only within the same conversation (the
saved procedure is echoed back into that thread), so a brand-new chat won't know
it — that's expected. Cross-conversation persistence is what the external
Intelligence backend provides. Names the symptom explicitly so reviewers aren't
surprised when a new conversation "doesn't know" the workflow in OSS mode.
Rework the dashboard's Pending approval view and fix two approval-gate UX bugs
on the banking demo (PR #5266):
- Table layout: replace the center-stacked per-row card with a scannable table
(Merchant / Amount / Policy / Actions). Actions are check / x icon buttons plus
a "more actions" overflow menu holding File policy exception. Status is its own
column (Over limit / Cleared / Within limit), and Approve is gated until the
charge is actually clearable.
- Fix the table shrinking when the more-actions menu opens: the Radix menu is
modal by default and engaged react-remove-scroll, whose scrollbar compensation
reflowed the table. Set modal={false} (a row menu needn't be modal) and add
whitespace-nowrap to the status badges.
- Fix "cannot approve after filing an exception": the inline card offered all
codes, including non-justifying ones that set activeExceptionId (flipping the
row to Cleared) but never lift the server gate, so the approve 422'd silently.
The card exists to clear an over-limit charge, so it now offers only justifying
codes. The gate itself is unchanged.
Verified live in OSS dev: file (justifying) -> Cleared -> approve succeeds; the
menu opens without reflow; lint + build green.
## Summary
- `showcase/docker-compose.local.yml:262` hardcodes
`LOCAL_SERVICES_JSON` to `showcase-langgraph-python` (intentional N=1
demo default).
- `showcase/bin/showcase test <slug> --d6 --isolate` was inheriting that
value verbatim into the iso1 stack, so the iso1 control-plane discovered
`showcase-langgraph-python` instead of `showcase-<requested-slug>`.
- Result: iso1 probes targeted the wrong service. Visible in iso1
harness logs as `discovery.railway-services.local-injection count:1
names:["showcase-langgraph-python"]` regardless of CLI arg.
- This PR teaches the iso1 compose generator (`apply_isolation` in
`showcase/scripts/cli/_common.sh`) to inject a per-slug
`LOCAL_SERVICES_JSON` override built from the slug's manifest.yaml demos
list. Fallback to `["agentic-chat"]` if manifest absent.
## Scope
Two files, ~55 LOC added: `showcase/scripts/cli/cmd-test.sh` (pass slug
arg), `showcase/scripts/cli/_common.sh` (inject regex sub in python
rewriter). Bash + embedded python only; no TypeScript changed.
## Verification
- **Local `--isolate` discovery confirmed correct:**
`showcase/bin/showcase test ms-agent-python --d6 --isolate` — iso1
harness log now shows `discovery.railway-services.local-injection
names:["showcase-ms-agent-python"]` (was `showcase-langgraph-python`
before this PR).
- Persistent stack default behavior (langgraph-python N=1) preserved via
fallback when no slug arg provided.
- **Heredoc hardening scope:** commit edc77f809 moves `$slug` from
bash-interpolation into the python rewriter to an env var. This is a
slug-only carve-out — `$slug` is the only value that originates from the
CLI arg path. The other bash-interpolated `$VAR`s embedded in the
heredoc (`$PORTS_FILE`, `$COMPOSE_FILE`, `$name`, `$SHOWCASE_ROOT`,
`$ISOLATE_PORT_OFFSET`) remain script-internal: each is constructed
inside `_common.sh` from validated sources (manifest reads, computed
offsets, fixed roots), not from user input, and CR Round 1 + Round 2
slot 5 both verified they are not user-tainted. A broader
env-var-pass-all-vars refactor would be a separate concern and is out of
scope for this PR.
## Out of scope
- Worker heartbeat clock-skew issue (`fleet.health.worker-unhealthy
lastHeartbeatAt N min stale`) is a separate bug; not touched.
- `buildLocalServicesJson` in `cli/control-plane-run.ts` has a similar
comment-vs-code disagreement (JSDoc says "filter", code returns env
verbatim) — flagged but not auto-fixed, as iso1 override is the correct
insertion point.
- Generalized env-var-pass for all heredoc-embedded $VARs (see
Verification) — separate refactor concern.
## Summary
- D5 e2e-deep probes for `tool-rendering-{default,custom}-catchall` send
`"forecast for Tokyo"` as the test input (see
`showcase/harness/src/probes/scripts/d5-tool-rendering-{default,custom}-catchall.ts`).
- aimock uses substring match on `userMessage`.
- The catchall fixtures on main had a stale `"check Tokyo weather
forecast"` string that couldn't substring-match the probe input →
fixture miss → probe falls through to live LLM → CV ✗ red D4 on the
dashboard.
- This PR renames the userMessage to the canonical `"forecast for
Tokyo"` across 28 catchall fixture files in 16 integrations.
## Scope
28 files × ~2 userMessage occurrences each = 67 line changes. **No code,
no agent, no page.tsx changes.** Pure fixture-data alignment.
Integrations covered (default-catchall and/or custom-catchall): ag2,
agno, built-in-agent, claude-sdk-python, claude-sdk-typescript,
crewai-crews, google-adk, langgraph-python, langroid, llamaindex,
mastra, ms-agent-dotnet, ms-agent-python, pydantic-ai, spring-ai,
strands.
## Commit history note
Commit b1f19bdc8 changes the rename target from the original `"weather
in Tokyo"` (commit 388c69e68) to `"forecast for Tokyo"`. This was a CR
Round 1 catch: `"weather in Tokyo"` was a substring of the chain pill
prompt (`"weather forecast chain in Tokyo"` / similar), which would have
caused the catchall fixture to incorrectly match chain-pill traffic.
`"forecast for Tokyo"` has no such substring collision with any other
probe input.
## Verification
- **Live red-green proof** was performed on the prior `"weather in
Tokyo"` rename (commit 388c69e68) against `ms-agent-python` via
`showcase/bin/showcase test ms-agent-python --d6 --isolate`:
- Before (origin/main): catchall featureTypes red (fixture miss → live
LLM → flaky)
- After: `d6:ms-agent-python/tool-rendering-default-catchall=green`,
`d6:ms-agent-python/tool-rendering-custom-catchall=green`
- **The current HEAD's `"forecast for Tokyo"` rename (b1f19bdc8) has NOT
been re-run live.** It is verified by static analysis only: substring
math (no collision with any known D5 probe input or chain pill prompt)
and a clean CR Round 2 across all reviewing agents.
- Post-merge dashboard re-probe will be the final runtime verification.
## Out of scope (separate follow-ups)
- LG-TS / LG-FastAPI catchall fixtures don't have the stale string — use
pill-aligned userMessages; need different fix
- `tool-rendering` (non-catchall) and `tool-rendering-reasoning-chain`
featureTypes have separate failure modes
- Other red cells in the dashboard (frontend-tools-cosmic timeout,
agent-config, auth, etc) are unrelated
The python rewriter in apply_isolation previously interpolated $slug
directly into the inline python source via bash. A slug containing a
single quote would break the python literal. Internal-tool risk only
(slug is developer-typed), but cheap to harden.
Pass slug via SHOWCASE_ISO_SLUG env var and read os.environ.get(...)
inside the python heredoc. Defense-in-depth; no behavior change for
valid slugs.
The previous 'weather in Tokyo' rename (388c69e68) was a substring of the
main tool-rendering chain pill prompt 'Chain a few tools in this single
turn: get the weather in Tokyo, search flights from SFO to Tokyo, and roll
a d20.' Because aimock loads fixtures alphabetically per integration dir
and uses substring match with first-match-wins, the catchall fixture
(loaded before tool-rendering.json) was intercepting chain pill matches
across 16 integrations.
Rename catchall fixture userMessage to 'forecast for Tokyo' — a phrase
not contained in any other pill prompt. Update the corresponding D5
catchall probe inputs in d5-tool-rendering-{default,custom}-catchall.ts
and the test assertions that pin those inputs.
Call-Site Enumeration: 'weather in Tokyo' remains intentionally in
page.tsx suggestions.ts pills (user-visible UX) and inside the chain
pill prompt itself — neither is in the substring-match path now.
The persistent stack's docker-compose.local.yml hardcodes LOCAL_SERVICES_JSON
to the langgraph-python sample for fast N=1 local demos. When --isolate
spawns an iso1 stack with a different slug (e.g. ms-agent-python), the
iso1 harness container inherited that hardcoded value, causing
discovery.railway-services.local-injection to enumerate the wrong service
(showcase-langgraph-python instead of showcase-<requested-slug>). The iso1
probe then targeted the wrong container, broke red-green verification, and
left D5 cells unwritten.
Inject a per-slug LOCAL_SERVICES_JSON override into the iso1 compose
generator so iso1 always probes the slug passed via --isolate.
The recorder was a floating HUD overlay, which read as a different UI element
from the in-chat cards (e.g. "Save this workflow?"). Fold the live step feed
into the demonstration card itself: the `awaitDashboardDemonstration` card is
now titled "Recording your workflow" (with a REC pulse), keeps the
non-directional copy, and embeds a reactive `<RecordingSteps/>` child that
narrates each captured action inside the card. Removes the floating panel and
its CSS. Same card chrome as the other cards, so it reads consistently in the
conversation.
`RecordingSteps` subscribes to the recording context itself (not via the host
card's render closure), so it updates live as each step is logged without a
stale-closure dep.
Address review feedback on the banking self-learning "teach a workflow" loop
(PR #5266):
- Recorder HUD: a floating "Recording your workflow" panel narrates each officer
action live (Opened Dashboard -> Opened Transactions -> Opened Pending approval
-> Opened the exception form -> Filed the policy exception -> Approved the
charge), driven by logStep on the nav / tab / file-exception / approve call
sites. New recording-feed.tsx; steps + logStep added to recording-context.
- Non-directional demonstration: the await card is retitled "Show me how" and no
longer lists the steps ("go ahead and do it yourself now and I'll watch and
learn"); the agent's spoken handoff is likewise generic.
- Fix the "nothing happens after I'm done" stall: the model sometimes asked
"should I save this?" in prose instead of calling saveLearnedWorkflow, leaving
no Save card to click. The await tool-result is now directive (call
saveLearnedWorkflow; the card is how you ask), reinforced in the prompt.
- Harden the ending: after saving, the agent treats the demonstrated charge as
already cleared and waits, instead of re-running the freshly-saved procedure
on it.
Verified end-to-end in OSS dev (taught Google Ads, recalled AWS); lint + build
green.
Measure wall-clock impact of a larger Depot runner on the e2e suites.
The long pole (langgraph-python, ~10.4min) spends ~50% on build/prep
(CPU-bound) and ~43% on the Playwright run. 2->4 vCPU + NX_PARALLEL 4
should speed build/prep; the test-phase gain reveals whether it is
CPU-bound (big win) or LLM-latency-bound (then sharding is the lever).
- Scope clickPill locator to data-message-role='user' bubble so the pill
button itself can no longer satisfy the dispatch guard
- Dedup clickPill retry: skip click if the user bubble already exists
- Hero pill: assert declarative-card count=0 (OSS-136 no-Card rule),
metric count >=4 (was >=3 — KPI strip is 4 tiles per composition rule)
- At-risk pill: assert no chart and no table testids (composition rule)
- Top-account pill: assert no data-table and no status-badge testids
- Rename hero test title to 'KPI strip + pie + bar (no surrounding card)'
so the title no longer falsifies the body
- QA docs: replace 'card + metrics + pie + bar' Expected Results with
'4 KPI metrics + 1 PieChart + 1 BarChart, no surrounding Card per OSS-136'
- Probe responseTimeoutMs derived from FIRST_SIGNAL_TIMEOUT_MS so it
matches the e2e 90s budget
- Migrate readDeclarativeTestIds from booleans to counts so leftover vs
newly-mounted is distinguishable
- everyNewlyMounted gate uses current[k] > baseline[k] (was boolean
!baseline[k] against a count, which falsely blocked at non-zero baseline)
- minCounts enforce newly-mounted delta, not raw current count
(fixes the cross-pill bleed: at-risk metric:3 floor used to pass on
hero's 3 leftover metrics with no fresh mount)
- Per-pill minCounts add the chart sibling asserts D5 was missing:
hero=4 metric+1 pie+1 bar, team=1 table+1 bar, top-account=1 info-row+1 pie
- 31/31 harness tests green
- DataTable rowKey uses first-column value + index instead of bare index,
with JSON.stringify(row) fallback (stops re-mount on dynamic A2UI re-emits)
- Card emits data-card-id={props.title} so multi-card pills no longer
collide on a single declarative-card testid
- PieChart/BarChart value coercion replaced 'Number(x) || 0' with
finite-number check + console.warn on drift (no longer masks legitimate 0)
Replace the misleading 'single source of truth' claim with an explicit
DUPLICATION NOTICE describing the per-integration parity convention and
a TODO(OSS-136) for the future shared-module extraction. Both copies
remain byte-identical.
The YAML manifest was missing the app_home block the JSON variant has, so
apps created from it showed "Sending messages to this app has been turned
off". Add app_home with the Messages tab enabled (not read-only), matching
slack-app-manifest.json.
examples/slack depends on @copilotkit/bot* at "~0.0.1" so the example stays
deployable (mirrors a real npm install). Under pnpm 10 (link-workspace-packages
defaults off) that resolved the PUBLISHED 0.0.1 from npm instead of the local
workspace packages, so the example couldn't exercise local changes. Add root
pnpm.overrides mapping the three @copilotkit/bot* packages to workspace:*, which
forces local installs to link the workspace copies while leaving the example's
published version range intact.
The D5 e2e-deep probe for tool-rendering-{default,custom}-catchall sends
"weather in Tokyo" as the test input (harness/src/probes/scripts/
d5-tool-rendering-{default,custom}-catchall.ts). The fixture userMessage
matcher uses substring match. Main's catchall fixtures had a stale
"check Tokyo weather forecast" string that could not substring-match
the probe input, causing the fixture to miss and the probe to fall
through to the live LLM — surfacing as CV x red D4 on the dashboard.
Rename to the canonical "weather in Tokyo" string across affected
integrations' tool-rendering-{default,custom}-catchall.json files.
No agent or page.tsx changes; fixture content otherwise unchanged.
Rework the banking demo's teach-a-workflow loop so the officer demonstrates the
over-limit unlock on the actual dashboard instead of an inline chat card, and so
the first over-limit approve request no longer shows an approval card that fails.
When asked to approve an over-limit charge it has no saved procedure for, the
agent now declines and offers to record (no approval card). The officer opens the
new Dashboard -> Transactions -> Pending approval view, files a policy exception
and approves the charge there; a waiting card holds the chat until they click
"I'm done". The agent then saves the procedure and applies it itself to a
different over-limit charge.
Move the teach/recall HITL tools (offerWorkflowRecording,
awaitDashboardDemonstration, saveLearnedWorkflow, openPolicyException,
finalizePolicyException, approveTransaction) and the agent data/permission
readables into the global CopilotContext. A route-scoped registration unmounts
when the officer navigates to the dashboard, which drops the in-progress card and
prevents the followUp from firing; global registration survives navigation and
renders on every route.
The demonstrated exception code is captured via the recording context and handed
to the Save step, so the saved procedure names the exact code used. The agent
prompt is updated for the decline+offer and dashboard handoff, and still never
spells out the unlock or names a justifying code.
Verified end-to-end in OSS dev (lint + build green, gate smoke 3/3): Beat 1 shows
no card, the dashboard demonstration clears the Google Ads charge, and recall
clears the AWS charge via the learned procedure.
Update package READMEs + ARCHITECTURE for the assistant pane, native streaming,
and the new onThreadStarted / setSuggestedPrompts / setTitle surface. Reverse
the slack.mdx callout that told users to delete the assistant scopes (now
required), and enable the pane in the examples/slack manifest (assistant_view +
assistant:write + assistant_thread_* events) with a dev-ex onThreadStarted
greeting and the assistant config.
Activate Slack's assistant pane and native streaming as the default experience,
with zero config and safe degradation. The portable surface lands in
@copilotkit/bot (onThreadStarted lifecycle, capability-gated
thread.setSuggestedPrompts/setTitle, two SurfaceCapabilities flags + optional
adapter methods); @copilotkit/bot-slack adds the Bolt Assistant middleware
(assistant.ts), the chat.startStream transport with automatic legacy fallback
(native-stream.ts), a per-thread no-double-delivery listener guard, and
renderer pane-status mode. Everything degrades, never throws.
lefthook invokes a multi-line `run` as `sh -c "<script>"` and (in lefthook
2.1.1 / Windows git-sh) does not escape the script's embedded double quotes,
so `"$@"` / `[ "$#" ]` / `x=""` close the `-c` string early and the shell
aborts with "unexpected EOF", failing every commit. Rewrite the lint-fix
script double-quote-free (unquoted $@/$# — staged paths have no spaces; JSON
is already glob-excluded so oxfmt never sees package.json; ruff scoped to .py
via case).
The self-learning recorder POSTs to the annotate endpoint, which only exists with an
Intelligence backend; in OSS mode it returns 422. Call sites logged that rejection with
console.error, which Next.js 16 surfaces as a full-screen dev error overlay mid-demo even
though the failure is expected and harmless.
Swallow the failure in the recorder seam (useRecordUserActionInCurrentThread): catch it and
log quietly via console.debug instead of letting it reject. Recording stays best-effort — a
no-op without an Intelligence backend, and unchanged (records normally) with one.
Drive the FOR-137 self-learning story as an agent-orchestrated, narrated loop. When an
over-limit approval is rejected, the agent offers to record a workflow; the officer
demonstrates by filing a policy exception; the agent summarizes and saves the procedure;
then it applies that procedure itself to a different over-limit charge. Same-session recall
works by echoing the learned procedure back into the thread.
page.tsx: three new human-in-the-loop tools (offerWorkflowRecording,
recordExceptionDemonstration, saveLearnedWorkflow) plus a transactions agent-readable so the
agent resolves a charge the user names to the right id instead of guessing.
openPolicyException now returns the new exception id, and the agent-driven exception tools
are followUp:true so the recall chain (open then finalize then approve) does not stall.
route.ts: TEACH & RECALL prompt rules and an ACTION DISCIPLINE carve-out. The prompt does
not restate the unlock procedure, preserving the learning invariant.
policy-exception-inline.tsx: surface the demonstrated exception code via onFiled(code).
scripts/over-limit-gate-smoke.mjs: regression guard proving only a finalized
justifying-code exception lifts the policy-limit gate.
Verified end-to-end in OSS dev mode (lint and build green): the demonstration clears the
Google Ads charge and recall clears the AWS charge with a single successful approve.
Martha Kelly Schumann