Four aggregated rules using groupBy=[dimension], windowMs=120s
(one 90s probe tick plus slack), minMatches=3. Each carries the
<!channel> escalation that moved off the per-service red-tick rules
so channel gets paged exactly once for a fleet-wide outage instead
of N times. rate_limit.window=10m prevents re-escalation across
consecutive aggregation windows.
Aggregated rules bypass per-match dispatch and ingress-side gates
(rate_limit, dedupe, bootstrap); the composite FLUSH callback still
honors bootstrap suppression and rate_limit against the composite
dedupe key so successive window expiries for the same groupValues
collapse to one dispatch.
Add errorDesc to SMOKE_SLACK_SAFE_FIELDS and introduce an inline
L1-L4 safe-field set (errorDesc only) for the agent/chat/tools
dimensions — they don't have dedicated probe modules, so the allow-
list is defined at the orchestrator's dimension-registration site.
rule-loader.test.ts loadRealRules() mirrors the orchestrator wiring
so the full-config-dir test path still validates successfully.
Migrate the smoke-red-tick "escalated block renders !channel"
assertion onto the new fleet rule — <!channel> now lives on
smoke-red-fleet's aggregation template, not on the per-service
red-tick template.
AggregationBucketStore collects matching signals keyed by rule-defined
groupBy, fires onFlush once the bucket reaches minMatches (threshold),
or at windowMs expiry when below threshold (timer); drains outstanding
buckets on beforeExit. buildCompositeDedupeKey stabilizes rate-limit
across consecutive windows with the same groupValues.
CompiledRule gains optional `aggregation` field; schema.ts adds a
strict AggregationSchema so a typoed aggregation key fails load rather
than silently arming a bucket with undefined.
Strip HTML tags, script/style bodies (including entity-encoded), Slack
mrkdwn controls (<!channel>, <!here>, <@U…>), backticks. Decode entities
first so entity-smuggled script payloads are caught. Cap at 120 chars
with U+2026 ellipsis. Apply at 8 call sites in smoke.ts.
Wrap each {{signal.links.<key>}} reference in a Mustache section guard
so absent links emit nothing. Switch {{signal.errorDesc}} to triple-brace
{{{signal.errorDesc}}}; the value is pre-sanitized in smoke.ts's
sanitizeErrorDesc. Drop <!channel> from per-service rules — fleet rules
(introduced in a following commit) carry it instead.
No Mustache partials are wired in the renderer (render/renderer.ts uses
Mustache.render directly with no `partials:` arg), so the guarded-link
pattern stays inline across four YAMLs rather than being extracted.
Cover all four red-tick alert YAMLs. Missing links should emit nothing
instead of empty <|label> brackets. Sanitized errorDesc should render
literal characters, not HTML-entity-escaped.
A docs-only commit expanded {staged_files} to a docs/** path; oxlint
silently processed 0 files (both configs ignore docs/**) and then
oxfmt exited 2 with "Expected at least one target file", blocking the
commit. Adding an exclude entry in lefthook.yml lets the existing
empty-{staged_files} guard skip the hook cleanly.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The capture job can take ~30 minutes, during which other commits
routinely land on main. Without a rebase-and-retry loop, the
devops-bot push loses the race and fails with "fetch first".
Observed in runs 24799601181 and 24809331709 on 2026-04-22,
both failing at the same step with identical remote-rejected-push
output. Bounded to 5 attempts so a persistent failure still
surfaces rather than looping forever.
## Summary
The `langgraph-python-threads` example's canvas renders "No todos yet"
on thread resume even when the agent has persisted todos — the chat
correctly shows the prior conversation, but the shared-state view is
empty.
Root cause: `<ExampleCanvas>` calls `useAgent()` with no args, which
resolves to the registry agent. `<CopilotChat threadId={threadId}>`
drives a **per-thread clone**. On resume, the `/connect` bootstrap
replay's `STATE_SNAPSHOT` events hydrate the clone — not the registry
agent the canvas is reading from.
Fix: wrap `<ExampleLayout>` (both chat and canvas) in a single
`<CopilotChatConfigurationProvider agentId="default"
threadId={threadId}>`. The canvas's `useAgent()` then inherits the
active `threadId` via the existing fallback at `use-agent.tsx:139` and
resolves to the same clone. Redundant `agentId`/`threadId` props are
removed from `<CopilotChat>` (single source of truth).
## Changes
- `examples/integrations/langgraph-python-threads/apps/app/src/App.tsx`
— wrap in `CopilotChatConfigurationProvider`.
- `packages/core/src/__tests__/intelligence-agent.test.ts` — add
regression coverage: `connectAgent` with a bootstrap plan containing
`STATE_SNAPSHOT` events must leave `agent.state` hydrated (both directly
on `IntelligenceAgent` and through `ProxiedCopilotRuntimeAgent` in
intelligence mode). These tests already pass — they lock in the
replay-pipeline contract the demo depends on.
## Summary
- Classify Railway showcase services as `package` vs `starter` from the
service name and thread a `shape` field through the discovery source;
drivers branch on it.
- `drivers/smoke.ts` probes `/api/health` for starters (reusing the
result for the health side-emit), `/smoke` + `/health` for packages, and
now follows 308 redirects on the `/api/copilotkit/` L2 probe so
unmounted routes (308→404) actually flip red.
- `drivers/e2e-smoke.ts` short-circuits with `l3: "skipped"` + `l4:
"skipped"` for starters before launching chromium — starters are
single-app integrations with no `/demos/*` routing.
## Why
Starters expose a different URL surface than shell-based packages. The
current discovery-driven probe contract assumed one shape and was firing
false-red alerts on every tick against the 17 deployed starter services:
1. **No `/smoke` route** — L1 primary probe 404s.
2. **Health at `/api/health`, not `/health`** — side-emit 404s.
3. **No `/demos/*` routing** — L3 navigates to `/demos/agentic-chat` and
404s; L4 likewise.
4. **Registry lookup mismatch** — `deriveSlug` strips `showcase-`
leaving `starter-<slug>`, which `registry.json` does not key on, so
`hasToolRendering` is silently false and L4 never runs.
5. **L2 accepted raw 308** — masked regressions where the redirect
target is actually unmounted (final 404).
Next probe tick without this fix would fire **17 × 3 = 51 false-red L1
alerts** plus **17 × 2 = 34 false-red L3/L4 alerts**. Verified against
three live starters (`ag2`, `mastra`, `langgraph-python`): all return
200 on `/api/health`, 404 on `/smoke` / `/health` / `/demos/*`, and 308
→ 400 on `POST /api/copilotkit/` (runtime rejects the empty body;
proof-of-life remains).
Shape is derived from the Railway service name (`showcase-starter-*` →
`"starter"`, else `"package"`), so adding a new starter requires no YAML
edit — the next tick picks it up automatically.
## Test plan
TDD — 12 new tests written first (verified red against existing code),
then implemented against:
- `railway-services.test.ts` (3 new): starter / package / mixed-batch
classification of the `shape` field.
- `drivers/smoke.test.ts` (6 new): starter primary GET → `/api/health`
green-on-200, starter `/api/health` 503 → red, `/smoke` + `/health`
never probed under starter shape, explicit package shape preserves
legacy contract, `/api/copilotkit/` POST uses `redirect: "follow"`,
followed-redirect-to-404 flips red.
- `drivers/e2e-smoke.test.ts` (3 new): starter shape returns
green-skipped aggregate, chromium never launched, `/demos/*` never
navigated, in-band `demos` field ignored under starter shape.
Full showcase-ops suite: **716 tests passing** (704 existing + 12 new).
Typecheck clean. Build clean.
## Follow-ups
None deferred — L2 308 handling is addressed in this PR alongside the
shape split.
- Widen classifyShape package regex to accept hyphen-bearing multi-segment
names (showcase-langgraph-python, showcase-claude-sdk-typescript,
showcase-ms-agent-dotnet, showcase-crewai-crews) without firing the
name-shape-unknown warn on every tick.
- Broaden classifyShape warn trigger: any name that matches neither the
starter nor widened-package pattern now warns, including non-showcase-
prefixed names (my-random-service, copilotkit-cloud) and mixed-case
variants. Returns package as a safe default; warn is the audit trail.
- Drop NormalisedSmokeInput discovery-arm index signature so input.url in
a mode==="discovery" branch is a TS error rather than silently unknown.
- Drop the normaliseMode early-return arm that trusted raw.mode verbatim.
Field presence is the sole discriminator, keeping schema-parsed and
raw-input paths consistent. Bad-shape inputs now throw loud.
- Restore parse-time invariants on smokeInputSchema via a union-level
superRefine that rejects bare {key}, discovery-missing-publicUrl, and
mixed-mode {key,url,name,publicUrl} payloads that the discovery arm's
passthrough would otherwise absorb.
- Add regression tests for the primary-key rewrite in discovery mode
(smoke:showcase-ag2 -> smoke:ag2, smoke:showcase-starter-ag2 ->
smoke:starter-ag2) and verbatim pass-through in static mode.
- Add end-to-end package-shape test coverage for deriveSlug feeding
demosResolver (showcase-langgraph-python -> langgraph-python registry
lookup) and the throwing-resolver swallow behaviour as a regression
guard against the deferred fail-loud change.
`CopilotChatView` rendered the attachment queue + input as flex siblings
beneath the scroll area, so long messages hit the input's flat top edge
and were sliced mid-line. Most visible in pin-to-send mode where the user
reads at their own pace. The previously-shipped feather gradient masked
this but clashed with host themes whose `--background` didn't match its
hard-coded white/near-black (b621e96ee defaulted it to an empty div).
Wrap attachments + input in a single absolute-positioned overlay so the
scroll content fills full height and passes behind the rounded pill. Pad
scroll-content bottom by the measured overlay height so the last line
clears the pill. Welcome-screen input is unchanged (stays inline). The
`feather` slot remains — hosts who want a themed fade supply their own
gradient.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Applies Round 2 code review findings to the starter-probe contract:
Critical:
- classifyShape now warns on every showcase-* name that fails both the
strict starter pattern and the single-segment package pattern; typos
(showcase-strater-*) and hyphen-bearing multi-segment names surface
audit warns instead of silently falling to package.
- C2 (add "skipped" state to the aggregate ProbeResult) deferred: the
aggregate state lives on the shared ProbeState type used by the
transition-detector, alert-engine, and status-writer; extending it to
a new literal ripples across the whole state machine and six files
outside scope, and would materially change alert semantics. Flagged
as follow-up.
Major:
- resolveShape consolidated into discovery/railway-services.ts; both
smoke and e2e-smoke drivers import it and thread ctx.logger so the
classifier's audit warn fires on the driver path.
- Silent fallback path (neither name nor shape supplied) now emits a
structured debug log (resolve-shape-fallback) so the assumption is
greppable if it breaks.
- E2eSmokeStarterSignal gains errorDesc?: never, turning the "no failure
reason on a skipped-green row" invariant into a compile error.
- smokeInputSchema refactored to a zod union over static + discovery
branches; runtime normaliseMode() maps legacy inputs onto a
NormalisedSmokeInput with a required mode discriminator, replacing
the non-null assertion on input.url and the ad-hoc refine()s.
- Tests and driver comments stripped of CR-process tags (C1:, R1
bundle, etc.); test-side two-GETs rationale shrunk to a one-line
pointer so the authoritative copy lives in the driver.
- Added classifyShape + resolveShape audit tests: warn fires on typo
names, not on well-formed roots or starters, and debug fires on the
name/shape-absent fallback.
Minor:
- Warn event key namespaced to discovery.railway-services.name-shape-unknown.
- deriveSlug JSDoc moved to sit directly above deriveSlug in e2e-smoke.
- SHOWCASE_SHAPES inlined into showcaseShapeSchema (no external consumers).
- Starter signal carries a structured skipReason: "starter-shape" enum
alongside the human-readable note.
- Removed dead "unreachable" discriminator asserts in the starter
contract test.
Wrap <ExampleLayout> in <CopilotChatConfigurationProvider agentId="default"
threadId={threadId}> so the canvas's useAgent() inherits the active threadId
via the existing fallback in use-agent.tsx. Without this wrapper, the canvas
calls useAgent() with no args and resolves to the registry agent instead of
the per-thread clone that the chat's /connect replay populates, so
STATE_SNAPSHOT events never reach it — todos rendered blank on thread resume
even though the final persisted snapshot contained them.
CopilotChat no longer needs explicit agentId/threadId props; it inherits
from the same provider, keeping one source of truth.
Critical
- smoke + e2e-smoke drivers resolve `shape` via the classifier when
`input.name` is present and throw on explicit-vs-classifier
disagreement. Silent-defaulting at the driver boundary inverted the
whole starter-shape fix; the driver now fails loud on wiring drift.
Majors
- E2eSmokeSignal is a discriminated union keyed on `shape`. Starter
variant carries `note` (never rendered as failure) instead of
`errorDesc`, so alert templates stop treating a green short-circuit
as a failure reason.
- Starter shape runs TWO independent GETs against `/api/health` (smoke
and health feed separate alert dimensions that dashboards correlate;
a single-GET reuse would make the rows byte-identical and defeat the
signal).
- smoke.test.ts uses a real `node:http` server to exercise genuine 308
redirects for the agent POST; prior tests accepted a synthetic 404
response and would have let a `redirect: "follow"` regression pass.
- New tests: starter 404 on /api/health, single-source shape enum
assertion on discovery+package and discovery+starter redirect option,
mixed-driver wiring via `.passthrough()` spread, shape-mismatch
throws on both drivers.
- e2e-smoke: starter short-circuit ordering locked in via a throwing
demos resolver; full starter signal contract asserted field-by-field.
Minors
- Single-source the shape enum: `SHOWCASE_SHAPES`, `ShowcaseServiceShape`,
`showcaseShapeSchema` exported from railway-services; both driver
schemas and `deriveUrls` now import the shared types.
- `classifyShape` emits a one-shot warning on umbrella-match-but-unusual
names (`showcase-`, `showcase-starter`) when a logger is supplied.
- Static-mode rejects `url` + `shape` combo via zod refine — shape
applies only to discovered `publicUrl`.
- Drop rot-prone counts (17, 34, 51), "CR A1", "Procedure 0" and
"previously"/"old contract" phrasing. Trim WHAT comments that restate
code; keep WHY that explains contract decisions.
## What does this PR do?
Renames the docs top-bar call-to-action from **"Copilot Cloud"** to
**"Free Developer Access"** to better reflect the value proposition of
the signup flow.
The change touches the default text of the `LinkToCopilotCloud`
component. Inline usages that pass explicit children (e.g., "Click here
to get your Copilot Cloud API key for free", "navigate to Copilot
Cloud", the MCP-servers reference) are unaffected — they keep their
existing copy. The navbar nav link (not a CTA) also keeps its "Copilot
Cloud" label.
Affected surface: the CTA button in `TopBar` at the top of every docs
page.
## Related PRs and Issues
- n/a
## Checklist
- [x] I have read the [Contribution
Guide](https://github.com/copilotkit/copilotkit/blob/master/CONTRIBUTING.md)
- [x] If the PR changes or adds functionality, I have updated the
relevant documentation
Covers the shared-state thread-resume path that was missing coverage:
on resume, the /connect bootstrap plan replays STATE_SNAPSHOT events
captured during the original run, and agent.state (both on the direct
IntelligenceAgent and through the ProxiedCopilotRuntimeAgent bridge)
must reflect the final snapshot so UI reading from agent.state renders
the persisted state.
Both tests pass against the current implementation, ruling out these
layers as the source of the shared-state demo regression.
Starters (17 Railway services showcase-starter-*) expose a different URL
surface than shell-based showcase packages, so the current discovery-driven
probe contract produces false-red alerts on every tick:
1. No /smoke route -> L1 primary probe 404s
2. Health lives at /api/health, not /health -> side-emit 404s
3. No /demos/* routing -> L3 navigates to /demos/agentic-chat and 404s;
L4 likewise 404s on /demos/tool-rendering
4. Registry lookup strips only the showcase- prefix, leaving starter-<slug>
which registry.json does not key on -> hasToolRendering silently false
5. L2 /api/copilotkit/ was accepted on a raw 308, masking regressions
where the redirect target is actually unmounted (404)
Fix via shape detection in the Railway discovery source. Classify each
service by name prefix:
- showcase-starter-* -> shape: "starter"
- showcase-* -> shape: "package"
Drivers branch on the shape:
drivers/smoke.ts:
- starter -> primary GET /api/health, reuses result for health
side-emit (same endpoint), POST /api/copilotkit/
- package -> legacy /smoke + /health + /api/copilotkit/
- both -> POST /api/copilotkit/ now follows 308 redirects and
judges the FINAL status, so an unmounted route (308->404)
correctly flips red instead of masquerading as green.
drivers/e2e-smoke.ts:
- starter -> short-circuit with l3: "skipped" + l4: "skipped"
BEFORE launching chromium; skips registry lookup
(starter-<slug> is not keyed). Dashboards see an
explicit skipped state rather than a silent red.
- package -> unchanged.
Classification is done by name prefix so adding a new starter requires no
YAML edit; the next tick picks it up automatically with the right contract.
Verified against three deployed starters (ag2, mastra, langgraph-python):
all return 200 on /api/health, 404 on /smoke / /health / /demos/*.
Before fix: next probe tick would fire 17x3=51 false red alerts across L1
(smoke + health 404s) plus 17x2=34 false reds on L3/L4 (chat + tools
404s on /demos/*). After fix: starters report green L1 on healthy
/api/health, L2 green via followed 308->400 runtime reply, L3/L4 cleanly
skipped with explicit status.
12 new driver + discovery tests cover the shape classifier, primary+health
+agent URL selection per shape, /smoke-never-called regression guard,
308-follow behavior for L2, and e2e-smoke short-circuit before chromium
launch. 716 tests pass (704 existing + 12 new).
The feather fade-to-background overlay at the bottom of CopilotChatView's
scroll area hard-coded `from-white` (light) and `from-[rgb(33,33,33)]`
(dark). Host apps whose `--background` didn't match those values saw a
visible band at the bottom of the scroll area — most obvious in dark
mode (e.g. the `langgraph-python` integration example uses `#010507`).
Default `CopilotChatView.Feather` now renders an empty div — no visual,
but the element stays in the tree so a `scrollView={{ feather: "my-class" }}`
shorthand still applies. The `feather` slot on `ScrollView` /
`PinToSendScrollContainer` / `ScrollContent` is preserved unchanged, so
consumers who want a custom overlay can still opt in via
`scrollView={{ feather: MyCustomFeather }}`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
<!--
Thank you for sending the PR! We appreciate you spending the time to
work on these changes.
Help us understand your motivation by explaining why you decided to make
this change.
**Please PLEASE reach out to us first before starting any significant
work on new or existing features.**
By the time you've gotten here, you're looking at creating a pull
request so hopefully we're not too late.
We love community contributions! That said, we want to make sure we're
all on the same page before you start.
Investing a lot of time and effort just to find out it doesn't align
with the upstream project feels awful, and we don't want that to happen.
It also helps to make sure the work you're planning isn't already in
progress.
As described in our contributing guide, please file an issue first:
https://github.com/ag-ui-protocol/ag-ui/issues
Or, reach out to us on Discord: https://discord.com/invite/6dffbvGU3D
You can learn more about contributing to copilotkit here:
https://github.com/copilotkit/copilotkit/blob/master/CONTRIBUTING.md
Happy contributing!
-->
## What does this PR do?
(Describe the changes introduced in this PR)
- Bumps `@copilotkit/react-core`, `@copilotkit/react-ui`, and
`@copilotkit/a2ui-renderer` in
`examples/integrations/langgraph-python-threads/apps/app` from `1.56.2`
→ `1.56.3`
- Bumps `@copilotkit/runtime` in
`examples/integrations/langgraph-python-threads/apps/bff` from
`1.56.2` → `1.56.3`
## Related PRs and Issues
- (Direct link to related PR or issue, if relevant)
## Checklist
- [ ] I have read the [Contribution
Guide](https://github.com/copilotkit/copilotkit/blob/master/CONTRIBUTING.md)
- [ ] If the PR changes or adds functionality, I have updated the
relevant documentation
## Release monorepo v1.56.3
**Scope:** `monorepo` | **Bump:** `patch`
---
### How this release process works
1. **This PR was created automatically** by the "release / create-pr"
workflow.
It bumped the `monorepo` packages to `1.56.3`
and generated AI-enhanced release notes.
2. **CI runs on this PR** — the full test suite (unit tests, lint, type
checks, build)
must pass before merging. This is the review gate.
3. **Review the release notes** in `release-notes.md` in this PR.
If a Notion draft was created, you can edit the release notes there
before merging.
4. **When this PR is merged**, the `release / publish` workflow
automatically:
- Builds all packages
- Publishes the `monorepo` packages to npm at version `1.56.3`
- Creates git tag `monorepo/v1.56.3`
- Creates a GitHub Release with the final release notes
### Before merging
- [ ] CI is green (tests, lint, types, build)
- [ ] Version bumps look correct
- [ ] Release notes are accurate (edit in Notion if a draft was created)
---
> **Do not merge until CI is fully green.** The full test suite runs
automatically on this PR.
<!--
Thank you for sending the PR! We appreciate you spending the time to
work on these changes.
Help us understand your motivation by explaining why you decided to make
this change.
**Please PLEASE reach out to us first before starting any significant
work on new or existing features.**
By the time you've gotten here, you're looking at creating a pull
request so hopefully we're not too late.
We love community contributions! That said, we want to make sure we're
all on the same page before you start.
Investing a lot of time and effort just to find out it doesn't align
with the upstream project feels awful, and we don't want that to happen.
It also helps to make sure the work you're planning isn't already in
progress.
As described in our contributing guide, please file an issue first:
https://github.com/ag-ui-protocol/ag-ui/issues
Or, reach out to us on Discord: https://discord.com/invite/6dffbvGU3D
You can learn more about contributing to copilotkit here:
https://github.com/copilotkit/copilotkit/blob/master/CONTRIBUTING.md
Happy contributing!
-->
## What does this PR do?
Fresh empty chats were hitting `/connect` against thread IDs the backend
had never seen (404s) and the
welcome screen stayed hidden. Root cause: the v1 `<CopilotKit>` wrapper
pipes a
`ThreadsProvider`-minted UUID down as `threadId`, and downstream
consumers were treating `!!threadId`
as "caller-chosen," so an auto-minted placeholder looked identical to a
real thread.
Fix: introduce an explicit `hasExplicitThreadId` signal and plumb it
end-to-end, so consumers that only
make sense against a real backend thread (`/connect`, switch-flash
suppression, welcome-screen gating)
can distinguish a caller-chosen thread from an internal placeholder.
## Changes
- **`ThreadsProvider`** — exposes `isThreadIdExplicit`. True when the
`threadId` prop is supplied or
`setThreadId()` has been called; false for the first-mount auto-minted
UUID. Explicitness via
`setThreadId` is sticky across prop churn.
- **`CopilotChatConfigurationProvider`** — accepts an optional
`hasExplicitThreadId` prop (overrides
inference), otherwise infers from `!!threadId`. OR-inherits from parent
config so explicitness
propagates through nested/bare providers.
- **v1 `CopilotKit` bridge** (`copilotkit.tsx`) — reads
`isThreadIdExplicit` from `useThreads()` and
forwards it to `CopilotChatConfigurationProvider`, so the auto-minted
UUID no longer masquerades as
caller-chosen.
- **`CopilotChat`** — `isConnecting` and the `/connect` effect now gate
on `hasExplicitThreadId`
instead of `!!providedThreadId`, so a caller-chosen thread from a
wrapping provider is honored while a
leaked auto-UUID is not.
## Test plan
- [x] `threads-context.test.tsx` — `isThreadIdExplicit` transitions:
auto→explicit via `setThreadId`,
prop-driven explicitness, stickiness after prop removal
- [x] `CopilotChatConfigurationProvider.test.tsx` — inference, explicit
`false` override (v1 bridge
case), OR-inheritance through nested chains
- [x] `v1-explicit-threadid-bridge.test.tsx` (new) — regression coverage
for the v1 wrapper
specifically
- [x] `CopilotChat.welcomeGate.test.tsx` (new) — welcome screen stays
visible on a bare mount;
`/connect` is not called without an explicit thread
- [x] `packages/react-core` full test suite passes (1149 tests)
- [ ] Manual smoke in `examples/integrations/langgraph-python-threads`:
fresh app → welcome screen
shows → first message starts a real thread
## Related PRs and Issues
- (Direct link to related PR or issue, if relevant)
## Checklist
- [ ] I have read the [Contribution
Guide](https://github.com/copilotkit/copilotkit/blob/master/CONTRIBUTING.md)
- [ ] If the PR changes or adds functionality, I have updated the
relevant documentation
30-minute schedule (half the smoke cadence — L3+L4 per tick pulls a
real chromium page so we trade freshness for memory + bandwidth
headroom). 180s outer timeout covers two 60s page timeouts plus
chromium launch overhead. max_concurrency 2 caps peak memory at ~600MB
(each chromium ~300MB RSS) to fit the orchestrator's Railway footprint.
Discovery uses the railway-services source with namePrefix + the new
nameExcludes filter to hit every user-facing showcase without the
7 infra services.
Runtime stage needs two things for e2e-smoke: chromium (headless) and
the shell registry so the driver can gate L4 on demos membership.
Build stage pulls the single registry.json file through rather than
fetching at probe-tick time, keeping the image hermetic. Runtime
stage is already on Debian-slim (needed by Playwright's glibc-linked
chromium); `playwright install --with-deps chromium` runs as root
before the USER node drop so apt-get succeeds. /ms-playwright is
chown'd to node so the runtime user can read the browser tree.
Wire the in-process Playwright driver so probe ticks resolving to
kind: e2e_smoke dispatch to it. Drops the 'deliberately NOT
registered yet' placeholder that flagged the driver as scaffold-only.
The e2e-smoke probe needs to discover every user-facing showcase
service without enumerating infra (aimock, ops, pocketbase, shell,
shell-{dashboard,docs,dojo}). Extend the discovery filter schema
with an optional nameExcludes array so a single namePrefix can pull
in all 34 services and explicitly drop the 7 that don't expose user
demos, rather than maintaining 27 separate namePrefix entries.
Replace the JSON-reporter scaffold with an in-process Playwright
driver. Each invocation launches one headless chromium, runs L3
against /demos/agentic-chat (any non-empty response) and, when the
service registry entry includes tool-rendering, runs L4 against
/demos/tool-rendering (weather vocabulary check). Primary return is
e2e-smoke:<slug>; chat:<slug> and tools:<slug> side-emit via
ctx.writer. AbortSignal is wired end-to-end so timeout teardown
kills the browser cleanly. Launcher + demos resolver are dep-injected
for unit testability.
Input schema accepts { key, backendUrl?, publicUrl?, name?, demos? }
with a zod refine requiring at least one URL. Slug derivation strips
the "showcase-" prefix so Railway service names align with registry
entries (bare slugs). The default demos resolver memoises a read of
/app/data/registry.json at first use, keeping probe-tick overhead at
a single JSON parse across the orchestrator lifetime.
Extends the smoke probe driver to emit a third side-emission per target:
`agent:<slug>` — POST `${backendUrl}/api/copilotkit/` with `{}` body,
green on non-404 response (runtime is mounted), red on 404 or transport
failure. Matches the `checkAgentEndpoint` contract in the e2e helpers
(integration-smoke.spec.ts L2 `@agent` assertion).
Switches `config/probes/smoke.yml` from a static 17-target list to the
`railway-services` discovery source with `namePrefix: "showcase-"` and
an explicit `nameExcludes` for infra services (aimock/ops/pocketbase/
shell*). New showcase services are picked up automatically on the next
tick without YAML edits.
- src/probes/discovery/railway-services.ts: add `nameExcludes: string[]`
filter field, applied AFTER the prefix check so excluded services skip
per-service env fetch entirely.
- src/probes/drivers/smoke.ts: accept both static (`{key,url}`) and
discovery (`{key,name,publicUrl,imageRef,env}`) input shapes. Derive
slug by stripping `showcase-` prefix from `name` (discovery) or
splitting key on `:` (static fallback). Rewrite primary result key to
`smoke:<slug>` in discovery mode so existing alerts keyed on
`smoke:ag2` / `smoke:starter-ag2` stay intact.
- Sequential probes kept for socket-count bound: at max_concurrency=6 *
34 services * 3 endpoints, parallel would push 612 inflight sockets.
- Coverage: smoke driver 98.55% lines, exceeds 95% gate.
Adds three new alert-rule YAMLs modeled on smoke-red-tick.yml so the
per-starter depth signals side-emitted by the smoke + e2e-smoke drivers
(agent:<slug>, chat:<slug>, tools:<slug>) produce Slack alerts on
green_to_red / sustained_red / red_to_green transitions with the same
!channel 1-hour escalation + 15-minute rate limit + 20-minute deploy
grace as the L1 smoke rule.
Also extends the DIMENSIONS enum with the three new closed values so
rule-side typos continue to fail at load with a listed-valid-members
Zod error instead of silently never matching any probe key.
No driver or discovery changes in this commit — pure rule YAML + the
single enum entry each rule's `signal.dimension` resolves against.
## Summary
- Add `curl` to the runtime `apt-get install` line in:
- `showcase/starters/template/dockerfiles/Dockerfile.typescript`
-
`showcase/packages/{claude-sdk-typescript,langgraph-typescript,mastra}/Dockerfile`
- 3 starter Dockerfiles (regenerated from template)
## Why
Multi-stage Docker refactor (#4147) moved TS showcase services to
`node:22-slim` runtime. That base image doesn't include `curl`. The
watchdog's liveness probe invokes `curl -sS --max-time 5
http://127.0.0.1:8124/ok` — which exits `rc=127 "command not found"`
every cycle. After 180s grace + 3×30s strikes the watchdog kills the
agent. Railway auto-restarts. Loop forever.
Verified via `railway ssh`:
```
sh: 1: curl: not found
rc=127
```
while `ss -tlnp` on the same container confirmed both :8123
(langgraph-api) and :8124 (liveness.mjs) LISTEN. The agent was always
healthy — the probe was broken.
## Test plan
- [ ] Local amd64 docker build for starter+package variants succeeds
- [ ] `docker run ... sh -c "which curl"` prints `/usr/bin/curl`
- [ ] After Railway redeploy of langgraph-typescript, steady-state: no
kill-loop, `/api/health` green
The multi-stage Docker refactor (#4147) moved TypeScript showcase services
to `node:22-slim` runtime. That base image does NOT include `curl`.
entrypoint.sh's watchdog runs `curl -sS --max-time 5 http://127.0.0.1:8124/ok`
every 5s to probe the liveness endpoint. Without curl the probe exits
rc=127 "command not found" every cycle. After 180s grace + 3x30s strikes
the watchdog kills the agent process. Railway auto-restarts. Kill-loop
forever.
Verified via `railway ssh --service showcase-langgraph-typescript`:
sh: 1: curl: not found
rc=127
while `ss -tlnp` on the same container confirmed BOTH 0.0.0.0:8123
(langgraph-api) and 0.0.0.0:8124 (liveness.mjs) LISTEN. The agent
servers were healthy the whole time — the probe tool was missing.
Fix: add `apt-get install -y --no-install-recommends curl` to the
runtime stage of:
- showcase/starters/template/dockerfiles/Dockerfile.typescript (source)
- showcase/packages/claude-sdk-typescript/Dockerfile
- showcase/packages/langgraph-typescript/Dockerfile
- showcase/packages/mastra/Dockerfile
3 starter Dockerfiles regenerated from the template via
`pnpm -C showcase/scripts generate-starters`.
Python/Java/.NET services already install curl (they need it for
NodeSource or similar) and are unaffected.
Smoke: local `docker build --platform linux/amd64` for both
starter + package variants succeeded; `docker run ... sh -c "which curl"`
prints /usr/bin/curl in both.
Threads-minted UUIDs were leaking through as if caller-chosen, so fresh
empty chats fired /connect against a backend that had never seen the
thread (404) and the welcome screen stayed hidden. Plumb an explicit
hasExplicitThreadId signal through ThreadsProvider,
CopilotChatConfigurationProvider, and the v1 CopilotKit bridge so
consumers can distinguish an auto-minted placeholder from a real
caller-supplied thread.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
## Summary
- Change `shell-dojo` matrix entry: `context: "."` + `dockerfile:
"showcase/shell-dojo/Dockerfile"`. Matches shell-dashboard pattern.
## Why
Build fails with `failed to walk /tmp/buildkit-mount.../showcase: no
such file or directory`. `showcase/shell-dojo/Dockerfile` has `COPY
showcase/scripts/...`, `COPY showcase/shared/...`, `COPY
showcase/packages/...` — expects repo root as build context. Matrix had
`context: showcase/shell-dojo` which scopes the context to just
shell-dojo's own directory, making those COPYs fail.
## Test plan
- [ ] Next deploy of shell-dojo succeeds
## Summary
- Replace `jq -n --rawfile text … > \$(mktemp)` + `payload-file-path`
with inline `payload:` + `toJSON(format(...))` on the scheduled
Slack-alert step in `test_smoke-starter.yml`.
- Pattern already in use in `test_smoke-starter-deployed.yml` (PR #4068)
and `showcase_validate.yml` — this brings the last holdout in line.
## Why
Every scheduled run of `test / smoke / starter` that legitimately fails
a matrix leg (upstream outage, floating-dep breakage, etc.) has been
crashing the downstream alert step with:
```
##[error]Invalid input! Failed to parse contents of the provided payload file
```
Two compounding causes:
1. `slackapi/slack-github-action@v2.1.0` rejects payload files whose
name does not end in `.json`/`.yaml`/`.yml`. `mktemp` produces
extensionless files.
2. `jq -n --rawfile text "$SLACK_MSG"` under `set -e` can abort on
edge-case summary input (missing tmpfile, non-UTF-8 bytes surviving the
sanitizer), leaving an empty or missing payload.
Either way, the notifier failed before firing, so genuine outages went
unreported in `#oss-alerts`. The user reported this as "Showcase: Drift
Detection" — that specific workflow was already retired to showcase-ops
on 2026-04-22 (commit `89eb0734b`), but the same
jq-rawfile-then-payload-file-path pattern still lived in
`test_smoke-starter.yml`.
Inline `payload:` with `toJSON(format(...))` sidesteps both failure
modes: quotes, backslashes, and newlines in the sanitized summary are
safely JSON-encoded at template-eval time, and there is no intermediate
file to mishandle.
Audit of remaining file-based payloads left intentionally unchanged:
- `test_smoke-starter-deployed.yml` recovery step writes to
`/tmp/starter-smoke-recovery.json` (valid extension, simple `jq --arg`
scalar) — safe.
- `showcase_docs-sync.yml` writes to `slack-payloads/*.json` (valid
extensions, `jq --arg` scalars) — safe.
## Test plan
- [ ] `actionlint .github/workflows/test_smoke-starter.yml` reports no
new findings (confirmed locally — one pre-existing line-64 warning
unrelated to this diff)
- [ ] Next scheduled run on `main` that has a genuinely failing matrix
leg posts a well-formed red alert to `#oss-alerts` instead of `Invalid
input! Failed to parse contents of the provided payload file`
- [ ] A scheduled run where all matrix legs pass remains silent (no
behavior change on the green path)
The failure-alert Slack step crashed every scheduled run with
'Invalid input! Failed to parse contents of the provided payload file'.
Two compounding causes: slackapi/slack-github-action@v2.1.0 rejects
payload files without a .json/.yaml/.yml extension (mktemp produces
extensionless files), and 'jq -n --rawfile' under 'set -e' can abort
on edge-case summary input, leaving an empty/missing payload.
Switch to the inline 'payload:' + toJSON(format(...)) pattern already
used in test_smoke-starter-deployed.yml (PR #4068) and
showcase_validate.yml. Summary text is sanitized into $GITHUB_ENV via
heredoc (handles embedded =, quotes, newlines), then JSON-encoded by
toJSON at template-eval time — no intermediate file, no jq crash path,
genuine outages surface in #oss-alerts instead of being silently
suppressed by a broken notifier.
test_smoke-starter-deployed.yml recovery path (/tmp/...-recovery.json,
jq --arg) and showcase_docs-sync.yml (static slack-payloads/*.json)
use safe patterns already — left unchanged.
## Summary
Consolidates the 5 independent fixes needed to return main CI to green
after #4113 (showcase-ops) merged. Supersedes individual PRs #4160,
#4161, #4162, #4163, #4164, #4165 — closing those in favor of this one.
## Fixes (5 grouped commits, one per area of concern)
1. **`fix(shell-docs): resolve findFrameworksWithCell + SidebarLink
scope type errors`**
- Exports shared 3-arg `findFrameworksWithCell` from `docs-render.tsx`,
drops dead 1-arg locals in both `[[...slug]]/page.tsx` files (the root
copy was unused and referenced an undeclared `demos`).
- Makes `SidebarLink.scope` optional (was already effectively unused
internally — destructured as `_scope`). Step-2 cards on the overview
page now typecheck.
- Supersedes #4160 (missing `demos` import) — the underlying dead-code
removal makes the import unnecessary.
2. **`chore(commitlint): skip merge commits from subject validation`**
- `commitlint.config.js` ignore for `Merge ` prefix.
- `.github/workflows/static_quality.yml` guard: `git rev-list --parents
-n 1 HEAD` short-circuit when parent count >1. Catches GitHub's "Create
merge commit" flow where the subject is the PR title but the body pulls
markdown list lines that commitlint reads as additional subjects.
3. **`fix(shell-dashboard): resync scripts lockfile + unsilence npm
ci`**
- `showcase/scripts/package-lock.json` drifted from `package.json`
(pinned old `vitest@4.1.4`, missing `@vitest/coverage-v8@4.1.5`) —
regenerated via `npm install --package-lock-only`.
- Removes `--silent` from the Dockerfile's builder-stage `npm ci` calls
so next same-class failure surfaces the error directly.
4. **`fix(showcase/scripts): capture-previews skips demos without a
route`**
- `langgraph-python/cli-start` has no `route` field (it's a CLI
copy-paste card, not a navigable page). Script was building
`${backendUrl}undefined` → 45s DNS timeout. 10-line guard in
`buildTargets` + visible `[SKIP]` log.
5. **`fix(showcase/langgraph-fastapi): pre-create .langgraph_api dir
with app ownership`**
- Commit `b9bcf2e6b` added `USER app` to the Dockerfile.
`langgraph_runtime_inmem/store.py` calls `os.makedirs(".langgraph_api")`
at import time against CWD=`/app` — the `app` user can't write →
`PermissionError [Errno 13]` → Next.js stays alive so `wait -n` never
fires → watchdog restart-loops until the 600s budget expires.
- #4113 was the first rebuild that actually ran the new image (other 16
starters kept their pre-USER-app cached containers via paths-filter).
## Test plan
- [x] Each fix was verified individually in its original PR (all 6 PRs
were green).
- [x] `npm run build` in `showcase/shell-docs` — clean (26 static pages
generated).
- [x] `docker build -f showcase/shell-dashboard/Dockerfile .` through
`--target builder` — clean on Depot.
- [ ] CI on this PR will reverify all four Docker builds + commitlint.
## Summary
Replaces the four separate intelligence containers (`db-migrations`,
`app-api`, `realtime-gateway`, `thread-culler`) in the
langgraph-python-threads example with a single `intelligence` composite
container that runs all of them under `s6-overlay` supervision. Compose
file drops from six services to three (`postgres`, `redis`,
`intelligence`). README updated to match.
Railway deploys on and after main@5ed233f01 failed health within 600s with:
PermissionError: [Errno 13] Permission denied: '.langgraph_api'
First deploy of langgraph-fastapi since the b9bcf2e6b USER app change.
langgraph_runtime_inmem/store.py does `os.makedirs(".langgraph_api", exist_ok=True)`
at import time, resolving against CWD (/app). WORKDIR creates /app as root;
--chown=app:app on COPY only chowns contents. USER app (uid 1001) can't write
to /app, the agent subprocess crashes, Next.js stays alive so wait -n never
fires, and Railway / the watchdog restart loops until the 600s budget elapses.
Pre-create /app/.langgraph_api (and chown /app itself) so the first-import
makedirs is a no-op.
Verified locally:
- docker build + --load clean
- agent /ok → 200
- next /api/health → 200
- no PermissionError in logs
Jordan Ritter