Pre-merge checklist for the promote-notify workflow: Slack-membership
checks (#team-showcase + #oss-alerts), SLACK_BOT_TOKEN scope verification,
gh workflow run dispatch commands for all three canonical fixtures, and
a schema-mismatch test that verifies the workflow emits a ::warning::
annotation with NO Slack API call.
Adds three canonical fixtures under showcase/test-fixtures/promote-notify/:
success.json (29 services green), partial.json (26 green / 3 red with
mixed exit codes + categories), and total-failure.json (29 red fleet
abort with truncation-suffix sentinel).
Includes validate.sh — strict schema enforcement (rc propagation, enum +
regex assertions for run_id, pre_staging, abort_reason, category, exit
codes). And a README documenting the fixture contract + how to dispatch
them via gh workflow run.
Adds .github/workflows/showcase_promote_notify.yml — workflow_dispatch-only
Slack notify workflow for promote results. Posts initiation + threaded reply
to #team-showcase; cross-posts to #oss-alerts on partial/total failure.
Also adds showcase_promote_notify.dry-run.sh — a local render-logic mirror
that decodes the same payload and prints the messages the workflow would
send to Slack, without making any Slack API calls. Used by the pre-merge
runbook and by future CI fixture tests.
Adds a cookbook recipe showing how to define an agent once in Oracle
Agent Spec, run it on LangGraph over AG-UI, give it long-term memory on
Oracle AI Database, and render it in CopilotKit. Registers the page in
the cookbook sidebar (meta.json) and overview (index.mdx).
Draft: live-demo video pending CDN upload; example source repo
(CopilotKit/oracle-cookbook) pending publish.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Throw when a catalog definition has no matching renderer so createCatalog
satisfies strict check-types under Angular's tsconfig.
Co-authored-by: Cursor <cursoragent@cursor.com>
Modern @a2ui/web_core and @angular/cdk subpath exports require bundler
resolution so angular:check-types can resolve the new A2UI imports.
Co-authored-by: Cursor <cursoragent@cursor.com>
Align surface operation normalization and catalog test fixtures with the
RendererProps contract so the changed A2UI web-components pass tsc.
Co-authored-by: Cursor <cursoragent@cursor.com>
Remove the hand-written types entry from exports["."] so ng-packagr can
generate deterministic published package metadata without warnings.
Co-authored-by: Cursor <cursoragent@cursor.com>
- Implemented audio transcription capabilities with error handling.
- Refactored CopilotChat component to utilize a directive for handling attachments.
- Improved CopilotChatReasoningMessage to manage streaming state and elapsed time more efficiently.
- Added new scroll view component for better message display and auto-scrolling behavior.
- Updated styles for A2UI surface components to enhance layout and scrolling.
- Enhanced tests for OpenGenerativeUIRenderer to ensure proper height measurement.
Apply the common Angular CLI schematics to code which has been added
by Soverius.
We can do a follow-up for the high amount of effects, but we have to be
careful. We need tests first to do a refactoring for that.
## Release monorepo v1.60.2
**Scope:** `monorepo` | **Bump:** `patch`
---
### How this release process works
1. **This PR was created automatically** by the "release / create-pr"
workflow.
It bumped the `monorepo` packages to `1.60.2`
and generated AI-enhanced release notes.
2. **CI runs on this PR** — the full test suite (unit tests, lint, type
checks, build)
must pass before merging. This is the review gate.
3. **Review the release notes** in `release-notes.md` in this PR.
If a Notion draft was created, you can edit the release notes there
before merging.
4. **When this PR is merged**, the `release / publish` workflow
automatically:
- Builds all packages
- Publishes the `monorepo` packages to npm at version `1.60.2`
- Creates git tag `monorepo/v1.60.2`
- Creates a GitHub Release with the final release notes
### Before merging
- [ ] CI is green (tests, lint, types, build)
- [ ] Version bumps look correct
- [ ] Release notes are accurate (edit in Notion if a draft was created)
---
> **Do not merge until CI is fully green.** The full test suite runs
automatically on this PR.
## Summary
- document the shell-docs hybrid docs architecture and `docs_mode`
meanings
- add a shell-docs README with npm local dev, validation commands, and
authoring recipes
- keep retired top-level `docs/` guidance intact while making shell-docs
the clear source of truth
## Verification
- `git diff --check -- .claude/docs/documentation.md
showcase/shell-docs/README.md`
- pre-commit: `nx run-many -t test --projects=packages/**` passed after
rerun; Nx flagged `@copilotkit/react-core:test` as flaky from an earlier
timeout
- pre-commit: `nx run-many -t publint,attw --projects=packages/**`
passed
## Notes
- Archive refs were created before this branch:
`archive/docs-save-do-not-prune` and `archive/docs-retired-2026-06-17`.
- This PR intentionally does not delete the retired top-level `docs/`
tree; that should be a follow-up cleanup PR.
examples/slack now starts a Slack bot and/or a Telegram bot from one platform-neutral app layer, env-conditional on which credentials are set. Neutralized Slack-specific rendering so the shared components work on both platforms: unicode glyphs instead of mrkdwn shortcodes, no Block Kit raw fallbacks, neutral context/tool wording. Migrated the Telegram e2e smoke harness and BotFather setup docs; removed the separate examples/telegram app.
grammY-based adapter implementing the PlatformAdapter interface. Long-polling by default with webhook/auto ingress opt-in. Topic-aware hybrid threading. HTML rendering with a plain-text format fallback, chunked-edit streaming, callback_query HITL, and inbound file ingestion. Ships a built-in Telegram user-lookup tool and platform context, plus a full unit test suite.
Add display-only generative-UI the copilot renders on request:
- showPendingApprovals — the dashboard's interactive approval table, in chat
- showSpendingTrend, showBudgetUsage, showSpendBreakdown, showIncomeVsExpenses —
brand-styled charts (dependency-free, hand-rolled SVG/CSS)
- showApprovalFlow — a diagram of how an over-limit charge gets cleared
Registered globally in copilot-context (so they work on any route) and routed
via the agent prompt.
- Cards page: the credit-card grid compressed each card (wrapping the holder /
valid-thru text) when the chat panel narrowed the content area. It's now a
horizontal-scroll row — each card holds a 300px minimum and grows to fill on
wide screens, scrolling instead of squishing.
- overflow-x:auto forces overflow-y:auto, which clipped the cards' soft drop
shadows; matching negative-margin/padding pairs give the shadows room inside
the scroll viewport without moving the cards.
- Chat: space stacked HITL/action cards so consecutive cards no longer share an
edge (CopilotKit stacks them with no row-gap).
## Summary
Harden the showcase harness producer against transient Railway-GQL 429 /
Cloudflare-WAF flaps. Today's incident: a ~25-min Cloudflare WAF
burst-block on `backboard.railway.com/graphql/v2` caused the producer's
catalog-enumerate to hard-fail every cron tick, zeroing out D4/D5/D6
writes and turning the entire staging dashboard red within one tick
window.
Three discrete behavior changes, one PR:
1. **Retry with exponential backoff in `source.enumerate`** — three
retries at 1s/4s/16s on HTTP 429, 5xx, Cloudflare 1015/1020/1022
markers, or transport-level errors. Does NOT retry on
`DiscoverySourceAuthError`, non-429 4xx, or schema errors
(operator-actionable, fail loud). Lives in
`showcase/harness/src/fleet/control-plane/catalog-enumerator.ts` (the
seam every family enumerator passes through).
2. **Cached-catalog fallback** — per-enumerator in-memory cache of the
last successful `services[]`. On persistent failure (all retries
exhausted) AND a cache present, the wrapper logs
`fleet.producer.enumerate-failed-using-cache` (warn, with `services`
count, `ageMs`, and `reason`) and returns the cached catalog. With NO
cache (fresh-boot first enumerate fails), the wrapper re-throws so the
producer's `enumerate-failed` short-circuit still runs — without a
catalog there's nothing to enqueue.
3. **3-tick family-silence threshold** —
`SILENCE_CONSECUTIVE_TICK_THRESHOLD = 3` layered ON TOP of the existing
`3 × period` elapsed-time gate. The silence alert now requires BOTH:
`now - lastSuccessAt > 3 × period` AND three consecutive evaluation
cycles observed silent. A single bad tick on a stale `lastSuccessAt` can
no longer page every family at once.
## Red-Green proof
**RED on main (`5a62acbf`)** — observed BEFORE the fix. The RED file
asserts the BUG (`calls === 1` after a 429 throw; `posts.length === 1`
after a single silent tick):
```
> vitest run src/fleet/control-plane/red-baseline-railway-gql-resilience.test.ts
RUN v3.2.4 .../showcase/harness
✓ src/fleet/control-plane/red-baseline-railway-gql-resilience.test.ts (2 tests) 3ms
✓ [BUG] one 429 + Cloudflare 1015 from the source aborts the whole enumerate (no retry)
✓ [BUG] silence alert posts on the FIRST silent evaluation tick (no consecutive-tick threshold)
Test Files 1 passed (1)
Tests 2 passed (2)
```
**GREEN on fix branch** — observed AFTER the fix (assertions inverted to
the fixed behavior; `calls === 4` after retries; `posts === []` until
the third silent tick):
```
> vitest run src/fleet/control-plane/red-baseline-railway-gql-resilience.test.ts src/fleet/control-plane/catalog-enumerator.test.ts src/fleet/control-plane/family-silence-monitor.test.ts
RUN v3.2.4 .../showcase/harness
✓ src/fleet/control-plane/red-baseline-railway-gql-resilience.test.ts (2 tests) 4ms
✓ src/fleet/control-plane/family-silence-monitor.test.ts (19 tests) 8ms
✓ src/fleet/control-plane/catalog-enumerator.test.ts (29 tests) 6ms
Test Files 3 passed (3)
Tests 50 passed (50)
```
Full harness suite — **131 files / 2812 tests pass** (`pnpm -F
@copilotkit/showcase-harness test`).
## Test plan
- [x] RED proof captured on `main` (single-attempt enumerate;
single-tick silence alert)
- [x] GREEN proof on this branch (retry to 4 calls; cached fallback;
3-tick threshold)
- [x] `pnpm -F @copilotkit/showcase-harness test` — 2812 passed
- [x] `pnpm -F @copilotkit/showcase-harness typecheck` — clean
- [x] `pnpm -F @copilotkit/showcase-harness build` — clean
- [x] `oxfmt --write` applied; `oxlint` clean
- [ ] CI green
- [ ] Staging redeploy verified via Railway CLI + Playwright dashboard
snapshot
## Files touched
- `showcase/harness/src/fleet/control-plane/catalog-enumerator.ts`
(+296, -3): retry+cache wrapper, exports `ENUMERATE_RETRY_BACKOFF_MS` +
`isRetryableEnumerateError` + `SleepFn`
- `showcase/harness/src/fleet/control-plane/catalog-enumerator.test.ts`
(+295): retry/cache/auth-not-retried/backoff-SSOT tests
- `showcase/harness/src/fleet/control-plane/family-silence-monitor.ts`
(+61): `SILENCE_CONSECUTIVE_TICK_THRESHOLD = 3`, per-family counter,
counter reset on healthy
-
`showcase/harness/src/fleet/control-plane/family-silence-monitor.test.ts`
(+218, -39): 3-tick threshold + counter-reset gate tests; updated
existing tests to advance through 3 silent ticks
-
`showcase/harness/src/fleet/control-plane/red-baseline-railway-gql-resilience.test.ts`
(new, +212): the literal RED→GREEN gate
## Operational notes
- No new env vars, feature flags, or backward-compat shims (per scope
directive).
- The cached-catalog warn surfaces in observability via
`fleet.producer.enumerate-failed-using-cache` (services count, ageMs,
reason).
- `SLACK_WEBHOOK_OSS_ALERTS` is intentionally unset (user config); not
touched.
Pre-existing repo-wide lefthook failures (`@copilotkit/core`,
`@copilotkit/runtime`, `@copilotkit/shared` etc.) reproduce on `main`
without my changes and are unrelated to harness code; harness-scoped
quality gates all passed before commit.
## Summary
Ports
[ag-ui-protocol/ag-ui#1914](https://github.com/ag-ui-protocol/ag-ui/pull/1914)
to CopilotKit — plus the two supporting guard files ag-ui already had:
- **`.github/workflows/canary.yml`** — discoverable **`canary /
publish`** `workflow_dispatch` orchestrator. Any maintainer can publish
a prerelease of the branch they're on straight from the Actions tab. It
is a thin orchestrator — it does **not** publish to npm itself:
1. Guards against `main` and non-branch refs.
2. Mints the devops-bot App token (app-id `1108748`,
`DEVOPS_BOT_PRIVATE_KEY`) with scoped `contents:write` +
`actions:write`.
3. Mirrors the dispatched ref to a unique
`canary/<slug>-<run_id>-<attempt>` branch via the GitHub API (no
checkout).
4. Dispatches **`publish-release.yml --ref canary/<slug> -f
mode=prerelease …`**, locates the run, and waits (`gh run watch
--exit-status` + explicit conclusion check).
5. Deletes the canary ref — status-gated (never yanks the ref under a
still-running delegated run) with a fresh cleanup token (90-min job
ceiling exceeds the 1h App-token TTL).
- **`scripts/release/verify-release-scope-dropdowns.sh`** — drift guard:
the hand-maintained `scope` dropdowns in `publish-release.yml` /
`stable-release.yml` / `canary.yml` must exactly match
`release.config.json`'s `.scopes` keys. Parsers fail loud and distinct
on structural changes instead of silently passing.
- **`.github/workflows/lint-release-workflows.yml`** — actionlint +
shellcheck + the dropdown-sync job over the release pipelines.
### Why a separate orchestrator (and not a flag in publish-release.yml)
- A GitHub Environment's deployment-branch policy is evaluated against
the ref a run is **triggered on** — not branches created mid-run. The
orchestrator exists to get the publish run *onto* a `canary/*` ref.
- `publish-release.yml` holds the **single npm OIDC trusted-publisher
binding**; a second publishing entry point would break OIDC for every
`@copilotkit/*` package. The orchestrator never touches npm.
- The cross-workflow dispatch uses the **App token, not `GITHUB_TOKEN`**
— `GITHUB_TOKEN`-authenticated events never start new workflow runs.
**Note:** the `npm` environment currently has *no* deployment-branch
policy, so the orchestrator is a convenience wrapper today. Tightening
the policy to `main` + `canary/*` + `release/publish/*` (matching
ag-ui's security posture) is being applied as repo configuration
alongside this PR — requires admin. This PR includes the prerequisite:
`publish-commit.yml` (pkg-pr-new) is removed from the `npm` environment,
since it runs on every PR and would be blocked by the policy (it
publishes to pkg.pr.new, not npm, and uses no environment secrets).
## Testing done
- Drift guard: positive run against all three real workflows; negative
tests (scope removed → drift FAIL with diff; bogus scope → FAIL; `case
"${SCOPE}"` quoting refactor → loud parser-degradation FAIL; whole case
block deleted → loud zero-block FAIL; quoted arm `"angular")` →
accepted; blank/comment lines inside `options:` → still parsed; prose
comments mentioning case/SCOPE/in → no false positive).
- `shellcheck` clean at all severities; `bash -n` on every workflow
`run:` block; YAML parses.
- 3 rounds of 7-agent code review converged to zero load-bearing
findings.
## ⚠️ Still to verify before first real use
- [ ] devops-bot App (id 1108748) has **Actions: write** — required for
the in-workflow `gh workflow run`. Safe first test: dispatch once with
`dry_run=true`.
- [ ] First `dry_run=false` run clears the `npm` environment end-to-end
via the App token once the deployment-branch policy is tightened.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
## Post-merge follow-ups (maintainer action required)
These need repo **admin** rights and must happen **in this order**:
1. **Merge this PR first.** `main`'s current `publish-commit.yml`
(pkg-pr-new) still sits in the `npm` environment and runs on every PR
touching `packages/**` — applying the branch policy before this PR lands
would block every snapshot publish. This PR removes that environment
association.
2. **Tighten the `npm` environment's deployment-branch policy** to
`main` + `canary/*` + `release/publish/*` (matching ag-ui). With an
admin-scoped token:
```bash
gh api --method PUT repos/CopilotKit/CopilotKit/environments/npm \
-F "deployment_branch_policy[protected_branches]=false" \
-F "deployment_branch_policy[custom_branch_policies]=true"
gh api --method POST
repos/CopilotKit/CopilotKit/environments/npm/deployment-branch-policies
-f name="main" -f type=branch
gh api --method POST
repos/CopilotKit/CopilotKit/environments/npm/deployment-branch-policies
-f name="canary/*" -f type=branch
gh api --method POST
repos/CopilotKit/CopilotKit/environments/npm/deployment-branch-policies
-f name="release/publish/*" -f type=branch
```
Or via UI: Settings → Environments → npm → Deployment branches and tags
→ "Selected branches and tags" → add the three patterns above.
Why these three: `main` covers stable `workflow_dispatch` retries and
`stable-release.yml`; `release/publish/*` covers the merged-release-PR
runs (the run's head branch is the release PR branch); `canary/*` covers
the orchestrator's delegated prerelease runs. After this, direct
`mode=prerelease` dispatches from arbitrary feature branches stop
working — the `canary / publish` orchestrator becomes the one-click path
(by design).
3. **Verify the devops-bot App (id `1108748`) has `Actions: write`**
(org/App settings). The orchestrator's `gh workflow run` dispatch 403s
without it. Safe end-to-end test, after step 2: Actions tab → **canary /
publish** → pick any feature branch, any scope, **`dry_run=true`** →
confirm the delegated `release / publish` run is created, watched, and
the `canary/*` ref is deleted afterward.
4. **First real canary** (`dry_run=false`) confirms the npm OIDC publish
clears the environment gate end-to-end on a `canary/*` ref.
Adds an integration-style test file that pins the BOTH layers of the
2026-06-17 Cloudflare-WAF-burst incident fix together:
- the enumerator retries 3× on a 429+Cloudflare-1015 burst before
bubbling (the original bug let one 429 abort the whole enumerate);
- the silence monitor requires THREE consecutive silent evaluation
cycles before posting an alert (the original bug fired on tick #1).
These assertions were the LITERAL red proof on `main`:
- on main both `it` blocks PASSED while asserting the buggy behavior
(calls === 1, posts.length === 1 after a single silent tick),
- on this branch the same gates re-pin the fixed behavior (calls === 4
after retries; posts === [] until the third silent tick).
Run-time output captured for the PR body confirms the inversion.
Layer a per-family consecutive-silent-tick counter ON TOP of the existing
3×period elapsed-time gate (`SILENCE_PERIOD_MULTIPLIER`). The silence
alert now requires BOTH:
- `now - lastSuccessAt > 3 × period` (existing elapsed-time gate), AND
- `SILENCE_CONSECUTIVE_TICK_THRESHOLD = 3` consecutive evaluation cycles
observed silent (new — the counter resets on any successful evaluation).
Without the new gate a single bad cron tick on a family whose
`lastSuccessAt` was already stale (e.g. after a long quiet window or a
deploy gap) tripped the alert immediately — the failure mode the
2026-06-17 Cloudflare-WAF-burst incident exposed where one ~25 min flap
on backboard.railway.com/graphql/v2 paged every family at once.
The counter is NOT incremented during boot grace, so a cold-start cycle
can't alone push it to threshold. The meta-alert (`family-silence-eval`)
path keeps its own clock and is unaffected. Existing tests advance
through three consecutive silent ticks before asserting the post.
Three-retry exponential backoff (1s/4s/16s) on `source.enumerate` against
Railway-GQL when the underlying error is transient (HTTP 429, 5xx, or a
Cloudflare 1015/1020/1022 WAF marker, or a transport-level reject). On
persistent failure, fall back to the last successful catalog from a
per-enumerator in-memory cache — LOUDLY logged via
`fleet.producer.enumerate-failed-using-cache` so the cache-use shows up
in observability.
A fresh-boot process with no cached entry preserves the current
hard-fail behavior (the producer's `enumerate-failed` short-circuit) —
without a catalog there is nothing to enqueue. Real config errors
(`DiscoverySourceAuthError`, non-429 4xx, schema rot) are NOT retried so
operator-actionable failures surface immediately.
Context: 2026-06-17 Cloudflare WAF burst-blocked
backboard.railway.com/graphql/v2 for ~25 min, hard-failing the producer
enumerate on every cron tick and zeroing out D4/D5/D6 writes — the
entire staging dashboard went red within one tick. Retries + cache ride
out the burst on the same tick and preserve job production across
longer outages.
Pre-existing repo-wide lefthook test failures (@copilotkit/core,
@copilotkit/runtime, @copilotkit/shared, etc.) are unrelated to the
harness; verified by stashing my changes and running the same hook on
main with identical failures. Harness suite (131 files / 2812 tests),
typecheck, and build pass on this branch.
- webhook: don't let GET / health shadow the verify handshake when WHATSAPP_PATH=/
- adapters: fail loud on a malformed PORT instead of binding NaN
- tests: cover the id-only senderContext branch, the 404-regression for the
health route, and the PORT guard
- docs: update README senderContext snippets to the 2-arg signature
Adds a cookbook recipe + runnable showcase that gives the Built-in Agent
OAuth-backed Arcade tools (Gmail, Google News) and renders Arcade's one-time
authorization step as a generative-UI "Connect" card in the chat.
- docs: showcase/shell-docs/src/content/docs/cookbook/arcade.mdx (+ meta.json, index card)
- app: examples/showcases/arcade-tools (Next.js App Router, single-route runtime)
Co-authored-by: Cursor <cursoragent@cursor.com>
- Add control-plane-run.test.ts (17 tests) covering buildLocalServicesJson,
expectedKeys, dedupeScopes, and runViaControlPlane error surfacing
- Export SlugScope, buildLocalServicesJson, expectedKeys, dedupeScopes
for unit-test coverage (factored inline dedup loop into dedupeScopes
helper at the same time)
- runViaControlPlane: surface scopeLabel (demo-aware) in the 0-enqueue
error instead of the bare-slug join, with an empty-targets guard so
the error never renders with a double-space gap
- runViaControlPlane: treat tick.enqueueFailures > 0 as fatal — partial
enqueue used to silently proceed and either mask missing cells or
hang the poll loop to timeout
- Eliminate a stray literal NUL byte in the source by switching the
dedup key separator to a \x00 escape
- lifecycle.up(): name the compose call (infra-up vs target rebuild)
in the health-fail error so an operator can tell which call left a
service unhealthy
(cherry picked from commit 9f35c64adfdf7f5ff2bf0a5ae57ed03818cea607)
CR Finding 1 (BLOCKER): d5-tool-rendering-default-catchall.ts used the
broken page.evaluate(fn, arg) second-arg form to pass the leak-phrase
needle into the browser-side closure. A11 proved empirically that the
arg arrives as undefined inside the closure, making 'if (needle)' guard
the entire leak-detection cascade as dead code — customLeakPhrasePresent
stayed false forever, rendering validateDefaultCatchall's leak branch
dead code as well. Mirrors the A11 fix on the sibling custom-catchall
probe by inlining the needle as a JS string literal inside the closure;
no page.evaluate(fn, arg) dependency at all. Both probes now share the
same inline-needle pattern and keep the canonical literal in lock-step
with their exported phrase constant.
CR Finding 2 (MAJOR): the A11 inline-needle fix on the sibling
custom-catchall probe had no regression test — fake Page.evaluate in
makePageReturning never executes the probe closure, so reverting the
fix would not be caught. Added regression tests that capture the
probe's function source via toString() and assert (a) the canonical
phrase appears as a literal inside the page.evaluate(...) closure and
(b) the closure takes no parameter / the evaluate call has no
second arg. Added the same coverage to default-catchall to protect
the new A25a fix.
CR Finding 3 (MAJOR): A7's requireContentPhrase=true branch in
validateCustomCatchall / assertCustomCatchall had zero coverage —
tests omitted the third arg and exercised only the default false
branch. Added coverage for the true branch (pass on phrase present,
fail on phrase absent, fail on phrase undefined, default-branch
preserved) plus assertCustomCatchall plumbing through the options
form. Also added coverage for default-catchall's customLeakPhrasePresent
branch in validateDefaultCatchall for symmetry.
Local proof:
- RED (fix reverted via git stash): 2 inline-needle regression tests
fail on d5-tool-rendering-default-catchall.test.ts
- GREEN (fix restored): 34/34 tests pass across both files
Out of scope (NOT touched this commit): showcase/harness/src/cli/
control-plane-run.ts and lifecycle.ts (A25b's scope).
(cherry picked from commit c409e3a8d99e16ad0bb05ee3c2e2051e5792049f)
github-actions[bot]