mirror of
https://github.com/CopilotKit/CopilotKit.git
synced 2026-09-14 16:26:20 +08:00
showcase-previews
6618 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Jordan Ritter
|
ed4789afae |
ci: isolate GHCR error output from API body parsing
`gh api -i 2>&1` merged stderr into stdout, so gh diagnostics (auth failures, rate limits, network errors) could be spliced ahead of the HTTP header block. That corrupted HTTP_STATUS/API_BODY parsing — a failing drift run would look clean while every call actually failed. Redirect stderr to a mktemp file instead; surface it only when `gh` returns a non-zero RC with no HTTP status (genuine connectivity/binary failure). Normal HTTP error paths (401/403/404/5xx) continue to parse stdout as the canonical header+body response. |
||
|
Jordan Ritter
|
0c92c3bb36 |
ci: make smoke-monitor starter check robust to non-starter count
Replaces the magic `-eq 19` sentinel that assumed a fixed non-starter count with a `grep -c '^starter-'` check that counts only the appended starter entries. Any future add/remove on the literal non-starter list would have silently disabled the previous guard. Also enables bash `nullglob` around the `showcase/starters/*/` loop so an empty/missing starters tree doesn't expand to the literal pattern and corrupt SERVICES with a bogus `starter-*` entry. |
||
|
Jordan Ritter
|
ddf2b63c59 |
ci: source starter slug list from filesystem in smoke-monitor
The SERVICES=(...) array in showcase_smoke-monitor.yml's 'Check image drift' step was a hardcoded copy of the 17 starter slugs already declared by showcase/starters/*/ directory names. Every new starter required a manual edit in three places; the parity validator caught drift after the fact but couldn't prevent it. This commit: - Adds a sparse actions/checkout step for showcase/starters/ only. - Replaces the literal starter-* entries with a filesystem enumeration (for dir in showcase/starters/*/; do ... done), skipping template/. - Fails loudly if the enumeration produces zero starters, so a broken checkout can't silently under-check drift. - Updates validate-workflow-starters.ts to drop the smoke-monitor check (drift is now structurally impossible) while keeping the two remaining literal-list checks against showcase_deploy.yml (workflow_dispatch options must be literal pre-checkout; ALL_SERVICES matrix carries per-starter deploy metadata like railway_id). Non-starter services stay literal — they don't live under showcase/starters/ and are provisioned differently. |
||
|
Jordan Ritter
|
9c42ec0813 |
ci: add tsconfig for showcase/scripts to fix LSP type resolution
The scripts under showcase/scripts/ run via tsx at runtime and had no
tsconfig.json, which left LSP falling back to no-config defaults and
flagging bogus errors ("Cannot find module 'fs' / 'path' / 'url'",
"Cannot find name 'process'"). Adds a minimal tsconfig that matches
tsx's runtime semantics: bundler resolution (so existing extensionless
relative imports keep working), ES2022 + DOM libs, Node types, strict
discriminated-union narrowing. Excludes __tests__/ to avoid surfacing
pre-existing type issues outside this PR's scope.
|
||
|
github-actions[bot]
|
3a22cbcb82 | style: auto-fix formatting | ||
|
Jordan Ritter
|
ecce840e2b |
ci: enforce starter list parity across workflows
The starter slug list is duplicated across at least three places: - .github/workflows/showcase_deploy.yml workflow_dispatch options - .github/workflows/showcase_deploy.yml ALL_SERVICES matrix entries - .github/workflows/showcase_smoke-monitor.yml SERVICES bash array Adding a new starter under `showcase/starters/` but forgetting any of these leaves the service deployable in theory but invisible to the dispatch UI and/or drift detection — exactly the failure mode this PR is trying to close. Add `showcase/scripts/validate-workflow-starters.ts`. It enumerates every directory under `showcase/starters/` (excluding `template/`) and confirms `starter-<slug>` is present in each of the three workflow locations, emitting a precise "missing from: <source>" diagnostic per gap. Wire it into showcase_validate.yml right after `validate-parity` so it gates every PR + main push touching `showcase/**` or the relevant workflow files. Also extend that workflow's `on.paths` filter to include `showcase_deploy.yml` and `showcase_smoke-monitor.yml` so edits to those files trigger the parity check. |
||
|
Jordan Ritter
|
820e3493ac |
ci: surface GHCR errors in drift detector
The drift detector queried GHCR with `gh api ... || true` and then
skipped the service whenever `$TAGS` was empty. That collapsed three
very different outcomes into one silent continue:
- 200 OK with no published versions yet (legitimate, quiet)
- 404 for a brand-new service (legitimate, quiet)
- 401/403/5xx transient error (NOT legitimate — looks clean and
hides a broken drift check from operators)
Capture the HTTP status via `gh api -i`, route 200-empty and 404 to
a quiet continue with a log line, and emit `::warning::` for any
other status so operators see the problem in the run UI without
failing the whole drift run over one flaky service.
Also tighten the SHA match from substring `grep -q "$SHA"` to word-
boundary `grep -qw "$SHA"`. The previous pattern could false-positive
if one SHA happened to be a prefix of another tag on the same image
(extremely unlikely at 40 chars, but the SHA-8 abbreviation shows up
in tags like `branch-<8char>` which is the failure mode we avoid by
anchoring).
|
||
|
Jordan Ritter
|
583db441ab |
ci: retry shell health check before failing
`shell` and `shell-dojolike` use `health_path="/"` because neither ships an `/api/health` endpoint — they're Next.js app shells serving the homepage as the liveness signal. With the new `exit 1` on persistent unhealth, a single transient 5xx or 301-chain glitch at the homepage during a cold-start would reset the HEALTHY_STREAK and, at the edge of the 24-attempt budget, could page us for nothing. Wrap the HTTP probe in a 3-attempt inner retry (2s apart) before the streak logic observes the code. Services whose homepage is genuinely dead still fail — all 3 tries must return non-200 — but a one-off blip no longer trashes accumulated progress. Non-shell services (which hit `/api/health`, `/health`, etc.) benefit from the same transient-tolerance for free. |
||
|
Jordan Ritter
|
c6ceb27b25 |
ci: distinguish partial from full failure in showcase deploy alert
The notify job's failure branch posted "FAILED — N service(s) targeted"
whenever `needs.build.result == 'failure'`, which rolls up to 'failure'
if even a single matrix leg failed. With fail-fast: false, one failed
leg out of 18 can roll up as 'failure' while 17 others shipped fine —
the old wording falsely implied all N failed.
Add a "Compute per-leg build results" step that queries the Actions
API for this run's jobs, buckets each `build (<leg>)` leg by its
conclusion, and exposes failed/succeeded counts + lists as outputs.
The payload step now picks between three shapes:
- Partial failure: "N/M service(s) failed (<failed>) — <succeeded> ok"
- Full failure: existing "FAILED — M service(s) targeted (<list>)"
- API lookup lost: softer "1+ of M service(s) failed (<list> targeted)"
so the alert never lies about how many were affected.
Also reword the stale "sleep-on-idle waking up" comment on the
success-but-no-HTTP-200 branch — Railway is on the Pro tier (see
rationale near line 412) and does not sleep on idle. Clarify the
notify cancel-handling comment accordingly.
|
||
|
Jordan Ritter
|
7f2a83f41e |
ci: fail showcase deploy on unhealthy service
Previously, when a deployed service never returned a 200 on its health path within the 360s window, the workflow emitted `::warning::` and exited 0. That masked silently-broken deploys: Slack saw green, smoke monitor saw no failing deploy run, and the stale image sat live. Emit `::error::` and exit 1 so the deploy job fails and the alert actually fires. Railway is on the Pro tier with no sleep-on-idle, so a persistent failure to become healthy in 6 minutes is a real failure, not a cold start. Evidence of silent failures: https://github.com/CopilotKit/CopilotKit/actions/runs/24616467033 |
||
|
Jordan Ritter
|
49f834d94f |
ci: include starter services in drift detector
The smoke monitor's SERVICES array only listed package slugs (mastra, llamaindex, ...) and did not include any starter-* slugs. As a result, GHCR showcase-starter-<svc> tags were never checked for drift and the monitor never dispatched showcase_deploy.yml for starter services when their images fell behind main. showcase_deploy.yml already accepts starter-* dispatch names and already redeploys any service with a railway_id via serviceInstanceRedeploy, so simply extending SERVICES to include the 17 starter slugs is sufficient to close the loop. Evidence of the gap: https://github.com/CopilotKit/CopilotKit/actions/runs/24616467033 |
||
|
Jordan Ritter
|
02907c7024 |
fix(ci): repair starter-deployed-smoke workflow (test discovery + Slack alert) (#4079)
## Summary The `Starter Deployed Smoke Tests` workflow has been red on every run since 2026-04-18 (>15 consecutive failures on main). Two independent bugs. Failing run cited: https://github.com/CopilotKit/CopilotKit/actions/runs/24610503288 ### Commit 1 — `fix(showcase): gate starter smoke tests on integration.deployed` **Primary failure**: Playwright errored `No tests found.` against `--grep "@starter-health|@starter-agent|@starter-chat"`. Root cause: `integration-smoke.spec.ts` filtered starters with `i.starter?.deployed === true`, but the manifest→registry bundler (`bundle-demo-content.ts` + `generate-registry.ts`) doesn't propagate that sub-field from the source YAML manifests. Commit `14537d8f3` regenerated the registry and silently dropped all `starter.deployed` values — so the filter matched zero entries, the `Deployed Starters` describe block generated zero tests, and Playwright's `--grep` had nothing to match. Fix: gate on integration-level `i.deployed === true`, which is the single deployment flag the registry actually carries, and which the `INTEGRATIONS` array in the same file already uses. Both filters now anchor to the same source of truth. Validated locally: ``` $ cd showcase/tests && npx playwright test e2e/integration-smoke.spec.ts \ --grep "@starter-health|@starter-agent|@starter-chat" --list Total: 17 tests in 1 file ``` ### Commit 2 — `fix(ci): repair starter-deployed-smoke Slack alert payload` **Secondary failure** (masked by `continue-on-error: true`): the Slack step failed with `SlackError: Invalid input! Failed to parse file extension /tmp/tmp.XXX`. `slackapi/slack-github-action@v2.1.0` rejects `payload-file-path` values without a `.json`/`.yaml`/`.yml` extension, and `mktemp` produces extensionless files. Result: smoke results never reached #oss-alerts. Fix: switch to the inline `payload:` + `toJSON(format(...))` pattern from `showcase_validate.yml` (PR #4068, commit `d4e75958`). Same pattern; extracts failed starter slug + first error line and emits via `$GITHUB_ENV` so the notify step references them safely through `env.*`. Distinguishes four failure modes (missing_report, jq_parse_failed, no_failures_in_report, real failures) in the excerpt. `continue-on-error` and the tmp-file cleanup step are removed — no temp file to clean, and a genuine Slack failure should now surface in job status. ## Test plan - [ ] CI on this PR goes green (checkout, npm ci, playwright install, test discovery) - [ ] On the next failing smoke (scheduled or manual dispatch), verify: - [ ] #oss-alerts receives a single actionable message carrying failed-starter slug + first error line - [ ] No `Invalid input! Failed to parse...` error in the workflow log - [ ] On a successful smoke, confirm no spurious Slack message is posted 🤖 Generated with [Claude Code](https://claude.com/claude-code) |
||
|
Jordan Ritter
|
44622440fe |
feat(showcase): add local smoke target (aimock + 17 packages in Docker)
Wires up a single-command path to run the full integration smoke suite against a local Docker stack instead of Railway. Useful when Railway is degraded (OOM, rate limits) or when testing changes that haven't been deployed yet. Additions: - showcase/docker-compose.local.yml: add `aimock` as 18th service so integration containers can reach http://aimock:4010 on the compose network, mirroring the Railway setup where they call showcase-aimock. - showcase/tests/e2e/integration-smoke.spec.ts: `LOCAL_PORTS=1` env rewrites each integration's Railway URL to http://localhost:<port> via showcase/shared/local-ports.json. Starters are skipped under this flag because they aren't in local-ports.json. - showcase/scripts/smoke-local.sh: orchestrates build → up → wait → run Playwright → tear down. Supports --level=L1/L2/L3/L4, --keep, --no-build. - showcase/tests/package.json: `pnpm smoke:local[:L1|:keep|:nobuild]` scripts delegate to the helper. - showcase/.env.example: document optional OPENAI_BASE_URL + ANTHROPIC_BASE_URL (route through local aimock) and package-specific GitHubToken + GOOGLE_API_KEY (ms-agent-dotnet, google-adk). Verified locally: `pnpm smoke:local:L1` → 17/17 L1 green against the local stack. |
||
|
Mike Ryan
|
04deb142bb |
refactor(examples): Switch Threads to use GHCR (#4076)
<!-- Thank you for sending the PR! We appreciate you spending the time to work on these changes. Help us understand your motivation by explaining why you decided to make this change. **Please PLEASE reach out to us first before starting any significant work on new or existing features.** By the time you've gotten here, you're looking at creating a pull request so hopefully we're not too late. We love community contributions! That said, we want to make sure we're all on the same page before you start. Investing a lot of time and effort just to find out it doesn't align with the upstream project feels awful, and we don't want that to happen. It also helps to make sure the work you're planning isn't already in progress. As described in our contributing guide, please file an issue first: https://github.com/ag-ui-protocol/ag-ui/issues Or, reach out to us on Discord: https://discord.com/invite/6dffbvGU3D You can learn more about contributing to copilotkit here: https://github.com/copilotkit/copilotkit/blob/master/CONTRIBUTING.md Happy contributing! --> ## What does this PR do? (Describe the changes introduced in this PR) ## Related PRs and Issues - (Direct link to related PR or issue, if relevant) ## Checklist - [ ] I have read the [Contribution Guide](https://github.com/copilotkit/copilotkit/blob/master/CONTRIBUTING.md) - [ ] If the PR changes or adds functionality, I have updated the relevant documentation |
||
|
Mike Ryan
|
1fcd3f205a | refactor(examples): Switch Threads to use GHCR | ||
|
Jordan Ritter
|
a2cb28eda0 |
fix(ci): repair starter-deployed-smoke Slack alert payload
The Alert Slack step used `payload-file-path` pointed at an
`mktemp`-produced extensionless tmpfile. slackapi/slack-github-action
v2.1.0 requires the payload file to have a `.json` / `.yaml` / `.yml`
extension and rejects extensionless paths with:
SlackError: Invalid input! Failed to parse file extension /tmp/tmp.XXX
SlackError: Invalid input! Failed to parse contents of the provided payload file
This silent failure (masked by `continue-on-error: true`) meant smoke
results never reached #oss-alerts.
Switch to the inline `payload:` + `toJSON(format(...))` pattern from
showcase_validate.yml (PR #4068, commit
|
||
|
Jordan Ritter
|
f773c51491 |
fix(showcase): gate starter smoke tests on integration.deployed
The STARTERS filter in integration-smoke.spec.ts gated on
`i.starter?.deployed === true`, but the manifest→registry bundler
(showcase/scripts/bundle-demo-content.ts + generate-registry.ts)
does not carry that field forward from the source YAML manifests.
At registry regen time (commit
|
||
|
Jordan Ritter
|
d4e75958ef |
chore(showcase): include failed job/step + error excerpt in validate Slack alert (#4068)
## Summary - Bare `❌ Showcase validate: failed | View run` forces a click-through to triage. Red alerts must carry triage-ready detail per the oss-alerts policy. - Adds a pre-notify step that extracts the failed step name and first meaningful error line, then interpolates both into the Slack payload. - Companion to #4065 (which quieted routine success noise); this PR makes the remaining failure-only posts actionable at a glance. ## Before / after **Before** ``` ❌ Showcase validate: failed | View run ``` **After** (against tonight's run `24598654559`, the validate-parity regression) ``` ❌ Showcase validate: failed — Run validate-parity (MUST checks gating): [FAIL] ag2: demo 'hitl-in-chat' declared in manifest but no src/app/demos/hitl-in-chat/ directory | View run ``` ## Extraction approach New `Extract failure details for Slack` step (runs only on `failure() && push && env.SLACK_WEBHOOK != ''`): 1. **Resolve current job ID** via `GET /repos/{repo}/actions/runs/{run_id}/jobs`. Match by `name == "Validate Showcase"` (primary) with a fallback to the first job containing a failed step (rename-drift tolerance). 2. **First failed step name** pulled from the same jobs response. 3. **First error line** via `gh run view --log-failed --job=<id>`. Pipeline: - `awk -F'\t'` to split TSV columns (`job \t step \t timestamp + content`) and strip the leading ISO timestamp + optional BOM from the content column. - `sed` strips ANSI escape codes. - `grep -vE` skips runner header noise (`##[group]`, `shell:`, `env:`, `Run`, blank lines). - `grep -m1 -E` grabs the first line matching `[FAIL]`, `[ERROR]`, `Error:`, `error:`, or `::error`. - `head -c 300` truncates to keep the payload under the ~800-char Slack budget even after JSON escaping. 4. **Writes `failed_step` + `error_excerpt` to `$GITHUB_ENV`** using heredoc delimiters (safe for values containing `=` or newlines). 5. **Never blocks the notify step**: `set +e`, stderr suppressed on all extraction commands, explicit `exit 0`, and safe fallbacks (`"unknown step"` / `"see workflow run for details"`). The notify step's payload now uses `toJSON(format(...))` around all four interpolated values (repo, run_id, failed_step, error_excerpt) so any quotes/backslashes/newlines in extracted content are JSON-encoded defensively — matches the pattern from `showcase_drift-report.yml`. ## Validation - `actionlint .github/workflows/showcase_validate.yml` — no findings in the edited region (lines 360-470). The only reported finding is the pre-existing `depot-ubuntu-24.04-4` label info on line 47, unrelated. - Dry-run of the extraction pipeline against run `24598654559`: ``` gh run view 24598654559 --repo CopilotKit/CopilotKit --log-failed --job=71933395025 \ | awk -F'\t' 'NF>=3 { sub(/^[\xEF\xBB\xBF]?[0-9T:.\\-Z ]+/, "", $3); print $3 }' \ | sed 's/\x1b\[[0-9;]*[a-zA-Z]//g' \ | grep -vE '^(##\[|shell: |env: |Run |[[:space:]]*$)' \ | grep -m1 -E '^\[(FAIL|ERROR)\]|^Error:|^error:|^::error' \ | head -c 300 ``` Output: ``` [FAIL] ag2: demo 'hitl-in-chat' declared in manifest but no src/app/demos/hitl-in-chat/ directory ``` Which is exactly the signal that was missing from tonight's bare alert. - `oxfmt --check` and `oxlint` on the edited file: clean. ## Scope - Only `.github/workflows/showcase_validate.yml` is touched. `showcase_deploy.yml` and `showcase_smoke-monitor.yml` are untouched. - Preserves the empty-webhook guard (`env.SLACK_WEBHOOK != ''`) added in #4065 and mirrors that pattern on the new extraction step. ## Test plan - [ ] Merge and wait for the next validate failure on `main` (or temporarily force one on a push-path) to confirm the Slack payload carries the expected `— <step>: <excerpt>` suffix. - [ ] If a subsequent run fails in a step with no matching error marker, confirm the fallback `"see workflow run for details"` renders cleanly rather than an empty string. |
||
|
Jordan Ritter
|
9559e15c04 |
chore(showcase): include failed job/step + error excerpt in validate Slack alert
Bare "❌ Showcase validate: failed | View run" forces a click-through to triage — the signal that would make the alert actionable is one click away. Per the oss-alerts policy, red alerts must carry triage-ready detail in the payload itself. Adds a pre-notify step that (on failure(), push, webhook set): - resolves the current job's ID via the runs/{id}/jobs API (matches by job name; falls back to "first job with a failed step" for rename drift tolerance) - extracts the first failed step name from the same jobs response - pulls the failed-step log via `gh run view --log-failed --job=<id>`, strips TSV prefix + leading timestamp + ANSI codes, skips runner header noise (##[group], shell:, env:, Run), then grabs the first line matching [FAIL]/[ERROR]/Error:/::error and truncates to 300 chars - emits failed_step and error_excerpt to $GITHUB_ENV via heredoc delimiters (safe for values containing = or newlines) - uses set +e and explicit exit 0 so extraction glitches never block the notify step — fallbacks yield "unknown step" / "see workflow run for details" Updates the Slack payload to interpolate both values via toJSON(format()) so dynamic content is JSON-encoded defensively (matches the pattern from showcase_drift-report.yml). Preserves the ❌ prefix, the "View run" link, and keeps total length well under the 800-char budget. Companion to #4065, which quieted routine per-run success posts; this PR makes the remaining (failure-only) posts actionable at a glance. Against run 24598654559 (tonight's validate-parity regression), the extracted excerpt is: [FAIL] ag2: demo 'hitl-in-chat' declared in manifest but no src/app/demos/hitl-in-chat/ directory which is exactly the triage signal that was missing from the bare "failed" alert. |
||
|
Jordan Ritter
|
1633d9ff18 |
chore(showcase): ratchet validate-pins baseline 111→109 (#4078)
## Why `showcase/scripts/fail-baseline.json` drifted **below** the actual FAIL count: real pin-fix PRs have landed on `main` since the baseline was last refreshed, reducing the validate-pins FAIL set from 111 to 109 and invalidating the stored hash. The ratchet gate now rejects every PR (including ones that don't touch pins, like #4068) with a "ratchet down" instruction. This is a pure **ratchet-down to match reality** — no policy change, no validator/workflow change, no actual pin fix. ## What One-line update to `showcase/scripts/fail-baseline.json`: | field | before | after | |---|---|---| | `validatePinsFailCount` | 111 | **109** | | `validatePinsFailHash` | `77b586b7…b474e03` | **`d03716b5…f597e81d`** | Values computed by running the canonical pipeline from `.github/workflows/showcase_validate.yml` ("Run validate-pins (ratchet)" step) against a clean `origin/main` worktree: ```bash pnpm exec tsx showcase/scripts/validate-pins.ts 2> stderr grep ^Summary stdout # -> Summary: OK=0 SKIP=5 WARN=17 FAIL=109 grep '^\[FAIL\]' stderr | LC_ALL=C sort -u | shasum -a 256 # -> d03716b5746ac43cd8d711fd51e866f92717e0c4abc5a99221f0c7f9f597e81d ``` Matches CI run `24608764563` (PR #4068), which observed the same `FAIL=109` from a clean rebase onto `origin/main`. ## Unblocks - #4068 (`chore/oss-alerts-validate-detail`) — currently failing `Validate Showcase` purely on this ratchet, despite only touching workflow files. - Any other PR blocked on the same gate. ## Not in scope - Fixing any of the 109 remaining pin-drift FAILs. That's a separate, larger effort (tracked under issue #4047). ## Test plan - [ ] `Validate Showcase` check goes green on this PR on first run. |
||
|
Jordan Ritter
|
5f6a864922 |
chore(showcase): ratchet validate-pins baseline 111→109 (fixes landed on main)
Real pin-fix PRs have landed on `main` since the baseline was last
refreshed, reducing the FAIL set from 111 to 109 and invalidating the
stored hash. The ratchet gate now rejects every PR (including ones
that don't touch pins) with a "ratchet down" instruction.
Refresh the baseline to reflect the actual state of `origin/main`:
validatePinsFailCount: 111 -> 109
validatePinsFailHash: 77b586b7 -> d03716b5
Values computed by running the canonical pipeline from
`.github/workflows/showcase_validate.yml` ("Run validate-pins (ratchet)"
step) against a clean `origin/main` worktree:
pnpm exec tsx showcase/scripts/validate-pins.ts 2> stderr
grep ^Summary stdout -> FAIL=109
grep '^\[FAIL\]' stderr | LC_ALL=C sort -u | shasum -a 256
-> d03716b5...f597e81d
This is a pure ratchet-down to match reality, not a policy change.
No validator behavior, workflow, or pin change is included. Actual
pin drift cleanup (109 -> 0) continues as a separate effort.
Unblocks #4068 and any other PR stalled on the same ratchet.
|
||
|
Atai Barkai
|
163bfe8263 |
feat(showcase/shell-internal): variant presentations at 5 experimental routes
To support exhaustive E2E testing via multiple variants per feature ×
framework, extend the status model with an optional `variants[]` array
per demo (each variant has the same demo/code/E2E/Smoke/QA/health
breakdown) and mount five different visual treatments so we can compare
side-by-side before committing to one:
- `/variants-stack` — each variant rendered as its own mini-row
in the cell; tall cells, all info visible.
- `/variants-tabs` — tabs at the cell top, click to switch
variant; cell stays compact.
- `/variants-aggregate` — pass/total rollups per signal +
"N variants ▾" expand button to drill down.
- `/variants-grid` — mini-matrix: rows = variants, cols =
demo/code/E2E/Smoke/QA/health.
- `/variants-strip` — one colored chip per variant per signal;
hover chip for variant name, click for URL.
Refactor: the grid chrome moves to `components/feature-grid.tsx`
(accepts a `renderCell` callback). Main `/` keeps the existing
single-variant layout via `components/cell-single.tsx`. Shared badge /
links helpers live in `components/badges.tsx` and
`components/variant-pieces.tsx`.
Mock variant data is seeded on 4 demos (langgraph-python's
agentic-chat, gen-ui-tool-based, hitl-in-chat; langgraph-typescript's
agentic-chat) so each option shows variants in context alongside
ordinary cells.
Variant-specific deep links append `?variant=<name>` to the shell
preview / code / hosted URLs — the shell routes can pick that up
later to highlight variant-specific files or payloads.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Atai Barkai
|
6e16befaff |
feat(showcase): wire real health probes + clickable health badge
- New `scripts/generate-status.ts` — probes every `integration.backend_url + demo.route` in parallel (~166 URLs) and writes health status per demo to `shell/src/data/status.json`. E2E, Smoke, and QA stay mock with explicit `TODO(wire-*)` comments; real readers for those ingest from `showcase_aimock-e2e.yml`, `showcase_smoke-monitor.yml`, and `showcase_qa-sync.yml` later. `GENERATE_STATUS_MOCK_HEALTH=1` offline override for dev. - Health badge in the feature-matrix cell is now clickable — opens the hosted URL (`integration.backend_url + demo.route`) in a new tab, with a tooltip noting the last probe time + status. - Ran the probe once against Railway: 10/22 langgraph-python demos up (pre-merge features), 12/22 down (new demos on this branch, not yet deployed). Other 16 integrations fully live. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Atai Barkai
|
09b29916a6 |
feat(showcase/shell-internal): rich status cell (E2E/Smoke/QA/health)
Each grid cell now renders a two-row readiness rollup: demo · code E2E ✓ Smoke ✓ QA 3d ● up Signals: - E2E / Smoke — pass/fail + freshness (green <6h, amber older, red fail or no suite, gray when bundle itself is stale) - QA — days since human sign-off (green <7d, amber <30d, red otherwise or never) - Health — live probe dot (up/down/unknown) Cells with no demo show a single centered ✗ (unsupported) or `—` (supported but no demo yet). Client-side staleness check: if `status.json.generated_at` is older than 24h, every signal degrades to a gray `?` and the header shows a stale-bundle warning — so a failed cron visibly announces itself instead of silently serving green badges. Data layer (`src/lib/status.ts`) is a single source of truth for the badge color/label logic. The cell component is pure presentation. Status data is currently mock (deterministic per slug+demo) so the visuals can be reviewed; the CI/Notion/health-probe pipeline that writes the real `status.json` is out of scope for this commit. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Jordan Ritter
|
ddbb0470f0 |
fix(showcase/test-integration): clean up test-integration-tmp between runs (#4071)
## Summary Three showcase test suites leak working-tree drift (workflow YAMLs + shell data JSONs) on every run. This fixes the leaks and adds a shared `FileSnapshotRestorer` + `restoreFromGitHead` harness so the suites are idempotent. Also moves `showcase/scripts/vitest.config.ts` from the thread pool to the fork pool, which is required under Node 20 for the test subprocess churn in `validate-pins` + the three generator-invoking suites. ## What this PR does NOT fix — vitest 3.2.4 RPC timeout on Node 20 (upstream) `unit (20.x)` still reports `Timeout calling "onTaskUpdate"` -> ELIFECYCLE **after** all 14/14 test files and 1011/1011 tests pass. Known upstream bug: https://github.com/vitest-dev/vitest/issues/6129. The birpc timeout is hardcoded at 60 s (`DEFAULT_TIMEOUT = 6e4` in vitest's bundled `index.B521nVV-.js`) and is NOT exposed to `vitest.config.ts`. No `poolOptions.forks.*` / `teardownTimeout` / `hookTimeout` knob influences it: - `singleFork: true` made the run STRICTLY WORSE — only 1/14 files completed (validate-pins consumes the whole 60 s budget on its own, run 24602985507). - `fork-per-file` (default) gives each file a fresh RPC channel, and every file passes — but the final pool-teardown RPC still races on Node 20 and surfaces as a process-level exit 1. Under vitest 3.2.4 the fix requires either (a) upgrading to vitest 4.x (out of scope — monorepo-wide upgrade), or (b) a pnpm patch against the bundled `DEFAULT_TIMEOUT` constant (cross-cutting change; declined here). The observable reality: this PR makes the showcase-scripts suites idempotent and green; the remaining `unit (20.x)` redness is a known vitest flake orthogonal to what this PR is trying to fix. ## CI-killer error (verbatim) ``` Error: Package directory already exists: /home/runner/work/CopilotKit/CopilotKit/showcase/packages/test-integration-tmp ⎯⎯⎯⎯⎯⎯ Unhandled Errors ⎯⎯⎯⎯⎯⎯ Error: [vitest-worker]: Timeout calling "onTaskUpdate" ELIFECYCLE Test failed. Failed tasks: - @copilotkit/showcase-scripts:test ``` ## Root cause (fixed here) Three test suites invoke real generator scripts that write to tracked files OUTSIDE any tmp dir, leaking drift on every `nx run-many -t test`: - `create-integration.test.ts` scaffolds `showcase/packages/test-integration-tmp/` AND mutates three CI workflow YAMLs (`showcase_deploy.yml`, `showcase_drift-detection.yml`, `starter-smoke.yml`). - `generate-registry.test.ts` rewrites `showcase/shell/src/data/registry.json` + `constraints.json`. - `bundle-demo-content.test.ts` rewrites `showcase/shell/src/data/demo-content.json`. ## Fix (9 commits, by area of concern) 1. **`test(showcase/scripts): add shared test-cleanup snapshot/restore helper`** — new `__tests__/test-cleanup.ts` + `__tests__/paths.ts` + direct unit coverage in `__tests__/test-cleanup.test.ts`: - `FileSnapshotRestorer` — snapshots file content as `Buffer` (byte-exact, preserves non-utf8), restores only files that drifted, writes atomically via temp-file + rename, recreates parent dirs on write ENOENT. Temp filenames use `crypto.randomBytes(8).toString("hex")` so concurrent writes can't collide and the `snapshot()` sweep can unambiguously identify stragglers. On `snapshot()`, sweeps `.<basename>.<16-hex>.tmp` stragglers **scoped to the snapshotted basenames only**. - `restoreFromGitHead(repoRoot, paths)` — partitions the input via `git ls-files --error-unmatch` BEFORE any destructive op. **Narrow catch**: only genuine exit-1 pathspec errors are treated as untracked; ENOENT / EACCES / non-exit-1 failures re-raise with captured stderr. Uses `execFileSync` (no shell), forces `LC_ALL=C` / `LANG=C`, scrubs all `GIT_*` environment overrides, frozen exec options. - `test-cleanup.test.ts` itself strips `GIT_*` from child env when it creates tmp repos — pre-commit hooks (lefthook) run with `GIT_DIR` / `GIT_INDEX_FILE` set, which would otherwise cause tmp-repo `git commit` calls to write to the HOST working-tree HEAD. 2. **`fix(showcase/test-integration): clean up test-integration-tmp between runs`** — `create-integration.test.ts` wires `FileSnapshotRestorer` + `restoreFromGitHead` into the suite, wraps `rmSync` in `try/finally` so workflow restoration always runs, and migrates `execSync(string)` -> `execFileSync("npx", [...args])` via a shared `runGenerator()` helper. 3. **`fix(showcase/test-integration): stop generate-registry + bundle-content leaks`** — same pattern applied to `generate-registry.test.ts` and `bundle-demo-content.test.ts`; drops a redundant bundler pre-run in the latter. 4. **`fix(showcase/scripts): switch vitest to forks pool for Node 20 stability`** — `vitest.config.ts`: thread -> fork pool. 5. **`docs(showcase/scripts): tidy test-cleanup comments and JSDoc`** — documentation cleanup. 6. **`fix(showcase/scripts): pin vitest to a single fork + bump teardown timeouts`** — SUPERSEDED by commit 9 below (left in history for auditability). 7. **`fix(showcase/scripts): add post-heal drifted-baseline guard`** — the PR had claimed a drifted-baseline guard on CI for `restoreFromGitHead`, but no post-heal `git diff --quiet` was actually running. Adds the missing check: on CI, any tracked path still drifted post-heal throws `drifted-baseline guard: post-heal diff failed`; off-CI warns. Red-green unit coverage via a counter-based git shim that selectively fails the N-th `diff --quiet` (so the post-heal diff is targeted independently of the off-CI pre-checkout diff). 8. **`fix(showcase/scripts): decouple generate-registry test 2 from test 1 output`** — `sorts integrations by sort_order` was reading `registry.json` without invoking the generator, so `afterEach(restore)` between tests meant it was exercising the committed baseline rather than live output. Adds a `runGenerator()` call at the top. 9. **`fix(showcase/scripts): revert singleFork — fork-per-file is strictly better`** — empirical data from run 24602985507 proved `singleFork: true` was worse than fork-per-file (1/14 vs 14/14 files completing before RPC timeout). Reverts the `poolOptions.forks.singleFork` change; keeps the 30 s `teardownTimeout` / `hookTimeout` bumps. Comments now accurately reflect that the RPC timeout is upstream-hardcoded in birpc and NOT tunable via vitest config. ## Proof of idempotence ``` pnpm nx run @copilotkit/showcase-scripts:test --skip-nx-cache Run 1: Test Files 14 passed (14), Tests 1011 passed (1011) Run 2: Test Files 14 passed (14), Tests 1011 passed (1011) git status after each: only the intentional test file edits. ``` Red-green verified for the HIGH CR-findings: - narrow `partitionTrackedPaths` catch: unit test with empty PATH reproduces ENOENT; pre-fix hid it as "untracked", post-fix throws. - basename-scoped tmp sweep: unit test places both target-basename and unrelated `.something-else.<hex>.tmp`; pre-fix swept both, post-fix sweeps only target. - post-heal drifted-baseline guard: counter-based git shim fails the 2nd `diff --quiet`; pre-fix tests pass (guard absent), post-fix tests throw on CI / warn off-CI with the advertised message. ## Test plan - [x] `pnpm nx run @copilotkit/showcase-scripts:test` passes 1011/1011 twice in a row (locally, Node 25) - [x] Red-green: disabling `restore()` fails the regression + safety-net tests - [x] Red-green: disabling the post-heal drift guard fails the new guard tests - [x] Working tree clean after full run - [x] `prettier --check` + `oxlint` clean on touched files - [x] `GIT_*` scrub in test harness prevents pre-commit-hook-induced pollution of real HEAD ## CI status - **`unit (22.x)`**: pass - **`unit (24.x)`**: pass - **`unit (20.x)`**: all 14/14 files + 1011/1011 tests pass; post-suite `onTaskUpdate` RPC timeout emits exit 1. Upstream bug https://github.com/vitest-dev/vitest/issues/6129; not fixable at `vitest.config.ts` level on vitest 3.2.4. ## Caveats - Local verification ran on Node v25.8.0. No Node 20 binary on this dev host; Node 20 CI was the final gate. - The residual `unit (20.x)` failure is orthogonal to this PR. To resolve it we would need to upgrade vitest to 4.x (monorepo-wide change) or apply a pnpm patch to bump `DEFAULT_TIMEOUT` in vitest's bundled `birpc`. Both are tracked separately. |
||
|
Atai Barkai
|
e44afa29d6 |
fix(showcase): per-demo backend file scoping in the code viewer
Problem: shell's /code viewer was showing every `.py` file under `src/agents/` for every demo of a package. Visually contaminating: opening gen-ui-tool-based (Controlled Gen-UI Display) showed a2ui_dynamic, a2ui_fixed, mcp_apps_agent, open_gen_ui_agent, reasoning_agent, interrupt_agent, and tool_rendering_agent in the file picker even though none of them are relevant to that demo. Root cause: `bundle-demo-content.ts` ran `discoverBackendFiles()` once per package and attached the same union of all agent files to every demo. This was fine when all demos shared one graph, but since we split demos into dedicated graphs the bundle stopped matching reality. Fix: - `manifest.schema.json`: add optional `backend_files` field per demo (string array, paths relative to the package root). - `bundle-demo-content.ts`: when `demo.backend_files` is present, bundle exactly those. Otherwise fall back to the legacy full-package scan so packages that haven't adopted the field still work as before. - `langgraph-python/manifest.yaml`: populate `backend_files` for every demo. Each demo bundles `src/agent_server.py` plus only the agent file its graph routes to (main.py for shared-graph demos; reasoning_agent.py / interrupt_agent.py / a2ui_dynamic.py / a2ui_fixed.py / mcp_apps_agent.py / open_gen_ui_agent.py / tool_rendering_agent.py for demos with dedicated graphs). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Jordan Ritter
|
423275b2af |
fix(showcase/langgraph-typescript): bind agent server to 0.0.0.0 (#4075)
## Summary The `langgraph-typescript` container's agent server was binding to `localhost:8123` which resolves IPv6-first (`::1`) on modern Node. When the Next.js frontend's `/api/health` probe hit `localhost:8123`, it could resolve to IPv4 `127.0.0.1` (unbound), producing a 503 with `agent:"error"`. ## Fix Pass `--host 0.0.0.0` to `@langchain/langgraph-cli dev` so the agent listens on both IPv4 and IPv6 loopback. ## Verification Rebuilt and ran locally: ``` BEFORE: Server running at ::1:8123 → /api/health = 503 (agent:error) AFTER: Server running at 0.0.0.0:8123 → /api/health = 200 (agent:ok) ``` ## Test plan - [ ] `Validate Showcase` CI still green - [ ] No unrelated CI regressions - [ ] Once merged + deployed: `curl https://showcase-langgraph-typescript-production.up.railway.app/api/health` returns `agent:"ok"` consistently ## Not included Scoped to the entrypoint fix only. Other packages' entrypoints were spot-checked and don't share this pattern (Python langgraph, mastra, ag2 etc all bind `0.0.0.0` or use frameworks that do by default). |
||
|
Jordan Ritter
|
63643cbdb8 |
fix(showcase/langgraph-typescript): bind agent server to 0.0.0.0
The LangGraph agent server was started without --host, defaulting to 'localhost' which resolves IPv6-first (::1) on modern Node. The Next.js frontend's /api/health probe hits 'localhost:8123' which can resolve to IPv4 127.0.0.1 — unbound — producing a 503 with agent:"error". Add --host 0.0.0.0 so the agent listens on both IPv4 and IPv6 loopback. Verified locally: /api/health returns 200 with agent:"ok". |
||
|
Jordan Ritter
|
c026e657a8 |
fix(showcase/validate-parity): resolve demo directory from demo.route (#4074)
## Summary Unblocks the `Validate Showcase` CI check which has been failing on `main` for all 17 packages (last 3 runs: |
||
|
Jordan Ritter
|
3a40b065fd |
test(showcase/scripts): cover parseManifest route validation + routeToDirName branches
parseManifest (lib/manifest.ts): - Shape errors for demos[i].route: number / null / object / empty string - Shape error for route not starting with /demos/ - Happy path persists frozen demo.route on the parsed entry - Backward-compat: demos[i] without route is accepted with undefined route validate-parity: - Negative-case regression: missing-demo-dir's expectedDir is derived from demo.route (not demo.id) when route is present; mismatched id + route was silently hiding drift. - Fallback-case: demo with no route resolves expectedDir from demo.id - routeToDirName unit tests: undefined / bare /demos/ / normal tail segment TDD verified: mutating the /demos/ prefix guard failed the relevant test (RED), restoring the guard passed it (GREEN). Mutating expectedDir to demo.id-only failed the negative-case test (RED), restoring passed (GREEN). |
||
|
Jordan Ritter
|
90a8172373 |
fix(showcase/validate-parity): resolve demo directory from demo.route
demo.id is the CATALOG identifier (matched to qa/spec filenames and shell registry entries). demo.route is the URL + filesystem path (/demos/<dir> → src/app/demos/<dir>/). They are deliberately separate — a manifest with id: hitl-in-chat and route: /demos/hitl lives at src/app/demos/hitl/. validate-parity.ts previously resolved the demo directory from demo.id, producing a spurious missing-demo-dir MUST for every such split. Fix: - lib/manifest.ts: add optional route field to ManifestDemo; if present, parser requires it to be a non-empty string beginning with "/demos/". - validate-parity.ts: introduce routeToDirName helper (matches bundle-demo-content.ts idiom); loop over demos resolving expected dir from route and falling back to id. missing-demo-dir PackageIssue now carries both demoId and expectedDir so deriveMessage can flag route-resolved paths distinctly. - __tests__/validate-parity.test.ts: red-green regression test — a package with id: hitl-in-chat, route: /demos/hitl, and dir src/app/demos/hitl/ must PASS (no missing-demo-dir error). |
||
|
Jordan Ritter
|
00c4fbb173 |
feat(showcase/crewai-crews): aimock toggle for dev + CI + smoke (#4070)
## Summary - Adds an env-controlled aimock toggle to `showcase/packages/crewai-crews` (and every starter whose package ships `.env.example`) so the Python agent can be redirected to a local [aimock](https://github.com/CopilotKit/aimock) server for dev / CI / smoke. **Production default is unchanged** — when `AIMOCK_URL` is unset, behavior is identical to today (real OpenAI via `OPENAI_API_KEY`). A production guard additionally refuses to apply the toggle when any of 8 prod-ish env vars (`NODE_ENV`, `ENV`, `ENVIRONMENT`, `APP_ENV`, `DEPLOY_ENV`, `PYTHON_ENV`, `RAILWAY_ENVIRONMENT`, `RAILWAY_ENVIRONMENT_NAME`) equals `production`/`prod`, so a misconfigured deploy that leaks `AIMOCK_URL` cannot silently redirect real traffic through a mock. - Extends `showcase/scripts/generate-starters.ts` to ship `aimock_toggle.py` alongside `agent_server.py` AND to propagate `.env.example` into every starter whose package ships one. The copy step rewrites `AGENT_URL=http://localhost:8000` → `:8123` during propagation (starter dev scripts bind on 8123 while package dev scripts bind on 8000). The port-rewrite regex is scoped to `localhost` / `127.0.0.1` AND is exported so `starter-consistency.test.ts` imports the same pattern — generator + test cannot drift. - Hardens `.github/workflows/showcase_aimock-e2e.yml` against script injection via PR comment bodies (attacker-controlled text passes through `env:` instead of expression interpolation; slug is validated against a strict whitelist before reaching `cd` / `pip install`; job is gated on `author_association` so third-party commenters cannot trigger runs at all). R4 additions: aimock pinned to `@copilotkit/aimock@^1.14.3` (was `@latest`); `pnpm install --ignore-scripts` (defense against postinstall on untrusted PR content); `workflow_dispatch` slug is a `choice` enum restricted to packages that ship `aimock_toggle.py`; package-type step short-circuits with a clear `::error::` for non-toggle slugs; slug parsing uses POSIX-safe `grep -oE` + `sed` (no PCRE); Python agent health-check extended to 90s for CrewAI cold imports; pip cached via `actions/setup-python`; `OPENAI_BASE_URL` is no longer pre-set for python agent start (toggle must do it); dead `AIMOCK_URL` exports removed from Next.js dev-server + Playwright steps; `Post result to PR` passes slug + job status via `env:` (not `${{ }}` interpolation). R7 additions: `npm bin -g` replaced with `npm prefix -g`/`bin/aimock` (the former was removed in npm 9 and hard-failed on the node 22 runner); `pip install --only-binary :all:` loosened to `--prefer-binary` to tolerate sdist-only transitive deps (author_association gate bounds the residual risk); aimock PID captured + `kill -0` in the readiness loop so a crash fails fast; `APP_MODULE` fallback dropped (unreachable); comment-trigger TOCTOU documented in the residual-trust-model header. - Splits pytest into its own `python-unit-tests` job in `showcase_validate.yml`. Now a **matrix on Python 3.10 + 3.12** (fail-fast disabled) — 3.10 exercises the `typing_extensions` `NotRequired` fallback path that was previously uncovered by CI. `pip install` per package is a HARD FAIL (was `::warning::` + continue) so a broken `requirements.txt` cannot ship green. - **94 pytest cases** covering the toggle contract (red-green verified; +6 in R7 for `RAILWAY_ENVIRONMENT_NAME` parametrize entries across both prod-guard "refuses" + "silent no-op" variants). 1013 showcase-scripts vitest cases pass. ## Context Tonight's outage traced to defects in `ag-ui-crewai` triggered by upstream LLM behavior. A parallel PR in `ag-ui-protocol/ag-ui` fixes those defects. This PR is the complementary dev/CI/smoke-side improvement: routing non-prod traffic through aimock removes one class of trigger and cuts dev API costs + external dependency. ## Interface contract | Scenario | Result | |---|---| | `AIMOCK_URL` set, `USE_AIMOCK` unset | **enabled** (default when URL is set) | | `AIMOCK_URL` set, `USE_AIMOCK=1\|true\|yes\|on` | **enabled** | | `AIMOCK_URL` set, `USE_AIMOCK=0\|false\|no\|off` | **disabled** (explicit opt-out) | | `AIMOCK_URL` set, `USE_AIMOCK=<unknown value>` | **disabled** + WARNING (fail safe) | | `AIMOCK_URL` set, pre-existing `OPENAI_BASE_URL` OR `LITELLM_API_BASE` differs (raw) | overwrite + WARNING on each | | `AIMOCK_URL` set, `USE_AIMOCK=""` | **enabled** (empty fall-through) | | `AIMOCK_URL` set, `OPENAI_API_KEY` unset/empty/whitespace | dummy key injected (`sk-aimock-dev-ci-only`) | | `AIMOCK_URL` set, any prod-guard var set to `production`/`prod` | **refused** + WARNING (production guard) | | `USE_AIMOCK=1` alone, no `AIMOCK_URL` | no-op + WARNING (reason: `AIMOCK_URL unset`) | | `AIMOCK_URL=" "` (whitespace-only) | no-op (reason: `... empty or whitespace-only`) | | Both unset | **no-op**, production path preserved | When enabled, the toggle populates `OPENAI_BASE_URL` + `LITELLM_API_BASE` and injects a dummy `OPENAI_API_KEY` (with `sk-` prefix, self-documenting value) if absent/empty/whitespace. ## Import ordering (load-bearing) `load_dotenv()` + `configure_aimock()` sit at the **very top** of `agent_server.py`, above the `from crewai.cli import crew_chat` import. crewai / litellm can construct clients at import time that latch onto `OPENAI_BASE_URL` / `OPENAI_API_KEY` as they were at import — mutating env after the fact is invisible to those clients. ## Python version compatibility `aimock_toggle.py` imports `NotRequired` from `typing` (Python 3.11+) with a fallback to `typing_extensions` (listed in `requirements.txt` as `typing_extensions>=4.6; python_version < "3.11"`, so the backport only installs on older Pythons). The 3.10 matrix entry in `python-unit-tests` exercises the real-stdlib-absent fallback end-to-end; an additional pytest case (`test_typing_extensions_fallback_path_importable`) forces the branch via `sys.modules` manipulation for defense-in-depth even on 3.11+. ## CI integration - **`showcase_aimock-e2e.yml`** (manual + `/test-aimock` comment trigger): slug arrives from an attacker-controlled comment body, so we route it through an `env:` block (not `${{ ... }}` expression interpolation) and validate it against `^[a-z0-9-]+$` AND confirm `showcase/packages/<slug>/src/aimock_toggle.py` exists before any `cd` / `pip install`. Job is additionally gated on `author_association` ∈ `{OWNER, MEMBER, COLLABORATOR}`. No silent-fallback slug — bare `/test-aimock` with no slug is rejected (R6). Auto-detects `agent_server:app`; hard-fails if the agent doesn't bind. R7: uses `npm prefix -g` (not the removed-in-npm-9 `npm bin -g`) and `pip install --prefer-binary` (not `--only-binary :all:`, which broke on sdist-only transitive deps). - **`showcase_validate.yml`** (every PR): dedicated `python-unit-tests` matrix job (3.10 + 3.12) discovers `showcase/packages/*/tests/python/`, installs each package's `requirements.txt` as a hard-fail, then runs pytest. ## Security hardening (cumulative through R7) - **Script injection via PR comment body**: blocked — comment body passes through `env:` (not expression interpolation), and the derived slug is validated against a strict regex + existing-directory check before any downstream shell consumes it. - **Path traversal via `cd showcase/packages/$SLUG`**: blocked — slug regex rejects `../`, `/`, and anything else that could escape the packages dir. - **Third-party comment triggers**: blocked by `author_association` gate on the job. - **Postinstall on untrusted PR content**: blocked — `pnpm install --ignore-scripts` and `npm install --ignore-scripts` for the aimock install. - **Unpinned `@latest` aimock**: blocked — pinned to `^1.14.3`. - **Production toggle leakage**: blocked — 8 prod-ish env vars (NODE_ENV, ENV, ENVIRONMENT, APP_ENV, DEPLOY_ENV, PYTHON_ENV, RAILWAY_ENVIRONMENT, RAILWAY_ENVIRONMENT_NAME) refuse the toggle with a WARNING when set to `production`/`prod`. - **Copy-pasteable `sk-...` placeholder in `.env.example`**: replaced with `replace-with-your-key` across every package + starter so users who skip editing get a loud 401 from real OpenAI instead of a silent aimock success masking misconfig. - **Known TOCTOU**: the gap between trusted `/test-aimock` comment and `pulls.get` HEAD resolution is an architectural limitation of GitHub Actions' comment-trigger model (no native comment-time SHA pin). Documented in the workflow header; mitigated by the author_association gate + code-review social contract. ## Verification ```bash # Unit tests — red-green verified cd showcase/packages/crewai-crews python3 -m pytest tests/python/ -v # 94 passed # Starter consistency + generator guards pnpm nx run @copilotkit/showcase-scripts:test # 1013 passed cd showcase/scripts && pnpm exec tsx generate-starters.ts --check # OK # Docker build (local verification per docker-ci-safety) cd showcase/packages/crewai-crews cp -R ../../shared/python shared_python # CI copies this in docker build -f Dockerfile -t test-crewai-crews . # SUCCESS cd ../../starters/crewai-crews docker build -t test-crewai-crews-starter . # SUCCESS # Confirm AGENT_URL port rewrite in all starters grep AGENT_URL showcase/starters/*/.env.example # every hit reads `AGENT_URL=http://localhost:8123` ``` ## Test plan - [x] `pytest tests/python/` — 94/94 passed on Python 3.10 (typing_extensions fallback) and 3.12 (stdlib) - [x] Red-green: R7 RAILWAY_ENVIRONMENT_NAME parametrize entries (6 new cases) fail pre-fix, pass after - [x] `pnpm nx run @copilotkit/showcase-scripts:test` — 1013 passed (20 starter-consistency + shared regex) - [x] `generate-starters.ts --check` — all 17 starters in sync - [x] `docker build` both package and starter — succeeds - [x] All 17 starter `.env.example` files: `AGENT_URL=http://localhost:8123` and `OPENAI_API_KEY=replace-with-your-key` - [ ] CI green on the PR (verify post-force-push) |
||
|
Atai Barkai
|
3332928321 |
refactor(showcase/langgraph-python): extract view components per demo
Review pass across all 18 langgraph-python demo columns. For each demo, verified strict minimality (no cross-contamination from other features) and extracted any inline view JSX >= 15 lines into dedicated sibling files. All 18 demos validated end-to-end via Playwright screenshot + visual review. View components extracted to dedicated files: - `a2ui-fixed-schema/flight-card.tsx` + `catalog.ts` - `agentic-chat-reasoning/reasoning-block.tsx` - `chat-slots/custom-welcome-screen.tsx`, `custom-assistant-message.tsx`, `custom-disclaimer.tsx` - `gen-ui-agent/InlineAgentStateCard.tsx` - `gen-ui-interrupt/InterruptCard.tsx` - `headless-complete/message-list.tsx`, `user-bubble.tsx`, `assistant-bubble.tsx`, `input-bar.tsx`, `typing-indicator.tsx` - `hitl-in-chat/approval-card.tsx` - `open-gen-ui/sandbox-functions.ts`, `suggestions.ts` - `tool-rendering/weather-card.tsx` Backend fixes: - `mcp_apps_agent.py`: tighten system prompt to prevent the LLM from hallucinating unrelated framework names in its reply. - `open_gen_ui_agent.py`: short-circuit subsequent runs when a `ToolMessage` is already in state. `generateSandboxedUi` is registered with `followUp: true` by the provider, which caused the agent to re-emit the tool call in a loop after each sandbox handler response. `agentic-chat/README.md`: removed stale references to tools that no longer exist in the minimized demo. Regenerated `shell/src/data/demo-content.json`. Each demo's page.tsx is now pure composition; presentational code lives in dedicated sibling files. Zero cross-contamination between demos (audited via grep for forbidden-feature imports). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Atai Barkai
|
1a4d39bc6d |
chore(showcase): coordinated rebuild helper for parallel review agents
Parallel review subagents each need a fresh showcase-langgraph-python build to screenshot. Running `dev-local.sh up langgraph-python` N times in parallel serializes on Docker and wastes time rebuilding identical source. `scripts/agent-notes/rebuild-coord.sh` aligns rebuild attempts to a 30-second clock tick with 0-5s jitter. First agent on a tick acquires a `mkdir`-based lock, rebuilds, and writes a per-tick done marker at `/tmp/showcase-rebuild.done-<tick>`. Peers on the same tick skip their own build and wait up to 30s for that marker. If no marker appears, they retry the next tick (up to 3 ticks). Agents call `bash scripts/agent-notes/rebuild-coord.sh` instead of `./scripts/dev-local.sh up langgraph-python` when they need the live container refreshed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Atai Barkai
|
938dde15f4 |
chore(showcase): move Reasoning under Generative UI + rename
- Move `agentic-chat-reasoning` from `chat-ui` category to `generative-ui`, positioned after `tool-rendering`. - Rename display "Agentic Chat (Reasoning)" -> "Reasoning". - Update langgraph-python manifest demo entry's name, description, and tag to match. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Atai Barkai
|
dabd936727 |
chore(showcase): remove phantom mobile/web-framework rows from registry
These 7 feature ids had zero manifest references, zero demo dirs, and zero matching packages backing them in the showcase. They added noise to the feature matrix without carrying substance: - mobile-react-native - mobile-swiftui - mobile-android - web-svelte - web-vue - web-tanstack — conceptually misplaced anyway; TanStack Start is a React meta-framework, not a React alternative - web-angular — `packages/angular/` exists as a published SDK, but the showcase has zero integration for it. Angular support is really its own matrix/story; removing this placeholder row until there's a real showcase integration. Also drops the `mobile` and `web-frameworks` categories now that they have no members. Regenerates registry.json + constraints.json. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Atai Barkai
|
10e6922dc9 |
chore(showcase): regenerate registry + demo-content bundle
Generated via `generate-registry.ts` and `bundle-demo-content.ts` from the new manifest + feature-registry + demo sources. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Atai Barkai
|
9891d32949 |
feat(showcase/langgraph-python): generative UI demos + dedicated backends
Minimal frontends + minimal dedicated langgraph graphs for each Generative UI / interactivity demo: - `hitl-in-chat/` — `useHumanInTheLoop` confirm-destructive-action flow; shared `sample_agent`. - `gen-ui-interrupt/` + `interrupt_agent.py` — `ask_confirmation` tool that calls langgraph `interrupt()`; frontend uses `useInterrupt` with `renderInChat: true`. - `declarative-gen-ui/` + `a2ui_dynamic.py` — emits A2UI `a2ui_operations` at runtime; frontend uses `createA2UIMessageRenderer`. - `a2ui-fixed-schema/` + `a2ui_fixed.py` — `search_flight` tool emits A2UI ops against a client-side fixed `FlightCard` catalog. - `mcp-apps/` + `mcp_apps_agent.py` — `show_mcp_app` stub tool; the TS `MCPAppsStubMiddleware` in route.ts synthesizes the activity + handles the `__proxiedMCPRequest` round-trip with a pre-baked HTML resource. - `open-gen-ui/` + `open_gen_ui_agent.py` — dispatches `copilotkit_manually_emit_tool_call` for `generateSandboxedUi`; frontend uses `openGenerativeUI` runtime config on the isolated `/api/copilotkit-ogui` endpoint. - `gen-ui-agent/page.tsx` — `useAgent` + `messageView.children` slot; inline agent-state card between messages and interrupts. - `tool-rendering/page.tsx` + `tool_rendering_agent.py` — `get_weather` backend tool; `useRenderTool` renders a weather card inline. - `reasoning_agent.py` — dedicated graph that manually streams a reasoning chunk via a synthetic `BaseChatModel` before delegating to gpt-4o-mini, so the v2 `messageView.reasoningMessage` slot populates reliably regardless of the underlying model. Each demo touches only its own feature's hooks/components. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Atai Barkai
|
16739dfe9b |
feat(showcase/langgraph-python): chat & UI demos
Minimal demo pages covering the Chat & UI category: - `agentic-chat/page.tsx` — stripped to pure chat (CopilotKit + CopilotChat, single "Write a sonnet" suggestion). No tool calls. - `chat-slots/page.tsx` — three slot overrides: custom welcome screen, custom disclaimer (input sub-slot), and a custom assistant message bubble so the slot is visible during active chat. - `headless-simple/page.tsx` — bare-bones custom chat built on `useAgent` (no CopilotChat) + a single `useComponent` for inline card rendering. - `headless-complete/page.tsx` — fuller custom chat: scrollable message list, user/assistant bubbles, auto-scroll, disabled state while running, inline tool-call rendering, cancel button. - `agentic-chat-reasoning/page.tsx` — custom `messageView.reasoningMessage` slot renders an amber "Reasoning" block when the agent emits `role: "reasoning"` messages. - `prebuilt-chat/page.tsx` — minimal `<CopilotChat />`. - `prebuilt-sidebar/page.tsx` — dummy main content + `<CopilotSidebar />` (defaultOpen). - `prebuilt-popup/page.tsx` — dummy main content + `<CopilotPopup />` (defaultOpen). Each outer function wraps provider + layout; inner `Chat` holds only the registrations for that demo's feature. No cross-contamination (no a2ui/HITL/interrupt/tool-rendering in demos that don't demonstrate them). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Atai Barkai
|
04820597a4 |
feat(showcase/langgraph-python): wire manifest + agent registrations for demo fleet
- manifest.yaml: add 12 new demo entries (chat-slots, headless-simple,
headless-complete, agentic-chat-reasoning, hitl-in-chat,
gen-ui-interrupt, declarative-gen-ui, a2ui-fixed-schema, mcp-apps,
open-gen-ui, prebuilt-chat/sidebar/popup).
- langgraph.json: register dedicated graphs for demos that need a
non-default backend (tool_rendering, interrupt_agent, reasoning_agent,
a2ui_dynamic, a2ui_fixed, mcp_apps, open_gen_ui).
- route.ts: register each new agent name; demos with dedicated graphs
are mapped explicitly outside the shared-graph loop. A2UI runtime
middleware scoped to `declarative-gen-ui` + `a2ui-fixed-schema` so
non-A2UI demos keep their `useComponent` renderers intact. MCP Apps
middleware wraps only the `mcp-apps` agent and synthesizes an
`ACTIVITY_SNAPSHOT` event when the backend `show_mcp_app` tool fires.
- New `/api/copilotkit-ogui/route.ts`: dedicated isolated runtime for
the `open-gen-ui` demo. `openGenerativeUI: {...}` advertises the
`openGenerativeUIEnabled` flag globally on the probe response, which
causes the CopilotKit provider's setTools effect to wipe per-demo
`useFrontendTool`/`useComponent` registrations; isolating to its own
endpoint keeps the default runtime clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Atai Barkai
|
93497aa3df |
feat(showcase): expand feature registry for langgraph-python demo fleet
Adds rows for the demos being built in this branch: - Split `chat-prebuilt` into `prebuilt-chat`, `prebuilt-sidebar`, `prebuilt-popup` (CopilotChat / CopilotSidebar / CopilotPopup). - Reorder `generative-ui` rows and add: `hitl-in-chat` (In-Chat HITL), `gen-ui-interrupt`, `declarative-gen-ui` (Dynamic Schema), `a2ui-fixed-schema` (new), `mcp-apps`, `open-gen-ui`, `gen-ui-agent`, `tool-rendering`. - `constraints.yaml`: expand `constrained-explicit` allowlist so the langgraph-python manifest validates against the new feature ids. - Bump expected langgraph-python feature/demo count in `generate-registry.test.ts` from 10 -> 22 to match the manifest. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Jordan Ritter
|
15e8cca4b4 |
fix(showcase/scripts): revert singleFork — fork-per-file is strictly better
Run 24602985507 (singleFork: true) was strictly worse than the prior fork-per-file run (24602657301): singleFork (24602985507): 1/14 files completed, 64.64s, then timeout fork-per-file (24602657301): 14/14 files passed, 169.56s, then teardown timeout The RPC timeout under Node 20 is a hardcoded 60 s DEFAULT_TIMEOUT in birpc (DEFAULT_TIMEOUT = 6e4 in vitest's index.B521nVV-.js, not tunable via config). validate-pins.test.ts alone takes ~60 s because of its 134 `npx tsx` subprocess invocations, so singleFork blows its RPC budget before a single file finishes. Fork-per-file gives each file its own fresh 60 s RPC channel, which is why the prior run got all 14/14 files to complete before the final teardown hiccup. Revert `poolOptions.forks.singleFork: true`. Keep the teardownTimeout / hookTimeout bumps to 30 s — they don't affect the RPC timeout but do protect against slow per-hook teardowns under the same load. Note in comments that the RPC timeout is upstream-hardcoded and tracked at https://github.com/vitest-dev/vitest/issues/6129. |
||
|
Jordan Ritter
|
40ec4f8fe2 |
fix(showcase/scripts): decouple generate-registry test 2 from test 1 output
`sorts integrations by sort_order` read registry.json without invoking the generator, making it dependent on whichever run last left disk state in place. afterEach(dataRestorer.restore()) restores the data files to HEAD between tests, so test 2 was reading the committed baseline registry — not the live generator output — and would have silently agreed with whatever state happened to be on main. Add a `runGenerator()` call at the top of test 2 mirroring the `runBundlerAndRead()` pattern that tests 2-5 of bundle-demo-content already use. Test now exercises the actual generator's sort behavior under live conditions. |
||
|
Jordan Ritter
|
e11625a35b |
fix(showcase/scripts): add post-heal drifted-baseline guard
The PR description advertised a drifted-baseline guard on CI for
restoreFromGitHead, but the implementation never actually ran a
post-heal `git diff --quiet HEAD -- <tracked>` check — the only
diff was the off-CI pre-checkout guard against clobbering developer
edits. A racing external mutator (or a parallel test suite we haven't
accounted for) could rewrite a tracked file between our `git checkout
HEAD --` and our subsequent snapshot, and we'd silently bake the drift
into the baseline and the afterEach restore loop would maintain it
forever.
This commit adds the missing post-heal check immediately after the
`git checkout HEAD --`. On CI it throws with `drifted-baseline guard:
post-heal diff failed` citing every offending path; off-CI it warns so
developers iterating on a dirty tree aren't blocked.
Also:
- Rephrases the `isBenignPathspec` catch comment to describe the
realistic case (belt-and-braces against a rm race, not the normal
flow) now that partitionTrackedPaths pre-filters untracked paths.
The branch is intentionally kept — cheap to tolerate, and it
guards against a race that's hard to rule out in a shared worktree.
- Red-green unit coverage: new tests use a counter-based git shim
that fails the N-th `diff --quiet` invocation, so we can target
the post-heal diff independently of the off-CI pre-checkout diff.
- CI throws on drift with the advertised message
- off-CI warns (does not throw) with the same message
- no false-positive on a clean tracked path
Verified red without the guard, green with it.
|
||
|
Jordan Ritter
|
4c1304ac60 |
fix(showcase/scripts): pin vitest to a single fork + bump teardown timeouts
Run 24602657301 showed unit (20.x) still emitting `Timeout calling "onTaskUpdate"` during @copilotkit/showcase-scripts:test teardown — 1008/1008 tests passing, then ELIFECYCLE. The previous fix switched the pool from threads to forks but left the default fork-per-file behavior, so every file teardown tore down and reestablished the parent↔child RPC channel. Under Node 20 the channel occasionally fails to reestablish within the 10s default teardownTimeout — a known vitest issue (https://github.com/vitest-dev/vitest/issues/6129). Two defenses: - poolOptions.forks.singleFork: true — one long-lived fork across all test files (combined with the existing fileParallelism: false, files still run sequentially). The RPC channel stays warm for the whole run instead of being torn down and reestablished between every file, which eliminates the per-file teardown as a surface for the race. - teardownTimeout / hookTimeout bumped from the 10s default to 30s, matching testTimeout. A slow teardown can't be the bottleneck that kills the run. This is the most conservative setting short of dropping to a single- thread pool entirely. Node 22 / 24 unaffected. |
||
|
Jordan Ritter
|
60836bee8f |
fix(showcase/crewai-crews): greppable prod-guard WARNING + CORS intent doc + test hardening (CR8)
Three small CR8 fixes bundled by area (crewai-crews Python): * aimock_toggle.py: prod-guard WARNING now uses a grep-friendly prefix `aimock_toggle: DISABLED (prod-guard: VAR=value)`. Rationale in-file: `ENV=production` is a common shell-profile default that can silently disable the toggle in dev/CI without the operator realizing; a consistent greppable prefix lets them find unexpected refusals without reading every WARNING line. Strengthened the refusal test to assert exactly one WARNING is emitted with that prefix (regression guard against future iteration of _PROD_ENV_VARS firing past the first match). * agent_server.py: added a `CORSMiddleware` comment documenting why `allow_origins=["*"]` is intentional for this LOCAL DEMO STARTER — the agent binds to localhost:8000 (or :8123 inside a generated starter container) and is only reached by the Next.js frontend during development. Pointed at `.env.example` / a future `CORS_ORIGIN` env var as the path for real deployments. No behavior change. * test_aimock_toggle.py: `test_default_arg_path_mutates_os_environ` now `monkeypatch.delenv`s every entry in `_PROD_ENV_VARS` before asserting `enabled is True`. A CI runner or dev shell that exports e.g. `ENV=production` or `RAILWAY_ENVIRONMENT=production` would otherwise silently flip this test from "toggle enabled" to "prod-guard refused" and the enabled-True assertion would fail for the wrong reason. monkeypatch restores on teardown so other tests are unaffected. Regenerated crewai-crews starter to mirror the package changes. |
||
|
Jordan Ritter
|
806294704c |
fix(showcase/starters): scope NODE_ENV=production to Next.js in non-Python Dockerfiles (CR8)
The R6 "scope NODE_ENV to Next.js invocation only" fix landed for the Python Dockerfile template but missed the .NET / Java / TypeScript templates — they still set `ENV NODE_ENV=production` at the image level, which leaks into every child process. That's surprising for the TS templates specifically (mastra dev, langgraph-cli dev, tsx) because those frameworks gate HMR / source maps / telemetry on `NODE_ENV` and should NOT see "production" when they're running as the agent backend behind Next.js. Fix: remove image-level `ENV NODE_ENV=production` from all three non-Python Dockerfile templates. The entrypoint template already binds `env NODE_ENV=production npx next start` so Next.js still sees the value — only the child agent processes get their environment back. Added a lang- specific comment to each template explaining the intent and calling out the affected child processes. Regenerated the 5 affected starter Dockerfiles (claude-sdk-typescript, langgraph-typescript, mastra, ms-agent-dotnet, spring-ai) via `tsx generate-starters.ts`; diffs are byte-identical to the template change. |
||
|
Jordan Ritter
|
c99d840571 |
ci(showcase/aimock-e2e): fix readiness probes, sync trust-gate layers, post-Playwright liveness (CR8)
Blocking fix: aimock serves `/__aimock/health` + `/v1/*` only — probing `/` returns HTTP 404 which fails `curl -sf`, so the readiness loop never broke and every run hard-failed at "Start aimock". Switched all three probes (startup loop, startup verify, pre-Playwright re-probe) to `/__aimock/health`. Also tightened the workflow in four smaller ways: * Added `--max-time 2 --connect-timeout 1` to every curl probe (aimock readiness + verify, agent readiness + verify, dev-server readiness + verify, pre-Playwright re-probe). A hung socket can't blow the loop's iteration budget now. * Shell slug extractor (`grep -oE`) and job-level `startsWith` gate now agree on "first token only": the extractor anchors at `^` instead of allowing a leading-whitespace alternative. Header comment updated so the two-layer defense-in-depth rationale reflects the now-identical behavior. * Captured aimock PID into `$GITHUB_ENV` and added an `if: always()` "Re-check aimock liveness after Playwright" step that `kill -0`s the PID after tests. If aimock OOM'd mid-Playwright, the job now fails loudly instead of trusting cached / fall-through results. * Removed dead `OPENAI_BASE_URL` / `OPENAI_API_KEY` env on the `Run Playwright tests` step (Next.js was already running from the earlier step with these inline on that process; env on `npx playwright test` didn't flow anywhere). Added comment explaining why setting them here would be misleading. * Expanded comment on `workflow_dispatch.inputs.slug.type: choice` to document that single-choice UI preselect IS intentional (refused the sentinel-value footgun). |
||
|
Jordan Ritter
|
789cd14a47 |
chore(showcase/starters): regenerate after CR7 prod-guard + docs sync
Propagates the crewai-crews aimock_toggle.py (adds RAILWAY_ENVIRONMENT_NAME to `_PROD_ENV_VARS` + updated header comment) and .env.example (full prod-guard var enumeration) from packages/ to starters/ via generate-starters.ts. Only crewai-crews ships aimock_toggle.py today so only that starter changes; other starters re-generate byte-identical. Verified: `tsx generate-starters.ts --check` OK across all 17 starters. |
||
|
Jordan Ritter
|
701d2996ca |
fix(showcase/crewai-crews): add RAILWAY_ENVIRONMENT_NAME to prod-guard; sync .env docs (CR7)
- Add RAILWAY_ENVIRONMENT_NAME to `_PROD_ENV_VARS`. Railway exposes BOTH the legacy `RAILWAY_ENVIRONMENT` and the newer canonical `RAILWAY_ENVIRONMENT_NAME`; both can hold "production" on a prod service, so both must be guarded. Missing this one was a production-aimock-leak footgun for anyone hosting on newer Railway service templates. - Update the parametrized prod-guard tests (both "refuses" + "silent no-op" variants) to cover RAILWAY_ENVIRONMENT_NAME so new vars added to `_PROD_ENV_VARS` can't drift from the test matrix. - Sync .env.example comment block to enumerate ALL 8 prod-guard vars (NODE_ENV, ENV, ENVIRONMENT, APP_ENV, DEPLOY_ENV, PYTHON_ENV, RAILWAY_ENVIRONMENT, RAILWAY_ENVIRONMENT_NAME). The old comment said "NODE_ENV or ENV" which was misleading — operators reading the .env scaffold had no way to discover the full guard surface. - Update outdated "Local dev on macOS ships Python 3.9" header comment in aimock_toggle.py. CI matrix is 3.10/3.12 and requirements.txt enforces 3.10+ via marker-gated typing_extensions. Tests: 94 pass (was 88, +6 from RAILWAY_ENVIRONMENT_NAME parametrize entries across both refuse + silent-no-op variants). |