Commit Graph

11 Commits

Author SHA1 Message Date
Mike Ryan 873cbb8b6c feat(showcase): checkpoint 1 - baseline and registry 2026-07-23 07:12:53 -07:00
Jordan Ritter d0497649c7 test(showcase/harness): turn-done signal completion/backstop matrix
Cover the data-copilot-running turn-done signal in waitForTurnComplete:
true->false transition completion, stayed-stopped quiescence, the
attr-gated early backstop, pre-send run-start baseline, and the
integration wait-for-turn-complete behavior.
2026-06-23 14:32:05 -07:00
Jordan Ritter 3d49d4c38f test(harness): bubble-race integration tests + defect reproductions
- bubble-race-repro.ts: shared driver harness for repro tests
- bubble-race-repro-defect-{1,2,3,4}.test.ts: real-browser RED tests
  that fail without the fix and pass with it
    - Defect 1: fast-replay false-positive settle on count
    - Defect 2: multi-turn flicker via list[last]
    - Defect 3: cascade-tier blindness
    - Defect 4: boot-time baseline staleness / pre-paint placeholder
- bubble-race-mechanisms.test.ts: GREEN tests pinning internal
  contracts (cascade-pollution guard, atomic readCascadeState,
  init-script idempotency)
- wait-for-turn-complete.test.ts: 3-conjunct classification matrix
- probe-contract.test.ts: ctx-bridge shape
- sse-counter.test.ts: counter increment + framenavigated reset +
  handle-cache invalidation
2026-06-15 18:37:26 -07:00
Jordan Ritter 4f9f5dba4c fix(showcase/shell): docs-host robustness, builder & matcher hardening (SU2-stragglers + SU5 + SU6-A/B)
Round-by-round CR convergence covering the redirect builder, the middleware
matcher, the docs-host self-loop guard, and the runtime-config env readers.

Highlights:
- Clear module-load warns after fresh middleware import
- Validate SET BASE_URL values (scheme-less/degenerate/garbage) with
  sentinel fallback + once-guarded FATAL log
- Normalize path/query/fragment-bearing DOCS_HOST to origin; reject
  non-http(s) schemes; branch rejection reasons
- Harden POSTHOG_HOST (degenerate-host/scheme rejection); expose
  posthogKey via readEnvPair semantics
- Reject a DOCS_HOST equal to the shell's own host (redirect-loop guard,
  authority compare)
- Warn on missing local-ports.json under SHOWCASE_LOCAL=1 and validate
  TCP port range; extract helper for tests
- backend-url hardening — slug charset guard, frozen local-backends memo,
  pattern path-segment warn, local-override URL validation
- Client config fail-loud covers all four URL fields with type checks
- Make RuntimeConfig.posthogKey optional — absence is a valid state, not
  a wiring bug
- Drop R15/R17 and guard /integrations from SEO redirects
- Dedup duplicate wildcard prefixes with first-match-wins warn
- Validate malformed SEO entries at lookup-build time
- Restore case-insensitive redirect matching parity
- Normalize trailing slashes before redirect matching
- Keep the framework segment on F13, pin MG3 case fix
- Read posthogKey from runtime config in middleware, not raw process.env
- Fall back to the default backend host pattern for degenerate values
- Disable docs redirects when the default fallback collides with the
  shell host
- Bring validateBaseUrl to parity with its sibling readers
- Strip query/fragment from POSTHOG_HOST while keeping reverse-proxy paths
- Restrict local backend overrides to http(s) URLs
- Add server-only guard to runtime-config
- Harden localBackendsEnv failure posture
- Hoist /integrations namespace guard above the docs-host redirect
- Validate seo-redirect sources and cross-kind shadowing in
  buildRedirectLookup
- Unify slash normalization for middleware matching
- Lowercase-normalize REGISTRY_FRAMEWORK_SLUGS at construction
- Escalate missing POSTHOG_KEY to console.error in production
- Skip all redirect steps when docs redirects are disabled (sentinel
  consumer)
- Reject userinfo credentials in DOCS_HOST, POSTHOG_HOST, and the backend
  host pattern
- Branch dev-vs-prod logging in readDocsHost and fatalPatternOnce
- Prepend http:// (not https://) to scheme-less loopback hosts
- Round-5 micro-finding batch across the URL config libs

SU5-A1..A7 — registry safety, // reject, builder lint batch (case-
insensitive :path*, same-destination twin allowlist, original-case
divergence remainder), matcher api boundary, generator+vitest infra, test
hygiene + empty docs-host guard, comment batch.

SU6-A1..A6 — reject miscased :path* tokens, warn on tokenless wildcards,
normalize redirect-destination comparisons like request time, reject
destinations containing "//", surface missing POSTHOG_KEY at config-
resolution time, compile matcher harness like Next's runtime, type
parse/tokensToRegexp in the path-to-regexp shim, keep buildRedirectLookup
JSDoc attached.

SU6-B1..B7 — reject query/fragment/userinfo in pattern and local-override
URL gates, return parsed-normalized URL form from validation success
paths, distinguish unset/blank/padded SHOWCASE_LOCAL states, warn when
SHOWCASE_LOCAL is set to a value other than 1, validate {slug} placeholder
in generate-registry, mirror middleware drop semantics in the wiring
test's registry re-derivation, pin the noStore spy and calls to one fresh
module instance in the Edge-path test.
2026-06-11 14:00:50 -07:00
Jordan Ritter f8aee59b62 feat(showcase): add fleet queue client, CAS claim, and role-config 2026-06-05 10:38:40 -07:00
Jordan Ritter d15d2637a2 showcase: re-ratchet validate-pins baselines under new invariant 2026-05-31 10:13:21 -07:00
Sam Julien e2bef7a0b7 fix(shell-docs): retarget R15/R17 to /integrations/built-in-agent legacy URLs
R15 and R17 sources were /builtin-agent (no hyphen), matching no real traffic. Real legacy URLs live under /integrations/built-in-agent/* (47 URLs in the upstream sitemap). Retarget the source patterns to match.
2026-05-07 14:41:34 -07:00
Sam Julien 1b0ffc0dd8 test(harness): refresh seo-redirects snapshot for retargeted catalog 2026-05-07 10:55:06 -07:00
Jordan Ritter 00462bccbc fix(showcase): fix pre-existing test failures from smoke-red-tick deletion and e2e-deep timeout
The rename PR (#4388) deleted smoke-red-tick.yml from config/alerts/
and test/fixtures/rules/valid/ but left 16 tests referencing it:

- render-red-tick.test.ts: removed smoke-red-tick.yml from YAMLS array
  and deleted two smoke-specific tests (endpoint link rendering)
- rule-loader.test.ts: replaced smoke-red-tick.yml fixture references
  with agent-red-tick.yml; created matching valid fixture; removed the
  smoke-red-tick describe block from full YAML contract coverage
- e2e-deep.test.ts: increased timeout on failure_turn test from 5s to
  15s -- fillAndVerifySend retry loop (3 x 2s) exceeds 5s when
  stallEvaluate prevents user message count growth
2026-04-28 18:06:18 -07:00
Jordan Ritter c62dd66dc4 Merge remote-tracking branch 'origin/main' into blitz/showcase-harness-rename/integration
# Conflicts:
#	pnpm-lock.yaml
#	showcase/harness/config/alerts/smoke-red-tick.yml
#	showcase/harness/scripts/test-notify-harness-jq.sh
#	showcase/harness/src/probes/drivers/e2e-chat-tools.test.ts
#	showcase/harness/src/probes/drivers/e2e-chat-tools.ts
#	showcase/harness/src/probes/drivers/e2e-demos.test.ts
#	showcase/harness/src/probes/drivers/e2e-demos.ts
#	showcase/harness/src/probes/drivers/e2e-readiness.test.ts
#	showcase/harness/src/probes/drivers/e2e-readiness.ts
#	showcase/harness/src/probes/drivers/e2e-smoke.test.ts
#	showcase/harness/src/probes/drivers/e2e-smoke.ts
#	showcase/harness/src/probes/drivers/liveness.test.ts
#	showcase/harness/src/probes/drivers/liveness.ts
#	showcase/harness/src/probes/drivers/smoke.test.ts
#	showcase/harness/src/probes/drivers/smoke.ts
#	showcase/harness/src/probes/liveness.test.ts
#	showcase/harness/src/probes/liveness.ts
#	showcase/harness/src/probes/smoke.test.ts
#	showcase/harness/src/probes/smoke.ts
#	showcase/harness/test/fixtures/rules/valid/smoke-red-tick.yml
#	showcase/ops/src/probes/drivers/e2e-demos.test.ts
#	showcase/ops/src/probes/drivers/e2e-demos.ts
#	showcase/ops/src/probes/drivers/e2e-smoke.test.ts
#	showcase/ops/src/probes/drivers/e2e-smoke.ts
#	showcase/ops/src/probes/drivers/smoke.test.ts
#	showcase/ops/src/probes/drivers/smoke.ts
#	showcase/ops/src/probes/smoke.test.ts
#	showcase/ops/src/probes/smoke.ts
#	showcase/scripts/verify-railway-image-refs.ts
2026-04-28 16:04:44 -07:00
Jordan Ritter 0522b7fa41 refactor(showcase): rename showcase/ops → showcase/harness
The monitoring/alerting service is a test harness (probes, assertions,
alerting), not an operations service. Rename the directory, package
name (@copilotkit/showcase-ops → @copilotkit/showcase-harness), all
internal references (Dockerfile, Prometheus metric prefix
showcase_ops_ → showcase_harness_, orchestrator log messages, probe
YAML nameExcludes, test fixtures), and regenerate pnpm-lock.yaml.

Wire protocol names (X-Ops-* headers) and shell-dashboard internal
API naming (OPS_BASE_URL, ops-api.ts) are intentionally unchanged —
they are stable contracts between sender and receiver.
2026-04-28 13:48:12 -07:00