4771 Commits

Author	SHA1	Message	Date
Jordan Ritter	edc59c476c	fix(showcase): cold-load gray chip was masking genuinely-red cells (#6156) ... > **Scope, stated up front.** This PR fixes **one** of the ways a genuinely-red > showcase cell can render as something other than red: the cold-load `signal` > polarity. It does **not** eliminate the symptom. **Nine further masking > mechanisms remain** and are enumerated, with measured counts, under > [What this PR does NOT fix](#what-this-pr-does-not-fix). Please read that > section before treating a green dashboard as evidence of a green fleet. ## The bug Absence of evidence about *why* a cell failed was erasing the fact that it failed. `classifyRung` opened its red branch with: ```ts if (!raw.signalKnown) { return { ...base, contribution: "NO_DATA", rawStatus: null }; } ``` `signalKnown` is `false` when the dashboard's bulk fetch projected the `signal` blob away. So a **genuinely RED rung whose infra-vs-product attribution was merely PENDING got classified `NO_DATA` and painted gray** — the muted "no live probe data" treatment — while the depth strip beside it, built from the raw fold, correctly rendered `1P ✗`. One model object simultaneously said `d5.status === "red"` and `chipColor === "gray"`. Two things make that polarity wrong: 1. **`signalKnown === false` never meant "known to have no signal."** PocketBase omits the `signal` key *only* under a `fields=` projection; a row that genuinely has no signal arrives as `null`, and `null !== undefined`. The flag is pure provenance: "this row came from a projected fetch." It was never evidence the failure was infra. 2. **The guess was wrong on the overwhelming majority.** Of the non-green rows in production, **94.2% are product-class and 5.8% infra-class** (measured today against prod PocketBase using the classifier's own imported `signalHasInfraErrorClass` predicate: 3082 rows, 359 red, 21 infra-class — `driver-error` 11, `abort` 10 — 338 product-class). Graying every unattributed red lost 338 real failures to suppress 21 false alarms. A masked red is never investigated; a false alarm is looked at once and closed. This was **not** a first-paint race. The gray state is a fully-rendered, confident-looking wrong answer that persists until the probe's next sweep rewrites that specific row and the SSE delta redelivers it with `signal`, and it recurs on **every** reload, because the projection strips `signal` unconditionally. **Worst case for the routine per-cell dimensions is ~60 min**, from the hourly `e2e-demos` / `starter_smoke` / `d6-all-pills-e2e` crons in `harness/config/probes/*.yml`; drift dimensions are effectively unbounded. *(An earlier revision of this body claimed "~29 min observed, ~14 min mean". Both were withdrawn during review: "~29 min" was derived from the `*/30` tier and missed the hourly tier entirely, and "~14 min mean" had no derivable basis. Do not cite them.)* ## Measured effect PR #6169's exhaustive sweep — **279,936 rung combinations** (7 status rows × 6 row variants), the same map fed to the pre-fix and post-fix engines and diffed field-by-field — puts the direction of this change beyond argument: - **74,256 cells change**, and **every one moves toward *more* red.** - **Zero** flips toward less red: no `red→gray`, no `red→green`, no `amber→green` anywhere in the diff. - INV2–INV6 hold across the full sweep on the branch engine (0 violations). Red-green on **live production data** (local `next dev` wired to prod PocketBase, driven with Playwright, screenshots visually inspected — DOM measurements alone are not trusted for visual verification; both captures back-to-back on the same underlying rows, only the code changing): | Band | before | after | |---|---|---| | green | 632 | **632 — unchanged, no green regression** | | amber | 0 | 0 | | **red** | **0** | **51** | | no data | 76 | **25** | `0 red → 51 red`, `76 → 25 no data` — 51 recovered, arithmetic closes. Red on **first paint**, no gray plateau. ## The fix — two parts ### (a) Flip the fail-safe polarity — `cell-model.contribution.ts` A red rung with unknown infra-ness now classifies `FAIL_FRESH`. Graying a red requires **positive infra evidence**: `hasNonInfraRed` is false only when every contributing red row's blob actually carries an `INFRA_ERROR_CLASSES` attribution, and signal-provenance is retained as an explicit second precondition so a missing blob can never be read as an infra attribution even if `signalHasInfraErrorClass` is loosened later. ```ts if (fold.redSignalKnown && !fold.hasNonInfraRed) { return { ...base, contribution: "INFRA_RED_FRESH", rawStatus }; } ``` `redSignalKnown` is derived inside `foldFamily`'s **red-row** bookkeeping, not family-wide, so a projected-away GREEN sibling cannot suppress a legitimate infra gray. (The original family-wide flag made `INFRA_RED_FRESH` effectively **unreachable in the browser for every multi-row family** — D4, any multi-pill D5/D6; only single-key D3 was unaffected, which is why every pre-existing fixture missed it. Fixed by round-1 unit G1.) This is the durable fail-safe: any future projection or fetch regression now degrades toward **over-reporting** failure rather than silently hiding it. ### (b) Make first paint correct — `useLiveStatus.ts` The supplemental `signal` fetch is widened from the comm-error aggregates to a **union**: the existing clause OR `state != "green"`. ``` before: filter=(dimension = "d6" || … || dimension = "d5-single-pill-e2e") && key !~ "%/%" after: filter=((dimension = "d6" || … || dimension = "d5-single-pill-e2e") && key !~ "%/%") || (state != "green") ``` The existing fetch was not a viable vehicle as-is: it excluded these rows twice over — dimension `d5` is absent from `FLEET_COMM_AGGREGATE_DIMENSIONS` entirely, *and* its `key !~ "%/%"` predicate excludes every per-cell ladder-rung key. Relaxing only the `%/%` predicate would still have missed them. **Why widen the existing fetch rather than add a second one:** it reuses the freshness guard (`supplementalRowIsOlder`) and the chimera-avoidance merge verbatim — that merge already reasons carefully about a newer bulk row vs an older signal-bearing twin, and duplicating it would be the risky part. It also keeps first paint at one extra request instead of two. **Why scope by STATE, not by key:** signal-provenance is consulted **only** in `classifyRung`'s red branch — green and degraded paths never touch it — so only non-green rows can change a chip verdict. `!= "green"` rather than an explicit red/degraded list is deliberate: it also catches an out-of-vocabulary state, which `rankOfState` ranks **worst** — so the rows that matter most can never fall outside the fetch. The comm-error clause is kept as an independent disjunct, not replaced: a **green** aggregate row can still carry an active comm-error blob. The fetch is page-capped at `MAX_INITIAL_FETCH_PAGES = 20`, projected to `STATUS_LIST_FIELDS,signal`, and non-fatal at the fetch site — truncation is handled **asymmetrically**: supplemental truncation degrades (warn, use what arrived), bulk truncation throws. (Round-1 unit FETCH; the fetch was originally unbounded, and its rejection propagated to `setRows([])`, so an *enrichment* failure blanked the whole matrix.) **Rejected: suppressing the chip while pending.** A blank or spinning chip for up to an hour on a large slice of the matrix hides reds exactly as effectively as a gray one, while additionally looking broken. ### What each part buys Real engine over all 3082 production rows, three wires built from the identical row set *(green / amber / red / gray over the 708 catalog cells the band counts)*: | Wire | pre-fix engine | post-fix engine | |---|---|---| | FULL (server / `/api/matrix`) | 596 / 37 / 51 / 24 | 596 / 37 / 51 / 24 | | COLD, pre-fix projection | **596 / 0 / 0 / 112** | 596 / 37 / 51 / 24 | | COLD, post-fix projection | 596 / 0 / 49 / 63 | 596 / 37 / 51 / 24 | - The pre-fix browser wire collapsed **88 cells** (51 red + 37 amber) into gray. The entire red *and* amber population was zeroed, because the `!signalKnown` early return preempted `FAIL_FRESH`, `FIRST_STRIKE_FRESH` **and** `INFRA_RED_FRESH` alike. - **(b) alone is not sufficient**: it recovers 49 of 51 reds and **zero** of the 37 ambers, leaving 39 cells still mis-painted. - **(a) is what delivers correctness** — with it, all three wires equal server truth exactly. - **(b) is what keeps (a) from having to guess.** Today (a)-alone happens to cost 0 false-positive cells, because no cell's verdict-determining rung is currently a pure infra red. That is luck, not a guarantee: prod holds 21 infra-classed red rows. (b) supplies the actual evidence so those stay correctly gray, and it also feeds `decodeCellCommError` for non-green per-cell rows. ## Value test — per cell, all 6 (≥3 required) Every affected cell verified individually, not inferred from a shared symptom. Live DOM on the real surface, plus the real engine driven offline over all 3082 production rows through faithful replicas of the pre- and post-fix browser wire: | Cell (`d5:<slug>/multimodal`) | PB row | Server truth | Cold PRE | Cold POST | Live chip PRE | Live chip POST | Flipped | |---|---|---|---|---|---|---|---| | `agno` | red, fc 1817 | red | **gray** | red | gray `D4` + `1P ✗` | **red `D4`** | ✅ | | `crewai-crews` | red, fc 1716 | red | **gray** | red | gray `D4` + `1P ✗` | **red `D4`** | ✅ | | `llamaindex` | red, fc 2005 | red | **gray** | red | gray `D4` + `1P ✗` | **red `D4`** | ✅ | | `mastra` | red, fc 1806 | red | **gray** | red | gray `D4` + `1P ✗` | **red `D4`** | ✅ | | `ms-agent-python` | red, fc 1710 | red | **gray** | red | gray `D4` + `1P ✗` | **red `D4`** | ✅ | | `built-in-agent` | red, fc 1803 | red | **gray** | red | gray `D4` + `1P ✗` | **red `D4`** | ✅ | 6/6 flip. All six are `errorClass: conversation-error` — a real probe conversation failure, not in `INFRA_ERROR_CLASSES`, so `FAIL_FRESH` is the correct classification. (Prod figures; `built-in-agent` is green on staging, so staging shows 5.) **Depth distribution is byte-identical across the pair** (`D6:632 D5:0 D4:71 D3:0 D0:5` both times). The fix changes only the *colour*, never the depth — the `D4` label on these chips was always correct, so this must not be mis-validated by looking for a depth change. ## Tally-site audit Every tally is downstream of `chipColor`, so all self-correct — none needed refactoring. All verified on the live surface post-fix: | Site | Mechanism | Pre-fix | Post-fix | Verified | |---|---|---|---|---| | `feature-grid.tsx` `computeColumnTally` | gray-skip | `✗ 0` | `✗ 8` (Agno) | ✅ header DOM | | `feature-grid.tsx` `computeColumnTallyDetail` | gray-skip | empty list | 8 features incl. **Attachments** | ✅ panel opened via `tally-trigger-red` | | `page-stats.ts` `computeHealthStats` | `case "gray": noData++` | `0 red / 76 no data` | `51 red / 25 no data` | ✅ band DOM + screenshot | | `page-stats.ts` `computeDepthDistribution` | — (no gray gating at all) | `D4:71` | `D4:71` | ✅ unaffected | | `page-stats.ts` `computeD6Stats` | `d6Effective` | `null` → gray | `"red"` → red | ✅ golden fixture | **No other `chipColor` consumer changes behaviour.** `d0-gone-predicate.ts` gates the outage alerter on `chipColor === "red"`, but `d0-gone-monitor` reads *full rows with `signal`*, so provenance is always known there — as it is for `/api/matrix`. The polarity flip is reachable **only** where rows are projected, i.e. exclusively the browser cold-load path. ## A test asserted the buggy behaviour `api-matrix-equivalence.test.ts` asserted: ```ts expect(serverChip).toBe("red"); expect(coldLoadChip).toBe("gray"); expect(serverChip).not.toBe(coldLoadChip); ``` commented as "the state the browser converges to after its supplemental `signal` fetch … the more accurate answer." That comment is factually wrong — the supplemental fetch resolved in ~1 s and structurally excluded those rows twice over — and the assertion pinned the defect as intended behaviour. **This is why CI never caught it.** Read in full, it is asserting the defect, not an adjacent legitimate case: the fixture is a genuine product red (`errorDesc: "assertion failed: wrong answer"`, no infra class) with `signal: undefined` on the red rung. Inverted to assert **red on both paths**, with the fail-safe direction explained inline. Same for the harness's `§7 I5` case (`cell-model-v2.test.ts`) and `classifyRung`'s unit case. Four golden-master fixtures (`pos-d{3,4,5,6}-red-signal-unknown`) re-frozen. Negative controls added so the flip can't over-reach: an infra-classed red **still** grays (positive evidence present), and `§7 I4` (infra-only red → gray, `isRegression` false) still passes untouched. ## The structural lever: coherence invariant INV7 `cell-model.coherence.test.ts` asserted INV1–INV6, all of which relate `chipColor` only to the other **chip-side** outputs. **None inspected the `d3`/`d4`/`d5`/`d6` depth pills.** So the engine could return one object saying both `d5.status === "red"` and `chipColor === "gray"` with every invariant intact — which is exactly what happened, on every cold load. INV7: **a gray chip above a red depth pill requires positive infra evidence on every contributing red row.** Red-greened against the old polarity — it fails on 3 fixtures with a legible message: ``` pos-d5-red-signal-unknown: INV7 gray chip over a red pill requires POSITIVE infra evidence on every red row — d5:acme/agentic-chat has none (signal STRIPPED (pending)). A red whose cause is merely unknown must render red, not gray. ``` It closes the gap in the **only** safe direction: by fixing the chip upward, never by muting the strip. Deriving the pills from the classified contributions — the superficially cleaner unification — would render `1P ✗` as `1P ?` on cold load and destroy the only on-page element that was telling the truth, converting a visible inconsistency into an invisible failure. INV7's own limitations are disclosed under [Known limitations of the guards themselves](#known-limitations-of-the-guards-themselves). --- # Review-round fixes folded in Three full 12-reviewer review rounds were run against this branch. Round 1 contributed five units plus an integration fix; round 2 contributed four more. The loop was **deliberately stopped** after round 3 by explicit decision rather than running a fourth; round 3's findings are catalogued (132 items in `DEFERRED-r3.md`) and routed to the follow-up epic, not silently dropped. ## Round 1 — five units | # | Commit | Concern | Surface | |---|---|---|---| | G1 | `2a68282d` | Signal-provenance was derived family-wide but gated a red-row-scoped predicate. Precondition moved into `foldFamily` as `redSignalKnown`; `RawRung.signalKnown` **removed entirely** with all its plumbing. | production + tests | | A5 | `ebd7ac68` | The autocancel fake servers ignored `filter`/`fields`, so the supplemental fetch's leftover-append silently repaired a dropped bulk page — three pagination guards passed under every mutation. Servers rebuilt to evaluate the query. | test-only | | FETCH | `f5178765` | The supplemental fetch was unbounded (`for(;;)`), unprojected, and its rejection propagated to `setRows([])`. Now capped, projected, and non-fatal at the fetch site. | production + tests | | INV7 | `3dea9f64` | INV7 was quantified over the whole live map rather than the cell's contributing keys, and had no `isStaleCell` carve-out — it failed two classes of legitimate rendering. Scoped, exempted, plus an anti-vacuity guard. | test-only | | DOCS | `2797618c` | Stale/unanchored numeric claims (row counts, byte estimates, re-delivery window) corrected against production and given their source queries. | comments only | | — | `938be2ac` | **Integration fix.** A5 and FETCH disagreed about how a supplemental request is *identified*; all three fake servers misclassified it as a bulk page and the whole autocancel suite failed 3/3. Both dashboard suites now key on `signal` being IN the projection. | test-only | ## Round 2 — the four integrated units Integrated in order Q2 → Q1 → Q3 → Q4. Every guard was re-mutated **after** integration and reproduced its pre-integration result; no guard was disarmed by the merge. | # | Commit(s) | Concern | What it proved | Surface | |---|---|---|---|---| | **Q2** | `b8b75b7d` | The dashboard's `useLiveStatus` test doubles carried **three divergent implementations** of the same PocketBase query evaluator, and the nominally-shared one was the weakest. Collapsed to exactly one, every semantic a measured observation from a real PocketBase v0.22.21 server. Deleted autocancel's 380-line private copy. `evaluatePbList` **cannot throw**, so a parse error can no longer send NO response and hang the hook to a 20 s `waitFor` death. | Six mutations (short page dropped in wave merge, boundary logic removed, merge order reversed, supplemental filter broken, supplemental filter widened to `""`, bulk filter selects nothing) each reproduced their pre-integration RED count exactly — 3/1/2/6/5/6 failures. The consolidated evaluator is strictly stronger, not merely shorter. | test-only (−488 lines net in the doubles) | | **Q1** | `75fdfb3a` + `ed10b61f` | `truncated` meant "cap exhausted", so a **complete** read that happened to end exactly on the cap page was reported as truncated — the bulk caller threw, burned its reconnects and landed on a blank dashboard behind an offline banner, on every retry and every reload. `truncated` now means exactly "there are rows we did not read", proven by ONE lookahead read of page cap+1 whose rows are discarded and only whose emptiness is load-bearing. Algorithm hoisted to an exported `paginateStatusPages(readPage)`. | Lowering the cap 20→10 and 20→5 each produces **10 failed / 6 passed**, where the pre-fix code was **6/6 GREEN** — i.e. the guard was previously disarmed. A complete 10,000-row dataset renders `live`, `truncated === false`, seeded red cell still `chipColor: "red"`, page set exactly `[1..21]`; 10,001 rows still throws with `status: "error"` and "exceeded" in the message. | production + tests | | **Q3** | `49dea099` | INV7 false-failed on the ordinary I1 ladder-gap shape ("no `e2e:` row + red `chat:<slug>`"). Added a carve-out keyed on `ladderGapDepth`, **one-directional** (a gap *above* the red excuses nothing) and never firing without a witness rung — plus a two-sided canary that fails loudly the moment the underlying gap-break is fixed. | Deleting the `isStaleCell` exemption fails exactly the U8 force-gray case (the exemption is load-bearing, not decorative). Changing the engine's gap-break `break` → `continue` fires the **two-sided canary** with its intended message (`expected 'red' to be 'gray'`) — so the allowance cannot go on silently excusing a shape the engine no longer produces. Gapless and gap-above-red incoherence still FAIL; the legitimate gap shape still PASSES. | test-only | | **Q4** | `b04330ab` | Three explicitly load-bearing family folds were invisible at golden-master level — `maxNonInfraRedFailCount`'s MAX, `allRedSoftClass`'s EVERY quantifier, and `redSignalKnown`'s RED-ROW scope — because no fixture anywhere carried two *differing* red rows. 5 new fixtures, baseline 56→61, 140 insertions / 0 deletions. | Each of three mutations now fails via **exactly one** fixture: `Math.max`→`Math.min` ⇒ `hetero-d4-red-failcount-max`; `every`→`any` ⇒ `hetero-starter-mixed-soft-hard`; `redSignalKnown` red-row→family scope ⇒ `hetero-d4-green-stripped-infra-red`. Precise attribution, not a shotgun. | test-only | One merge conflict, in `useLiveStatus.supplemental-bounds.test.tsx`: Q2 deleted the file-local `classifyRequest` helper on the same lines where Q1 added its new `PAGES_THROUGH_LOOKAHEAD` constant. Resolved by keeping **both** intents rather than taking a side. Baseline golden-master key count is **61**, and the completeness check (`has exactly one baseline entry per fixture (no stale/missing)`) runs, so a fixture added without a baseline entry fails rather than being skipped. --- # What this PR does NOT fix **This PR does not fully eliminate the original symptom.** It fixes the `signal`-polarity masking mechanism and makes the cold load correct. A genuinely-red cell can still render **gray, green, or amber** by nine other routes. All nine were found by review of this branch and are catalogued, not speculative. ### Quantified by the #6169 sweep (279,936 combinations) | Mechanism | Rung combinations | |---|---| | **M1** — ladder gap-break: `scanWorst` folds the FIRST `ABSENT`/`STUB` rung as `ABSENT` and **`break`s**, so a fresh non-infra RED on a rung *above* the gap is never folded and paints gray | **17,088** | | **M2** — D6 outside the `scanWorst` walk: a red D6 never reaches the chip | **2,048** | | **M3** — gap not below the red: the remaining gray-over-red shapes | **192** | | | **19,328 total, 0 unexplained** | M1 is highly reachable in practice: `chat:<slug>` / `tools:<slug>` are integration-scoped, so one red `chat:` row plus an unemitted `e2e:<slug>/<featureId>` row (new feature, rotation slot, D3 driver never ran) is a gray chip over a red D4 pill. `combine.ts` is deliberately untouched here — fixing any of M1–M3 re-verdicts cells across the matrix and therefore needs its own PR with a golden-master **baseline re-freeze** and a **live value test**. Q3's INV7 carve-out is silent about M1 **on purpose and only about that**, and its two-sided canary is the tripwire that forces the carve-out to be deleted the moment M1 is actually fixed. ### Found in round 3, not quantified by the sweep 4. **Prototype-chain property lookups invert the anti-masking guard.** `STATE_RANK` / `rankOfState`, `WORST_STATE_RANK` / `worstStateRank`, `firstStrikeConfig[kind]` and `CATALOG_TO_D5_KEY` are plain object literals read without an own-property guard, so a runtime value colliding with an `Object.prototype` member (`constructor`, `toString`, `valueOf`, …) resolves to an inherited function that the `??` fallback then never replaces. For `rankOfState` the out-of-vocabulary guard **inverts** — an unknown state, which the code deliberately ranks worst, instead classifies as an infra red and grays; `worstStateRank` silently swallows the row entirely. `CATALOG_TO_D5_KEY` is unguarded at four further sites in `cell-model.ts`, where the consequence is a **whole-matrix render crash**. Fix is `Object.prototype.hasOwnProperty.call` (or a prototype-free table) at each site. 5. **Wave truncation bypasses the fail-loud throw entirely.** In the fan-out merge, a short page inside a wave causes every page issued *after* it in the same wave to be discarded — sound for a stable collection, but the pages resolve concurrently against a table the harness writes continuously. The discard produces no log, no warning, no counter, no `degraded` flip, and **`truncated` stays `false`** because it is only ever set by the cap lookahead. So the bulk caller's deliberate "a truncated BULK read cannot degrade gracefully … throw" posture fires only for cap exhaustion, never for this class of loss. The missing rows render as *absent*, which `buildCellModel` resolves to `ABSENT` / `NO_DATA` and paints gray — the exact polarity this PR exists to remove. `INITIAL_FANOUT_BATCH` is documented as "purely a WIRE-EFFICIENCY knob" but is load-bearing for correctness: at a larger batch the loss is `INITIAL_FANOUT_BATCH - 1` pages. 6. **`INITIAL_PAGE_SIZE` above the server clamp grays ~6/7 of the matrix.** Termination is the client-side comparison `first.length === INITIAL_PAGE_SIZE`, correct only because PocketBase clamps `perPage` to 500 server-side. Raise the constant — the obvious "fewer round-trips" tuning, which the surrounding comment invites — and every page returns 500, i.e. *short*, so the loop stops after page 1 and returns ~500 of ~3100 rows with `truncated: false`. The bulk caller sees a complete read, never throws, and ~6/7 of the matrix renders no-data gray with no warning and no banner. The `truncated` contract cannot catch it because the read genuinely never hit the cap. 7. **An empty bulk page 1 publishes `live` with 0 rows.** `paginateStatusPages` returns `{rows: [], truncated: false}` for an empty collection and `connect()` then does `setRows([])` + `setStatus("live")` + `setError(null)` with no empty-collection guard. Every cell renders no-data gray behind a healthy "live" indicator. Untested — the smallest fixture anywhere in these files is 3 rows. The server-side analogue is already a known deferred item. 8. **First-strike de-amplification fires at `fail_count === 0`, painting a genuine red AMBER.** The gate is `maxNonInfraRedFailCount < threshold` with `threshold` 2, so `fail_count` of **0** de-amplifies exactly like 1 → `FIRST_STRIKE_FRESH` → amber, i.e. "transient, not yet actionable". The threshold's semantics are valid only if a red row's `fail_count` is always ≥ 1, which is asserted as a *producer* property and enforced nowhere on the read side; out-of-vocabulary states (the harness can persist `"error"` as a no-data marker, which `rankOfState` ranks above red) and back-filled or externally-written rows both reach the classifier at 0. A `fail_count` of 0 on a red row is not positive evidence of a first strike. The golden master cannot see it: every red-bearing fixture uses `fail_count: opts.failCount ?? (isRed ? 1 : 0)`, so no fixture puts a red row at 0. 9. **An infra-red or `degraded` D1/D2 over a green ladder paints chip=green, achieved=6.** `combine.ts`'s §F gate fires only on `contribution === "FAIL_FRESH"` in both the achieved-depth walk and `computeChip` step 1; `INFRA_RED_FRESH` and `DEGRADED` fall through as non-gating, so the cell keeps its green ladder verdict with `d6Effective: "green"`. Measured on the branch engine: `D1 infra-red (driver-error) + green D2 → chip=green ach=6 reg=false d6Eff=green`, and the same for `D1 degraded`. **Mutation-proven invisible to the golden master:** patching *both* §F gate sites to also fire on `INFRA_RED_FRESH` and `DEGRADED` — a materially different verdict for every column whose health probe is failing — leaves **all 61 baseline entries byte-identical**. `health:<slug>` / `agent:<slug>` are integration-scoped, so this is whole-column. The polarity flip closed the `signal: undefined` hole here and left the `driver-error` / `abort` hole wide open — and `driver-error` / `abort` on a health probe is precisely what an unreachable container or a cancelled sweep produces. # Known limitations of the guards themselves Disclosed rather than omitted. These are properties of this PR's own tests. - **INV7's Q3 carve-out does not cover the M2 shape.** It is keyed on `ladderGapDepth` and excuses only a gap strictly *below* the shallowest red pill, which is M1. The D6-outside-the-walk shape (M2, 2,048 combos) is a different mechanism and is not addressed by the carve-out. - **The starter-axis INV7 canary short-circuits and proves nothing.** `assertChipStripCoherent` returns immediately on `m.chipColor !== "gray" || redDepth === null`. The hard-red starter cell the test builds has `chipColor === "red"`, so the function returns on the **first** disjunct — not on the strip condition the comments claim. The call would pass identically if a starter cell *did* expose a red depth strip. The canary role is actually carried by the four `lvl.exists === false` / `lvl.status === null` assertions beside it; the `assertChipStripCoherent` call adds nothing. Compounding it, `stripReadsRed` — the identifier the comments reason about — does not exist anywhere in the repo. - **The five INV7 "teeth" assert `/INV7/`, which the drift-guard message also matches.** `assertChipStripCoherent` raises two structurally different failures and both messages contain the literal `INV7`: the keyspace-drift guard, and the real coherence failure. Every teeth test asserts only `.toThrow(/INV7/)`. So if `contributingKeys` ever drifts from the engine keyspace — `collectAgentLadder` gains a rung, `CATALOG_TO_D5_KEY` fan-out changes shape, a `keyFor` segment changes — `redRows` goes empty, the drift guard fires, and **all five teeth still pass** while INV7 has gone vacuous over the entire fixture matrix. That is precisely the state the drift guard was written to make impossible; it is silenced inside the only tests that reach it. The fix is to assert the specific message (`/requires POSITIVE infra evidence/`) and pin the drift message in its own case. - **The test double cannot express absent-vs-null projection provenance.** `applyPbFields` silently drops projection tokens it does not model, so the fake cannot distinguish a key *omitted by a `fields=` projection* from a key present with value `null` — which is the single distinction `redSignalKnown` rests on. Fixtures therefore cannot validate the provenance semantics the fix is built on; that validation lives only in the live red-green capture above. # Follow-ups - **Epic: "A red must render red" — a single enforcement point for the showcase cell-model read model**, on Notion under CopilotKit *Plans / Proposals*: https://app.notion.com/p/3a83aa38185281b99f7bfdd1d0f46bb9 — carries M1–M3 and the round-3 mechanisms above toward one enforcement point rather than nine point-fixes. - **`DEFERRED-r3.md` — 132 catalogued items** from review round 3 (comment accuracy, test-double conformance, coverage gaps, altitude). Nothing in round 3 was discarded; it was catalogued and routed. # Quality gate Re-run at head `b04330ab` in a clean detached worktree. - **`oxfmt@0.36.0 --check`**: clean — "All matched files use the correct format" (16 changed TS/TSX files) - **`oxlint@1.51.0`** (repo `.oxlintrc.json`): **5 warnings, 0 errors.** 4 of the 5 reproduce verbatim on the `origin/main` versions of the same files (`unicorn(consistent-function-scoping)` on `c`, `eslint(no-shadow)` on `rows`, `unicorn(no-array-reverse)`, `typescript-eslint(consistent-type-imports)`). The one net-new against `origin/main` is a second `unicorn(consistent-function-scoping)` warning, on the test-local helper `lvl` in `cell-model.coherence.test.ts`. Zero errors; the repo's `oxlint` CI job is green. - **`tsc --noEmit`**: **0 errors** in `shell-dashboard`; `harness` clean under `tsconfig.build.json`. `harness`'s `tsc --noEmit` reports one pre-existing error, `frontend-matrix.test.ts` resolving `../../../shell/src/data/frontend-catalog.json` — a stale path in a file byte-unchanged from `origin/main`. - **`shell-dashboard` tests** (measured locally at `b04330ab`): **71 files / 1407 passed / 1 skipped / 0 failed.** Scoped to `src/`; `tests/runtime-env-switch.spike.test.ts` was deliberately not run (it spawns `next build` plus two `next start` invocations and is not a unit test). - **`harness` tests** (measured locally at `b04330ab`): **3659 passed / 18 skipped / 3 failed** — the same three pre-existing D5 fixture-registry-drift failures, in files **byte-unchanged from `origin/main`** and driven entirely by inputs none of this PR's 17 files touch: - `frontend showcase matrix > plans every runnable catalog cell without loss or duplication` — `expected 664 to have a length of 660` - `d5-mapping-drift > dashboard CATALOG_TO_D5_KEY structurally mirrors harness REGISTRY_TO_D5` — the drift regex cannot locate `CATALOG_TO_D5_KEY` in `shell-dashboard/src/lib/live-status.ts` (a 675-byte re-export shim on `main`, not modified here) - `D5_REPRESENTATIVES > covers every D5FeatureType that has a registered script` — missing `browser-use-smoke` Those three are now **explicitly quarantined** by the `test / unit-showcase` workflow that landed on `main` in `7282ecddfb` (`showcase/harness/vitest.quarantine.json`), and that workflow's **quarantine ratchet** re-runs each one to confirm it *still* fails — so they neither hide nor block. At this PR's head the ratchet passes: `Quarantine ratchet OK: all 3 quarantined file(s) still fail.` - Diff hygiene: no lockfiles, no generated `src/data/*.json`. ## CI provenance — this is the PR's first honest CI `main` has been **merged in** at `7282ecddfb` (merge commit `12ea621a`, textually clean, zero conflicts; the 13 commits above are unrewritten). This matters for more than being up to date: the `harness unit suite` and `shell-dashboard unit suite` jobs **did not exist** at `b04330ab`, because the workflow that enables them landed on `main` afterwards and GitHub runs `pull_request` workflows from the PR head. The earlier "17 pass" therefore ran **none** of the ~5,000 tests covering this PR's own subject matter (`cell-model/*` and `useLiveStatus*`). - CI at this head (`12ea621a`): **21 pass / 0 pending / 0 fail** across 24 registered rows. - Both new jobs run and both pass — `shell-dashboard unit suite` **70 files / 1404 passed / 1 skipped**, `harness unit suite` green, quarantine ratchet OK. - The **Q1–Q4 guard battery (16 re-proofs)** was re-run at the merge head and reproduced its pre-merge numbers exactly, so nothing was disarmed by the merge: Q1 cap→10 and cap→5 each **10 failed / 6 passed** (unmutated **16/16 green**); Q2 M1–M6 = **3 / 1 / 2 / 6 / 5 / 6** (M4 flips the supplemental cap test, M6 the bulk cap test, M5 red in both worlds by design); Q3 `isStaleCell` deletion = **1 failure** (the U8 force-gray case) and the engine `break`→`continue` fires the two-sided canary with `expected 'red' to be 'gray'`; Q4 each of the three fold mutations = **1 failed / 61 passed** via exactly one fixture (`hetero-d4-red-failcount-max`, `hetero-starter-mixed-soft-hard`, `hetero-d4-green-stripped-infra-red`), baseline key count **61**, completeness check present. # Review guidance **Surface: 17 files, +4880 / −382.** Most of that volume is tests, fixtures, and the regenerated golden-master baseline. Only three files carry production behaviour: `cell-model.contribution.ts`, `cell-model.ts`, and `useLiveStatus.ts`. **The four round-2 units are separable commits** — `b8b75b7d` (Q2), `75fdfb3a` + `ed10b61f` (Q1), `49dea099` (Q3), `b04330ab` (Q4) — and round 1's are too, so this can be read unit by unit rather than as one diff. **Highest-value review targets, in order:** 1. **`cell-model.contribution.ts` — the gate.** `if (fold.redSignalKnown && !fold.hasNonInfraRed)`. This is the whole polarity decision. Worth checking: that `redSignalKnown` really is red-row-scoped inside `foldFamily` (a family-wide derivation is what made `INFRA_RED_FRESH` unreachable in the browser), and that the two conjuncts are genuinely independent evidence rather than one restated. 2. **`useLiveStatus.ts` — `truncated` semantics.** Q1 redefined `truncated` from "cap exhausted" to "there are rows we did not read", proven by one lookahead read of page cap+1 whose rows are discarded and only whose emptiness is load-bearing. Worth checking: the asymmetry (bulk throws, supplemental warns) is applied at the right call sites, and that the lookahead genuinely cannot be confused with a data page. Note limitation 5 above — this contract does **not** cover wave-level loss. Everything else is fixtures, doubles, and the coherence suite; the disclosed weaknesses in those are listed under *Known limitations of the guards themselves* rather than left for a reviewer to rediscover.	2026-07-25 23:57:03 -07:00
Jordan Ritter	db75a04837	chore(showcase): mark multimodal unsupported for llamaindex and crewai-crews (#6158) ... ## What this is `multimodal` (Attachments) has never worked on **`llamaindex`** or **`crewai-crews`**, but both manifests listed it under `features`, so the fleet probed it and reported **red**. A red chip says "this regressed". The truth is "this was never built". This PR marks both cells **unsupported** instead. It does **not** implement the feature, and it does **not** suppress the cell — the cell still exists, the demo stays wired, and the chip renders the 🚫 unsupported glyph. ## Mechanism used (existing, not invented) The repo already has exactly one way to declare a feature unsupported for an integration: **`not_supported_features` in the integration's `manifest.yaml`**. The full derivation chain: | Step | Location | |---|---| | Declaration | `showcase/integrations/<slug>/manifest.yaml` -> `not_supported_features:` | | Schema | `showcase/shared/manifest.schema.json` — *"feature IDs that this integration's framework cannot architecturally support … excluded from parity computation"* | | Status fold | `showcase/harness/src/shared/catalog/catalog-flatten.ts:239` — `determineCellStatus()` checks `not_supported_features` **first**, returns `status: "unsupported"` | | Input mapping | `showcase/harness/src/shared/cell-model/catalog-input.ts:53` — `isSupported: cell.status !== "unsupported"` | | Model | `showcase/harness/src/shared/cell-model/cell-model.ts:847` — `if (!isSupported) return UNSUPPORTED;` (the frozen singleton at `:551`: `supported: false`, `chipColor: "gray"`, `isRegression: false`) | | `/api/matrix` | `showcase/harness/src/http/matrix.ts:206` -> `matrix-compute.ts:53` — projects that same model, so the API value **is** the rendered chip by construction | | Render | `showcase/shell-dashboard/src/components/unified-cell.tsx:308` — `if (!model.supported)` renders `data-testid="unified-cell-unsupported"` with 🚫 and `title="Not supported by this framework"` | The mechanical guard at `catalog-flatten.ts:169` rejects a feature that appears in **both** `features` and `not_supported_features`, so each entry was **moved**, not duplicated. Note this mechanism is strictly stronger than a probe-side skip: `buildCellModel` returns `UNSUPPORTED` regardless of what the live PocketBase row says. Verified against the existing not-supported cells on these same two integrations, which carry **green** PB rows and still render 🚫: ``` llamaindex/gen-ui-interrupt matrix: chip=gray supported=False | PB rows: d5=green d6=green e2e=green llamaindex/shared-state-streaming matrix: chip=gray supported=False | PB rows: d5=green d6=green e2e=green crewai-crews/mcp-apps matrix: chip=gray supported=False | PB rows: d5=green d6=green e2e=green ``` So the cell can never read green *or* red once declared here — which is the property we want. ## Per-integration reason (recorded inline in each manifest) **`llamaindex` — upstream gap.** The pinned `llama-index-protocols-ag-ui==0.2.2` (`llama_index/protocols/ag_ui/utils.py:82-85`) passes an AG-UI `UserMessage`'s `content` straight into `ChatMessage(...)`. A text-only turn passes a plain string (fine — every other llamaindex cell is green); an attachment turn passes a **list** of AG-UI content-part models, which pydantic routes into `ChatMessage.blocks`, a union discriminated on `block_type` — a field AG-UI's `TextInputContent` / `ImageInputContent` / `BinaryInputContent` do not carry. Live backend error: ``` pydantic_core._pydantic_core.ValidationError: 3 validation errors for ChatMessage blocks.0 Unable to extract tag using discriminator 'block_type' [type=union_tag_not_found, input_value=TextInputContent(type='te... image I just attached'), input_type=TextInputContent] ``` Fixing it needs a content-part -> `TextBlock`/`ImageBlock`/`DocumentBlock` conversion, upstream or on our side of `get_ag_ui_workflow_router`. This PR also corrects that module's docstring, which asserted the router *"normalizes them via the OpenAI `input_file` path"* — it does not. **`crewai-crews` — never implemented, ours.** `src/agent_server.py` registers a dedicated AG-UI endpoint for every other demo but **no `/multimodal` route** (the block ends at the catch-all `add_crewai_crew_fastapi_endpoint(app, LatestAiDevelopment(), "/")`), and there is no `src/agents/multimodal_agent.py`. So `src/app/api/copilotkit-multimodal/route.ts` aliases the generic shared crew, which has no vision handling and dies on a content-part message: ``` [CopilotKit] Error (agent_run_error_event): Error: thread=… run=…: CrewAI flow failed; see server logs ``` That route file's own header comment already concedes the gap ("A dedicated per-demo crew with vision-tuned agent prompts is tracked as follow-up work"). ## Proof Method: the **real** `GET /api/matrix` handler (`registerMatrixRoute`) driven over the **real** production PocketBase `status` collection (all 3082 rows, fetched verbatim from `showcase-pocketbase-production.up.railway.app`) and the **real** on-disk manifests (default `loadCells` = `buildCatalogCells`, the single flattening authority). Same rows, same fixed clock (`now = max(observed_at) + 60s = 1784932566438`) for both runs — the only variable is the manifest diff. ### BEFORE — real `GET /api/matrix`, real prod PocketBase rows, manifests at `origin/main` (38613623f4) ``` llamaindex/multimodal {"chipColor": "red", "supported": true, "achievedDepth": 4, "ceilingDepth": 6, "isRegression": true, "surfaceState": "red", "isStaleCell": false} crewai-crews/multimodal {"chipColor": "red", "supported": true, "achievedDepth": 4, "ceilingDepth": 6, "isRegression": true, "surfaceState": "red", "isStaleCell": false} ``` ### AFTER — same route, same rows, same clock, manifests with this PR ``` llamaindex/multimodal {"chipColor": "gray", "supported": false, "achievedDepth": 0, "ceilingDepth": 0, "isRegression": false, "surfaceState": "gray", "isStaleCell": false} crewai-crews/multimodal {"chipColor": "gray", "supported": false, "achievedDepth": 0, "ceilingDepth": 0, "isRegression": false, "surfaceState": "gray", "isStaleCell": false} ``` ### Full multimodal column, AFTER (control) ``` ag2 chip=green supported=True depth=6/6 [unchanged] agno chip=red supported=True depth=4/6 [unchanged] built-in-agent chip=red supported=True depth=4/6 [unchanged] claude-sdk-python chip=green supported=True depth=6/6 [unchanged] claude-sdk-typescript chip=green supported=True depth=6/6 [unchanged] crewai-crews chip=gray supported=False depth=0/0 [CHANGED] google-adk chip=green supported=True depth=6/6 [unchanged] langgraph-fastapi chip=green supported=True depth=6/6 [unchanged] langgraph-python chip=green supported=True depth=6/6 [unchanged] langgraph-typescript chip=green supported=True depth=6/6 [unchanged] langroid chip=green supported=True depth=6/6 [unchanged] llamaindex chip=gray supported=False depth=0/0 [CHANGED] mastra chip=red supported=True depth=4/6 [unchanged] ms-agent-dotnet chip=green supported=True depth=6/6 [unchanged] ms-agent-harness-dotnet chip=green supported=True depth=6/6 [unchanged] ms-agent-python chip=red supported=True depth=4/6 [unchanged] pydantic-ai chip=green supported=True depth=6/6 [unchanged] spring-ai chip=green supported=True depth=6/6 [unchanged] strands chip=green supported=True depth=6/6 [unchanged] strands-typescript chip=green supported=True depth=6/6 [unchanged] ``` ### CONTROL — an already-supported multimodal cell is untouched `langgraph-python/multimodal` (the reference integration) is `chip=green supported=true depth=6/6` **before and after**, byte-identical. So is every other integration's multimodal cell, including the four that are red for unrelated reasons (`agno`, `mastra`, `ms-agent-python`, `built-in-agent`) — those stay **red**, they were not swept up. ### Complete set of cells whose state changed — exactly 2 of 1000 Diffing every field of every cell in the `/api/matrix` body, before vs after (identical 1000-cell keyset): ``` crewai-crews/multimodal red -> unsupported llamaindex/multimodal red -> unsupported ``` Nothing else. Aggregate confirms no green was manufactured: ``` BEFORE AFTER chipColor green=632 gray=317 red=51 green=632 gray=319 red=49 supported=false 94 96 ``` `green` is **unchanged at 632** — this PR turned two reds into unsupported and created zero greens. Second, independent derivation (the dashboard's own generated `catalog.json`, via `showcase/scripts/generate-registry.ts`, which runs full AJV validation first) agrees, and also changes exactly 2 cells: ``` crewai-crews/multimodal (integrated): status wired -> unsupported, max_depth 4 -> 0 llamaindex/multimodal (integrated): status wired -> unsupported, max_depth 4 -> 0 metadata BEFORE: total_cells 980, wired 688, stub 0, unshipped 198, unsupported 94, docs_only 20 metadata AFTER : total_cells 980, wired 686, stub 0, unshipped 198, unsupported 96, docs_only 20 ``` `parity_tier` is unchanged on both columns (already `partial`), and no other integration's cells moved. ### Live dashboard render `shell-dashboard` run locally against the prod PocketBase, Playwright over the real DOM. On the `feature-row-multimodal` ("Attachments") row, exactly two of twenty columns carry `data-testid="unified-cell-unsupported"`: ``` CrewAI (Crews) unsupportedGlyph=TRUE text="🚫" LlamaIndex unsupportedGlyph=TRUE text="🚫" LangGraph (Python) unsupportedGlyph=false text="Demo ↗ Code </> D6 UI ✓ BE ✓ 1P ✓ D6 ✓" Agno unsupportedGlyph=false text="Demo ↗ Code </> D4 UI ✓ BE ✓ 1P ✗ D6 —" … 16 more, all unsupportedGlyph=false ``` Visually: a grey outlined 🚫 badge in those two columns — plainly not a green `D6` pill, and not a red `✗`. Control row `feature-row-agentic-chat` has `unsupportedGlyph=false` in all twenty columns. ## Quality gates - `oxfmt --check` on all three changed files — clean - `oxlint` — 0 warnings, 0 errors - `generate-registry.ts --validate-only` (AJV against `manifest.schema.json`) — passes; confirms no `features` / `not_supported_features` overlap - `shell-dashboard`: production build ✓, **68 test files / 1333 tests passed**, 1 skipped - `harness`: `tsc --noEmit` clean; **171 test files / 3625 tests passed**. 3 pre-existing failures (`d5-mapping-drift`, `frontend-matrix`, `d5-representatives` — the last complains about `browser-use-smoke`) fail **identically on clean `origin/main`**, verified by stashing this diff and re-running. Unrelated to this change. - Diff is 3 files, no lockfile churn, no generated artifacts, no stray worktree files. ## Deliberately not done - Not implementing the feature for either integration (the upstream conversion for llamaindex and the vision crew for crewai-crews remain open work). - Not touching `agno` / `mastra` / `ms-agent-python` / `built-in-agent`, whose multimodal cells are red for four unrelated reasons and stay red here. - Not loosening the D5 assertion, not dropping `skipSend`, not deleting the cell, not skipping the probe, and not special-casing the probe to pass — every one of those would green a broken cell. ## Follow-up commit: CI-caught wired-count bound The first push failed **Showcase: Validate** -> "Run build pipeline tests" (the `showcase/scripts` vitest, which I had not run locally — my mistake; the harness and dashboard suites both passed): ``` FAIL __tests__/generate-catalog.test.ts > parity tier: crewai-crews wired cells render at_parity or partial against the elected reference AssertionError: expected 29 to be greater than or equal to 30 ❯ __tests__/generate-catalog.test.ts:257:32 ``` Real and caused by this PR: reclassifying `crewai-crews/multimodal` from `wired` to `unsupported` drops that integration's wired-cell count 30 -> 29. That `30` is a **snapshot floor, not an invariant** — the assertion's own comment states the partial parity tier requires only `intersection >= 3` with the reference's wired set. 29 clears that by a wide margin, and the two tier assertions immediately below it (`parity_tier` in `["at_parity","partial"]`, uniform across the column) still pass untouched. So the floor was updated to 29 with a comment recording exactly which cell moved and why, rather than being deleted or loosened to a no-op. Nothing else in that suite moved: `metadata.total_cells` is still 980 and the `wired + stub + unshipped + unsupported == total_cells` sum invariant still holds (686 + 0 + 198 + 96 = 980). Re-run locally after the fix: **72 test files / 2323 tests passed, 0 failed.** `oxfmt --check` and `oxlint` clean on the changed test file.	2026-07-25 23:18:35 -07:00
Jordan Ritter	59f275eedc	fix(showcase/mastra): restore shared-tools symlink, shrink erosion ratchet (#6161) ... ## What `showcase/integrations/mastra/shared-tools` was a **real committed directory** where a **symlink into the single source of truth** belongs. This restores the symlink (`-> ../../shared/typescript/tools`) and shrinks the erosion ratchet accordingly. This is the iron-rule violation described in `showcase/AGENTS.md` → "The single-source symlink mechanism": `*/shared-tools`, `*/tools`, and `*/_shared` are meant to be symlinks into `showcase/shared/...`, so edits to the shared source silently do NOT reach an integration whose symlink has been clobbered by a real directory. ## Root cause | | | |---|---| | Symlink originally added | `93d5815cdb` — *"refactor: add shared tools symlink to mastra, update path alias"* (target `../../shared/typescript/tools`) | | Symlink clobbered | `534cd1efa7` — *"fix(showcase): D5 integration fixes across 12 frameworks"* — deleted the symlink (`shared-tools \| 1 -`) and committed **14 real files** in its place | That commit clobbered `shared-tools` for **three** TS integrations at once: `mastra`, `langgraph-typescript`, and `claude-sdk-typescript` — all three grandfathered in `showcase/scripts/validate-shared-symlinks.baseline.json`. mastra's Dockerfile already documented the symlink as the expected state, so the tree had drifted from its own documented contract: ``` # shared-tools/ is a symlink to ../../shared/typescript/tools — resolved at # build time by CI which copies the target into the build context. ``` ## Divergence inventory — **NONE** Diffed the real directory against `showcase/shared/typescript/tools` exhaustively **before** changing anything. **Zero divergence**: identical file set, identical git blob hashes, identical sha256s across all 14 files. | File | git blob (main) | classification | |---|---|---| | `__tests__/generate-a2ui.test.ts` | `b3c8223e` | identical copy | | `__tests__/get-weather.test.ts` | `57dd5b94` | identical copy | | `__tests__/query-data.test.ts` | `ef284b0b` | identical copy | | `__tests__/sales-todos.test.ts` | `8c5c0d6e` | identical copy | | `__tests__/schedule-meeting.test.ts` | `ab470b3b` | identical copy | | `__tests__/search-flights.test.ts` | `f64b8782` | identical copy | | `generate-a2ui.ts` | `7d76c00c` | identical copy | | `get-weather.ts` | `0b444f7a` | identical copy | | `index.ts` | `3b0e2a45` | identical copy | | `query-data.ts` | `48a2cbdc` | identical copy | | `sales-todos.ts` | `96b1874f` | identical copy | | `schedule-meeting.ts` | `d5e626e8` | identical copy | | `search-flights.ts` | `7a943023` | identical copy | | `types.ts` | `f87f26d1` | identical copy | Classification totals: - **stale** (shared moved on, mastra behind): none - **mastra-specific** (intentional or accidental local edit): **none** - **additive** (file only in mastra's copy): none - files present in one and not the other: **none** ``` $ git diff --no-index --stat integrations/mastra/shared-tools shared/typescript/tools (no output — identical) ``` Because the divergence set is empty, restoring the symlink is a **pure structural fix with zero content change** — nothing mastra-specific existed in the copy, so nothing can be silently lost. The restored link blob is `ddc634b6` — **the same git object** the pre-erosion symlink had. ## AFTER — structural proof ``` $ ls -la showcase/integrations/mastra/shared-tools lrwxr-xr-x shared-tools -> ../../shared/typescript/tools $ git ls-files -s showcase/integrations/mastra/shared-tools 120000 ddc634b6d9 0 showcase/integrations/mastra/shared-tools ``` Verified on a **fresh checkout of this commit** (not merely the working tree): mode `120000`, link resolves, and the content mastra resolves to is byte-identical to `showcase/shared/typescript/tools`. Validator on a fresh checkout — what CI sees: ``` ℹ 2 single-source slot(s) are ERODED • [known] showcase/integrations/claude-sdk-typescript/shared-tools (real-dir) • [known] showcase/integrations/langgraph-typescript/shared-tools (real-dir) ✔ single-source symlinks OK — no NEW erosion (2/2 baselined slot(s) still eroded) ``` `mastra/shared-tools` removed from `validate-shared-symlinks.baseline.json` per the shrink-only ratchet — the validator reports stale entries specifically so this step is mechanical. Baseline: **3 → 2**. ## Docker-image symlink verification A `COPY` can dereference or break these symlinks, and there is prior art in this repo of `docker build` eroding these exact links — so this was tested directly rather than assumed. **1. A naked `docker build` with the symlink FAILS** — expected, and exactly why `stage_shared()` exists (the target is outside the build context): ``` ERROR: failed to compute cache key: "/shared-tools": not found ``` **2. After the `stage_shared()` dereference the build succeeds and REAL FILES land in the image** — no dangling symlink: ``` === /app/shared-tools inode type === drwxr-xr-x shared-tools <- real dir, not a link === file count === 8 ``` **3. Verified inside the actual running showcase container** built from this branch: ``` $ docker exec showcase-iso2-mastra sh -c 'test -L /app/shared-tools && echo IS_SYMLINK || echo IS_REAL_DIR' IS_REAL_DIR ``` All 8 in-container `sha256sum` values match `showcase/shared/typescript/tools` byte-for-byte. **4. Every build path stages first** — this is not a new mechanism, it is the one 13 other integrations already rely on: - `stage_shared()` in `showcase/scripts/cli/_common.sh` (called by `cmd-recreate.sh` under `trap restore_symlinks EXIT`) - `stageSharedModules()` in `showcase/harness/src/cli/lifecycle.ts` (4 call sites, each paired with `restoreSymlinks()`) - `.github/workflows/test_showcase-frontend-matrix.yml` runs `stage_shared` in the step immediately before its raw `docker build` Both stagers iterate `["tools", "shared-tools", "_shared"]` generically, so `mastra/shared-tools` is handled with **no code change**. 13 other integrations already `COPY` a symlinked shared dir in their Dockerfile (`ag2`, `agno`, `claude-sdk-python`, `crewai-crews`, `google-adk`, `langgraph-fastapi`, `langgraph-python`, `langroid`, `llamaindex`, `ms-agent-dotnet`, `ms-agent-harness-dotnet`, `ms-agent-python`, `pydantic-ai`, `strands`) and depend on this same staging. Confirmed the full `stage_shared` → build → `restore_symlinks` round-trip: after a complete probe run the symlink is intact and the tree is clean. **Consequence worth stating plainly:** because the staged bytes are identical before and after this change, the Docker build context is **bit-identical** either way — the COPY layer even cache-hits. The built image cannot differ, which is the structural reason no cell can flip. ## Behavior proof — probe verdicts Ran the **real D5 probes** (`bin/showcase test <slug>:<feature> --d5 --direct`, real Docker containers + Playwright), on two features, from two separate worktrees: a pristine `origin/main` checkout (BEFORE) and this branch (AFTER). Not unit tests against fakes. | feature | BEFORE (`origin/main`, real dir) | AFTER (symlink) | delta | |---|---|---|---| | `mastra:tool-rendering` | **green** — 1 passed, 0 failed | **green** — 1 passed, 0 failed | **unchanged** | | `mastra:multimodal` | **red** — dom-missing | **red** — dom-missing (identical signature) | **unchanged** | ``` BEFORE tool-rendering: {"slug":"mastra","passed":1,"failed":0,...,"state":"green","durationMs":6443} AFTER tool-rendering: {"slug":"mastra","passed":1,"failed":0,...,"state":"green","durationMs":4697} BEFORE multimodal: {"slug":"mastra","passed":0,"failed":1,...,"state":"red","durationMs":122339} AFTER multimodal: {"slug":"mastra","passed":0,"failed":1,...,"state":"red","durationMs":121494} ``` **No cell changed state.** `tool-rendering` is the load-bearing one — it is precisely the feature that consumes `shared-tools`. Its manifest highlights `shared-tools/{get-weather,query-data,schedule-meeting,search-flights}.ts`, and `src/mastra/tools/index.ts` imports the impls through the alias that resolves into the converted directory: ```ts import { getWeatherImpl, queryDataImpl, scheduleMeetingImpl, searchFlightsImpl, generateA2uiImpl, buildA2uiOperationsFromToolCall, } from "@copilotkit/showcase-shared-tools"; // tsconfig alias -> ./shared-tools (now the symlink) ``` Its staying green is direct evidence that the symlink resolves, the shared tool impls are found and executed, and Next.js bundles them correctly. ### On the multimodal RED `mastra:multimodal` is **already red on `origin/main`** — I reproduced it on the pristine BEFORE worktree with the byte-identical failure signature: ``` waitForTurnComplete: turn 1 did not complete within 60000ms (reason=dom-missing, runsFinished=0, count=0, attrPresent=true, runningNow=false, runStartCount=0) ``` Its cause is unrelated to `shared-tools`: `mastra` ships **no `public/demo-files/` directory at all**, so the probe's `sample.png` / `sample.pdf` attachment 404s. Verified independently — `mastra/public/` contains only `.gitkeep`, two logo SVGs, `demo-audio/` and the `angular` symlink. `agno` has the same gap. This is a separate pre-existing packaging bug and is explicitly **not** addressed here. Also confirmed the symlink survives a full probe run: after `stageSharedModules()` dereferenced it tree-wide and `restoreSymlinks()` ran, the link is intact and the tree is clean. ## Other slots / other integrations `mastra` has **no** `tools/` or `_shared/` slot at all, so `shared-tools` was its only exposure. Full audit of all three slots across all 17 integrations on a clean checkout — the only remaining real-dirs are the two still-baselined: | integration | slot | state | |---|---|---| | `mastra` | `shared-tools` | **symlink (fixed here)** | | `langgraph-typescript` | `shared-tools` | real dir — still baselined | | `claude-sdk-typescript` | `shared-tools` | real dir — still baselined | | all 14 others | `tools` / `_shared` | healthy symlinks | `langgraph-typescript/shared-tools` and `claude-sdk-typescript/shared-tools` are eroded the same way, by the same commit, and are **also byte-identical** to the shared source — so both are trivially convertible. They are deliberately **left for follow-up** rather than folded in here so that each conversion carries its own behavior proof; `langgraph-typescript` in particular imports via a relative `from "../../shared-tools"` rather than mastra's tsconfig path alias, which warrants its own verification. Healing those two takes the baseline to `[]`, at which point the guard becomes **fully enforcing**. ## Control `langgraph-python` (an integration that already had correct symlinks) is untouched by this diff — its `tools`/`_shared` symlinks remain intact and resolve to content identical to the shared source. This commit touches only `showcase/integrations/mastra/shared-tools` and `showcase/scripts/validate-shared-symlinks.baseline.json`. I also verified my own worktree checkout had not itself mangled symlinks (a real risk with `git worktree`) **before** drawing any conclusion about mastra — every other integration's symlinks were intact in the same checkout, so the mastra real-dir was genuine and not an artifact. ## Tests - `showcase/scripts/__tests__/validate-shared-symlinks.test.ts` — **11/11 pass** on a clean checkout, including the case asserting zero fresh erosion and zero stale baseline entries against the real repo - `showcase/scripts/__tests__/extract-starter.test.ts` — pass - `npx tsx showcase/scripts/extract-starter.ts mastra` — correctly dereferences the symlink into real files in the extracted starter (its docstring names `mastra` as the example; that path is symlink-aware by design) - `prettier --check` clean on the changed baseline JSON > Operational note: running the validator test **while** a `showcase test` run is in flight reports 27 spurious erosions, because `stageSharedModules()` has temporarily dereferenced every symlink tree-wide mid-run. Not a defect — but the validator test is not safe to run concurrently with a probe run.	2026-07-25 23:18:18 -07:00
Jordan Ritter	c2e9264dde	Merge branch 'main' into fix/ms-agent-python-multimodal-prompt	2026-07-25 23:04:20 -07:00
Jordan Ritter	4a303f8bed	Merge branch 'main' into chore/multimodal-unsupported-llamaindex-crewai	2026-07-25 23:04:14 -07:00
Jordan Ritter	2825b10ba1	Merge branch 'main' into fix/mastra-shared-tools-symlink	2026-07-25 23:04:09 -07:00
Jordan Ritter	b907a2660a	Merge branch 'main' into fix/showcase-integration-page-titles	2026-07-25 23:04:04 -07:00
Jordan Ritter	d28384a2eb	fix(showcase/ci): make a partially-cancelled build loud, and verify what it actually shipped (#6171) ... ## The hole A partially-cancelled showcase build was **indistinguishable from a clean one in both directions**: it emitted no alert, and it suppressed post-deploy verification — while having actually redeployed real images to staging. Three merges landed on `main` on 2026-07-25 with **zero post-deploy verification**. Reconstructed from the API, with run IDs: | # | build run | conclusion | what happened | |---|---|---|---| | #5483 | `30162754730` | cancelled | `shell-docs` killed at 10m04s. Alert **did** fire (all-failed path). No verify. | | — | `30162770765` | cancelled | `shell` killed at 10m04s. **Silent.** No verify. | | #6160 | `30162773601` | cancelled | 5 slots killed, **23 redeployed to staging**. **Silent.** No verify. | | #6168 | `30162784491` | success | clean — and its verify was then cancelled by a sibling. | ### The chain, for run `30162773601` 1. #6160's workflow edit forced a full-fleet rebuild. Five slots died — `shell`, `shell-docs`, `shell-dashboard`, `shell-dojo`, `showcase-aimock`. 2. The intersection guard correctly excluded them. `redeploy-staging` **succeeded** and uploaded `redeploy-summary`, redeploying the 23 that built. 3. The run concluded **`cancelled`** (5 legs cancelled, 0 failed → GitHub rolls the run up to `cancelled`). 4. `notify` was **skipped** → no Slack, no PR comment. 5. `showcase_deploy.yml` requires `conclusion == 'success'` → never started. 23 services redeployed, unverified. 6. The one legitimate verify — run `30163309977`, from #6168's successful build — started 15:17:13 and was **cancelled at 15:17:22** by run `30163312882`, which was triggered by a *different* partially-cancelled build and then skipped every job anyway. ## Which links I verified in the YAML, and where the brief was wrong | Link | Verdict | |---|---| | Intersection guard excludes cancelled slots | ✅ Confirmed — `showcase_build.yml` `redeploy-staging` intersects matrix ∩ `status == "success"`. | | Run concluded `cancelled`, not failure/success | ✅ Confirmed via API on all three runs. | | `showcase_deploy.yml` requires `conclusion == 'success'` | ✅ Confirmed verbatim on `resolve-matrix`. | | Global `showcase-verify-deploy` concurrency group preempted a sibling | ✅ Confirmed — group had no commit key, `cancel-in-progress: true`; timestamps above show the 9-second preemption. | | **`notify`'s condition is `if: failure()`** | ❌ **Wrong.** It is already `!cancelled() && (failure() \|\| any_success == 'false')`. | | **The suppressor is `failure()` not matching `cancelled`** | ❌ **Wrong, and the proposed fix would not have worked.** See below. | | **Depot `Step canceled by GitHub` is a flake** | ❌ **Wrong.** It is a `timeout-minutes` kill. See "Root cause". | ### Why `if: failure() || cancelled()` would NOT have fixed this I built a throwaway probe workflow on a scratch branch — a matrix with one leg killed by `timeout-minutes` and one green leg, plus dependent jobs guarded by each candidate expression. **Probe run [`30166429073`](https://github.com/CopilotKit/CopilotKit/actions/runs/30166429073)**: | probe | result | conclusion | |---|---|---| | killed leg's `job.status` | `cancelled` | a timeout kill reports as *cancelled* | | matrix rollup `needs.legs.result` | `cancelled` | | | `if: cancelled()` | **SKIPPED** | `cancelled()` evaluated **FALSE** | | `if: failure()` | **SKIPPED** | `failure()` evaluated **FALSE** | | pre-fix `notify` condition | **SKIPPED** | ← the bug, reproduced | | post-fix `notify` condition | **RAN** | ← the fix, proven | | workflow run conclusion | `cancelled` | matches production | `cancelled()` is **workflow-scoped**: ["Returns `true` if the workflow was cancelled."](https://docs.github.com/en/actions/reference/evaluate-expressions-in-workflows-and-actions) Individual legs dying does not cancel the *run*, so it is false. And `failure()` ["returns `true` if any ancestor job fails"](https://docs.github.com/en/actions/reference/evaluate-expressions-in-workflows-and-actions) — a **cancelled** ancestor did not *fail*, so it is false too. With 23 slots green, `any_success` is `'true'`. Every clause the old guard had went the wrong way. Independently corroborated in production: in run `30162773601`, both `!cancelled()`-guarded jobs (`aggregate-build-results`, `redeploy-staging`) **ran**, which is only possible if `cancelled()` was false. ## Root cause of the cancellations: a timeout, not a flake From the Depot log of the killed `shell` leg: ``` 15:04:03 depot build --file showcase/shell/Dockerfile ... --push 15:13:29 #25 [builder 15/15] RUN cd scripts && ... generate-registry.ts DONE 54.3s 15:13:32 failed to solve: Canceled: context canceled 15:13:32 ##[error]Step canceled by GitHub 15:13:32 BUILD_STATUS: cancelled ``` Started 15:03:28, killed 15:13:34 — **exactly `timeout-minutes: 10`**. Same for the other three shell slots; `showcase-aimock` died at exactly its 5. The build was making real but slow progress the whole time (repeated docker.io base-image pull stalls, `#9 ...` for seven minutes) and was 2 steps from done. So **a retry is the wrong lever twice over**: a cancelled job cannot run further steps, and the build was not erroring. The underlying cause is Depot builder contention — three full-fleet rebuilds ran concurrently (~84 simultaneous amd64 builds). Measured budgets: | slot | warm | contended | budget | |---|---|---|---| | shell / shell-docs / shell-dashboard / shell-dojo | 2–5 min | killed at 10 | **10 → 20** | | showcase-aimock | 2 min | killed at 5 | **5 → 12** | | showcase-pocketbase | — | **8.8 min** (near-miss) | **10 → 15** | | webhooks | 0.6 min | — | 5 (unchanged, `skip_build`) | The `context: "."` slots build the whole monorepo root and are the heaviest in the fleet, yet carried the *smallest* budgets — inverted. ## The design **1. Stop laundering the signal** (`build-outputs.ts`, per-slot writer). `cancelled` becomes a first-class `BuildOutcome` instead of being mapped to `skipped`. The per-slot result is the *only* place the cancellation survives, because the status functions can't see it. `successSet` still excludes it, so the redeploy intersection is unchanged — a slot that pushed no image still cannot enter the redeploy CSV. **2. Alert on it** — aggregator publishes `any_cancelled` + `cancelled_services`; `notify` gains an `any_cancelled == 'true'` clause (the exact expression proven by the probe), and its PR comment now distinguishes *incomplete* from *failed*. **3. Red the run, don't leave it `cancelled`** — new `notify-cancelled-builds` job exits non-zero, so the conclusion is `failure`. A slot killed by its timeout budget *is* a failure, and `cancelled` is precisely what suppressed everything downstream. **4. Verify the partial deploy anyway** — the `conclusion == 'success'` gate becomes a terminal-conclusion allowlist (`success|failure|cancelled|timed_out`). > **Tradeoff, stated plainly.** Relaxing the gate rather than only failing loudly is deliberate: a build that redeployed 23 services *needs* verifying, and gating on `success` guaranteed it never would. It is safe because the decision of *what* to verify never depended on the rollup — no `redeploy-summary` artifact still means `has_services=false` and verify is skipped, and the redeploy gate still narrows to the per-service success set, so a cancelled slot can never be probed against a stale `:latest` and reported healthy. The cost is one extra no-op deploy run for builds that die before redeploying. Note this *also* fixes the same hole for `conclusion == 'failure'`, which was equally unverified. **5. Key verification per commit** — `showcase-verify-deploy-${{ github.event.workflow_run.head_sha || github.sha }}`, so a later cancelled run can no longer destroy an earlier successful run's verification. `cancel-in-progress: true` is kept, so re-verifying the *same* commit still supersedes. (`github` context is [documented as available](https://docs.github.com/en/actions/reference/workflows-and-actions/contexts) in `concurrency`.) **6. Realistic timeout budgets** — separate commit, droppable independently. Mitigation, not the root fix; the durable lever is a non-cancelling concurrency queue on the build workflow, noted inline as a follow-up since this workflow deliberately has *no* concurrency group. ### On alerting for a genuine manual cancel It stays **silent**, and that is correct. `!cancelled()` is retained on both notify jobs as the flake-vs-intentional discriminator, and the probe proves it discriminates: a human cancelling the whole RUN makes `cancelled()` true → silent; a leg-level timeout leaves it false → alerts. Belt and braces — on a run-level cancel, `aggregate-build-results` is itself skipped by its own `!cancelled()` guard, so `any_cancelled` resolves to `''` rather than `'true'`. Two independent reasons for silence, zero manual-cancel noise. ## What I proved - **Real end-to-end red-green on GitHub Actions** — probe run [`30166429073`](https://github.com/CopilotKit/CopilotKit/actions/runs/30166429073), table above. Old condition skipped, new condition ran, on a genuinely timeout-killed matrix leg. Scratch branch deleted; the run is permanent. - **Local red-green on the TS layer** — 8 tests failing before the fix (`cancelledSet is not a function`, `invalid "status" (must be success|failure|skipped)`), 46 passing after. - **Local red-green on the live workflow expression.** `redeploy-guard.test.ts` parses the **actual `if:` strings out of the YAML** and evaluates them. Pointed at `origin/main`'s YAML, the new `GREEN: the live guard FIRES on the production partial-cancel` test fails (`expected false to be true`) and is the *only* failure; pointed at this branch, all 31 pass. The pre-fix guard is pinned as a literal so the test permanently encodes the difference rather than just the current behaviour. - **No new alert noise** — explicit tests that `notify` stays silent on a clean build, on a human run-cancel, and on a no-changes push. That last one exposed a latent inaccuracy in the test harness's own model (a skipped aggregator's outputs are `''`, not `'false'`), now fixed. - **Lint** — `actionlint` finding count unchanged at 10 vs the `origin/main` baseline (all pre-existing: the unknown `depot-ubuntu-24.04-4` label and shellcheck `info` notes on untouched scripts). `zizmor` with the repo's own config at `min-severity: low`: *No findings to report*. - **Suite** — 77 tests green across the three touched files. Full `showcase/scripts` suite has 5 pre-existing failing files (`emit-railway-envs-json`, `generate-catalog`, `generate-registry`, `integration-smoke-registry`); I reproduced the identical 5 files / 8 tests on a pristine `origin/main` worktree, so they are unrelated to this change. ## What remains unproven until it runs on `main` - The **production** partial-cancel path end-to-end. The probe reproduced the expression semantics on a synthetic matrix, not the real 28-slot fleet with real Depot builds; the Slack message and PR-comment rendering have not been exercised against a live webhook. - The relaxed deploy gate firing on a real `cancelled`/`failure` build — including the "no artifact → skipped, no noise" path. - The per-commit concurrency group under real rapid-fire merges. - Whether 20 minutes is actually enough for `shell` under peak Depot contention. It was ~3 steps from done at 10; 20 is inference from that, not measurement. `\n` handling in the new Slack payload follows the existing `fromJSON('"\n"')` convention in this file, but is not verified against a live webhook.	2026-07-25 23:00:37 -07:00
Jordan Ritter	12de577952	fix(showcase/ci): alert and red the run when build slots are cancelled ... Build run 30162773601 (merge of #6160) forced a full-fleet rebuild; 5 of 28 slots were killed by their `timeout-minutes` budget, the other 23 built and WERE redeployed to staging, and the run emitted no signal at all: `notify` was skipped, so no Slack alert and no PR comment, and the run rolled up to conclusion `cancelled`. No existing guard could catch it. Measured on purpose-built probe run 30166429073 (matrix leg killed by `timeout-minutes`, sibling leg green): killed leg `job.status` ........ cancelled matrix rollup `needs.*.result` . cancelled `if: cancelled()` .............. SKIPPED (evaluated FALSE) `if: failure()` ................ SKIPPED (evaluated FALSE) pre-fix `notify` condition ..... SKIPPED <- the bug post-fix `notify` condition .... RAN <- the fix run conclusion ................. cancelled So `failure() || cancelled()` would NOT have fixed this. The signal has to come from the per-slot build results. - stop laundering `cancelled` into `skipped` in the per-slot writer - expose `any_cancelled` / `cancelled_services` from the aggregator job - add `notify-cancelled-builds`: exits non-zero so the run concludes `failure` rather than `cancelled` (a slot killed by its timeout budget is a failure, and `cancelled` is what suppressed everything), and Slacks the affected service names - add the `any_cancelled` clause to `notify` so the merge author gets the PR comment, with wording that distinguishes incomplete from failed `!cancelled()` is retained on both jobs as the intentional-vs-flake discriminator: a human cancelling the whole RUN makes `cancelled()` true and stays silent, while a leg-level timeout leaves it false and alerts. Extends redeploy-guard.test.ts, which evaluates the LIVE `if:` strings from the workflow, with the exact production scenario. It pins the pre-fix guard string as a literal so the test proves the difference the fix makes, not merely the current behaviour.	2026-07-25 10:05:12 -07:00
Jordan Ritter	62474be1dd	fix(showcase/ci): make a cancelled build slot a first-class outcome ... `job.status` for a matrix slot is success|failure|cancelled, but the per-slot writer laundered cancelled into `skipped` before publishing its result, so a slot killed by `timeout-minutes` became indistinguishable from one that legitimately never built. That erased the only signal that could tell a partially-cancelled fleet build from a clean one. GitHub's status functions cannot recover it: `cancelled()` is documented as "returns true if the workflow was canceled" (workflow-scoped, and FALSE for a leg-only cancel), and a cancelled ancestor is not a FAILED ancestor so `failure()` is false too. Add `cancelled` to the BuildOutcome contract, add `cancelledSet()`, and have the aggregator publish `any_cancelled` + `cancelled_services` alongside `any_success`. `successSet` still excludes cancelled slots, so the redeploy intersection is unchanged — a slot that pushed no image still cannot enter the redeploy CSV.	2026-07-25 10:04:58 -07:00
Jordan Ritter	1c24e0644b	chore(showcase): store demo assets uniformly as Git LFS pointers (#6163) ... ## What Demo assets under `showcase/integrations/*/public/{demo-files,demo-audio}/` were stored two different ways. This makes all of them LFS pointers — the root `.gitattributes` convention that the majority already follow. **Before this PR, on `main`:** | form | integrations | |---|---| | **LFS pointers** (10) | `ag2`, `built-in-agent`, `claude-sdk-python`, `claude-sdk-typescript`, `crewai-crews`, `langroid`, `llamaindex`, `ms-agent-harness-dotnet`, `pydantic-ai`, `spring-ai` | | **Raw blobs + per-integration `.gitattributes` override** (8) | `google-adk`, `langgraph-fastapi`, `langgraph-python`, `langgraph-typescript`, `ms-agent-dotnet`, `ms-agent-python`, `strands`, `strands-typescript` | | **Missing entirely** (2) | `agno`, `mastra` — being added as pointers on #6157 | Committed sizes: pointer-mode `sample.png` 130 B / `sample.pdf` 129 B / `sample.wav` 130 B; raw-mode `sample.png` 10083 B / `sample.pdf` 2486 B / `sample.wav` 87078 B. ## Why the overrides existed, and why they no longer need to Each override re-declared the same paths `-filter -diff -merge` so the files would be committed as real binaries. The stated reason (in their own comments) was that the image build didn't run `git lfs pull`, so a pointer stub shipped into the image and the multimodal sample-attachment magic-bytes guard rejected it. That premise no longer holds. The deploy build's Checkout step in `showcase_build.yml` hardcodes `lfs: true` (added in `7bde1eef3aa`), so every integration image gets real binaries regardless of storage form. The 10 pointer-mode integrations already ship through that exact pipeline today. The overrides are now dead weight that only buys divergence. ## What changed - Deleted all 8 override `.gitattributes`. Each contained **nothing but** demo-asset exemptions (the `langgraph-*`/`strands*` ones also exempted `public/demo-audio/sample.wav`), so each is removed in full. - Renormalized the 21 affected assets through the LFS clean filter. - Root `.gitattributes` untouched. `.github/` untouched. `agno`/`mastra` untouched. Deleting an override necessarily un-exempts that integration's `demo-audio/sample.wav` too, so the 5 raw `.wav` files are converted in the same pass — leaving them raw under an inherited `filter=lfs` would produce a permanently dirty checkout. ## Byte-identity Storage form changes, content does not. Every asset's sha256 **already equals** the LFS OID the pointer-mode integrations reference, so each renormalized blob is bit-for-bit the pointer blob already committed on `main`: ``` sample.png 10083 B oid 01aa5681de99461247543e9215c1e4da3242e26b2bee11593fcdbe209672d973 sample.pdf 2486 B oid 3da2afae36a1a81fd2c02f15e54bfc38b6c22e41655c31a5b54ff1e0e3daab41 sample.wav 87078 B oid bd4aa7b049f1c3e324dfd15af4068d7f8fbf2eae1dd044df270dddc5f38a5c57 ``` Verified per file (21/21): worktree sha256 before == worktree sha256 after == the pointer's OID, and each staged blob hash equals the canonical pointer blob (`a780acbd` png / `6446acbb` pdf / `f156ab54` wav) already on `main`. **No new LFS objects are introduced at all**, so a dangling pointer is impossible by construction. Confirmed anyway: - LFS batch API returns `download` actions for all three OIDs. - All three objects downloaded and confirmed to hash to their OID. - `git lfs push --dry-run origin <branch>` reports nothing to upload. - `git lfs fsck --pointers HEAD` reports no new violations (only the two pre-existing `examples/teams/appPackage/*.png` ones, also present on `main`). ## On the CI signal, and the real proof Being precise about what CI here does and does not prove: - **`showcase_build.yml`** (deploy, `lfs: true`) is the job that determines whether the deployed image gets real assets. It runs **only on push to `main`**, not on PRs. It is unchanged by this PR. - **`showcase_build_check.yml`** (the PR job) checks out with `lfs: ${{ matrix.service.lfs }}`, which is `false` for all 8 converted integrations. Its images therefore contain pointer stubs. It is build-only (`push: false`, no smoke test, no asset assertion), so it will pass — but **a green `build_check` here does not prove real assets ship.** I am not resting the case on it. I did **not** rely on a local Docker build either — it smudges LFS from the local checkout and would prove nothing. ### The real proof: prod already serves pointer-mode assets correctly Instead of a weak build signal, here is what the **live deployed prod images** serve today. Pointer-mode integrations are unchanged by this PR and demonstrate that the pointer storage form already ships real bytes through the `lfs: true` deploy checkout — for all three asset types: ``` --- .png : POINTER-mode on main (unchanged) --- ag2 200 10083 01aa5681de REAL-exact-match spring-ai 200 10083 01aa5681de REAL-exact-match llamaindex 200 10083 01aa5681de REAL-exact-match pydantic-ai 200 10083 01aa5681de REAL-exact-match langroid 200 10083 01aa5681de REAL-exact-match --- .png : RAW-mode on main (converted by this PR) --- langgraph-python 200 10083 01aa5681de REAL-exact-match strands 200 10083 01aa5681de REAL-exact-match google-adk 200 10083 01aa5681de REAL-exact-match ms-agent-python 200 10083 01aa5681de REAL-exact-match langgraph-typescript 200 10083 01aa5681de REAL-exact-match --- .wav : ALREADY pointer-mode on main (precedent for the 5 wav conversions) --- google-adk 200 87078 bd4aa7b049 REAL-exact-match ms-agent-python 200 87078 bd4aa7b049 REAL-exact-match ms-agent-dotnet 200 87078 bd4aa7b049 REAL-exact-match ag2 200 87078 bd4aa7b049 REAL-exact-match spring-ai 200 87078 bd4aa7b049 REAL-exact-match --- .wav : RAW on main (converted by this PR) --- langgraph-python 200 87078 bd4aa7b049 REAL-exact-match strands 200 87078 bd4aa7b049 REAL-exact-match langgraph-typescript 200 87078 bd4aa7b049 REAL-exact-match langgraph-fastapi 200 87078 bd4aa7b049 REAL-exact-match strands-typescript 200 87078 bd4aa7b049 REAL-exact-match --- .pdf : pointer-mode vs raw-mode --- ag2 (ptr) 200 2486 3da2afae36 REAL-exact-match spring-ai (ptr) 200 2486 3da2afae36 REAL-exact-match langgraph-python(raw) 200 2486 3da2afae36 REAL-exact-match strands (raw) 200 2486 3da2afae36 REAL-exact-match ``` Both storage forms already serve **byte-identical** content in prod (sha256 equals the LFS OID in every case), including `.wav` where 3 integrations are already pointer-mode. Converting raw to pointer therefore cannot change what prod serves: both forms resolve to the same bytes through the same deploy checkout. That is the safety argument — verified, not asserted. ## Sequencing - **#6160** (`ci(showcase): fetch Git LFS in every workflow that serves integration assets`) should merge **first**. It makes `showcase_build_check.yml` fetch LFS and removes the dead per-slot `lfs` flag; after it lands, `build_check` on this kind of change becomes a real signal instead of a weak one. No file overlap with this PR (it touches only `.github/workflows/`), so no merge conflict either way. - **#6157** (agno/mastra assets) is independent — disjoint paths, any order, no conflict. This PR assumes the deploy build's hardcoded `lfs: true` already on `main` and complements #6160's non-deploy workflow coverage. ## Cell impact **Changed-cell set: empty (expected and predicted).** This changes storage form only. The prod probes above show both storage forms already serve byte-identical content, so the multimodal cells for the 8 converted integrations have no mechanism by which to change state. The cells cannot be re-measured against this branch pre-merge (the deploy build runs only on push to `main`), so this is a prediction grounded in the byte-identity and prod-parity evidence rather than a post-deploy dashboard reading — worth a dashboard confirmation after merge. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01PYdjeveT8Xof9TyHWMLoJr	2026-07-25 09:59:39 -07:00
Jordan Ritter	8a726e701a	Merge branch 'main' into chore/showcase-demo-assets-uniform-lfs	2026-07-25 09:42:45 -07:00
Jordan Ritter	fb28e93473	Merge branch 'main' into fix/multimodal-missing-demo-files	2026-07-25 09:42:29 -07:00
Jordan Ritter	12ea621a77	Merge branch 'main' into fix/cold-load-gray-masks-red	2026-07-25 08:25:50 -07:00
Jordan Ritter	7282ecddfb	ci(showcase): run the harness and shell-dashboard unit suites in CI (#6168) ... ## The gap **No CI job ran the showcase unit suites.** Not "ran them partially" — no job invoked them at all. Verified per suite against `91f0321397` (the tip of `main` this branched from), by reading every workflow's actual `run:` lines rather than trusting job names: | Suite | Tests | Ran in CI before? | Where | |---|---|---|---| | `showcase/harness` | 3646 (3625 pass / 3 fail / 18 skip) | **NO** | nowhere | | `showcase/shell-dashboard` | 1331 (1330 pass / 1 skip) | **NO** | nowhere | | `showcase/shell` | ~57 (8 files) | **NO** | nowhere | | `showcase/scripts` | — | yes | `showcase_validate.yml` → "Run build pipeline tests" | Why each candidate workflow misses them: - **`test_unit.yml`** — the one whose name implies it covers this — excludes them *twice over*: `paths-ignore` lists `showcase/**`, **and** its nx selection is hard-scoped to `--projects='packages/**'`. - **`showcase_validate.yml`** runs `pnpm exec vitest run` only in `showcase/scripts`. It has two `working-directory: showcase/harness` steps, but they are a CVDIAG perf bench (`vitest bench src/cvdiag/emit-perf.bench.ts`) and an ESM boot-smoke — neither runs the unit suite. - **`static_quality.yml`** does run `nx run-many -t check-types`. This resolves the reported name/invocation divergence: the *job* is called `check-types` and the harness's script is called `typecheck`, so no `check-types` target exists on the harness and **the harness is not typechecked either**. Net effect: ~5000 showcase unit tests gated nothing. A PR could carry real defects in `showcase/harness/src/**` and still show an all-green check list, because no job structurally could have caught them. ## What this wires New workflow **`test / unit-showcase`** (`.github/workflows/test_unit-showcase.yml`), two parallel jobs on PR + push-to-main filtered to `showcase/**`: - **`harness unit suite`** — `nx run @copilotkit/showcase-harness:test:ci`, then `nx run @copilotkit/showcase-harness:test:quarantine-ratchet`. Driven through nx per the root `CLAUDE.md` task convention. - **`shell-dashboard unit suite`** — `vitest run` directly; the dashboard is deliberately outside the pnpm workspace (see the note in `pnpm-workspace.yaml`) so it is not an nx project and has no target to run. New nx targets are declared in **`showcase/harness/package.json`'s `nx.targets` block**, not in `nx.json` — they are harness-specific, and a `nx.json` `targetDefaults` entry would silently apply to any future project that happens to share the script name. Both are `cache: false` on purpose: the nx `test` named-input covers `src/**` and `*.test.*` but **not** `vitest.ci.config.ts` or `vitest.quarantine.json`, so a cached result could survive an edit to the quarantine list. ### Why a separate workflow rather than steps in `showcase_validate.yml` That file is a single ~900-line job already budgeted at 25 minutes and is the busiest merge-path workflow in the repo. Splitting the unit suites out gives them their own name in the check list, their own concurrency group, and — since the two suites need different package managers — lets them run in parallel instead of lengthening the critical job. Naming follows the existing convention (`test_unit.yml`, `test_unit-python-sdk.yml`, `test_unit-spring-ai.yml`). **No `continue-on-error` and no `|| true` anywhere in the file.** A gate that cannot fail is not a gate. ## Hazards, and how each is handled ### 1. The hanging spike test *Correction to the reported location:* it is `showcase/shell-dashboard/tests/runtime-env-switch.spike.test.ts`, not under `showcase/harness/`. It reaches the dashboard suite via that package's `include: ["src/**/*.test.{ts,tsx}", "tests/**/*.spike.test.ts"]`. **Reproduced locally.** vitest sat at **0.0% CPU with zero output** and never even spawned the `next build` its `beforeAll` calls — no worker fork, nothing. Killed at ~2 minutes; consistent with the reported ~25-minute stall. **Handled by excluding it from the unit gate**, not by a hard timeout. Justification: it is an integration spike by construction — one `next build` plus two `next start` boots on fixed ports 3801/3802 — and this package's own config comment already says it is *"too heavy for the per-file unit suite"* even though the `include` glob pulls it in anyway. A hard timeout would convert a 25-minute stall into a red gate with nothing actionable in it; this suite has to stay a fast, trustworthy unit signal. The spike needs its own job with a real server budget, which is out of scope for wiring up the unit gate. Note the `--exclude` values restate the package's existing config excludes (`tests/visual/**`, `node_modules/**`) because vitest's CLI `--exclude` **replaces** the config value rather than extending it — dropping them would silently re-enable them. ### 2. Gitignored generated artifacts `showcase/.gitignore` ignores `shell/src/data/*.json` and `shell-dashboard/src/data/*.json`, but the suites consume them: - `harness/src/probes/frontend-matrix.test.ts` **statically imports** `shell/src/data/frontend-catalog.json` — a missing file is a module *load error*, not a test failure. - `harness/src/fleet/control-plane/d0-gone-predicate.test.ts` reads `shell/src/data/registry.json` at runtime. - `shell-dashboard/src/lib/docs-status.ts` **statically imports** `@/data/docs-status.json`. Generated as an explicit job step (`generate-registry.ts`, plus `probe-docs.ts` for the dashboard), mirroring how `showcase_validate.yml` drives the same generators. The dashboard job invokes vitest directly rather than `npm test` specifically so the `pretest` hook does not re-run `probe-docs.ts` and double its ~50 outbound HEAD requests. Also load-bearing: `npm ci --ignore-scripts` in the dashboard job. That package's `postinstall` is `cd ../scripts && npm install`, which would lay an npm-resolved `showcase/scripts/node_modules` over the pnpm-managed one. ### 3. The 3 pre-existing failures — quarantined behind a ratchet, none deleted or weakened Confirmed byte-identical to `origin/main` and pre-existing. A job that is red on arrival gets ignored or disabled, so the three files are excluded via `showcase/harness/vitest.quarantine.json` + `vitest.ci.config.ts` — **and `scripts/quarantine-ratchet.ts` re-runs each one and requires it to STILL FAIL.** The moment somebody fixes a quarantined test, the ratchet goes red and names the entry to delete. An entry therefore cannot outlive the failure it excuses. The ratchet also fails on an entry pointing at a missing file, on an entry whose filter matches more than one file, and on a malformed or under-documented manifest (every entry must carry `file` / `since` / `reason` / `unquarantineWhen`). Local `pnpm test` still uses the unfiltered config, so a developer sees the quarantined failures. | File | Disposition | Why | |---|---|---| | `src/probes/frontend-matrix.test.ts` | quarantine | Asserts hard-coded per-frontend counts (`react=660`, `angular=636`) against the **generated** `frontend-catalog.json`, which has grown to 1302 runnable cells (`react=664`). Stale literals, not a product defect — the two assertions that express the test's stated intent ("without loss or duplication": `length === metadata.runnable`, ids unique) both pass. Exit criterion: derive the counts from the catalog; re-hard-coding fresh numbers just resets the clock. | | `src/probes/helpers/d5-mapping-drift.test.ts` | quarantine | **Stranded by a completed refactor.** It regex-scrapes `CATALOG_TO_D5_KEY` out of `shell-dashboard/src/lib/live-status.ts` to prove the dashboard copy hasn't drifted from the harness's `REGISTRY_TO_D5`. That file is now a one-line `export *` barrel over the canonical `harness/src/shared/cell-model/live-status.ts` — there is no second copy to drift and nothing for the regex to find, so the test throws its own "could not locate" guard. Exit criterion: cell-model owner deletes it (single-source is a strictly stronger guarantee) or rewrites it to assert the barrel re-export. | | `src/probes/helpers/d5-representatives.test.ts` | quarantine | **Genuine product gap, deliberately not fixed here.** `d5-browser-use-smoke` registers `browser-use-smoke` in `D5_REGISTRY` but `D5_REPRESENTATIVES` has no entry, so `getD5Representative('browser-use-smoke')` returns `undefined` and the D5 driver has no representative fixture. The test is correct and reporting a real hole; the fix belongs to the D5 owner, and this PR touches no product source. | ### 4. New finding: a load-sensitive flaky test Not one of the 3, and worth fixing separately: `src/probes/loader/probe-invoker.test.ts` → *"times out at the invoker level when enumerate() ignores abortSignal"* asserts a **wall-clock** `elapsed < 150` against a 100ms invoker timeout racing a 200ms enumerate. 50ms of headroom does not survive CPU contention — it passed **5/5 in isolation** but failed twice at ~217ms while the machine was running other vitest suites, and a 4-vCPU runner executing 173 test files in parallel is exactly that condition. Mitigated with `retry: 2` **in the CI config only** (base config unchanged, so local runs still surface non-determinism). This is not suppression: a genuinely broken test fails all three attempts and the gate stays red, and vitest prints every retried test. The real fix is for the probe-loader owner to drive that assertion off fake timers or widen the bound, after which the `retry` should come out. Flagging the tradeoff explicitly rather than burying it. ## Deliberate-breakage proof A job that has never failed is not yet a gate. Scratch commit `e100df27a7` broke two product-source files, then `4f092eccb1` reverted it. The final branch diff is **578 insertions, 0 deletions, and zero product source touched.** | Run | SHA | `harness unit suite` | `shell-dashboard unit suite` | Quarantine ratchet | |---|---|---|---|---| | baseline [`30140087665`](https://github.com/CopilotKit/CopilotKit/actions/runs/30140087665) | `452a26115e` | **success** (3m33s) | **success** (1m35s) | success | | **breakage** [`30140218159`](https://github.com/CopilotKit/CopilotKit/actions/runs/30140218159) | `e100df27a7` | **FAILURE** (3m31s) | **FAILURE** (1m28s) | success | The breakage — both reproduced locally before pushing: - `showcase/harness/src/shared/cell-model/staleness.ts`: `D4_STALE_AFTER_MS` 1h → 10h (a frozen-green D4 row would credit D4 for ten hours instead of one) - `showcase/shell-dashboard/src/lib/sort-order.ts`: `sortOrder` `mastra: 8` → `88` What the RED run actually caught: ``` harness unit suite — Test Files 3 failed | 168 passed | 2 skipped (173) Tests 4 failed | 3612 passed | 18 skipped (3634) FAIL src/http/matrix.test.ts > stale green ladder: raw state='green' but /api/matrix folds to gray (U8) FAIL src/shared/cell-model/cell-model.contribution.test.ts > family fold: one stale-green sibling forces STALE_DEGRADED AssertionError: expected 'GREEN_FRESH' to be 'STALE_DEGRADED' FAIL src/shared/cell-model/cell-model.equivalence.test.ts > golden-master identity > fixture: pos-d4-stale-green FAIL src/shared/cell-model/cell-model.equivalence.test.ts > golden-master identity > fixture: stale-column shell-dashboard unit suite — Test Files 3 failed | 64 passed (67) Tests 4 failed | 1326 passed | 1 skipped (1331) FAIL src/components/__tests__/depth-utils.test.ts > does not credit D4 when its green chat row is stale AssertionError: expected 4 to be 3 FAIL src/lib/__tests__/cell-model.test.ts > comm-error staleness window > scopes the window PER ROW FAMILY (G3e) AssertionError: expected 36060000 to be less than 21600000 ``` Three things this establishes beyond "the job can go red": 1. **It catches exactly the class of defect that went undetected on #6156** — a change to `showcase/harness/src/shared/cell-model/**` landed 4 failures in the harness gate, including two golden-master equivalence fixtures. 2. **The dashboard job also caught the harness defect**, through the `live-status.ts` re-export barrel. The shared cell-model now has two independent gates over it. 3. **`retry: 2` is not suppression.** Every `FAIL` line above appears three times in the log — the test was retried the full budget and failed all three attempts. A genuinely broken test cannot pass through it. And the **quarantine ratchet stayed green on the broken run**, confirming the quarantine is scoped to its three files and does not swallow unrelated regressions. ## Runtime Measured on CI (Depot 4-vCPU), baseline run [`30140087665`](https://github.com/CopilotKit/CopilotKit/actions/runs/30140087665) — **both jobs green**: | Job | CI wall-clock (whole job) | Suite only (local) | Files | |---|---|---|---| | `harness unit suite` | **3m33s** | 63.5s | 173 (171 pass / 2 skip, 3616 tests) | | `shell-dashboard unit suite` | **1m35s** | 7.2s | 67 (1330 pass / 1 skip) | They run in parallel, so the added critical-path cost is ~3m33s — comfortably under `showcase_validate.yml`'s existing 25-minute budget and nowhere near irritating-enough-to-disable. In both jobs the budget is dominated by install and (for the harness) nx `^build`, not by the tests. **No sharding or `nx affected` needed.** `nx affected` would in fact be actively wrong here: the entire point is that a change anywhere under `showcase/**` should run the suite covering it, and the historical failure mode was scope-narrowing, not slowness. If install time later dominates, the lever is pnpm/npm store caching (already configured) — not cutting test scope. ## Not covered `showcase/shell`'s 8 test files (~57 tests) are **also ungated** and I left them out to keep this change to one concern. They need a third install path (`showcase/shell` is likewise outside the pnpm workspace) and its `vitest.config.ts` already has a `globalSetup` that generates its own `registry.json`. Cheap follow-up. ## Coordination Concurrent work is adding a vacuity/drift text gate under `showcase/scripts` and compile-time completeness exports. This PR owns only the **suite invocation**. The one plausible cherry-pick conflict is `showcase/harness/package.json` (I added two `scripts` entries and an `nx.targets` block); `.github/workflows/test_unit-showcase.yml` is a new file and `nx.json` is deliberately untouched.	2026-07-25 08:00:59 -07:00
Varun Nuthalapati	9f7fef178b	docs(langgraph): fix emit-api-hook-mapping links and placeholder model ... Point "See also" links at the /integrations/langgraph/ prefixed routes (the bare /langgraph/... links 404), drop the dead Python SDK reference link (no target page), and swap the placeholder "gpt-5.4" model string for a real one (gpt-4o) in both code samples. Addresses review feedback on #5483. The unrelated use-agent.tsx isReady change was dropped by rebasing this branch onto upstream/main and carrying forward only the docs commit — isReady already ships on main via #6041.	2026-07-24 21:45:31 -07:00
Varun Nuthalapati	dda8754f4a	docs(langgraph): document emit API to v2 hook mapping end-to-end ... Adds a new page under integrations/langgraph/advanced/ explaining how LangGraph SDK backend emit functions (copilotkit_emit_tool_call, copilotkit_customize_config) connect to React v2 frontend hooks (useFrontendTool, useComponent). Includes a mapping table, two concrete end-to-end examples, and a decision guide. Closes #3301	2026-07-24 21:44:42 -07:00
Jordan Ritter	b04330abdb	test(showcase): pin the polarity flip's D1/D2 case and make family folds mutation-detectable ... The fail-safe polarity flip's widest case was frozen by nothing. A stripped- `signal` red on a `health:`/`agent:` row flips a whole column green->red (achieved 6->0, isRegression true) because those keys are integration-scoped, but `red-signal-unknown` was only applied by `positionSweep` (D3-D6), and `liveness-fresh-red-d1/-d2` use `row()`'s default `signal: null` -- which is not `undefined`, so they classified FAIL_FRESH under the old polarity too. Measured: restoring `!redSignalKnown -> NO_DATA` killed 4 golden-master entries, all D3-D6, and left the entire D1/D2 leg byte-identical. Underneath that sat a structural blind spot: every fixture variant maps ONE uniform row shape over ALL keys of a rung, so no family anywhere contained two DIFFERING red rows and no family contained a green row with an absent `signal` key. A uniform set has nothing to aggregate, so three explicitly load-bearing family folds were invisible at golden-master level -- `Math.max`->`Math.min` on `maxNonInfraRedFailCount`, `allRedSoftClass` `every`->`any`, and re-widening the provenance flag from red-row to family scope each left all 56 entries identical. Adds 5 fixtures (56 -> 61 baseline entries): - `liveness-red-signal-unknown-d1` / `-d2` -- a red D1/D2 row with `signal: undefined` over a green D3-D6 ladder. Both now fail under a polarity re-widening, joining the 4 existing entries. - `hetero-d4-red-failcount-max` -- D4 {chat red fc1, tools red fc3}, straddling D4_FIRST_STRIKE_THRESHOLD. MAX(1,3)=3 -> FAIL_FRESH/red; MIN(1,3)=1 -> FIRST_STRIKE_FRESH/amber, i.e. a fresh sibling re-arming tolerance for an already-confirmed failure, which the MAX doc explicitly forbids. - `hetero-d4-green-stripped-infra-red` -- D4 {chat green signal-omitted, tools red driver-error}, the production shape `classifyRung` cites verbatim and the first fixture with a green row whose `signal` key is absent. Red-row scoping keeps this gray; a family-scoped flag paints it red. - `hetero-starter-mixed-soft-hard` -- starter {transport-error (soft), smoke-failed (hard), rest green}, both below SOFT_MISS_TOLERANCE_THRESHOLD so the soft-class quantifier is the only deciding factor. Each mutation is now caught by exactly one fixture while its pre-existing sibling stays blind, so the teeth come from the new shapes rather than incidental coupling. Baseline entries were derived by hand from the source first and inserted additively (140 insertions, 0 deletions) -- no wholesale regeneration, so no result is self-approving. Also closes the D1/D2 gap in the dashboard equivalence suite: the `fail-safe polarity` fixtures seeded no `health:`/`agent:`/D5/D6 rows, so those rungs contributed ABSENT -- also gray -- and the gray assertions passed via the no-data path while naming the infra path. A `greenScaffold` helper makes every rung but the one under test green-fresh, discriminating pill assertions prove the gray sits over a present red pill, and a new test pins the whole-column D1/D2 stripped-red case api-vs-browser (equal chip, achieved 0, regression true). Fixtures and baseline only -- no classification logic, `combine.ts`, or coherence invariant touched.	2026-07-24 19:57:35 -07:00
Jordan Ritter	49dea09973	test(showcase): stop INV7 false-failing on the I1 ladder-gap shape ... INV7 (`assertChipStripCoherent`) fired on a legitimate engine shape: with no `e2e:` row, `scanWorst` breaks at the ABSENT D3 rung (invariant I1) and never consults D4, so a fresh product-red `chat:<slug>` renders a gray chip over a red D4 pill. INV7 demanded infra evidence for a red the chip provably never read. Round-1 finding a4(iii), unfixed by the round-1 INV7 fix; re-raised as a8. Adds a third, narrow carve-out beside the U8 all-stale one: the gray is excused only when the ladder walk stopped for want of any observation STRICTLY BELOW the shallowest red pill. `ladderGapDepth` tests exactly that ("no row for any of the rung's keys"), which is provably equivalent to `classifyRung`'s ABSENT for a D3/D4/D5 rung — deliberately not the strip's `status === null`, which also covers the `anyExpectedMissing` NO_DATA collapse (a rung that does have an observation and does not stop the walk). `combine.ts` is untouched: the gap-break's own masking is a separate change needing a golden-master re-freeze. The allowance is one-directional and canaried. A gap ABOVE the red excuses nothing, and when the gap-break is eventually fixed the new case's chip stops being gray, so its precondition assertion fails loudly rather than letting the carve-out excuse a shape the engine no longer produces. Also removes INV7's unreachable starter branch (b24). A starter cell has no depth strip, so INV7's precondition is structurally unsatisfiable there; the branch implied coverage that does not exist. It is now a loud throw, and a new test pins the structural reason and fails if the starter axis gains a strip. New cases: the gap shape (green, plus an in-suite mutation proof that the allowance is what passes it), a gap above the red, a gapless gray-over-red, and an UNPERTURBED engine model INV7 still rejects (an infra-red D4 grays a product-red D6 — D6 is outside `scanWorst`'s scan). That last one corrects the file's claim that the engine no longer produces a gray-over-red incoherence. Exhaustive sweep over 295,245 rung combinations: 25,504 INV7 failures without the allowance, 1,456 with. Every remaining one has its gap at or above the shallowest red pill, i.e. none is gap-induced; they are two other pre-existing engine masking mechanisms and INV7 keeps its teeth on them.	2026-07-24 19:57:35 -07:00
Jordan Ritter	ed10b61ffb	test(showcase): give the initial-fetch page cap a two-sided guard ... The cap "drift guard" did not guard. Every assertion in useLiveStatus.supplemental-bounds.test.tsx was an UPPER bound (toBeLessThanOrEqual / not.toContain(cap + 1)), so only an increase was caught. Measured on the pre-fix branch, MAX_INITIAL_FETCH_PAGES could be halved to 10 or cut to 5 with all 6 tests still green. A lowered cap is just as much a silent behavior change -- it moves the cap toward the legitimate read and starts truncating real collections. The bound assertions were also vacuous, because Math.max of an empty array is -Infinity, so a supplemental fetch that never ran at all satisfied them. Both sites now assert the EXACT page sequence a full walk of the cap produces (1..cap plus the one terminal lookahead), sorted because fan-out waves are concurrent, and DISTINCT on the fail-loud bulk path because the throw re-runs the whole loop once per reconnect attempt. Zero pages now fails, a halved cap now fails. The literal is additionally pinned MECHANICALLY against the exported constant, so the "MUST match" claim in its doc is checked rather than trusted. New file useLiveStatus.page-cap-boundary.test.tsx covers the boundary the suites never reached -- a COMPLETE read that ends exactly on the cap page -- in both directions and at both levels. - DIRECT on paginateStatusPages. Exactly cap x 500 rows is NOT truncated (and issues one lookahead, then stops); one row past the cap IS truncated; a read ending on a short page never issues the lookahead; a failed lookahead fails safe to truncated. - THROUGH the hook. A complete 10,000-row collection renders (status live, all rows, red cell still RED from a signal-less bulk row) instead of the permanent blank-dashboard outage; one row past the cap still fails LOUD with an error and no rows. It carries its own ../lib/pb mock rather than reusing the pb-query-eval fake servers. The boundary needs 10,000 rows in the BULK response, which no shared fixture serves, and nothing this file proves should be able to go green or red because of a change to those servers' filter fidelity.	2026-07-24 19:57:35 -07:00
Jordan Ritter	75fdfb3aba	fix(showcase): make truncated mean "rows unread", not "cap exhausted" ... `fetchStatusPages` inferred truncation from cap exhaustion with a full last page. That is not evidence: the loop learns where the collection ends only from a SHORT page (`skipTotal: true` drops `totalItems`/`totalPages`), so a collection of exactly MAX_INITIAL_FETCH_PAGES x INITIAL_PAGE_SIZE rows is read COMPLETELY and still ends on a full page. The bulk consumer THROWS on truncation, so at exactly 10,000 rows the dashboard went offline deterministically and permanently -- every retry, every reload, because the condition is a pure function of the row count. The function's own JSDoc asserted the opposite of what it did. This is the same false-outage class matrix.ts already fixed server-side (any full final page read as truncation took /api/matrix down on every 500-boundary), reintroduced client-side. `truncated` now means exactly "there are rows we did not read", and it is proven rather than inferred: on cap exhaustion with a full last page the loop issues ONE lookahead read of page MAX_INITIAL_FETCH_PAGES + 1 whose rows are discarded and only whose emptiness is load-bearing. Chosen over `skipTotal: false` on page 1 (the matrix.ts shape) because that puts a COUNT(*) on the first-paint path of EVERY load -- the exact overhead `skipTotal: true` exists to drop -- to disambiguate a case that can only arise after 20 consecutive full pages; the lookahead puts the whole cost on the pathological path, where it is a rounding error against the 20 requests that preceded it. A FAILED lookahead fails safe (unproven completeness is reported as truncated), keeping the bulk path's fail-loud posture. The asymmetric on-hit semantics are unchanged and remain correct: supplemental truncation DEGRADES (reds stay red), bulk truncation THROWS (unread rows are whole cells that would render gray = masked failure). The false positive is fixed; the throw is not weakened. The pagination algorithm is hoisted out of the effect as `paginateStatusPages` over an injected page reader, so the `truncated` contract can be asserted directly rather than only through the hook's observable state -- and so every page of a read, INCLUDING the lookahead, provably carries the same `listOpts`. MAX_INITIAL_FETCH_PAGES is exported (for a mechanical, two-sided test pin) and its value is now justified against both bounds it actually has to satisfy: ~3x the measured production worst case below, and bounded runaway damage above; the value is unchanged at 20.	2026-07-24 19:56:10 -07:00
Jordan Ritter	b8b75b7d45	test(showcase): collapse three PocketBase query evaluators into one, verified against a real server ... The dashboard's `useLiveStatus` test doubles carried THREE divergent implementations of the same PocketBase query evaluator — the nominally-shared `__tests__/pb-query-eval.ts`, a second complete private copy inside `useLiveStatus.autocancel.test.tsx`, and a third partial one in the supplemental-bounds fakes. They disagreed on four axes, and the shared one was the weakest of the three. Three definitions of "what PocketBase does" is no definition at all. There is now exactly one, and every semantic in it is a measured observation from a real PocketBase v0.22.21 server (the version `showcase/pocketbase/Dockerfile` pins) plus the real SDK's own `filter()`. `pb-query-eval.test.ts` pins that oracle case by case, with the live result recorded next to each expectation. Fidelity defects fixed, all of which the module's own docs asserted backwards: - Single-quoted literals. `pb.filter()` wraps string params in SINGLE quotes with `\'` escaping and emits numbers/booleans/null BARE. The shared tokenizer handled only double quotes, so every dimension-scoped filter the hook builds threw `unsupported character "'"`. - LIKE is case-INSENSITIVE (it compiles to SQLite `LIKE`). - The implicit `%…%` wrap is triggered by `%` alone, and it ESCAPES `%` and `_` so they match literally. `_` does not suppress the wrap. Both prior copies got a different half of this wrong. - Ordering and equality follow the COLUMN type: `fail_count > 2` selects `fail_count` 10, which the string comparison in the private copy excluded. - `sort=-key` descends instead of silently no-opping. And it now genuinely fails loud rather than claiming to. An unterminated literal, a bare word (which real PB reads as a column reference and 400s on), a dangling operator or escape, an unbalanced paren, trailing input, an unsupported operator, a sort it cannot apply, a non-numeric `page`, and a field the row does not carry all throw — matching the real server, which answers every one of them with a 400. Serving that contract over HTTP is structural now, not per-fake discipline. `matchesPbFilter` used to be called bare inside two `createServer` handlers, so a parse error produced NO response: the hook hung and the test died on a 20 s `waitFor` timeout with the parse message nowhere in sight. Fakes go through `evaluatePbList`, which cannot throw — it returns either a 200 or the real server's own 400 carrying the message. All three supplemental-bounds fakes now answer through it, on both their bulk and their supplemental legs. Two of them ignored `filter` entirely while that file's preamble claimed all of them evaluated it, which left the page-cap tests proving themselves on fixture shape: with the fakes repaired, breaking the supplemental filter turns the supplemental cap test red, and breaking the bulk filter turns the bulk cap test red. Both were green before.	2026-07-24 19:56:02 -07:00
Jordan Ritter	4f092eccb1	Revert "test(showcase): SCRATCH deliberate breakage to prove the new gate fails" ... This reverts the scratch commit. Its only purpose was to prove the two new jobs fail on a real defect in product source — see the PR body for the RED run ID. The branch is back to a strictly additive CI change with no product diff.	2026-07-24 19:23:38 -07:00
Jordan Ritter	e100df27a7	test(showcase): SCRATCH deliberate breakage to prove the new gate fails ... TEMPORARY — reverted in the next commit. A job that has never failed is not yet a gate, so this proves both new jobs actually catch a defect in product source. harness: D4_STALE_AFTER_MS 1h -> 10h in shared/cell-model/staleness.ts (a frozen-green D4 row would credit D4 for ten hours instead of one) dashboard: sortOrder mastra 8 -> 88 in lib/sort-order.ts Both reproduce locally before pushing: harness 3 failed / 177 passed in src/shared/cell-model, dashboard 4 failed / 1326 passed.	2026-07-24 19:19:15 -07:00
Jordan Ritter	f5ff535ed6	test(showcase): add ratcheted quarantine config for the harness unit suite ... Wiring the harness unit suite into CI (next commit) found it already RED on main with exactly three failures. A job that is red on arrival gets ignored or disabled, so this adds a CI-only vitest config that excludes those three files — and a ratchet that stops the exclusion from becoming permanent. vitest.quarantine.json the three entries, each with a date, a reason, and an explicit exit criterion vitest.ci.config.ts `test` minus the quarantined files scripts/quarantine-ratchet.ts re-runs each quarantined file and requires it to STILL FAIL The ratchet is what makes the exclusion defensible: the moment somebody fixes a quarantined test, it goes red and names the entry to delete, so an entry can never outlive the failure it excuses. It also fails on an entry that points at a missing file, on an entry whose filter matches more than one file, and on a malformed manifest. No test is deleted and no assertion is changed. The three failures are a stale hard-coded cell count, a drift test stranded by a completed single-source refactor, and a genuine `D5_REPRESENTATIVES` gap for `browser-use-smoke` that belongs to the D5 owner. `test:ci` and `test:quarantine-ratchet` are declared in the harness package.json `nx.targets` block (project-local, not a workspace default) with `cache: false`: the nx `test` named-input covers `src/**` but not the quarantine manifest, so a cached result could survive an edit to the exclusion list. Local `pnpm test` still uses the unfiltered config — a developer should see the quarantined failures.	2026-07-24 19:13:35 -07:00
Jordan Ritter	938be2accf	test(showcase): reconcile the supplemental-fetch discriminator across both dashboard suites ... Integration defect, introduced by folding the autocancel fake-server fidelity repair together with the bounded/projected supplemental fetch. Each was correct alone; together they disagreed about how a supplemental request is IDENTIFIED. The fidelity repair's `isSupplementalRequest` keyed on the ABSENCE of a `fields` projection. That was true when it was written and is now false: the supplemental fetch sends `STATUS_LIST_FIELDS,signal`. So all three faithful fake servers classified the supplemental request as a BULK page, which: - failed `expect(fields).not.toContain("signal")` on the "every bulk page projects `signal` away" wire contract (the supplemental query was in the bulk bucket), and - left `supplementalQueries` / `firstElemSupplementalPages` EMPTY, so the two `toEqual([1])` bounds assertions compared `[]` against `[1]`. Whole autocancel suite: 3 failed / 3. Not a flake and not a pre-existing failure — it is exactly the two-conventions collision the integration had to resolve. Resolved toward the convention the mocked sibling suite already uses after the projection landed: a request is supplemental iff `signal` IS in its `fields`. That is the property that actually defines the request rather than an artifact of how it happened to be built, so it survives both the projection being added and the bulk projection changing width. Both dashboard suites now assert the SAME convention, and dropping `fields: STATUS_SIGNAL_FIELDS` from the hook turns both of them red (verified), so neither can silently drift back. Also corrected the three comments that still described the supplemental fetch as unprojected. Tests only — no production change. `git diff` against the hook is empty.	2026-07-24 16:37:03 -07:00
Jordan Ritter	51cc63d974	chore(showcase): store demo assets uniformly as Git LFS pointers ... Demo assets under showcase/integrations/*/public/{demo-files,demo-audio}/ were stored two different ways. Ten integrations committed them as LFS pointers (the root .gitattributes convention); eight carved themselves out with a per-integration .gitattributes that re-declared the same paths `-filter -diff -merge`, committing raw binaries instead. Those carve-outs were added when the image build did not fetch LFS, so a pointer stub shipped into the image and the multimodal sample-attachment magic-bytes guard rejected it. That premise no longer holds: the deploy build's Checkout step hardcodes `lfs: true` (7bde1eef3a), so every integration image now gets real binaries regardless of storage form. The overrides are dead weight that only buys divergence. Delete all eight override files and renormalize the 21 affected assets through the LFS clean filter. Each override contained nothing but demo-asset exemptions, so each is removed in full; the root .gitattributes is untouched. Storage form changes, content does not. Every asset's sha256 already equals the LFS OID the pointer-mode integrations reference, so each renormalized blob is bit-for-bit the pointer blob already committed on main -- no new LFS objects are introduced and no pointer can dangle: sample.png 10083 B oid 01aa5681de99461247543e9215c1e4da3242e26b2bee11593fcdbe209672d973 sample.pdf 2486 B oid 3da2afae36a1a81fd2c02f15e54bfc38b6c22e41655c31a5b54ff1e0e3daab41 sample.wav 87078 B oid bd4aa7b049f1c3e324dfd15af4068d7f8fbf2eae1dd044df270dddc5f38a5c57 All three OIDs return download actions from the LFS batch API and were downloaded and confirmed to hash to their OID.	2026-07-24 16:33:14 -07:00
Jordan Ritter	2797618c5e	docs(showcase): correct and anchor stale cold-load fetch doc claims ... Every numeric claim below is re-derived from production PocketBase (showcase-pocketbase-production.up.railway.app) or from repo config on 2026-07-24, not copied from a neighbouring comment. - Row/page counts: 2455 rows / 5 pages -> 3082 rows (~3100) / 7 pages, with the exact source query recorded so the number can be refreshed. The INITIAL_FANOUT_BATCH no-over-fetch note now says that it holds because the CURRENT page count is odd, not as an algorithm property. - Byte estimate: ~1.29 MB -> ~1.6 MB. The retired figure applied the (correct) 371 KB -> 113 KB per-500-row basis to the OLD 2455-row collection. Measured all 7 pages both ways: 2.34 MB full vs 0.70 MB projected = 1.64 MB of signal. - Worst-case re-delivery window: ~29 min -> ~60 min. staleness.ts and harness/config/probes/*.yml schedule e2e-demos, starter_smoke and d6-all-pills-e2e HOURLY (10/40 * * * *); aimock-wiring is 6-hourly and the drift probes are weekly/monthly. - Restored the coverage-gap acknowledgement this PR deleted, rewritten for what is true today: the supplemental union is neither minimal nor complete. It is green-blind outside clause 1 (2646 rows carry a signal but sit outside it; the 731 green per-cell rows under the aggregate dimensions are excluded by BOTH clauses). Currently latent (0 uncovered rows carry __fleetCommError) but structurally open, and it is the hole that let the mis-scoped signal-provenance misreport ship green. - Anchored the 94%/6% product-vs-infra split with its query, date and partitioning predicate (336 vs 21 of 357 red rows), and flagged it as a snapshot the polarity argument does not depend on. Softened the unreproducible "~20% of the matrix" to the derivable bounds. - Version-qualified the STATUS_LIST_FIELDS omission-vs-null claim to the pinned PocketBase 0.22.21 and recorded the upgrade hazard: PB >=0.23's PublicExport isVisible/GetHidden path lets a field-level hidden flag omit signal independently of our projection, reintroducing the ambiguity the provenance flag exists to avoid. INTEGRATION RECONCILE (folded in when this was cherry-picked on top of the signalKnown-scope fix). Three conflicts in useLiveStatus.ts, all row/page-count prose, resolved to the measured 3082-row / 7-page anchor and to this commit's corrected INITIAL_FANOUT_BATCH note (the "page 7 ends its own wave" claim is only true at an ODD page count, which the superseded text asserted as an algorithm property). Additionally, every doc reference to `RawRung.signalKnown` — a field the scope fix DELETED — was rewritten to name the surviving red-row-scoped `FamilyFold.redSignalKnown`, so no comment describes a flag that no longer exists: live-status.ts STATUS_LIST_FIELDS doc (3 sites), useLiveStatus.ts coverage-gap note, api-matrix-equivalence.test.ts (2 sites). Comments only - no logic, no reformatting, no reordering.	2026-07-24 16:31:30 -07:00
Jordan Ritter	3dea9f6424	test(showcase): make INV7 chip/strip coherence sound (scope + U8 stale exemption) ... INV7 (added with the cold-load gray-masks-red fix) was unsound in two ways that only failed to bite by fixture accident: 1. WRONG QUANTIFICATION. Its doc says "every CONTRIBUTING red row", but it filtered over every row in `live.values()` — a whole-matrix map. Any red row belonging to a different column or feature could fail the invariant for a cell it contributes nothing to. Now quantified over the exact keyspace `buildCellModel` collects (`contributingKeys`). 2. MISSING isStaleCell EXEMPTION. `buildCellModel` deliberately force-grays an all-stale cell on every path (agent, starter, null-feature), folding ANY stale colour — red included — to the "re-sweep pending" gray. That gray is a recency claim about the cell, not an infra attribution for the red pill, so an all-stale cell containing a red row legitimately renders gray over a red pill. INV7 failed it. Exempted, with the reasoning inline. Both holes were latent: the current `stale-*` fixture variants are green/degraded, and the matrix has no cross-cell rows, so nothing hit them — the next fixture added would have tripped a false positive. Also adds an anti-vacuity guard (the narrowed quantification must still find at least one contributing red row whenever the strip reads red, so a future drift between `contributingKeys` and the engine's keyspace fails loudly instead of silently passing everything), and fixes the synthetic contribution helper `c()`, which set `freshestAgeMs: 0` for `ABSENT` — a shape `classifyRung` never produces (it returns `null`: an absent rung has no observation to age). New tests cover both false positives, a control, and four teeth cases proving INV7 still fails a genuine non-stale gray-chip-over-red-pill.	2026-07-24 16:27:40 -07:00
Jordan Ritter	f5178765fc	fix(showcase): bound the dashboard's supplemental signal fetch and stop its failure blanking the matrix ... PR #6156 widened the browser's supplemental `signal` fetch from "comm-error aggregate rows only" to "comm-error aggregates ∪ every non-green row, all dimensions", and in doing so removed the early return that had bounded the work. What was left was a `for(;;)` with no page cap, unprojected, awaited before any row is returned — so it degrades exactly when the dashboard matters most. Measured against production: nominally 357 of 3082 rows (one page, 429 KB), but under a full-column outage the non-green set trends toward the whole collection — 7 pages, 1.73 MB — all of it on the critical path to first paint. Against a server that never returns a short page it does not terminate at all. Worse than slow: a supplemental rejection propagated through fetchInitial → connect() → the retry chain → `setRows([])` + `status: "error"`. A failure of an ENRICHMENT fetch blanked the ENTIRE dashboard, which directly contradicts the fail-safe rationale the PR was written to establish. (It also leaked an unhandled rejection per attempt, because the promise rejects while the bulk pages are still in flight and nothing is awaiting it yet.) Three changes, all on the dashboard hook: PAGE CAP. Both initial-fetch loops now share one bounded implementation (`fetchStatusPages`) and one cap, `MAX_INITIAL_FETCH_PAGES = 20`. Sized as a runaway guard rather than a budget: ~3x the real collection, so it never truncates a legitimate read including the degenerate all-non-green case. The supplemental loop also picks up the wave-concurrency and short-page merge guards the bulk loop already had and it did not, which is what made the parity structural instead of a promise two comment blocks made to each other. On-cap behavior is deliberately ASYMMETRIC: supplemental truncation DEGRADES (warn, use what arrived — the unreached rows keep their signal-less bulk copy, which classifyRung paints RED), while bulk truncation THROWS, because rows the bulk never read are whole cells that would render as no-data GRAY, i.e. exactly the masked-failure polarity this PR exists to remove. NON-FATAL ENRICHMENT. The rejection handler is attached at the fetch site, so a supplemental failure yields an empty row set, the merge degrades to "bulk rows, unenriched", and the dashboard RENDERS. Reds stay red via the classifier's fail-safe polarity; the residual cost is that a comm-error overlay on a green aggregate waits for its next SSE delta — the pre-CF7-F3-#1 behavior for the duration of one failed request, instead of a blank matrix. Attaching at creation also removes the unhandled rejection. PROJECTION. The supplemental fetch now sends `STATUS_LIST_FIELDS + signal` instead of no projection. That is exactly the declared StatusRow shape, derived from the drift-guarded constant so the two cannot diverge, which drops the columns PocketBase adds and nothing reads (collectionId, collectionName, created, updated, state_written_at, written_by): 429,085 → 362,982 bytes measured on the real non-green set, ~260 KB at incident scale. Deliberately NOT narrower — the merge treats a supplemental row as a COMPLETE row (replaces its bulk twin wholesale, appends one the bulk snapshot missed), so `key,observed_at,signal` would have saved another 58 KB by forcing a graft onto a row of a different vintage, which is the chimera `supplementalRowIsOlder` exists to prevent. It stays on the first-paint path. Enriching after first render would have removed it from the critical path, but a non-green row's chip can move once its signal lands (red → gray for a positively-infra red, red → amber for a soft-class first strike), and a cold-load verdict that corrects itself a beat later is the class of symptom this PR set out to eliminate. With the fetch bounded, wave-concurrent and projected it runs inside the bulk fetch's own round-trip budget, so gating first paint costs nothing on the happy path. Tests are REAL-SDK over real sockets, with fake endpoints that actually evaluate `filter`/`fields`/`sort` (new `__tests__/pb-query-eval.ts`) so a response is a function of the query the hook sent rather than of fixture layout. Verified by mutation: restoring the unbounded loop, the fatal rejection, or the missing projection each turns its own test red.	2026-07-24 16:27:25 -07:00
Jordan Ritter	ebd7ac681d	test(showcase): restore PB filter/fields fidelity to the autocancel fake servers ... PR #6156 silently invalidated the dropped-tail pagination guards added by PR #4504. Regression in test COVERAGE, not production code — which is why CI stayed green. The three fake PB servers in the autocancel suite ignored the `filter` query param (and `fields`). #6156 made the supplemental `signal` fetch UNCONDITIONAL (dropping the `commAggregateFilter === null` early return and widening the filter to `state != "green"`), so a filter-ignoring server answered that now-always-issued fetch with the ENTIRE dataset — and `fetchInitial`'s leftover-append then REPAIRED whatever rows the bulk pagination dropped. Mutating the merge to drop the entire short page kept all three tests GREEN. The servers now answer a list request the way real PocketBase does: filter, then sort, then the page slice, then the `fields` projection, with the COUNT(*) envelope omitted only when `skipTotal` is set. The filter evaluator covers exactly the grammar `useLiveStatus` emits and THROWS on anything else (answered as 400) — a silently-unparsed clause would degrade to "match everything", i.e. straight back into the masking bug. Real-SDK quoting is matched (single-quoted string params, bare numbers/booleans, verified against the installed SDK); the sibling suite's double-quoting `pb.filter` stub flaw is deliberately NOT copied forward, and a wire assertion pins the convention. The supplemental fetch is discriminated by the ABSENCE of a `fields` projection — the sibling `useLiveStatus.test.tsx` convention, reused rather than reinvented. This replaces a `filter.includes("state !=")` substring check that the union filter had already made unreliable. Broken assertions repaired: - over-fetch instrumentation no longer conflates the supplemental serial walk with the bulk fan-out (separate page arrays), and the split is now asserted rather than assumed - the FIRST-wave-element server serves POISON rows on the page after the short page. Real PB pagination is monotonic, so that page is empty and appending it changes nothing — which is exactly why deleting the merge's boundary logic left the test green. Poison makes a merge that runs past the boundary observable - `.some(r => r.id === "f0699")` replaced with `rows.at(-1)?.id`, the LAST-row boundary check the comment already claimed, plus explicit poison-absence - the mid-wave suite no longer claims a boundary property its fixture cannot measure (its short page is the wave's last element); name and comment now state what it does measure, and it gained a real tail-boundary assertion Mutation red-green, both directions. Drop-the-entire-short-page: PASSED 3/3 before, now FAILS 3/3 (1300 to 1000, 1250 to 1000, 700 to 500). Boundary-logic-removed: PASSED before, now FAILS on the suite that claims it (700 to 1000). Reversed-merge-order (same row set): PASSED before, now FAILS on both new `.at(-1)` assertions. Mutations reverted: 3/3 pass. Test-only change — no production code touched.	2026-07-24 16:27:21 -07:00
Jordan Ritter	2a68282d08	fix(showcase): scope the infra-red gray precondition to the RED rows ... `classifyRung`'s U7 gray branch reads the `signal` blobs of a family's RED rows and nothing else, but the precondition qualifying what it read — "was `signal` actually delivered, or projected away?" — was `raw.signalKnown`, a boolean AND over EVERY present row of the family. A predicate about red-row evidence was answerable "no" by a row the branch never looks at. That broke exactly the case the precondition was written to protect. The dashboard's supplemental cold-load fetch restores `signal` for `state != "green"` only (by design — the classifier needs it solely in the red branch), so in a MIXED-state family the red rows arrive WITH attribution and the green siblings arrive WITHOUT it. The family-wide flag therefore read `false` precisely when it should have read `true`: D4 = green `chat` + infra-red `tools` skipped the gray branch, fell through first-strike (its only red is infra, so `maxNonInfraRedFailCount` is null), and landed on FAIL_FRESH — the browser painting RED a cell that `/api/matrix`, which always has the full `signal`, reports as gray. That is the §11.4 api-vs-render drift the read-model exists to forbid, and it made INFRA_RED_FRESH effectively unreachable in the browser for every multi-row family: D4, and any multi-pill D5/D6 (whose per-cell `d5:<slug>/<pill>` keys the supplemental filter's clause-1 `key !~ "%/%"` excludes). Only single-key D3 was unaffected — there the family's one row IS the red row and the two scopes coincide, which is why every fixture missed it. The fix derives the flag where the rows are, at the scope that consumes it: `foldFamily` now reports `redSignalKnown` over the RED rows only, alongside the red-scoped `hasNonInfraRed` / `maxNonInfraRedFailCount` / `allRedSoftClass` it already computed. `RawRung.signalKnown` and its `gatherRows` plumbing are removed rather than left dead — a family-scoped flag named `signalKnown` sitting next to a red-scoped consumer is the trap that produced this bug. `anyExpectedMissing` stays family-scoped, correctly: a missing sub-key IS a family property and it gates a family-scoped verdict. `redSignalKnown` is today IMPLIED by `!hasNonInfraRed`, since `signalHasInfraErrorClass(undefined)` is false and a stripped red row forces `hasNonInfraRed` on its own. It is kept explicit anyway — this is the masks-real-red guard and it must not rest on a coincidental property of a helper two modules away — and pinned directly on the fold so the scope cannot regress silently. Coverage. The previous tests named `signalKnown: false` but were insensitive to it: `raw.signalKnown &&` could be deleted outright and all 181 harness cell-model tests plus 58 dashboard equivalence tests stayed green, because every fixture stripped `signal` from ALL rows of the target rung and never from a green sibling. The new tests construct the mixed-state family that ships the bug, on the real derivation surface (`buildCellModel`, not a hand-passed flag): D4 chip, multi-key D5 chip, and the D6 `d6Effective` badge — D6 is the soft-parity top so its chip is amber either way, but its badge drifts red-vs-null. Each asserts browser == server == `/api/matrix`. `api-matrix-equivalence`'s `coldLoadChip === serverChip` guard is extended off single-key D3 to the D4 family. Five `foldFamily` tests pin the flag's scope directly, and the fail-safe polarity keeps its negative control: a red row whose OWN `signal` was stripped still renders RED.	2026-07-24 16:27:17 -07:00
Jordan Ritter	510bb88553	fix(showcase): use each integration's own name in demo page titles ... 14 integrations' `src/app/demos/layout.tsx` hardcoded "LangChain - Python" in `generateMetadata` — a copy-paste leftover from langgraph-python, which the file was cloned from. Every `/demos/*` page in mastra, strands, ag2, agno and 10 others rendered `<title>LangChain - Python</title>`. Each now uses the display name from its own `manifest.yaml` `name:` field, matching the convention the already-correct integrations use (langgraph-typescript -> "LangGraph (TypeScript)", strands-typescript -> "AWS Strands (TypeScript)"). langgraph-python itself is included: its manifest name and root layout both say "LangGraph (Python)", so "LangChain - Python" (the legacy Notion partner-column label) was stale there too.	2026-07-24 16:19:17 -07:00
Jordan Ritter	e1c801313b	fix(showcase): bundle multimodal sample assets for agno and mastra ... agno and mastra were the only 2 of 20 integrations with no `public/demo-files/` directory ever committed. Their Attachments demo fetches `/demo-files/sample.png` before building the message, so the demo failed at the fetch with: [multimodal-demo] sample-attachment send failed Error: Could not fetch sample "sample.png" — HTTP 404. Is the file bundled under public/demo-files/sample.png? Nothing was ever sent to the agent — `userMsgCount: 0` — so the D5 probe timed out on turn 1. Both integrations already have working vision-capable multimodal agents and correct aimock fixtures; this was purely a missing-asset/packaging bug. Copies sample.png / sample.pdf / README.md byte-for-byte from langgraph-python (the canonical source) into each integration's own `public/demo-files/` — a per-integration duplicated directory, not part of the `shared/` symlink mechanism, which is exactly how two integrations came to be missing it. The assets are committed as plain Git LFS pointers, governed by the repo-root `*.png` / `*.pdf` LFS rules, with no per-integration `.gitattributes` override. LFS tracking stays uniform across the fleet; the deploy-time problem (integration image builds do not fetch LFS) is fixed in the build, not by carving these files out of LFS. The pointer OIDs are the ones already stored for the identical assets in the other integrations, so the backing LFS objects exist server-side and are not dangling: sample.png sha256:01aa5681de99461247543e9215c1e4da3242e26b2bee11593fcdbe209672d973 10083 sample.pdf sha256:3da2afae36a1a81fd2c02f15e54bfc38b6c22e41655c31a5b54ff1e0e3daab41 2486 NOTE: this change does not make agno/mastra pass on its own. It requires the build-side LFS fetch fix to land first or together.	2026-07-24 16:15:45 -07:00
Jordan Ritter	233b509928	fix(showcase/mastra): restore shared-tools symlink, shrink erosion ratchet ... `showcase/integrations/mastra/shared-tools` was a real committed directory where a symlink into the single source of truth belongs, violating the single-source symlink mechanism documented in showcase/AGENTS.md. Root cause: commit 534cd1efa7 ("fix(showcase): D5 integration fixes across 12 frameworks") deleted the symlink and committed 14 real files in its place — the classic clobber the guard in showcase/scripts/validate-shared-symlinks.ts was written to catch. The symlink was originally added by 93d5815cdb and pointed at `../../shared/typescript/tools`. Divergence inventory: NONE. All 14 files were byte-identical to showcase/shared/typescript/tools (identical git blob hashes and sha256s), so nothing mastra-specific, stale, or additive was carried in the copy. Restoring the symlink is therefore a pure structural fix with zero content change — the restored link blob is the same object (ddc634b6) the pre-erosion symlink had. Also removes the healed `mastra/shared-tools` key from validate-shared-symlinks.baseline.json, per the shrink-only ratchet (the validator reports stale entries specifically so this is mechanical). langgraph-typescript/shared-tools and claude-sdk-typescript/shared-tools are eroded the same way by the same commit and are also byte-identical to the shared source; they stay baselined here and are left for follow-up so each conversion carries its own behavior proof.	2026-07-24 16:09:29 -07:00
Jordan Ritter	006a42aeb6	test(showcase): track crewai-crews wired-count bound after multimodal unsupported ... `generate-catalog.test.ts` pinned crewai-crews' wired-cell count at >= 30. Moving `multimodal` from `features` to `not_supported_features` reclassifies that cell from `wired` to `unsupported`, so the real count is 29 and CI's `showcase/scripts` vitest failed. The bound is a snapshot floor, not an invariant: the test's own comment says the partial parity tier only requires intersection >= 3 with the reference's wired set. 29 still clears that by a wide margin, so the floor tracks the manifest. Updated to 29 with a note recording which cell moved and why, so the next reader does not read the decrement as a regression.	2026-07-24 16:02:26 -07:00
Jordan Ritter	4a40604aca	chore(showcase): mark multimodal unsupported for llamaindex and crewai-crews ... Attachments (`multimodal`) has never worked on either integration, and both manifests were over-claiming it under `features`, so the D5/D6 probe correctly reported red — a red chip that read as "this regressed" when the truth is "this was never built". Use the repo's existing unsupported mechanism — `not_supported_features` in the integration manifest, which `determineCellStatus` (showcase/harness/src/shared/catalog/catalog-flatten.ts:239) folds to `status: "unsupported"` -> `isSupported: false` -> the `UNSUPPORTED` CellModel (gray, `supported: false`, 🚫). Both cells now render as genuinely unsupported: not green (no false claim of working) and not red (no false claim of regression). Each entry carries an inline note recording WHY, so a future reader does not mistake it for an oversight. - llamaindex: upstream gap in `llama-index-protocols-ag-ui==0.2.2` (`utils.py:82-85` passes AG-UI content parts straight into `ChatMessage`, which rejects them on the `block_type` discriminator). - crewai-crews: never implemented on our side — `src/agent_server.py` registers no `/multimodal` route, so the demo route aliases the generic crew and fails. Also corrects the llamaindex `multimodal_agent.py` docstring, which asserted a normalization the upstream router does not perform. Demo entries stay wired on both, matching every other `not_supported_features` entry in these manifests.	2026-07-24 15:56:31 -07:00
Jordan Ritter	49e7b2174c	fix(showcase/ms-agent-python): keep the user's prompt on the multimodal PDF turn ... The multimodal PDF turn dropped the user's question out of the final outbound user message, so the model was handed a document dump with no question attached. Against aimock's strict mode that surfaced as `503 no_fixture_match` on turn 2 (turn 1, the image, passed); against a real LLM it would have silently answered the wrong thing. Root cause is a serialisation mismatch, not a fixture gap. `agent_framework_openai._chat_completion_client._prepare_message_for_openai` emits ONE OpenAI message per `Content` — it builds a fresh `args` dict on every iteration of its content loop. `_PdfFlattenChatMiddleware` appended the flattened `[Attached document]` text as a SECOND text `Content` beside the prompt, so one logical user turn serialised to two consecutive user messages: prompt-only, then document-only. Anything reading "the current user turn" off the tail of the list saw only the document. Merge the flattened document INTO the message's existing prompt text content instead, so the turn stays a single text content and serialises to a single user message reading `"<prompt>\n[Attached document]\n<body>"`. langgraph-python's equivalent agent is green precisely because LangChain keeps multiple text parts inside one message rather than splitting them. The merge copies the prompt `Content` rather than mutating it: the middleware restores the original `contents` list after the model call, and that restore only undoes the list swap — an in-place mutation would leak the raw PDF body into the AG-UI MESSAGES_SNAPSHOT and render it in the user's chat bubble. Also dedupe identical flattened blocks. The page's `LegacyConverterShim` appends a legacy `binary` mirror alongside every modern attachment part, so the same PDF arrives twice and its body was being sent to the model twice. No fixture change: the existing `userMessage` match key is correct and is what the corrected request shape satisfies.	2026-07-24 15:56:22 -07:00
Jordan Ritter	3198b6bc2e	fix(showcase): re-fetch signal for non-green rows on dashboard cold load ... The dashboard's bulk initial fetch projects the heavy `signal` blob away (`STATUS_LIST_FIELDS`) for a real payload win — measured at ~70% of the response, 371 KB → 113 KB per 500-row page. A supplemental fetch then restored `signal` for the rows that are read at render time, but it only ever covered the comm-error AGGREGATE rows: dimensions `d6`/`d4`/`e2e-demos`/ `d5-single-pill-e2e`, narrowed by `key !~ "%/%"`. `classifyRung` also reads `signal`, to tell an INFRA red from a PRODUCT red, and the rows it needs were excluded twice over: they are mostly dimension `d5`/`e2e`/`health` (absent from FLEET_COMM_AGGREGATE_DIMENSIONS entirely) AND they are per-cell `<dim>:<slug>/<featureId>` keys that `key !~ "%/%"` filters out. So on every cold load those rungs arrived with no attribution at all, and self-corrected only when the probe's next sweep rewrote that specific row and the SSE delta redelivered it with `signal` — up to a full sweep interval (~29 min observed, ~14 min mean), again on every reload. The supplemental filter becomes a UNION: the existing comm-error clause OR `state != "green"`. Scoping by STATE is what keeps it cheap — `classifyRung` consults `signal` only in its red branch, so green rows never need it, and in production the non-green rows are ~360 of ~3100 (~430 KB, about a third of what shipping `signal` on every row would cost) with no schema change. `!= "green"` rather than an explicit red/degraded list also catches an out-of-vocabulary state, which `rankOfState` ranks WORST — the rows that matter most can never fall outside the fetch. Widening the EXISTING fetch rather than adding a second one reuses its freshness guard (`supplementalRowIsOlder`) and chimera-avoidance merge verbatim, keeps first paint to one extra request, and keeps the fail-loud retry posture. A dimension-scoped hook still narrows the comm-error clause to the matched literal, and drops it entirely when the scope sits outside the aggregate set — but the non-green clause always applies, so such a scope no longer skips the supplemental fetch altogether. Its test is updated accordingly. Also fixes two test doubles that identified the supplemental request by `filter` containing `key !~`. That marker now disappears for an out-of-set dimension scope, so both would silently misclassify a supplemental request as a BULK page and corrupt the fan-out instrumentation. They key off the absence of a `fields` projection instead — the property that structurally distinguishes the two requests and the whole reason this one exists.	2026-07-24 15:25:33 -07:00
Jordan Ritter	3549f3e370	fix(showcase): a red rung with unknown infra-ness must render RED, not gray ... `classifyRung` opened its red branch with `if (!raw.signalKnown) return NO_DATA` — so whenever the dashboard's bulk fetch had projected the `signal` blob away, a genuinely-failing rung was classified "no data" and painted the muted gray "nothing to see here" chip, while the depth strip beside it correctly rendered `1P ✗`. That polarity is backwards twice over: - `signalKnown === false` never meant "known to have no signal". PocketBase omits the `signal` key ONLY under a `fields=` projection; a row that genuinely has no signal arrives as `null`, and `null !== undefined`. So the flag means "this row came from a projected fetch" — a PENDING attribution, the absence of evidence about WHY the rung failed. It was never evidence that the failure was infra. - Measured against production, ~94% of reds are product-class and only ~6% are infra-class. Graying every unattributed red lost 94 real failures to suppress 6 false alarms — and a masked red is never investigated, while a false alarm is looked at once and closed. Graying a red now requires POSITIVE infra evidence: `hasNonInfraRed` is false only when every contributing red row's blob actually carries an INFRA_ERROR_CLASSES attribution, and `signalKnown` is retained as an explicit second precondition so a missing blob can never be read as an infra attribution. The residual cost is over-reporting, which is the direction a health dashboard has to fail in. Tests: the harness `§7 I5` case and the dashboard's api/render equivalence case both ASSERTED the buggy gray as intended behaviour — which is why CI never caught this. Both are inverted, with the reasoning recorded inline. Four golden-master fixtures (`pos-d{3,4,5,6}-red-signal-unknown`) are re-frozen; the regenerated baseline diff touches those 4 of 56 and nothing else. Adds coherence invariant INV7: a gray chip sitting above a red depth pill requires positive infra evidence on every red row. INV1-INV6 only relate chipColor to the other CHIP-side outputs and never inspect the d3/d4/d5/d6 pills, so the engine could return one object saying both `d5.status === "red"` and `chipColor === "gray"` with every invariant intact. INV7 fails on the old polarity and closes the gap in the only safe direction — by fixing the chip upward, never by muting the strip.	2026-07-24 15:25:15 -07:00
David McKay	188bc6cb79	docs(react-native): expand into a production guide ... Rewrites the React Native page into a production guide (Metro, polyfills, provider options, runtime/model wiring, device + bench connectivity, frontend tools, run lifecycle, voice), and corrects the source-backed inaccuracies found in review. Every claim below was re-verified against the package source on main (@copilotkit/react-native 1.63.2), not inferred. Import surfaces (was "What's included" + "Headless imports") - Documents all THREE entry points separately with the native peers each forces Metro to resolve: /headless (none), root (expo-document-picker, expo-file-system via useAttachments), /components (@gorhom/bottom-sheet, react-native-streamdown). - Removes the claim that the root barrel pulls @gorhom/bottom-sheet. It does not -- the only import is src/components/CopilotModal.tsx, reachable solely from /components. (The package's own source comments state this incorrectly; that is what the previous draft was written from.) - Warns that root's CopilotChat/CopilotModal are HEADLESS wrappers rendering only children, while the same-named /components exports are the rendered chat -- a silent blank-screen trap. - The quickstart now imports from /headless throughout, matching its own advice and keeping readers out of the release-bundle failure the page warns about. - Corrects "the provider re-exports hooks" to the package, enumerates the shared hooks, and drops "behave identically to the web SDK": React Native ships its OWN useRenderTool (requires parameters, accepts handler, returns ReactElement | null, no wildcard) and omits the web-only rendering hooks. Metro - Corrects the cause: jose is not a JWT/license dependency. It arrives via telemetry (@copilotkit/shared -> @segment/analytics-node -> jose). - Replaces the global unstable_conditionNames array with resolveRequest scoped to jose (Metro's own documented recipe). unstable_conditionNames is an UNORDERED SET of asserted conditions -- target priority comes from each package's own exports key order, so reordering that array does nothing. The previous config also applied browser globally and dropped Metro's default react-native condition. - Fixes version bounds: package exports arrived opt-in in RN 0.72 / Metro 0.76.1 and is default-on from Metro 0.82 / RN 0.79 (not "0.70+"). Polyfills - Makes the quickstart canonical instead of correcting it later. Drops "optional belt-and-suspenders". - Documents the crypto ordering as a HARD requirement: CopilotKit's polyfill and react-native-get-random-values are both first-writer-wins, so any CopilotKit import evaluated first permanently locks in the non-cryptographic Math.random fallback and silently no-ops the secure library. Provider / runtime - headers: a function form is evaluated when the PROVIDER RENDERS, memoized, and pushed to the core -- never per request. Adds rotating-token guidance (drive from state, or call copilotkit.setHeaders on refresh). - cors: corrects the causal claim. CORS is browser-enforced; React Native's native stack sends no Origin and never consults Access-Control-Allow-Origin, so cors:true is irrelevant to native reachability (it matters for Expo Web). Device connectivity - Labels the adb reverse row Android-only and adds an iOS device row. - Adds the missing `npx expo install expo-build-properties` step. - Adds the iOS path, which was absent: ATS applies to debug and release alike, expo-build-properties has no iOS ATS option, and since iOS 17 ATS rejects raw IPs unless listed in NSExceptionDomains -- plus NSLocalNetworkUsageDescription for the local-network permission gate. Notes why NSAllowsLocalNetworking alone is insufficient. Frontend tools - Replaces the incorrect "useFrontendTool can carry a render function drawn as the tool runs". The rendered RN chat reads a registry populated only by RN's useRenderTool; a render passed to useFrontendTool is never drawn there and yields the fallback "Called: <toolName>" stub. Other - Removes the useAgent({ threadId }) example: unsupported on main, does not typecheck (TS2353), silently ignored. Recorded as a known limitation pointing at #6141 instead, so this page no longer depends on that PR. - Known limitations: drops the "No pre-built UI is required" positioning line; corrects the markdown entry (the rendered chat DOES render markdown via CopilotMarkdown/react-native-streamdown -- the old entry recommended the wrong library); adds the threadId and web-only-hooks entries. - Broadens the frontmatter description, which still promised only "get started". Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-24 14:39:37 -07:00
Alem Tuzlak	d7ba17277c	fix(showcase/built-in-agent): real beautiful-chat backend (verified vs real LLM) (#6150) ... ## Why Follow-up to #6106 / #6149. Prod testing (real LLM) revealed the flagship **beautiful-chat** cell was a **stub**: it was wired to the generic all-tools agent on a bare route, so the model reported `query_data` / Excalidraw / gen-UI tools "not available." It only passed local aimock D6 because fixtures scripted the tool calls — **aimock can't validate real-LLM tool availability**, which is why this slipped through. ## Fix Port the real beautiful-chat backend, mirroring `langgraph-python/src/agents/beautiful_chat.py`: - **New `createBeautifulChatAgent()`** — server tools `query_data` (sales data), `manage_todos`, `get_todos`, `search_flights` (A2UI fixed-schema flight cards); declares the runtime-injected tools (`render_a2ui`, `generateSandboxedUi`, Excalidraw MCP) so the model can actually call them; `manage_todos` → `STATE_DELTA /todos` for the app-mode board. Model `gpt-5.4`. - **Route** configures the combined runtime like LGP: `openGenerativeUI: true`, `a2ui.injectA2UITool: true` (catalog `copilotkit://app-dashboard-catalog`), `mcpApps` (Excalidraw). - Shared factory defaults unchanged (new opt-in factory). ## Verified against a REAL OpenAI LLM (not aimock) Drove a headless browser against the demo pointed at real OpenAI — all 6 flows fire the tool + render the component: | Flow | Result | |---|---| | Pie chart (`query_data`→`pieChart`) | ✅ real data | | Bar chart (`query_data`→`barChart`) | ✅ real data | | Calculator (`generateSandboxedUi`) | ✅ sandboxed iframe | | Flights (`search_flights` A2UI) | ✅ 2 FlightCards | | Excalidraw (`create_view` MCP) | ✅ MCP iframe | | Todos (`enableAppMode`→`manage_todos`→`get_todos`) | ✅ board populated | Zero "not available" apologies. ## Follow-up (tracked, not in this PR yet) Auditing the other generic-agent-backed demos against a real LLM for the same class of under-wiring; fixes will be added here or in a follow-up.	2026-07-24 20:23:28 +02:00
Ben Taylor	a00592f115	docs(channels): add Channels SDK documentation (#6031) ... ## What does this PR do? Adds documentation for the **`@copilotkit/channels` SDK** — running a CopilotKit agent as a bot inside Slack, Teams, WhatsApp, Telegram, and Discord. New `---Channels---` nav group under `showcase/shell-docs`. **Guides** - Overview (agent + JSX + channel, end to end), Quickstart (TanStack factory agent on the Intelligence adapter) - UI Library (JSX component rendering, callbacks, component registration, `{ raw }` escape hatch) - Interactive flows: Human-in-the-loop (+ agent-initiated `onInterrupt`), Reactions (+ emoji helpers), Modals - Commands & global reactions, Files & multimodality, MCP (agent-side), Channel configuration, Tools & context - Platforms: overview of what Intelligence handles for you, plus per-adapter pages (Teams / WhatsApp / Slack / Telegram / Discord) with built-in tools/context **API reference** (signatures pulled from the installed `0.2.1` type definitions): Channel, Thread, JSX callbacks. All examples use the Intelligence adapter except the platform-specific pages, which use each platform's own adapter. Every internal link resolves; MDX validated. ## Related PRs and Issues - N/A ## Checklist - [x] I have read the Contribution Guide - [x] If the PR changes or adds functionality, I have updated the relevant documentation - [x] "Allow edits by maintainers" is checked 🤖 Generated with [Claude Code](https://claude.com/claude-code)	2026-07-24 11:42:08 -05:00
Benjamin Taylor	4f3c820908	docs(channels): add required agents:{} to runtime snippets + keep-alive note (adversarial review) ... Adversarial pass caught that every new CopilotRuntime({...}) snippet omitted the required non-optional `agents` field (TS2741 on copy-paste) — added `agents: {}` matching examples/slack/app/managed.ts. Also noted that the managed index/mcp snippets end at ready() with no server, so a reader's process would exit — added a keep-alive pointer to the Quickstart's full listener mount. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-24 10:59:04 -05:00
Benjamin Taylor	6d1205d0f5	docs(channels): reconcile with Plan C — runtime-driven lifecycle, Intelligence key required ... Channels run only through the Intelligence runtime now (PR #6145): public channel.start()/stop()/addAdapter() are removed. Updated the run path to new CopilotRuntime({ intelligence, channels }) + handler/listener.channels.ready()/stop() across quickstart, index, mcp, configuration, the 5 platform pages, and the channel reference; reframed the managed-vs-direct table (both require an Intelligence key — the difference is who holds the platform credentials); name is now required (the runtime keys lifecycle by it); surfaced the Intelligence-key (free tier) prerequisite. Standardized the managed pattern on zero-adapter createChannel({ name }) (runtime attaches the managed transport), matching examples/slack/app/managed.ts. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-24 10:44:21 -05:00
Lukas Moschitz	9ab49609f0	fix(showcase): make langgraph-fastapi get_stock_price scriptable ... tool-rendering-custom-catchall renders the ticker quote in the wildcard card. fastapi's get_stock_price(ticker) had no scriptable args, so when the aimock fixture emitted price_usd/change_pct the @tool silently dropped the unknown kwargs and returned random numbers instead of the scripted quote — the card showed nondeterministic values, diverging from the north star. Port LGP's scriptable signature: get_stock_price(ticker, price_usd=None, change_pct=None) — echoes scripted values verbatim when supplied, random when omitted (backward-compatible with the other tool-rendering pills). Behavior byte-aligned to langgraph-python. D6 tool-rendering-custom-catchall green (two real ~8s runs); shared-backend siblings tool-rendering, tool-rendering-default-catchall, headless-complete all re-verified green.	2026-07-24 17:15:14 +02:00
Lukas Moschitz	797d30ca63	fix(showcase): port langgraph-fastapi gen-ui-headless-complete to parity ... Two compounding defects kept this cell red: 1. Backend missing get_revenue_chart. headless_complete.py never registered the revenue-chart tool, so the chart-turn never emitted a tool call and the card never mounted. Port LGP's get_revenue_chart tool + its system-prompt routing rule (tool return shape byte-identical to LGP). 2. Stale turnIndex fixtures caused turn-2 and turn-4 loops to the recursion limit. Align to LGP's canonical shape: - tool-rendering.json (shared): AAPL first-leg turnIndex:0 -> hasToolResult:false so it stops re-firing at turnIndex>=2 in the multi-pill thread. - headless-complete.json: rewrite to LGP's structure (narration/toolCallId fixtures first, tool-call legs after with userMessage+context only, no stale turnIndex); drop the divergent fastapi-only highlight fixtures, subsumed by the broader substring fixture. D6 langgraph-fastapi:gen-ui-headless-complete now green (two real ~29s runs); tool-rendering re-verified green (shared-fixture no regression).	2026-07-24 16:39:55 +02:00
Lukas Moschitz	243acc7771	fix(showcase): add langgraph-fastapi gen-ui-agent graph + recursion limit ... gen-ui-agent had no dedicated backend graph: langgraph.json lacked a gen_ui_agent entry and route.ts routed the name through the neutral-assistant loop to sample_agent, which has no steps state or set_steps tool, so the progress card never mounted (agent hit the default recursion limit of 25). - Port LGP's gen_ui_agent.py (byte-identical) and register it in langgraph.json. - route.ts: bind gen-ui-agent to createAgent("gen_ui_agent") and bake assistantConfig.recursion_limit (default 100) into every LangGraphAgent — the graph's Python with_config isn't visible to the server runs API, so the multi-step set_steps walk overran 25. Mirrors langgraph-python. - aimock: regenerate d6/gen-ui-agent.json from LGP (adds chunkSize:9999 on the 24 tool-call fixtures) and narrow the over-broad d4 chat.json "summarize" key to "Summarize the sales pipeline" so it stops substring-shadowing the competitor set_steps chain (and other summarize prompts). Matches LGP. D6 langgraph-fastapi:gen-ui-agent now green (two real ~14s runs); agent-config re-verified green (no regression).	2026-07-24 16:23:48 +02:00
Alem Tuzlak	2f2a703add	fix(showcase/built-in-agent): real-LLM backend fixes from prod audit ... Fixes the demos a real-LLM audit found broken (aimock had masked them). Each verified against real OpenAI (plain next dev / next build; turbopack dev is broken on Windows — unrelated): - shared-state-read / shared-state-read-write: seed agent state on [agent] deps (not []). useAgent returns a provisional agent during /info sync; the real runtime-synced agent swaps in as a new ref, so a []-deps seed effect never re-ran and runAgent serialized empty state. NOTE: diverges from LGP's byte-identical frontend — BIA's stateless in-process backend needs re-seeding; LGP masks it via server-side state. Documented in PARITY_NOTES. - set_notes -> STATE_DELTA add /notes in tanstack-factory convertStream (mirrors set_steps), so shared-state-read-write notes render. - declarative-gen-ui / a2ui-recovery: drop response_format:json_object (the Responses-API openaiText ignores it and returned empty) — enforce JSON via system prompt + stripJsonFences, like the byoc factories. Surfaces now paint. - reasoning-custom / -default / tool-rendering-reasoning-chain: switch to the Responses API with reasoning summaries (reasoning:{effort:high,summary:auto}) + custom STEP_* -> REASONING_MESSAGE_* converter; real OpenAI chat-completions never streams reasoning_content. Reasoning blocks now render. - cvdiag/*: drop .js import extensions so /api/copilotkit resolves under dev bundlers. - PARITY_NOTES: document all fixes + that auth/voice 500 is next-dev-turbopack only (verified working under next build/start in prod).	2026-07-24 15:54:10 +02:00
Lukas Moschitz	71f44d46f7	fix(showcase): port langgraph-fastapi agent-config to v2 create_agent ... The fastapi agent_config_agent still used the v1 StateGraph pattern reading tone/expertise/responseLength from RunnableConfig[configurable][properties]. The frontend (identical to LGP) publishes those knobs via the v2 useAgentContext hook and the route uses a plain LangGraphAgent, so the old graph received nothing on configurable and the run errored (turn never completed, D6 red). Port the backend to LGP's v2 shape: create_agent + CopilotKitMiddleware with a single static system prompt that reads the injected context entry. Update the route comment to describe the useAgentContext path (runtime code unchanged: plain LangGraphAgent + AGENT_URL env fallback preserved). D6 langgraph-fastapi:agent-config now green (two real ~28s runs).	2026-07-24 15:51:42 +02:00