After the npx fix each publish is ~4.7s and almost entirely a registry
round-trip, so a 26-package scope=all canary still spent ~125s waiting
serially. Publish 4 at a time (CANARY_PUBLISH_CONCURRENCY=1 restores
serial for debugging).
This weakens no ordering invariant. prerelease.ts's own header already
documents that the cross-scope graph has cycles (runtime ->
channels-intelligence, channels-core -> core), so no serial order avoided
publishing a package before the same-run version it pins.
Per-package output is captured and replayed as one block rather than
inherited, since a pool would otherwise interleave several npm publishes
line-by-line — and that log is the only forensic record when a canary
half-publishes. Every package is attempted even if others fail, so one
report names all of them; main() now exits non-zero on failure rather
than letting an unhandled rejection pass the step.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The canary flow took ~11.5 min steady-state (and 20 min in an observed
run). Measured from run 30473499191, the time went to five avoidable
places rather than to real work.
1. `npx --yes npm@11.15.0 publish` ran per package, and npx re-resolves
the spec against the registry on EVERY invocation: ~16s of each
package's ~21s. A 9-package channels canary paid ~2.4 min of pure npx
overhead; a 16-package monorepo release paid over 4 min. Hoist the
pinned npm into lib/npm-cli.ts, install it once into a throwaway
prefix, and reuse the binary.
2. publish-release.yml was the only workflow in the repo with no pnpm
store cache, so all three jobs installed 4608 packages cold every
time. Usually ~45s each, but registry-bandwidth bound and heavy
tailed: the observed run spent 9m08s here on tarballs arriving at
2-49 KiB/s. Add the same node-version-keyed cache the rest of CI uses.
3. The notify job ran for canaries only to compute "post nothing" — the
builder already returns should_post=false for mode=prerelease and the
self-watchdog is already gated off. ~85s of dead work on the critical
path, since canary.yml waits for the whole run. Skip the job, keeping
it reachable for a python_publish dispatch.
4. The build job fetched full history for canaries, which need none (no
tag, no GH Release, no release-note commit range, and `nx run-many`
resolves no merge base). That rode along in the 837 MiB workspace
artifact too. Shallow-fetch prereleases; stable keeps depth 0 because
its publish job pushes tags out of that artifact's .git.
5. Two smaller ones: the artifact was gzipped and then re-deflated into
the artifact zip (compression-level: 0), and the orchestrator's
run-discovery loop slept 6s before its first poll.
Verified: 143 release-script tests pass (6 new for the npm-cli helper),
actionlint + shellcheck + the scope-dropdown guard are clean, the
prerelease dry-run path still enumerates all 9 channels packages, and a
live probe confirms the helper installs npm 11.15.0 once (3.2s) and
memoizes thereafter (0ms).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Adversarial pass on the previous commit. The warning only inspected
`workspace:` ranges, so it missed the OTHER way a cross-scope pin goes stale: a
literal version range naming a package in another scope. `bumpPackages` rewrites
literal ranges for in-scope packages only, so such a pin survives every bump —
`scope=all` publishes the canary and the artifact still resolves the dependency's
last stable release, silently, which is the exact failure this warning exists to
surface.
`findCrossScopeWorkspaceDeps` becomes `findCrossScopePins`, reporting both shapes
tagged with a `reason`, and the literal case carries its own remedy (convert to
`workspace:`) instead of the useless "re-run with scope=all".
No such pin exists in the tree today — every cross-scope edge is `workspace:` —
so this closes a latent hole rather than a live one, in the one place a future
refactor would reintroduce it.
Also documents the multi-scope partial-failure mode in prerelease.ts: the
cross-scope graph has cycles, so no publish order avoids a package shipping
before the same-run version it pins, and npm's no-republish rule means a failed
run must be retried under a new suffix.
Two release-tooling defects turned a pair of canary publishes into a broken
combination for consumers (a canary runtime resolving the last STABLE
channels-intelligence, which still called the removed `channel.addAdapter`).
1. Canary versions were prereleases of an ALREADY-PUBLISHED version. A stable
release leaves the working tree on the version it just published, and
computePrereleaseVersion appended `-canary.<id>` to exactly that, so the
canary sorted BELOW its own release (`0.2.1-canary.x < 0.2.1`): the `canary`
dist-tag pointed behind `latest`, and no dependent range could ever resolve
it. Base the canary on the next unreleased version instead (patch bump,
reusing computeNextStableVersion's prerelease rule).
2. A canary published one scope at a time, but the scopes are only independent
on the version axis. `@copilotkit/runtime` carries
`"@copilotkit/channels-intelligence": "workspace:*"`, and `pnpm pack`
resolves that against the working tree — so a `monorepo` canary pinned the
channels family to its last stable release even when the commit changed both
sides of the contract. Add a prerelease-only `all` selector that bumps and
publishes every scope from one commit under one shared canary id, and warn
loudly when a single-scope canary leaves a cross-scope pin behind.
`all` is a selector, never a scope: stable releases stay single-scope (their
tag, release branch, and npm/Slack links all derive from one scope name), which
publish-release.yml enforces in both jobs and the dropdown guard enforces per
workflow.
Round-2 review fixes for the GITHUB_OUTPUT helper and the release scripts
that emit through it.
emitGithubOutputs (scripts/release/lib/github-output.ts):
- Replace the key newline/CR check with a full GitHub-Actions-safe charset
check: /^[A-Za-z_][A-Za-z0-9_-]*$/. A key containing "=" or whitespace
would silently corrupt the key=value line; rejecting up-front is
strictly safer. Value validation (single-line) is unchanged — "=" in
values is legal because GitHub splits on the first "=".
- Update the docblock accordingly.
prerelease.ts:
- Remove the dead `?? getCurrentVersion(scope)` fallback. The empty-list
guard above makes packages[0] guaranteed, and the fallback would have
masked a package.json missing its version field by emitting a version
divergent from what the loop publishes. Fail loudly with an explicit
exit instead.
- Drop the now-unused getCurrentVersion import.
- Add a comment above the dry-run emitGithubOutputs call explaining that
emitting in dry-run is safe — the publish workflow gates publish + the
verify guard on inputs.dry-run != true, so the dry-run emission only
serves local/e2e contract verification.
publish-release.ts:
- Hoist getPackagesForScope + empty-list guard above the prerelease-suffix
and registry checks. A misconfigured scope now fails with the clear
"no packages found" error instead of a misleading "not greater than
published" one. Loop is unchanged.
github-output.test.ts:
- Loosen the key-newline assertion from the JSON.stringify-coupled
/bad\\nkey/ to the stable /alphanumeric/ phrase from the new message.
- Add tests: "=" in key throws, space in key throws, empty key throws,
and "=" in value is accepted and written verbatim (note=a=b).
- Move vi.restoreAllMocks() to the top of afterEach so spies cannot leak
into env restore + rmSync cleanup.
Call sites audited:
- emitGithubOutputs: only ever called with {version, scope} (prerelease,
publish-release) — all valid under the new charset.
- publishVersion derivation: only used inside prerelease.ts main().
- getCurrentVersion: still imported by publish-release.ts, bump-prerelease.ts,
prepare-release.ts; only the prerelease.ts import was removed.
- getPackagesForScope hoist in publish-release.ts: `packages` was only
read inside the publish loop below; nothing earlier depended on it.
Hardens the new GITHUB_OUTPUT emission path so a malformed value can't smuggle
extra `key=value` lines into the workflow's step outputs, and so the workflow's
"Verify publish step emitted version" guard can't be fooled by a publish that
did nothing.
emitGithubOutputs now validates every key/value for `\n`/`\r` BEFORE the
GITHUB_OUTPUT early-return — a malformed value is a caller bug and should fail
loudly even when running locally. A multi-line value would need the heredoc
form, which this helper deliberately does not support.
prerelease.ts and publish-release.ts now fail loud when getPackagesForScope
returns an empty list. Without this, the new GITHUB_OUTPUT emission would make
the workflow's "Verify publish step emitted version" guard pass on a run that
published nothing — previously the missing output made such a run fail. The
guard runs BEFORE the dry-run branch in prerelease.ts. In publish-release.ts,
the inline iteration of getPackagesForScope(scope) is hoisted to a `packages`
const so the same guard fires before the publish loop.
The "no-op when GITHUB_OUTPUT is unset" test now spies on fs.appendFileSync
and asserts it wasn't called (the previous read of the unrelated temp file
was vacuously true). New tests cover newline/CR in value and newline in key.
The prerelease.ts usage string previously advertised `[--suffix <label>]`,
but the script never parses --suffix (suffix handling lives in
bump-prerelease.ts per the header comment). Removed.
Call sites enumerated:
- emitGithubOutputs: prerelease.ts (dry-run + post-publish), publish-release.ts
- getPackagesForScope: prerelease.ts, publish-release.ts (this commit);
bump-prerelease.ts, prepare-release.ts, versions.ts (not changed — out of
scope for this hardening)
Verification:
- npx vitest run --config scripts/release/vitest.config.mts → 91 passed
- Red-green for the newline validation: temporarily removed the validation,
the 3 new newline/CR tests failed (assertion: expected fn to throw); restored,
back to green.
- E2E: GITHUB_OUTPUT="$OUT" pnpm release:prerelease:dry succeeded and the
output file contained `version=1.59.5` and `scope=monorepo`.
Note: Fix 2's empty-list guard fires only on a misconfigured scope (no unit
test reachable — prerelease.ts is outside the vitest include glob and the
guard is boundary validation against a misconfigured scope, not a behavior
worth contriving a test harness for).
prerelease.ts published canaries successfully but never wrote the
version output the publish-release workflow's "Verify publish step
emitted version" guard reads, so every canary dispatch ended red after
a successful publish. Extract the GITHUB_OUTPUT append (previously
inline in publish-release.ts) into a shared lib/github-output.ts helper
and call it from both publish scripts.
Call-site enumeration:
- emitGithubOutputs: declared lib/github-output.ts; called from
prerelease.ts (dry-run path + after publish) and publish-release.ts
(replaces the inline appendFileSync block, same version=/scope= keys).
- No symbols removed; fs import in publish-release.ts still used (3
remaining call sites).
Replace pnpm publish with pnpm pack + npx npm@11.15.0 publish to
enable OIDC authentication. Set NODE_AUTH_TOKEN='' to prevent the
expired secret from blocking OIDC. Removes test workflow.
Remove pnpm run build (and pnpm run test) calls from publish-release.ts
and prerelease.ts so that repository build code never executes with
NPM_TOKEN in the environment. The CI build job (contents: read, no
secrets) now handles all building and testing, uploading pre-built
artifacts for the publish job to consume.
For prerelease: extract version bumping into a new bump-prerelease.ts
script that runs in the build job before the build step, ensuring the
built artifacts contain the correct canary version numbers.
Also fix prerelease build job to use persist-credentials: false,
matching the publish-release build job pattern.
Each release scope has its own packages, version source, and
independent version track:
- monorepo: 12 core @copilotkit/* packages (shared version)
- cli: copilotkit CLI (independent version)
- angular: @copilotkitnext/angular (independent version)
Branch pattern is now release/publish/<scope>/v<version> and git
tags use <scope>/v<version> for non-monorepo scopes.
Replace changesets with a simple, stateless release system:
Stable release (PR-gated):
Actions → "create release PR" → pick patch/minor/major → CI runs →
merge → publishes to npm, creates git tag + GitHub Release
Prerelease (ad-hoc):
Actions → "publish / prerelease" → publishes current version with
-canary.<suffix|timestamp> to npm under "canary" tag
Key features:
- All 12 core @copilotkit/* packages share a single version
- AI-generated release notes via Anthropic API
- Notion draft for team editing before merge
- Notion link commented on the release PR
- Guards: concurrent release PR check, version > npm check, clean
semver check, canary-only prerelease tag
- release/publish/v* branch pattern (hard to accidentally match)
- TypeScript throughout (tsx runner)
- release.config.json with versionedTogether/versionedIndependently
Tyler Slaton