mirror of
https://github.com/CopilotKit/CopilotKit.git
synced 2026-09-14 16:26:20 +08:00
Alem Tuzlak
4880afc846
## Summary - extend the existing public-skill drift suite to validate maintained setup assets against the generated public API manifest - fail when a skill imports an unpublished CopilotKit package or entrypoint, or a manifest-deprecated API - run the guard in the existing plugin-skills workflow when skills or the manifest change ## Why this matters Coding agents copy these skill assets directly into user projects. Mirror-sync tests prove that our duplicated skill files match, but they do not prove that the examples still reference packages and APIs we actually publish. A stale import can make CopilotKit fail at the first install or build step, which is exactly the kind of failure that prevents agents from choosing and successfully adopting us. This PR adds the smallest deterministic guard for that risk. It reuses our existing Vitest suite and canonical public API manifest; it does not introduce an eval harness, run agents, score behavior, collect metrics, add a provider, or add dependencies. ## Scope This is package-contract validation, not behavioral evaluation. Broader questions such as whether an agent follows a skill well, how many attempts it needs, and whether the generated application behaves correctly remain separate work and should start with a concrete decision the deterministic checks cannot answer. ## Verification - `pnpm exec vitest run scripts/__tests__/public-skill-drift.test.ts scripts/__tests__/sync-plugin-skills.test.ts` (17 tests) - `pnpm check:plugin-skills` - `pnpm check:public-api-manifest` - targeted TypeScript, oxfmt, and oxlint checks - mutation check: replacing `BuiltInAgent` with deprecated `BasicAgent` fails with the manifest-provided replacement Linear: PDX-320