Lockfiles committed in 8ba692c42 were generated inside the monorepo
while pnpm's hoisted node_modules tree was present. npm-arborist
resolved transitive deps against pnpm's symlinks and wrote ~40
`../../../node_modules/.pnpm/...` paths into each lockfile's
`packages` map.
npm 10 can parse the JSON, but its arborist bombs out walking the
tree at those pnpm-relative entries with the misleading error:
npm error code EUSAGE
npm error The `npm ci` command can only install with an
npm error existing package-lock.json or npm-shrinkwrap.json
npm error with lockfileVersion >= 1.
`npm install --dry-run` surfaces the real cause:
Cannot read properties of undefined (reading 'extraneous')
A fresh lockfile generated in an isolated container works.
- broken: 1259 packages, 43 with `../../../node_modules/.pnpm/...`
- fresh: 1321 packages, all `node_modules/...` paths
This commit regenerates every integration's lockfile inside an
isolated `node:22-slim` container via `npm install
--package-lock-only --legacy-peer-deps` and verifies with `npm ci`.
Glob form 'COPY package*.json ./' didn't fix CI -- only package.json
ended up in /app, despite the build context transferring 1.38 MB
(lockfile is 705 KB so it's clearly in the source).
This commit:
1. Splits the COPY into two unambiguous lines.
2. Adds a 'RUN ls -la /app/' probe before npm ci.
If the probe shows package-lock.json present in /app, the issue is in
npm ci discovery. If absent, the issue is in build context upload.
Probe to be reverted once root cause is known.
CI failed on the 16 integrations whose explicit two-file COPY
`COPY package.json package-lock.json ./` hit a poisoned Depot remote
BuildKit cache entry: the cached layer reported CACHED but only
contained `package.json`, so the subsequent `npm ci` failed with
"command can only install with an existing package-lock.json".
Depot's cache had a layer indexed against the prior `COPY package.json
./` instruction; the new two-file instruction was matching it by some
internal cache-key collision. Two of 18 integrations (langgraph-python,
langgraph-typescript) passed only because they had a fully-cached
`RUN npm ci` layer from a sibling build that short-circuited the
broken COPY.
The glob form `COPY package*.json ./` produces an instruction string
that has never appeared in Depot's cache, so the layer is computed
fresh against the actual build context and includes both files. It
also reads cleaner than the explicit two-file enumeration.
No-Op when no cache poisoning is present -- the glob expands to exactly
package.json and package-lock.json on every integration (verified
locally; only those two files match per directory).
## Root cause
17 of 18 integration Dockerfiles copy `package.json` but NOT
`package-lock.json`, then run `npm install --legacy-peer-deps`. Despite a
~700KB lockfile sitting in every directory, none of them are consulted at
build time. Only `built-in-agent` was already doing it right.
Effect on Windows / WSL2:
1. `npm install` re-resolves package versions from scratch on every
rebuild, downloading ~1.1 GB into the build container's writable layer
plus ~hundreds of MB of `~/.npm/_cacache` that lives in the same
layer (BuildKit can't dedupe across builds because the layer hash
varies with each non-deterministic resolution).
2. The npm install layer's BuildKit cache key is just `package.json`'s
hash + base image — but with `npm install` (not `npm ci`) the install
itself is non-deterministic, so a cached layer that resolved
successfully can produce different node_modules trees than a fresh
resolution. Worse, intermediate state from interrupted rebuilds
(e.g. host OOM during `npm install`) is not reclaimed by `docker
builder prune` until 24h later.
3. WSL2's `docker_data.vhdx` grows monotonically — it never shrinks
until `wsl --shutdown` + `Optimize-VHD`. Repeated rebuilds compound
into a VHDX that can reach hundreds of GB on the Windows host
filesystem before any reclaim happens.
## Fix
Two-part:
1. **Lockfile-pinned, deterministic install** in all 18 Dockerfiles:
```
COPY package.json package-lock.json ./
RUN npm ci --legacy-peer-deps
```
- `npm ci` is faster, deterministic, and writes ~half the temporary
state of `npm install`.
- The lockfile in COPY makes the install layer's BuildKit cache key
stable across rebuilds, so once the layer is warm it actually stays
warm.
- Matches the pattern `built-in-agent` already uses.
2. **Reclaim dangling BuildKit cache in `bin/showcase build`** with a
24h-window `docker builder prune --filter "until=24h"`. Keeps the
warm cache for day-of work, reaps orphans from interrupted builds.
## Verification
```
for d in showcase/integrations/*/; do
grep -E "^(COPY package|RUN npm)" "$d/Dockerfile" | head -2
done
```
now prints identical:
```
COPY package.json package-lock.json ./
RUN npm ci --legacy-peer-deps
```
for every integration.
## Out of band (cannot land in this PR)
- `docker volume prune -af` -- one-time recovery, ran locally, reclaimed
16.11 GB from 236 anonymous Postgres volumes dating back to 2023.
- `Optimize-VHD` to compact the WSL2 docker_data.vhdx -- requires elevated
PowerShell after `wsl --shutdown`. Each developer runs this themselves
when their host drive gets tight; not something CI or this script can
do.
Replace sys.path.insert hacks in Python agent files with direct
imports via symlinks to shared/{python,typescript}/tools.
Update Dockerfiles, entrypoints, and configs to support the new
symlink-based tool resolution. Add PARITY_NOTES for frameworks
that have known gaps.
The showcase framework directories better reflect their role as
integration examples rather than distributable packages.
Renames showcase/packages/ -> showcase/integrations/ and updates
the test docker-compose file reference accordingly.
Jordan Ritter