Files
Devin Foley 44f6312cd8 fix(ci): reuse one available Cloud registry cache (#13334)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Paperclip Cloud needs a verified image for each merged source
commit.
> - Fresh builders restore compiled native dependencies from registry
caches.
> - The current workflow imports up to eleven historical cache manifests
at once.
> - Live builds missed native layers that a fresh builder reused from
one manifest.
> - This PR selects the nearest available cache and tests reuse across
fresh builders.

## Linked Issues or Issue Description

Refs #13329 and #13330. A search of open cache PRs found no duplicate of
this change.

**What existing behavior does this improve?**

Remote Docker cache reuse on fresh Cloud image builders.

**Current behavior**

[Cloud run
34714483272](https://github.com/paperclipai/paperclip/actions/runs/34714483272/job/103609096836)
imported the previous cache manifest successfully but rebuilt
`cargo-chef` and Rust dependencies. The dependency compile took 3m43s.
The preceding image build had already exported those layers.

A controlled [fresh-builder
diagnostic](https://github.com/paperclipai/paperclip/actions/runs/34715336530)
used the same source and registry cache. The single-manifest job reused
both layers immediately. The multiple-manifest job rebuilt them and
failed the cache assertion. Both jobs used GitHub-hosted runners with
read-only access.

**Proposed behavior**

Inspect cache manifests in first-parent order and import only the
nearest available one. Keep full-SHA cache exports, the ten-commit
search bound, and the legacy fallback. If caches cannot be read, permit
a cold build.

**Reason and benefit**

Avoid the observed cache misses without changing image contents or
builder sizes. Expected savings include about four minutes of native
tool/dependency compilation when those inputs are unchanged. The final
merge-to-deployable gain still needs a post-merge measurement.

**Breaking changes**

No image, artifact, deployment, or runner-routing contract changes.

## What Changed

- Select one available ancestor cache after Docker login and Buildx
setup.
- Preserve separate writable cache tags for each full source SHA.
- Test cache ordering, missing caches, registry errors, and workflow
integration.
- Add the selector tests to the existing release-registry suite.
- Export a local test cache, remove the first builder, and verify a
source rebuild on a fresh builder.
- Document cache selection and the stronger Docker check.

## Verification

- Passed 456 focused workflow, routing, readiness, preview-artifact, and
cache-selector tests.
- Passed shell syntax, ShellCheck for the changed probe, actionlint
workflow validation, and `git diff --check`. actionlint's shell checks
were disabled for the workflow validation because unchanged
migration-label commands trigger existing SC2012 notes.
- The fresh-builder registry diagnostic proves the single-cache
behavior. The [permanent two-builder probe
passed](https://github.com/paperclipai/paperclip/actions/runs/34715771048/job/103612624090),
including a changed real binary and dependency-declaration invalidation.
- Passed all 35 latest-head checks (green or intentionally skipped),
including full typecheck, test, build, and browser suites in [PR CI run
34715771217](https://github.com/paperclipai/paperclip/actions/runs/34715771217).
- The real selector CLI inspected registry metadata and chose the
nearest available ancestor cache.
- Fresh Greptile review is 5/5 with no open findings. The PR title was
corrected to meet the source-change naming rule; the review check passed
after that correction.
- Local full-suite runs and Docker builds are unavailable because the
local Docker daemon is unresponsive after disk exhaustion. CI provides
the Linux verification.

## Risks

- Missing or unreadable caches cause a slower cold build. The selector
logs that condition and preserves image publication.
- Inspecting several missing ancestors adds lookup time. Each lookup has
a ten-second timeout and the search is bounded.
- The Docker test now exports a local cache. It removes the first
builder before starting the second to release disk space, then cleans up
its builders and files.

## Model Used

OpenAI GPT-6 through Codex, with reasoning, repository tools, and code
execution. The exact serving model ID and context window are not exposed
by this environment.

## Checklist


- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-09-12 13:20:18 -07:00

67 lines
2.4 KiB
JavaScript

#!/usr/bin/env node
import { execFileSync } from "node:child_process";
import { appendFileSync } from "node:fs";
import { pathToFileURL } from "node:url";
export function cloudCacheCandidates(image, commits) {
if (!/^ghcr\.io\/[a-z0-9._-]+\/[a-z0-9._-]+$/.test(image ?? "")) {
throw new Error("Expected a GHCR owner/repository cache image.");
}
if (!Array.isArray(commits) || commits.length === 0 || commits.some((sha) => !/^[a-f0-9]{40}$/.test(sha))) {
throw new Error("Cloud cache ancestry requires full commit SHAs.");
}
return [
...[...new Set(commits)].slice(0, 10).map((sha) => `${image}:buildcache-cloud-${sha}`),
`${image}:buildcache-cloud`,
];
}
export async function selectCloudCache(image, commits, {
exists = registryCacheExists,
log = console.log,
} = {}) {
for (const ref of cloudCacheCandidates(image, commits)) {
try {
if (!await exists(ref)) continue;
log(`Using cloud cache: ${ref}`);
return `type=registry,ref=${ref}`;
} catch {
// Cache availability must not turn an otherwise valid build into a
// failure. A later ancestor may still be available during a rollout.
log(`Could not inspect cloud cache ${ref}; trying the next ancestor.`);
}
}
log("No cloud cache is available; this build will populate one.");
return "";
}
function registryCacheExists(ref) {
try {
// Use the preceding Docker login, including for private registry caches.
// Inspect metadata only: no layer download and no image execution.
execFileSync("docker", ["buildx", "imagetools", "inspect", "--raw", ref], {
timeout: 10_000,
maxBuffer: 1024 * 1024,
stdio: ["ignore", "pipe", "pipe"],
});
return true;
} catch (error) {
if (/manifest unknown|not found|NAME_UNKNOWN/i.test(String(error.stderr ?? ""))) return false;
throw error;
}
}
if (process.argv[1] && import.meta.url === pathToFileURL(process.argv[1]).href) {
try {
if (!process.env.GITHUB_OUTPUT) throw new Error("GITHUB_OUTPUT is required.");
const commits = execFileSync("git", ["rev-list", "--first-parent", "--max-count=10", "HEAD"], {
encoding: "utf8",
}).trim().split("\n");
const source = await selectCloudCache(process.env.CACHE_IMAGE, commits, { exists: registryCacheExists });
appendFileSync(process.env.GITHUB_OUTPUT, `source=${source}\n`);
} catch (error) {
console.error(error.message);
process.exitCode = 1;
}
}