Prep for the repo rename to runpod/runpod-plugins-official. Updates
install/marketplace refs and plugin.json homepage/repository URLs.
Filesystem paths (plugins/runpod/skills/) are unchanged.
getting-started 'Get the tools' had a per-lane table but no install-everything
option. Add a copy-paste block (skills + runpodctl + flash + hosted MCP; drop
any line you won't use), with companion CLIs called out as separate/OS-specific.
Link it from the README Quick start's optional-upgrades note.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The key-resolution order already lists ~/.runpod/config.toml, but never named
the field, so an agent needing the raw key for a direct API call (Authorization:
Bearer) had to reverse-engineer it. Name the 'apikey' field and give the
$RUNPOD_API_KEY-first extraction one-liner (matches GP23's invoke).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
command-lint review: the inspect block said 'paste ssh_command verbatim' then
prefixed it with 'ssh ', yielding 'ssh ssh -i ...' (connects to host 'ssh').
Show a standalone 'ssh -i <key> root@<ip> -p <port>' form like GP22. Also
label the observed-output block as trimmed (real ls lists SOURCE.txt; df|awk
prints a header row) so a reader diffing against live output isn't surprised.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
README:
- Rebrand title to "Official Runpod Agent Skills".
- Add a 3-step "Quick start (Claude Code)" with plain-English example prompts.
- Restructure Authentication around "auth in two places, one API key covers
both" (table + numbered fastest-path), leading with OAuth as the easy MCP
option instead of burying it under the Bearer-header case.
Golden paths (onboarding review — a newcomer couldn't start unaided):
- GP22/23/25: add a "Prerequisites" section (Runpod auth, docker login +
push namespace, SSH key for pods) linking the shared "Before you run any
path" block + getting-started + companion-clis — the template contract's
Prerequisites section was missing and the shared block was unlinked.
- GP23 invoke: use $RUNPOD_API_KEY (the recommended setup) with config.toml
as fallback — the old config.toml-only read 401'd for anyone who followed
`export RUNPOD_API_KEY`.
- GP22: add "no key yet? ssh-keygen -t ed25519" fallback before add-key.
- Fix companion-clis relative link depth (../../ -> ../../../).
CI hooks pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Third reviewer (golden-paths) found the docs had drifted from the actual
live runs; fixed:
- HIGH (GP25): the "inspect" command was condensed and dropped the mkdir +
writes, so it couldn't produce the pasted "observed" output (and failed as
written — no /workspace/mounted-model). Restored the real command that was
run (mkdir, write note + weights, df -hT|awk, stat -f) so the evidence is
reproducible from the documented steps.
- MED (GP22, GP25): stop hardcoding `-i ~/.runpod/ssh/runpodctl-ssh-key` (the
key filename varies per account) — tell the reader to paste the
`ssh_command` that `runpodctl ssh info` prints, matching GP06's convention.
- MED (GP25): add `--volume-mount-path /workspace` to the pod-create, matching
every other volume-attaching path (was relying on the platform default).
- LOW (GP25): prose said `df -T`, command uses `df -hT` — aligned.
- LOW (GP23): add `--workers-min 0` for consistency with sibling endpoints.
- LOW (README): document the ✅ live-verified / 📘 documented badges the table
actually uses (legend previously only defined stub/drafted/spec/covered).
No dangling GP24 refs / broken links (reviewer confirmed). CI hooks pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The main skill's "Worked examples" task->path router didn't list the new
paths. Added two rows — "build a minimal image for a target (pod vs queue)"
-> 22/23 (+ building-images concepts) and "bake into image vs mount a
network volume" -> 25 — and refreshed the stale scenario count.
All CI hooks pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
GP24 was a framing/pointer doc for the load-balanced image contract, but
GP14 already live-verifies the identical minimal LB image (stdlib HTTP
worker, type:"LB", /ping+/echo+/stats 200). Removed the redundant path and
rewired all references (golden-paths table + intro + complementary-pairs
note, GP23 contrast link, building-images.md contract link) to GP14.
All 4 CI hooks pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Second agent-friendliness audit (separating-context-from-constraints)
returned 8 findings; applied all:
- building-images.md base-image guidance: signal strength + reachable
exceptions — "FROM runpod/pytorch for GPU workloads" with explicit CPU-slim
and non-Runpod-base(reproduce-SSH) exceptions, so it no longer reads as an
absolute that the shipped examples contradict (R2/R5).
- SSH section: promote the load-bearing obligation to a marked Rule ("custom
CMD must invoke /start.sh; serverless exempt") instead of stating it only as
a consequence in prose (R1/R2); give pattern 2 its trigger condition (R5).
- Best-practices bullets: demote the unverifiable "keep it small" goal to
context with the concrete steps as the rules (R3); split bundled obligations
(pin-tag vs platform; PYTHONUNBUFFERED vs pin-tag) and drop the duplicate
(R4).
- GP24 LB rule 2: split "set PORT + PORT_HEALTH + expose port" into an (a)(b)(c)
checklist so each is independently checkable (R4).
- GP23: align base-image note with building-images.md as the single source (R2).
All 4 CI hooks pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Agent-friendliness audit (runnable-examples criterion) flagged the local
handler-test command as the one snippet not actually run. Ran it against the
pushed image: `--test_input` works and returns the handler output. Added the
required `--platform linux/amd64` (image is amd64) and the observed output so
the command is runnable-as-written.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Runnable proof of the storage model behind building-images.md: a baked-in
file rides the image on the host's LOCAL disk (overlay fs), while a mounted
network volume at /workspace is a NETWORK filesystem — observed live as
`mfs#euro-3.runpod.net:9421` (FUSE / MooseFS) via `df -T` on a CPU pod with a
standard volume attached. Built, pushed, launched, inspected, torn down
(pod + volume deleted).
Cross-linked from building-images.md (bake-vs-mount) and registered in the
golden-paths table + intro note. All 4 CI hooks pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add the "minimal image per contract" trio, doubling as evals for
building-images.md:
- GP22 minimal pod image — live-verified: built, pushed, launched a CPU
pod, SSH'd in (PUBLIC_KEY -> authorized_keys, sshd up), HTTP proxy 200,
torn down. Demonstrates the from-scratch SSH-preservation pattern.
- GP23 minimal serverless queue image — live-verified: built, pushed,
deployed a CPU endpoint, /runsync -> COMPLETED, torn down.
- GP24 minimal serverless load-balanced image — the image-contract framing
of the already-live-verified GP14 (no redundant re-deploy).
building-images.md: add "Don't clobber the base image's startup (SSH /
start.sh)" — the #1 pod footgun — with three safe patterns; cross-link the
trio from the image-contract section. Register 22-24 in the golden-paths
table + complementary-pairs note.
All 4 CI hooks (links/branding/versions/marketplace) pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- runpod-usage/reference/building-images.md: official runpod/pytorch base (host-cached)
+ layering, bake-in vs mount, pod/queue/load-balanced image contracts, local testing.
- golden-paths/21-storage-tiers.md: launch standard vs high-performance network volumes
(standard via runpodctl; high-performance via v2 REST type:HIGH_PERFORMANCE or the
console — CLI/MCP cannot set the tier), attach to pod/serverless, storage recommendation
for training.
- Document the tier caveat consistently in runpodctl, runpod-mcp, and storage.md; link the
new guide from runpod-usage and register GP21 in the golden-paths index.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Path 10's multi-region fallback created the endpoint via POST rest.runpod.io/v1/endpoints
with computeType:CPU — the known footgun that silently provisions a GPU endpoint. Mirror
path 19: omit computeType from the REST create and express CPU via instanceIds + empty
gpuIds in the GraphQL saveEndpoint step. Also make docker-setup.md's WSL2 note
self-contained (dangling 'WSL2 setup' reference after the install extraction).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Addresses second-round review from @jeremyjacobs-runpod and @briandconnelly.
- Reword plugin/marketplace descriptions to lead with 'Official Runpod agent skills
and MCP server' (skills-first, not MCP-led); displayName -> 'Runpod (Official)'.
- Move one-time CLI install out of the companion-CLI reference docs into per-CLI
reference/*-setup.md, leaving a one-line pointer; github-setup.md also holds SSH-key
generation + registration. Drop the inline hf install from the always-loaded
companion-clis/SKILL.md (the file that loads every invocation).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Ran briandconnelly's agent-friendly-docs, agent-friendly-cli, and
separating-context-from-constraints skills against this repo and resolved the findings:
- AGENTS.md: fix stale golden-path status (defer to golden-paths/README); add a
Contributor rules section; de-duplicate the capability-matrix maintenance rule.
- CLAUDE.md: use an @AGENTS.md include so the instructions are always loaded.
- runpodctl: promote the CPU-endpoint REST footgun into Decision Rules; split the
GPU-pool PATCH into its own rule; make the SSH rule verifiable (no interactive
shell); replace 'actively-maintained worker' with observable hub signals; clarify
'active worker' = --workers-min 1 (not max) + its continuous-billing cleanup caveat.
- model-caching: reframe the runtime-path note as an agent-facing 'don't guess' rule;
disambiguate the documented model-caching path from the undocumented Model Repo
upload path; drop an internal ticket reference.
- networking: make the public-URL rule actionable (tell the user it's unauthenticated).
- docker: give the --platform/never-latest rule one authoritative home.
- CHANGELOG: mark it release-please-generated. Add a PR template + CODEOWNERS as a
doc-freshness gate.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The wait-for-ready loops used unbounded `until …; do sleep N; done`, which
contradicts the skills' own "bound any poll loop" rule and can spin forever on a
genuinely broken pod/worker. Converted to bounded `for i in $(seq …)` loops.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Reformat plugin manifests with JSON.stringify(obj,null,2)+newline so the
release-please json updater only ever changes the version line (no array
reformat churn in future release PRs).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Two progressive-disclosure trims from the audit (#1, #2):
- runpodctl/SKILL.md 307 -> 229 lines: moved the exhaustive per-resource
command/flag menus + the send/receive runbook into a new
reference/command-reference.md (nothing lost). SKILL keeps the 80%
essentials per resource plus the decision-critical prose (CPU-serverless,
model-cache, multi-region, create-from-hub). Registry/billing/SSH-key
management now live in the reference.
- runpod/SKILL.md auth section ~60 -> ~28 lines: deduped the "prefer key
over OAuth" message (was restated ~5x) into one block — check → get a key
(unlocks all; OAuth is MCP-only, a half-setup) → ordered options → then.
Kept inline (not relocated) so the agent still reads it; made the
getting-started ref a real link.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Correctness/consistency (verified against the live tree before editing):
- C1 (GP10): GraphQL saveEndpoint CAN make a CPU endpoint via instanceIds +
empty gpuIds (live-verified in GP19) — GP10 wrongly said "defaults to GPU,
use runpodctl for CPU," which could leave a reader silently GPU-billed.
Fixed both notes + cross-linked GP19.
- C2 (aws.md): the S3 --endpoint-url host must be LOWERCASED
(s3api-eu-ro-1…), while --region takes the DC id as-is. The doc reused one
uppercase DATACENTER placeholder for both -> a bad host. Fixed the rule,
examples, and preamble.
- C4: noted --compute-type casing (pods `cpu`, serverless `CPU`).
Structure (progressive-disclosure fixes):
- P4 (runpod-mcp): was over-split — Connect had no runnable command. Inlined
the 80% Bearer-key connect one-liner.
- P2 (router): pod-workflows.md / endpoint-workflows.md were bare filename
mentions; made them explicit links to runpod-usage/reference/.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Feedback from a fresh install test: the agent defaulted to MCP "Sign in with
Runpod" OAuth, which auths the MCP only — runpodctl/flash stayed blocked, and
it hit a wall when a CLI-only capability was needed.
Fix the bias across the auth guidance:
- runpod/SKILL.md "First run": lead hard with getting a RUNPOD_API_KEY (flash
login or Console) — it unlocks runpodctl + flash + hosted MCP (Bearer).
Explicit ⚠️ that OAuth-only is a half-setup; reordered preference (key first,
OAuth last resort); step 2 now recommends key-as-Bearer even when the MCP is
the unauthed lane present.
- runpod-mcp SKILL + reference/connect.md: present hosted+Bearer key as the
recommended full-access connect; frame OAuth as MCP-only.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Make the router tell the agent to confirm auth BEFORE the first infra
action in a session, and what to do if nothing is set up.
- runpod/SKILL.md "First run": explicit check (RUNPOD_API_KEY env > .env >
~/.runpod/config.toml; `runpodctl user`; MCP /mcp Connected), and if
nothing's set up, stop and ask the user to either `flash login` or provide
an API key. Clarify one key = full control (runpodctl + flash + hosted MCP
via Bearer); OAuth sign-in is an MCP-only convenience.
- Document that `flash login` does browser OAuth and saves a real Runpod API
key to ~/.runpod/config.toml (shared with runpodctl) — verified in flash's
credentials.py (save_api_key + resolution order). Noted in getting-started.md.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
STO-370 added a requirement after a Model Repo user test where an agent
started on an old runpodctl (2.1.9), hit errors, and flip-flopped between
versions. Add an explicit up-front instruction: run `runpodctl update` +
`runpodctl version` first, pin one recent build for the whole task, and
don't switch old<->new mid-task. Noted in the Install section, the
Quick-start sequence, and reference/install.md. Matches the real trap hit
verifying golden path 20 (local 2.3.0 lacked --model-reference until update
to 2.7.1).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Third live run (2026-07-15), diagnosed through the Runpod MCP
stream-worker-logs. The --model-reference flow works end to end:
- Deploy pinned :main to commit 7ae557604adf; no baked image, no volume.
- Async /run returned COMPLETED with real output ("Hello. That's all there
is to it."), delayTime ~105s, executionTime ~1.5s.
- Cache hit confirmed in worker logs: vLLM loaded the 0.92 GiB checkpoint in
0.55s at the pinned revision (already present, not a fresh HF pull).
The earlier failures were infra timing, not the feature: first-ever ~10 GB
image pull (>20 min) and a sync /openai 524 (>100s edge timeout on cold
load). Reworked the gotcha: "ready" can precede vLLM being up — read worker
logs (MCP) to tell slow boot from a real stall; use async /run for cold
calls. Status ⚠️ partial -> ✅; router count 19 -> 20 live-verified.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Second live attempt at golden path 20. Deploy + model-reference pinning
confirmed again; inference still not captured, but with richer real data:
- Worker readied in 162s once the worker-vLLM image was warm on the pool
(vs >20 min first-ever pull in run 1).
- Sync /openai and /runsync hit Cloudflare 524 on cold load (>100s edge
timeout during first-request model load) — switch step 4 to async
/run + /status.
- A ready worker left an async job IN_QUEUE ~15 min (inProgress:0), only
reaching running at teardown — the documented "ready but mis-dispatching"
symptom; no COMPLETED output, so the on-host cache path stays unconfirmed.
Status stays partial with these gotchas recorded. Total GPU spend ~$0.64;
all test endpoints torn down.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two consistency fixes from the contradiction swarm:
- Note that serverless defaults differ by layer: flash SDK (idle_timeout 60s,
workers (0,1), execution_timeout unlimited) vs platform/runpodctl (5s, 3,
600s). Added cross-referenced notes in flash/reference/api.md and
runpod-usage/reference/concepts.md so a default is read against its layer.
- Clarify large-HF-model delivery: naming/streaming works at any size but
re-downloads on every cold start; for reused large models cache via
NetworkVolume / --model-reference / bake. Same tradeoff as the
model-caching delivery table — not a contradiction.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Architecture pass on the runpod plugin:
- Move golden-paths/ under skills/runpod/ so they travel with the router
skill on single-skill installs; rewrite all cross-links (link check green).
- Slim always-loaded SKILL.md files (~1558 -> ~854 lines) by extracting
one-time setup and exhaustive reference into on-demand reference/*.md:
companion-clis (per-tool hf/gh/docker/aws), flash (setup-and-cli/api/
patterns), runpod-mcp (connect), runpodctl (install).
- Add runpodctl/reference/model-caching.md covering all four model-delivery
methods, --model-reference HF cache, and the Model Repository (STO-370);
expand the Models section; add the 4th row to storage.md; add golden
path 20 (host-cached HF model endpoint).
- Fix CPU serverless endpoint bug (verified live): public REST
POST /v1/endpoints with computeType:CPU silently provisions a GPU;
use `runpodctl serverless create --compute-type CPU` or MCP instead.
Fixed golden paths 16 & 18 and added a canonical note in the runpodctl skill.
- Document REST v1 (control, buggy for CPU) vs the v2 runtime/invoke API,
and how to read the MCP server version from the initialize handshake
(serverInfo.version = "2.0.0 [RUNPOD_REST_VERSION=v2]").
- Fix datacenter IDs in the AWS reference to region-prefixed form
(EU-CZ-1, US-CA-2, ...) per `runpodctl datacenter list`.
Golden path 20 is marked partial: deploy + model-reference resolution
verified live; inference not confirmed (worker cold start exceeded 20 min).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Remove the pinned "version":"1.0.0" from both plugin.json manifests so the plugin is
commit-tracked — per Claude Code marketplace docs, omitting version makes every git
commit a new version, so users get fixes on the next update/background pull instead of
being stuck until a manual version bump. Add a README 'Updating' section with the
per-client update commands (Claude Code /plugin marketplace update, Codex plugin
marketplace upgrade, skills.sh re-add) and the rationale.
Refine the router 'First run' guidance per feedback: proactively get the user set up
rather than silently limping on a partial fallback.
- Lead with the single unifying step: export RUNPOD_API_KEY unlocks runpodctl + flash
+ local-stdio MCP at once; the hosted MCP is the one exception (its own /mcp OAuth,
or same key as Bearer). Corrects the 'MCP auth = everything authed' misconception
(OAuth keeps no key on disk, so it doesn't set up the CLIs, and vice-versa).
- Reframe: use a working lane to stay unblocked, but don't quietly drop to a fallback
when a quick setup gives the intended lane — nudge toward proper setup; if nothing is
connected, stop and set them up.
- getting-started: same one-key-covers-CLIs + hosted-MCP-is-separate clarification.
- router SKILL: add 'First run — is a control plane connected?' — the agent checks
whether MCP tools are present / installed-but-unauthed / runpodctl available, and
GUIDES the user through setup (auth the bundled MCP, or install runpodctl) instead
of silently failing. Falls back to runpodctl when MCP isn't configured.
- runpod-mcp SKILL: add a 'verify it's live' step (/mcp shows Connected vs Needs
authentication; confirm with list-endpoints) — the plugin registers the server but
it's inert until sign-in.
- README: control-plane MCP heads-up — the plugin installs a hosted MCP that needs a
one-time sign-in; the runpod skill auto-detects + guides; runpodctl is the fallback.
Address the reproducibility audit's remaining findings so every path is turnkey:
- 04/07: spell out the elided `runpodctl ssh info` parse (use 06's proven pattern);
04 now writes train.py to the pod + runs it detached with a log to poll.
- 08: detached training launch is now a real runnable command (write script, setsid,
log, poll) instead of narration.
- 09: add the explicit step that copies the template files up to the pod /app.
- 10/13/17: add ## Prerequisites sections (link the shared note + getting-started);
17 gains explicit 17→14 prerequisite-reading cross-links; 10 gets a buildable image
source (points to 19's reusable handler+Dockerfile).
- 14: add 14→17 see-also (shared type:"LB" substrate).
- 18: create the serial-contrast endpoint/template the walkthrough compares against
(was only in cleanup) — create/cleanup now consistent.
- 19: placeholder coherence across sync loops + GraphQL + cleanup; capture-the-id note.
Doc-only (no new live runs); all live-verified results/observed output preserved.
Hooks: links + branding + marketplace all pass.
- 10: unify the CLI-path verify to <vol-ro>/<vol-cz> (was <vol-a>/<vol-b> from a
different sub-run) so a reader tracks one consistent set of ids.
- README: add 'Before you run any path' shared prerequisites — auth/SSH (link
getting-started), docker login + companion creds (link companion-clis), the
placeholder + '<your-registry> is yours, you can't push to the live-run namespace'
convention, and the runpodctl >=v2.4.0 note. Complements the registry/ID
genericization so readers know what to substitute.
Genericize per-run template/endpoint/volume ids (e.g. p8b3v1prf1, d4qw56wbx9,
ie59vtopn776j2, ...) to readable placeholders (<template-id>, <endpoint-id>,
<vol-ro>/<vol-cz>/<vol-is>, ...) across paths 05/10/13/14/16/17/18/19 so commands are
copy-paste-safe on a fresh account. Kept: the official reusable ComfyUI template id
(02), worker-hash + public-endpoint job/worker ids (observed-run evidence), image tags.
Genericize all Docker image references in the golden paths (build/push commands +
prose) so a new agent uses their own registry instead of a hardcoded account name.
Backlog of candidate golden paths not yet written: Public Endpoints, serverless
streaming (/stream), autoscaling tuning, load-balancing endpoint, monitor/debug +
alerts, webhooks, WebSocket worker, concurrent handler, and a 3-region same-file
endpoint (extends 10). Each with what it should prove + key references.
- 10: headless multi-volume attach documented as runpodctl >=v2.4.0
`--network-volume-ids v1,v2` (verified live), with the GraphQL saveEndpoint
object-shape path + Console as fallbacks. Reframed CLI guidance to a clean version
requirement (check runpodctl version; install from GitHub releases if the Homebrew
tap lags). Removed transient platform-status notes from the skill per maintainer
request (those are tracked/flagged separately with the team).
- runpodctl SKILL: add multi-region `--network-volume-ids` note (>=v2.4.0).
- README: TOC row 10 updated to the runpodctl lane.
- 09 (custom dual-mode dev loop): live-verified end to end — RTX 4090 pod transcribed
the JFK sample via python handler.py, built image (runpod/pytorch base) reproduced it
on clean /app, serverless flip returned identical transcription (pod<->serverless
parity). Template fixes hit live: PEP-668 --break-system-packages, and a two-step
install (--ignore-installed cryptography runpod) because runpod pulls a newer
cryptography the base can't uninstall. Base pinned to the exact runpod-torch-v280 image.
- 10 (multi-region HA serverless): live-verified — 2 volumes (EU-RO-1 + EU-CZ-1),
S3-synced identical, 16/16 requests COMPLETED across both DCs with identical output.
Real correction: headless multi-volume attach is GraphQL saveEndpoint with
networkVolumeIds:[{networkVolumeId}] OBJECTS — runpodctl --network-volume-ids and REST
networkVolumeIds:[strings] both reject bare strings. Also: api.runpod.io/graphql needs
a browser User-Agent; computeType is REST-only (GraphQL round-trip reset to GPU).
- router (skills/runpod/SKILL.md): add 'Worked examples (golden paths)' table (audit P1)
so a cold agent discovers the 10 worked paths and the HA journey stops dead-ending.
- gaps folded into docker.md / endpoint-workflows.md / gotchas.md.
- All test resources torn down (pods/volumes empty; only pre-existing whisper endpoints).
- 10: high-availability serverless by attaching one network volume per DC across
several DCs so workers spread across regions (not hostage to one DC's GPU pool).
Core lesson: volumes do NOT auto-sync — you must replicate identical data to every
volume or workers serve divergent data. Documents the three ways to populate/sync
each per-DC volume (S3 API no-compute / CPU pod / cheapest GPU pod) with a tradeoff
table, the HA setup walkthrough, sync discipline, gotchas, cost. Grounded in the
official docs (network-volumes multi-attach + S3 API). Marked spec.
- companion-clis: register the community Runpod Network Volume Storage Tool as the
optional resumable-transfer helper (aws s3 stays the zero-dep baseline).
- README TOC + intro: add 10 (spec).
- 09: new golden path — one image, two modes (MODE_TO_RUN), iterate a handler in a
GPU pod then flip to serverless with no code change. Worked example: whisper
speech->text. Vendored trimmed dual-mode template (handler.py/start.sh/Dockerfile/
requirements.txt), grounded in justinwlin/Runpod-GPU-And-Serverless-Base and the
verified paths 03/05/07. Marked spec (document-only, not yet live-run).
- README TOC + intro: add 09 (spec); 01-08 remain live-verified.
- fix 06 broken self-anchor link to the VS Code section (CI link check).
- branding hook: allowlist the literal 'RunPod-Key-Go' (runpodctl's own generated
SSH key filename) so CI stops flagging a real artifact name.
Fixes red CI on the branch.
- 05 model→endpoint: built distilbert-sst2 CPU handler image, pushed
justinrunpod/rp-gp05:v2, deployed CPU serverless endpoint, COMPLETED job
with real output (cold ~73s+0.86s, warm 0.16s)
- 06 dev pod: RTX 4090 + 10GB volume at /workspace in EU-RO-1; SSH-exec real
GPU/torch output; marker survived pod stop→start; VS Code Remote-SSH transport
connected and fetched the matching vscode-server tarball
- 04 fine-tune-on-pod: rewritten as the standalone training-half path, cites
08's real train run (TinyLlama LoRA, loss 2.07→1.68, ~5min/4090)
- README: all 8 golden paths now live-verified; intro/legend updated
- fold-back: docker.md + gotchas.md updated with gaps hit during live runs
All test resources torn down (pods/volumes empty; only pre-existing whisper
endpoints remain).
Ran the full loop live 2026-07-10: a pod LoRA-trained TinyLlama-1.1B (minimal peft +
transformers Trainer, loss 2.07->1.68, ~5 min), wrote the adapter to a network volume;
a flash serverless worker loaded base + adapter from /runpod-volume and generated
(cold ~48s queue + 9.7s exec, base reused from the volume's HF cache). Flip 08 to
COVERED.
- Document the minimal peft/transformers trainer as Option A (what ran; robust, reuses
template torch) alongside axolotl as Option B (richer/scale).
- Replace the serve handler with the exact verified function-form load-once pattern
(global cache, **kwargs, inlined paths — only the function body ships).
- Record real cold-start timings + generated output. TOC -> live-verified.
The smallest train-then-serve validation: LoRA-fine-tune a tiny LLM (TinyLlama-1.1B)
on a pod with axolotl, write the adapter to a network volume, then load base+adapter
from the same volume on a flash serverless endpoint and generate. Composes the
verified network-volume handoff (07) with the axolotl training flow (04).
Built to scale by changing only three knobs (model id, GPU, dataset) — the loop
(one volume, /workspace write -> /runpod-volume read, same DC, **kwargs handler,
non-empty input) stays fixed. Includes a scale-up table, QLoRA/vLLM notes, and the
inherited gotchas (mount-path swap, flash handler contract, load-once, PEP 668,
MCP worker-log diagnosis). Marked spec (documentation only; grounded in verified
pieces). TOC updated.
Ran the pod -> network-volume -> serverless handoff live (2026-07-10): a pod wrote
/workspace/hello.txt, a flash CPU worker read it back at /runpod-volume/hello.txt and
returned the exact contents. Flip 07 to COVERED.
Root cause of the earlier 'job timed out' failures (found via the Runpod MCP
stream-worker-logs, once prod v2 was up): the worker was healthy and REJECTING the
job, not broken. flash calls the handler as read(**job_input) — a def read(input:
dict) fails with 'unexpected keyword argument', and an empty {} input is rejected as
'missing input'. Fix: handler takes **kwargs and invoke with a non-empty input.
- 07: verified status + real root-cause section + corrected handler/invoke; TOC ✅.
- flash SKILL gotcha #10 corrected: QB spreads input as kwargs (param names must
match input keys; use **kwargs); empty input rejected. Removes the wrong 'name the
param input for the plain contract' claim.
- gotchas.md: reframe the IN_PROGRESS-then-timeout note — often a job-reject by a
healthy worker; read worker logs before assuming a broken image.
Registered SSH keys are injected at pod BOOT, so a key added after the pod is
running won't work until a restart. Add an explicit pre-flight to the pod
development loop (Provision step) and sharpen getting-started's SSH section:
check 'runpodctl ssh list-keys', and register first via 'runpodctl doctor'
(human) or 'ssh-keygen' + 'runpodctl ssh add-key --key-file ...' (scripted).
New golden path for the pod-produces / serverless-consumes pattern (the basis for
'fine-tune on a pod, serve the adapter from an endpoint').
Live run 2026-07-08 verified the MECHANICS: volume created; pod mounts it at
/workspace and writes+reads a file; pod removed with the volume persisting; a flash
serverless endpoint attaches the SAME volume by id (networkVolumeId confirmed) with
the fixed /runpod-volume mount path. NetworkVolume(id=...) attaches an existing
volume deterministically.
The serverless read-back did NOT complete: the flash reader worker returned 'job
timed out after 1 retries' (~30-50s, no output) reproduced 6x across GPU sync/async,
CPU, +/- deps, and raw curl + flash SDK client. Documented honestly as a known
blocker (needs worker-log diagnosis via MCP/Console or a different DC) and recorded
the IN_PROGRESS-then-timeout variant in gotchas.md. Variant B (LoRA fine-tune ->
serve) captured as spec. All resources torn down; TOC + status legend updated.
Fold open PR #27's substance into the flash skill (adapted + deduped against the
current file): Python 3.14 install caveat; RUNPOD_API_KEY-overrides-login auth
trap; 'when to use image= vs your own code'; 'choosing a model' + fp16 VRAM->GPU
table; 'loading ML models (warm workers)' with class __init__ + NetworkVolume
cache; QB-vs-LB request body shape unified into one gotcha; base64/volume return
guidance; mapped Resources index. Gotchas renumbered coherently (1-15).
Justin