mirror of
https://github.com/redwoodjs/local-ci.git
synced 2026-09-19 07:54:29 +08:00
Peter Pistorius
b9dbddcf7b
Address agent-readiness findings from isitagentready.com: - apps/website/public/robots.txt — RFC 9309 crawl rules with explicit entries for GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-Web, anthropic-ai, Google-Extended, PerplexityBot, Applebot-Extended, CCBot, Meta-ExternalAgent, Bytespider, DuckAssistBot (and standard search bots), a Sitemap reference, and Content Signals (search=yes, ai-input=yes, ai-train=no). - apps/website/src/app/pages/sitemap.ts — dynamic /sitemap.xml handler registered outside render(Document, ...) so it returns application/xml. Enumerates static routes, the copied CLI docs, and blog posts from content-collections with lastmod. - apps/website/src/app/headers.ts — adds a RFC 8288 Link response header on all routes: sitemap, agent-skills (extension), service-doc. - apps/website/scripts/copy-docs.mjs — sync script that copies the public CLI docs (README.md, compatibility.md, runner-image.md, SKILL.md) into apps/website/public/docs/ and regenerates apps/website/public/.well-known/agent-skills/index.json with fresh sha256 digests. Hooked into predev/prebuild so the website's static tree cannot drift from the CLI. The generated public/docs/ and public/.well-known/agent-skills/ trees are gitignored. Verified end-to-end against 'npm run preview': all endpoints return the expected status, content-type, and body, and the Link header appears on both HTML and XML responses. No changeset: this change is entirely under apps/website/ and does not affect published packages (per .claude/rules/changeset-required.md).
