mirror of
https://github.com/getsentry/sentry-for-ai.git
synced 2026-09-14 14:28:22 +08:00
Johannes Daxböck
5402298cfc
Add vitest-evals based eval system for testing SDK install skills against realistic fixture projects with real Claude API calls. Includes sandboxed runner with file tools (no shell/network), LLM judge for soft assertions, auto-discovered scenarios, and CI workflow gated on eval/skill path changes or run-evals label. Initial scenario: sentry-ruby-sdk rails-basic (Rails 8, Puma, SQLite). Assertions validated against real telemetry using a file-writing transport. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
19 lines
450 B
JSON
19 lines
450 B
JSON
{
|
|
"compilerOptions": {
|
|
"target": "ES2022",
|
|
"module": "NodeNext",
|
|
"moduleResolution": "NodeNext",
|
|
"lib": ["ES2022"],
|
|
"strict": true,
|
|
"esModuleInterop": true,
|
|
"skipLibCheck": true,
|
|
"forceConsistentCasingInFileNames": true,
|
|
"resolveJsonModule": true,
|
|
"isolatedModules": true,
|
|
"noUncheckedIndexedAccess": true,
|
|
"noImplicitOverride": true,
|
|
"types": ["node", "vitest"]
|
|
},
|
|
"include": ["src/**/*"]
|
|
}
|