mirror of
https://github.com/getsentry/sentry-for-ai.git
synced 2026-09-14 14:28:22 +08:00
Johannes Daxböck
5402298cfc
Add vitest-evals based eval system for testing SDK install skills against realistic fixture projects with real Claude API calls. Includes sandboxed runner with file tools (no shell/network), LLM judge for soft assertions, auto-discovered scenarios, and CI workflow gated on eval/skill path changes or run-evals label. Initial scenario: sentry-ruby-sdk rails-basic (Rails 8, Puma, SQLite). Assertions validated against real telemetry using a file-writing transport. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
11 lines
116 B
Plaintext
11 lines
116 B
Plaintext
.claude/*
|
|
!.claude/settings.json
|
|
.DS_Store
|
|
|
|
# Evals
|
|
evals/node_modules/
|
|
eval-results.json
|
|
.env
|
|
.env.local
|
|
.env.test
|