mirror of
https://github.com/getsentry/sentry-for-ai.git
synced 2026-09-14 14:28:22 +08:00
Johannes Daxböck
5402298cfc
Add vitest-evals based eval system for testing SDK install skills against realistic fixture projects with real Claude API calls. Includes sandboxed runner with file tools (no shell/network), LLM judge for soft assertions, auto-discovered scenarios, and CI workflow gated on eval/skill path changes or run-evals label. Initial scenario: sentry-ruby-sdk rails-basic (Rails 8, Puma, SQLite). Assertions validated against real telemetry using a file-writing transport. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
18 lines
306 B
Ruby
18 lines
306 B
Ruby
source "https://rubygems.org"
|
|
|
|
ruby "3.3.6"
|
|
|
|
gem "rails", "~> 8.1.3"
|
|
gem "puma", ">= 5.0"
|
|
gem "sqlite3"
|
|
# gem "solid_queue"
|
|
gem "bootsnap", require: false
|
|
|
|
group :development, :test do
|
|
gem "debug", platforms: %i[ mri windows ], require: "debug/prelude"
|
|
end
|
|
|
|
group :development do
|
|
gem "web-console"
|
|
end
|