Tests that the agent detects an existing OpenTelemetry setup and
chooses the OTLP integration path — config.otlp.enabled instead of
config.traces_sample_rate, plus sentry-opentelemetry gem.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Tests that the agent detects an existing Rollbar setup, removes it,
and replaces it with Sentry including migrating Rollbar API calls in
application code to Sentry equivalents.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds a scenario that validates the agent detects Sidekiq in a Rails 8
project and recommends sentry-sidekiq alongside sentry-rails. Verified
with a custom file transport that all three telemetry types (event,
transaction, log) flow correctly.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Tests the non-Rails code path: sentry-ruby only, Rack middleware
in config.ru, no Rails initializer. Validated with file-writing
transport (event + transaction envelopes confirmed).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add vitest-evals based eval system for testing SDK install skills
against realistic fixture projects with real Claude API calls.
Includes sandboxed runner with file tools (no shell/network), LLM judge
for soft assertions, auto-discovered scenarios, and CI workflow gated
on eval/skill path changes or run-evals label.
Initial scenario: sentry-ruby-sdk rails-basic (Rails 8, Puma, SQLite).
Assertions validated against real telemetry using a file-writing transport.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Johannes Daxböck