gastownhall__beads/docs/reference/observability.md

title, description

title	description
Observability (OpenTelemetry)	Exporting bd metrics and traces over OpenTelemetry (OTLP), with a local VictoriaMetrics and Grafana stack, env vars, and a metric reference.

Beads exports metrics via OTLP HTTP. Telemetry is disabled by default — zero overhead when no variable is set.

Recommended local stack

Service	Port	Role
VictoriaMetrics	8428	OTLP metrics storage
VictoriaLogs	9428	Reserved for future OTLP log storage
Grafana	9429	Dashboards

# From your personal stack's opentelemetry/ folder
docker compose up -d

Configuration

Telemetry is explicit opt-in. Set BD_OTEL_ENABLED=true and configure the exporter via standard OpenTelemetry SDK environment variables — both go in your shell profile or workspace .env:

export BD_OTEL_ENABLED=true
export OTEL_EXPORTER_OTLP_METRICS_ENDPOINT=http://localhost:8428/opentelemetry/api/v1/push

A standard OTEL_* variable alone will not turn bd telemetry on — bd won't auto-activate from a machine-global OTEL_* setting that was set for some other instrumented tool.

Log export is not implemented yet. BD_OTEL_LOGS_URL is reserved for a future VictoriaLogs exporter and does not activate telemetry today.

Shell profile (recommended)

# ~/.zshrc or ~/.bashrc
export BD_OTEL_ENABLED=true
export OTEL_EXPORTER_OTLP_METRICS_ENDPOINT=http://localhost:8428/opentelemetry/api/v1/push

Environment variables

Variable	Example	Description
BD_OTEL_ENABLED	true	Master switch. Activates telemetry. Without it, the variables below are ignored.
OTEL_EXPORTER_OTLP_METRICS_ENDPOINT	http://localhost:8428/opentelemetry/api/v1/push	Push metrics to an OTLP HTTP receiver.
OTEL_EXPORTER_OTLP_LOGS_ENDPOINT	http://localhost:9428/insert/opentelemetry/v1/logs	Push logs to an OTLP HTTP receiver (reserved for future log export).
OTEL_TRACES_EXPORTER	console	Write spans to stderr (dev/debug).
OTEL_METRICS_EXPORTER	console	Metric exporter selection, comma-separated (otlp, console, none). Unset defaults to otlp; console writes to stderr (dev/debug); a selection without otlp never exports to an OTLP endpoint, even if one is configured machine-globally.
OTEL_SERVICE_NAME	bd	Override the service.name resource attribute.
OTEL_RESOURCE_ATTRIBUTES	deployment.environment=workstation,team=infra	Extend or override resource attributes (comma-separated key=value).
OTEL_SDK_DISABLED	true	Force telemetry off even when BD_OTEL_ENABLED=true is set.

Resource attributes

Every metric and span carries the OTel resource describing the bd process:

Attribute	Value	Notes
service.name	bd	Override with OTEL_SERVICE_NAME.
service.version	bd version

Add anything else via OTEL_RESOURCE_ATTRIBUTES.

Local debug mode

BD_OTEL_ENABLED=true OTEL_TRACES_EXPORTER=console OTEL_METRICS_EXPORTER=console bd list

Legacy environment variables (deprecated)

The earlier BD_OTEL_* data variables are honored for backwards compatibility. Setting any of them activates telemetry on its own (no BD_OTEL_ENABLED=true required) and translates to the standard OTLP equivalent — a legacy value wins over a pre-existing OTEL_* value so a machine-global OTEL_* setting cannot silently redirect bd telemetry. Each bd invocation that sees one logs a one-line deprecation warning to stderr:

Legacy	Standard equivalent
BD_OTEL_METRICS_URL	OTEL_EXPORTER_OTLP_METRICS_ENDPOINT
BD_OTEL_LOGS_URL	OTEL_EXPORTER_OTLP_LOGS_ENDPOINT
BD_OTEL_STDOUT=true	OTEL_TRACES_EXPORTER=console + OTEL_METRICS_EXPORTER=console

Verification

bd list   # triggers metrics → visible in VictoriaMetrics

Verification queries in Grafana (VictoriaMetrics datasource):

bd_storage_operations_total                              # raw counter
sum(rate(bd_storage_operations_total[5m]))               # operation rate

Confirm storage instrumentation locally

To verify the storage decorator chain is wired up without standing up a collector, run bd with stdout exporters and look for bd.storage.* records on stderr:

BD_OTEL_STDOUT=true bd list 2>&1 | grep -F bd.storage.operations

Expect at least one line per storage call (GetReadyWork, GetIssue, …). If bd.storage.* and bd.issue.count are absent but bd.db.pool_* is present, the storage decorator is not in the chain — check wireStorageDecorators in cmd/bd/storage_chain.go.

---

Metrics

Storage (bd_storage_*)

Metric	Type	Attributes	Description
bd_storage_operations_total	Counter	db.operation	Storage operations executed
bd_storage_operation_duration_ms	Histogram	db.operation	Operation duration (ms)
bd_storage_errors_total	Counter	db.operation	Storage errors

These metrics are emitted by InstrumentedStorage, the beads SDK wrapper.

Dolt database (bd_db_*)

Metric	Type	Attributes	Description
bd_db_retry_count_total	Counter	—	SQL retries in server mode
bd_db_lock_wait_ms	Histogram	dolt_lock_exclusive	Wait time to acquire database locks

Issues (bd_issue_*)

Metric	Type	Attributes	Description
bd_issue_count	Gauge	status	Number of issues by status

status values: open, in_progress, closed, deferred.

AI (bd_ai_*)

Metric	Type	Attributes	Description
bd_ai_input_tokens_total	Counter	bd_ai_model	Anthropic input tokens
bd_ai_output_tokens_total	Counter	bd_ai_model	Anthropic output tokens
bd_ai_request_duration_ms	Histogram	bd_ai_model	API call latency

---

Traces (spans)

Spans are only exported when OTEL_TRACES_EXPORTER=console — there is no trace backend in the recommended local stack.

Span	Source	Description
bd.command.<name>	CLI	Total duration of the command
dolt.exec / dolt.query / dolt.query_row	SQL	Each SQL operation
dolt.commit / dolt.push / dolt.pull / dolt.merge	Dolt VC	Version control procedures
ephemeral.count / ephemeral.nuke	SQLite	Ephemeral store operations
hook.exec	Hooks	Hook execution (root span, fire-and-forget)
tracker.sync / tracker.pull / tracker.push	Sync	Tracker sync phases
anthropic.messages.new	AI	Claude API calls

Notable attributes

bd.command.<name>

Attribute	Description
bd.command	Subcommand name (list, create, ...)
bd.version	bd version
bd.args	Raw arguments passed to the command (e.g. "create 'title' -p 2")
bd.actor	Actor (resolved from git config / env)

hook.exec

Attribute / Event	Description
hook.event	Event type (create, update, close)
hook.path	Absolute path to the script
bd.issue_id	ID of the triggering issue
event hook.stdout	Script standard output (truncated to 1 024 bytes)
event hook.stderr	Script error output (truncated to 1 024 bytes)

The hook.stdout / hook.stderr events carry two attributes: output (the text) and bytes (original size before truncation).

---

Architecture

cmd/bd/main.go
  └─ telemetry.Init()
      ├─ OTEL_TRACES_EXPORTER=console        → TracerProvider stdout
      ├─ OTEL_METRICS_EXPORTER=console       → MeterProvider stdout
      └─ OTEL_EXPORTER_OTLP_METRICS_ENDPOINT → MeterProvider HTTP → VictoriaMetrics

internal/storage/dolt/        → bd_db_* metrics + dolt.* spans
internal/storage/ephemeral/   → ephemeral.* spans
internal/hooks/               → hook.exec span
internal/tracker/             → tracker.* spans
internal/compact/             → bd_ai_* metrics + anthropic.* spans
internal/telemetry/storage.go → bd_storage_* metrics (SDK wrapper)

When no OpenTelemetry SDK environment variable selects an exporter, telemetry.Init() installs no-op providers: hot paths execute only no-op calls with no memory allocation.