mirror of
https://github.com/google/adk-docs.git
synced 2026-09-14 16:16:59 +08:00
Debu Sinha
8be243255b
* Add MLflow scorers integration page Adds a new integration page at docs/integrations/mlflow-scorers.md covering MLflow's five Google ADK scorers (ToolTrajectory, ResponseMatch, ResponseEvaluation, Safety, Hallucination) for agent evaluation. The integration wraps ADK's TrajectoryEvaluator, RougeEvaluator, FinalResponseMatchV2Evaluator, SafetyEvaluatorV1, and HallucinationsV1Evaluator behind MLflow's scorer interface, so ADK users can evaluate agents inside mlflow.genai.evaluate() runs without leaving the ADK ecosystem. Complements the existing MLflow Tracing and MLflow AI Gateway integration pages by covering evaluation, the third leg of the MLflow stack for ADK. Signed-off-by: debu-sinha <debusinha2009@gmail.com> * Trigger CLA re-check Signed-off-by: debu-sinha <debusinha2009@gmail.com> * Use version-agnostic Gemini aliases and refine copy * Update category tags for other pages --------- Signed-off-by: debu-sinha <debusinha2009@gmail.com> Co-authored-by: Kristopher Overholt <koverholt@google.com>
97 lines
3.2 KiB
Markdown
97 lines
3.2 KiB
Markdown
---
|
|
catalog_title: Datadog
|
|
catalog_description: Develop, evaluate, and monitor LLM applications
|
|
catalog_icon: /integrations/assets/datadog.png
|
|
catalog_tags: ["observability", "evaluation"]
|
|
---
|
|
|
|
# Datadog Observability for ADK
|
|
|
|
<div class="language-support-tag">
|
|
<span class="lst-supported">Supported in ADK</span><span class="lst-python">Python</span>
|
|
</div>
|
|
|
|
[Datadog LLM
|
|
Observability](https://www.datadoghq.com/product/llm-observability/) helps AI
|
|
engineers, data scientists, and application developers quickly develop,
|
|
evaluate, and monitor LLM applications. Confidently improve output quality,
|
|
performance, costs, and overall risk with structured experiments, end-to-end
|
|
tracing across AI agents, and evaluations.
|
|
|
|
## Overview
|
|
|
|
Datadog LLM Observability can [automatically instrument and trace your agents
|
|
built on Google
|
|
ADK](https://docs.datadoghq.com/llm_observability/instrumentation/auto_instrumentation?tab=python#google-adk),
|
|
allowing you to:
|
|
|
|
- **Observe agent executions and interactions** - Automatically capture every
|
|
agent run, tool call, and code execution within your agents
|
|
- **Capture LLM calls and responses** made with the underlying Google GenAI SDK
|
|
- **Debug issues** by providing error rates, token usage and cost, and
|
|
out-of-the-box evaluations on your LLM calls and tool usage
|
|
|
|
## Prerequisites
|
|
|
|
Sign up for a [Datadog account](https://www.datadoghq.com/) if you do not have
|
|
one and [get your API
|
|
key](https://docs.datadoghq.com/account_management/api-app-keys/#api-keys).
|
|
|
|
## Installation
|
|
|
|
Install the required packages:
|
|
|
|
```bash
|
|
pip install ddtrace
|
|
```
|
|
|
|
## Setup
|
|
|
|
### Create an Application using ADK
|
|
|
|
If you do not have an application using ADK, follow the steps in the [ADK
|
|
Getting Started Guide](https://adk.dev/get-started/) to create a sample ADK
|
|
agent.
|
|
|
|
### Configure Environment Variables
|
|
|
|
You will need to specify an ML Application name in the following environment
|
|
variables. An ML Application is a grouping of LLM Observability traces
|
|
associated with a specific LLM-based application. See [ML Application Naming
|
|
Guidelines](https://docs.datadoghq.com/llm_observability/instrumentation/sdk?tab=python#application-naming-guidelines)
|
|
for more information on limitations with ML Application names.
|
|
|
|
```shell
|
|
export DD_API_KEY=<YOUR_DD_API_KEY>
|
|
export DD_SITE=<YOUR_DD_SITE>
|
|
export DD_LLMOBS_ENABLED=true
|
|
export DD_LLMOBS_ML_APP=<YOUR_ML_APP_NAME>
|
|
export DD_LLMOBS_AGENTLESS_ENABLED=true
|
|
export DD_APM_TRACING_ENABLED=false # Only set this if you are not using Datadog APM
|
|
```
|
|
|
|
These variables must be exported before running your application so the
|
|
following `ddtrace-run` command can use them, as opposed to putting them in the
|
|
agent's `.env` file.
|
|
|
|
### Run Your Application
|
|
|
|
Once you have configured your environment variables, you can run your
|
|
application and start observing your LLM-based applications.
|
|
|
|
```shell
|
|
ddtrace-run adk run my_agent
|
|
```
|
|
|
|
## Observe
|
|
|
|
Navigate to the [Datadog LLM Observability Traces
|
|
View](https://app.datadoghq.com/llm/traces) to see the traces generated by your
|
|
application.
|
|
|
|
|
|
|
|
## Support and Resources
|
|
- [Datadog LLM Observability](https://www.datadoghq.com/product/llm-observability/)
|
|
- [Datadog Support](https://docs.datadoghq.com/help/)
|