mirror of
https://github.com/google/adk-docs.git
synced 2026-09-14 16:16:59 +08:00
George Weale
71dc8b0b61
* docs(models): fix unparseable snippets and stale Claude setup steps * docs(models): correct apigee retry claim, LlmResponse.text, McpToolset spelling --------- Co-authored-by: Joe Fernandez <931947+joefernandez@users.noreply.github.com>
323 lines
13 KiB
Markdown
323 lines
13 KiB
Markdown
# Agent Platform hosted models for ADK agents
|
|
|
|
For enterprise-grade scalability, reliability, and integration with Google
|
|
Cloud's MLOps ecosystem, you can use models deployed to Agent Platform Endpoints.
|
|
This includes models from Model Garden or your own fine-tuned models.
|
|
|
|
**Integration Method:** Pass the full Agent Platform Endpoint resource string
|
|
(`projects/PROJECT_ID/locations/LOCATION/endpoints/ENDPOINT_ID`) directly to the
|
|
`model` parameter of `LlmAgent`.
|
|
|
|
## Agent Platform Setup
|
|
|
|
For more details on connecting ADK agents to Google Cloud hosted models and services,
|
|
including Gemini Enterprise Agent Platform, see the
|
|
[Connect to Google Cloud and Agent Platform](/get-started/google-cloud/) guide.
|
|
|
|
## Model Garden Deployments
|
|
|
|
<div class="language-support-tag">
|
|
<span class="lst-supported">Supported in ADK</span><span class="lst-python">Python v0.2.0</span><span class="lst-java">Java v0.1.0</span>
|
|
</div>
|
|
|
|
You can deploy various open and proprietary models from the
|
|
[Model Garden](https://console.cloud.google.com/vertex-ai/model-garden)
|
|
to an endpoint.
|
|
|
|
**Example:**
|
|
|
|
=== "Python"
|
|
|
|
```python
|
|
from google.adk.agents import LlmAgent
|
|
from google.genai import types # For config objects
|
|
|
|
# --- Example Agent using a Llama 3 model deployed from Model Garden ---
|
|
|
|
# Replace with your actual Agent Platform Endpoint resource name
|
|
llama3_endpoint = "projects/YOUR_PROJECT_ID/locations/us-central1/endpoints/YOUR_LLAMA3_ENDPOINT_ID"
|
|
|
|
agent_llama3_vertex = LlmAgent(
|
|
model=llama3_endpoint,
|
|
name="llama3_vertex_agent",
|
|
instruction="You are a helpful assistant based on Llama 3, hosted on Agent Platform.",
|
|
generate_content_config=types.GenerateContentConfig(max_output_tokens=2048),
|
|
# ... other agent parameters
|
|
)
|
|
```
|
|
|
|
=== "Java"
|
|
|
|
```java
|
|
import com.google.adk.agents.LlmAgent;
|
|
import com.google.adk.models.Gemini;
|
|
import com.google.genai.types.GenerateContentConfig;
|
|
|
|
// ...
|
|
|
|
// Replace with your actual Agent Platform Endpoint resource name
|
|
String llama3Endpoint = "projects/YOUR_PROJECT_ID/locations/us-central1/endpoints/YOUR_LLAMA3_ENDPOINT_ID";
|
|
|
|
LlmAgent agentLlama3Vertex = LlmAgent.builder()
|
|
.model(Gemini.builder()
|
|
.modelName(llama3Endpoint)
|
|
.build())
|
|
.name("llama3_vertex_agent")
|
|
.instruction("You are a helpful assistant based on Llama 3, hosted on Agent Platform.")
|
|
.generateContentConfig(GenerateContentConfig.builder()
|
|
.maxOutputTokens(2048)
|
|
.build())
|
|
// ... other agent parameters
|
|
.build();
|
|
```
|
|
|
|
## Fine-tuned Model Endpoints
|
|
|
|
<div class="language-support-tag">
|
|
<span class="lst-supported">Supported in ADK</span><span class="lst-python">Python v0.2.0</span><span class="lst-java">Java v0.1.0</span>
|
|
</div>
|
|
|
|
Deploying your fine-tuned models (whether based on Gemini or other architectures
|
|
supported by Agent Platform) results in an endpoint that can be used directly.
|
|
|
|
**Example:**
|
|
|
|
=== "Python"
|
|
|
|
```python
|
|
from google.adk.agents import LlmAgent
|
|
|
|
# --- Example Agent using a fine-tuned Gemini model endpoint ---
|
|
|
|
# Replace with your fine-tuned model's endpoint resource name
|
|
finetuned_gemini_endpoint = "projects/YOUR_PROJECT_ID/locations/us-central1/endpoints/YOUR_FINETUNED_ENDPOINT_ID"
|
|
|
|
agent_finetuned_gemini = LlmAgent(
|
|
model=finetuned_gemini_endpoint,
|
|
name="finetuned_gemini_agent",
|
|
instruction="You are a specialized assistant trained on specific data.",
|
|
# ... other agent parameters
|
|
)
|
|
```
|
|
|
|
=== "Java"
|
|
|
|
```java
|
|
import com.google.adk.agents.LlmAgent;
|
|
import com.google.adk.models.Gemini;
|
|
|
|
// ...
|
|
|
|
// Replace with your fine-tuned model's endpoint resource name
|
|
String finetunedGeminiEndpoint = "projects/YOUR_PROJECT_ID/locations/us-central1/endpoints/YOUR_FINETUNED_ENDPOINT_ID";
|
|
|
|
LlmAgent agentFinetunedGemini = LlmAgent.builder()
|
|
.model(Gemini.builder()
|
|
.modelName(finetunedGeminiEndpoint)
|
|
.build())
|
|
.name("finetuned_gemini_agent")
|
|
.instruction("You are a specialized assistant trained on specific data.")
|
|
// ... other agent parameters
|
|
.build();
|
|
```
|
|
|
|
## Anthropic Claude on Agent Platform {#anthropic-claude}
|
|
|
|
<div class="language-support-tag">
|
|
<span class="lst-supported">Supported in ADK</span><span class="lst-python">Python v0.2.0</span><span class="lst-java">Java v0.1.0</span>
|
|
</div>
|
|
|
|
Some providers, like Anthropic, make their models available directly through
|
|
Agent Platform.
|
|
|
|
**Example:**
|
|
|
|
=== "Python"
|
|
|
|
**Integration Method:** Uses the direct model string (e.g.,
|
|
`"claude-3-sonnet@20240229"`).
|
|
|
|
**How Resolution Works:** ADK's registry automatically recognizes `gemini-*`
|
|
strings and standard Agent Platform endpoint strings
|
|
(`projects/.../locations/.../endpoints/...`) and routes them via the `google-genai`
|
|
library. Claude model strings matching `claude-3-*` or `claude-*-4*` route
|
|
to the `Claude` wrapper class the same way. For a Claude model identifier
|
|
that does not match those patterns, import `Claude` from
|
|
`google.adk.models` and pass an instance instead of a string:
|
|
`LlmAgent(model=Claude(model="..."), ...)`.
|
|
|
|
**Setup:**
|
|
|
|
1. **Agent Platform Environment:** Ensure the consolidated Agent Platform setup (ADC, Env
|
|
Vars, `GOOGLE_GENAI_USE_ENTERPRISE=TRUE`) is complete.
|
|
|
|
2. **Install Provider Library:** Install the necessary client library configured
|
|
for Agent Platform.
|
|
|
|
```shell
|
|
pip install "anthropic[vertex]"
|
|
```
|
|
|
|
3. **Create the Agent:** Pass the Claude model string to `LlmAgent`:
|
|
|
|
```python
|
|
from google.adk.agents import LlmAgent
|
|
from google.genai import types
|
|
|
|
# --- Example Agent using Claude 3 Sonnet on Agent Platform ---
|
|
|
|
# Standard model name for Claude 3 Sonnet on Agent Platform
|
|
claude_model_vertexai = "claude-3-sonnet@20240229"
|
|
|
|
agent_claude_vertexai = LlmAgent(
|
|
model=claude_model_vertexai, # Pass the direct model string
|
|
name="claude_vertexai_agent",
|
|
instruction="You are an assistant powered by Claude 3 Sonnet on Agent Platform.",
|
|
generate_content_config=types.GenerateContentConfig(max_output_tokens=4096),
|
|
# ... other agent parameters
|
|
)
|
|
```
|
|
|
|
=== "Java"
|
|
|
|
**Integration Method:** Directly instantiate the provider-specific model class (e.g., `com.google.adk.models.Claude`) and configure it with an Agent Platform backend.
|
|
|
|
**Why Direct Instantiation?** The Java ADK's `LlmRegistry` primarily handles Gemini models by default. For third-party models like Claude on Agent Platform, you directly provide an instance of the ADK's wrapper class (e.g., `Claude`) to the `LlmAgent`. This wrapper class is responsible for interacting with the model via its specific client library, configured for Agent Platform.
|
|
|
|
**Setup:**
|
|
|
|
1. **Agent Platform Environment:**
|
|
* Ensure your Google Cloud project and region are correctly set up.
|
|
* **Application Default Credentials (ADC):** Make sure ADC is configured correctly in your environment. This is typically done by running `gcloud auth application-default login`. The Java client libraries use these credentials to authenticate with Agent Platform. Follow the [Google Cloud Java documentation on ADC](https://cloud.google.com/java/docs/reference/google-auth-library/latest/com.google.auth.oauth2.GoogleCredentials#com_google_auth_oauth2_GoogleCredentials_getApplicationDefault__) for detailed setup.
|
|
|
|
2. **Provider Library Dependencies:**
|
|
* **Third-Party Client Libraries (Often Transitive):** The ADK core library often includes the necessary client libraries for common third-party models on Agent Platform (like Anthropic's required classes) as **transitive dependencies**. This means you might not need to explicitly add a separate dependency for the Anthropic Vertex SDK in your `pom.xml` or `build.gradle`.
|
|
|
|
3. **Instantiate and Configure the Model:**
|
|
When creating your `LlmAgent`, instantiate the `Claude` class (or the equivalent for another provider) and configure its `VertexBackend`.
|
|
|
|
```java
|
|
import com.anthropic.client.AnthropicClient;
|
|
import com.anthropic.client.okhttp.AnthropicOkHttpClient;
|
|
import com.anthropic.vertex.backends.VertexBackend;
|
|
import com.google.adk.agents.LlmAgent;
|
|
import com.google.adk.models.Claude; // ADK's wrapper for Claude
|
|
import com.google.auth.oauth2.GoogleCredentials;
|
|
import java.io.IOException;
|
|
|
|
// ... other imports
|
|
|
|
public class ClaudeVertexAiAgent {
|
|
|
|
public static LlmAgent createAgent() throws IOException {
|
|
// Model name for Claude 3 Sonnet on Agent Platform (or other versions)
|
|
String claudeModelVertexAi = "claude-3-7-sonnet"; // Or any other Claude model
|
|
|
|
// Configure the AnthropicOkHttpClient with the VertexBackend
|
|
AnthropicClient anthropicClient = AnthropicOkHttpClient.builder()
|
|
.backend(
|
|
VertexBackend.builder()
|
|
.region("us-east5") // Specify your Agent Platform region
|
|
.project("your-gcp-project-id") // Specify your GCP Project ID
|
|
.googleCredentials(GoogleCredentials.getApplicationDefault())
|
|
.build())
|
|
.build();
|
|
|
|
// Instantiate LlmAgent with the ADK Claude wrapper
|
|
LlmAgent agentClaudeVertexAi = LlmAgent.builder()
|
|
.model(new Claude(claudeModelVertexAi, anthropicClient)) // Pass the Claude instance
|
|
.name("claude_vertexai_agent")
|
|
.instruction("You are an assistant powered by Claude 3 Sonnet on Agent Platform.")
|
|
// .generateContentConfig(...) // Optional: Add generation config if needed
|
|
// ... other agent parameters
|
|
.build();
|
|
|
|
return agentClaudeVertexAi;
|
|
}
|
|
|
|
public static void main(String[] args) {
|
|
try {
|
|
LlmAgent agent = createAgent();
|
|
System.out.println("Successfully created agent: " + agent.name());
|
|
// Here you would typically set up a Runner and Session to interact with the agent
|
|
} catch (IOException e) {
|
|
System.err.println("Failed to create agent: " + e.getMessage());
|
|
e.printStackTrace();
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
### Adaptive thinking
|
|
|
|
<div class="language-support-tag">
|
|
<span class="lst-supported">Supported in ADK</span><span class="lst-python">Python v1.34.0</span>
|
|
</div>
|
|
|
|
Newer Claude models support *adaptive* extended thinking, where the model chooses
|
|
its reasoning depth itself rather than using a fixed token budget. On the native
|
|
Claude path, a negative `thinking_budget` maps to adaptive thinking.
|
|
|
|
The recommended way to control reasoning depth is the `effort` field on
|
|
`AnthropicGenerateContentConfig`:
|
|
|
|
```python
|
|
from google.adk.agents import LlmAgent
|
|
from google.adk.models import AnthropicGenerateContentConfig
|
|
|
|
agent = LlmAgent(
|
|
model="claude-sonnet-4@20250514", # Your Agent Platform Claude model ID.
|
|
name="claude_reasoning_agent",
|
|
instruction="You are a helpful assistant.",
|
|
generate_content_config=AnthropicGenerateContentConfig(
|
|
effort="high", # One of: "low", "medium", "high", "xhigh", "max".
|
|
),
|
|
)
|
|
```
|
|
|
|
* The standard `thinking_config.thinking_level` is not supported for Claude.
|
|
Setting it on `AnthropicGenerateContentConfig` raises a validation error; on
|
|
a plain `types.GenerateContentConfig` it is ignored with a warning. Use
|
|
`effort` instead.
|
|
|
|
## Open Models on Agent Platform {#open-models}
|
|
|
|
<div class="language-support-tag">
|
|
<span class="lst-supported">Supported in ADK</span><span class="lst-python">Python v0.1.0</span><span class="lst-java">Java v0.1.0</span>
|
|
</div>
|
|
|
|
Agent Platform offers a curated selection of open-source models, such as Meta Llama, through Model-as-a-Service (MaaS). These models are accessible via managed APIs, allowing you to deploy and scale without managing the underlying infrastructure. For a full list of available options, see the [Agent Platform open models for MaaS](https://docs.cloud.google.com/vertex-ai/generative-ai/docs/maas/use-open-models#open-models) documentation.
|
|
|
|
=== "Python"
|
|
|
|
You can use the [LiteLLM](https://docs.litellm.ai/) library to access open models like Meta's Llama on Agent Platform MaaS
|
|
|
|
**Integration Method:** Use the `LiteLlm` wrapper class and set it
|
|
as the `model` parameter of `LlmAgent`. Make sure you go through the [LiteLLM model connector for ADK agents](/agents/models/litellm/#litellm-model-connector-for-adk-agents) documentation on how to use LiteLLM in ADK
|
|
|
|
**Setup:**
|
|
|
|
1. **Agent Platform Environment:** Ensure the consolidated Agent Platform setup (ADC, Env
|
|
Vars, `GOOGLE_GENAI_USE_ENTERPRISE=TRUE`) is complete.
|
|
|
|
2. **Install LiteLLM:** ADK requires `litellm>=1.84`.
|
|
```shell
|
|
pip install "litellm>=1.84"
|
|
```
|
|
|
|
**Example:**
|
|
|
|
```python
|
|
from google.adk.agents import LlmAgent
|
|
from google.adk.models.lite_llm import LiteLlm
|
|
|
|
# --- Example Agent using Meta's Llama 4 Scout ---
|
|
agent_llama_vertexai = LlmAgent(
|
|
model=LiteLlm(model="vertex_ai/meta/llama-4-scout-17b-16e-instruct-maas"), # LiteLLM model string format
|
|
name="llama4_agent",
|
|
instruction="You are a helpful assistant powered by Llama 4 Scout.",
|
|
# ... other agent parameters
|
|
)
|
|
|
|
```
|