google__adk-docs/docs/agents/models/vertex.md

Vertex AI hosted models for ADK agents

For enterprise-grade scalability, reliability, and integration with Google Cloud's MLOps ecosystem, you can use models deployed to Vertex AI Endpoints. This includes models from Model Garden or your own fine-tuned models.

Integration Method: Pass the full Vertex AI Endpoint resource string (projects/PROJECT_ID/locations/LOCATION/endpoints/ENDPOINT_ID) directly to the model parameter of LlmAgent.

Vertex AI Setup

Ensure your environment is configured for Vertex AI:

1. Authentication: Use Application Default Credentials (ADC):

   gcloud auth application-default login
   

2. Environment Variables: Set your project and location:

   export GOOGLE_CLOUD_PROJECT="YOUR_PROJECT_ID"
   export GOOGLE_CLOUD_LOCATION="YOUR_VERTEX_AI_LOCATION" # e.g., us-central1
   

3. Enable Vertex Backend: Crucially, ensure the google-genai library targets Vertex AI:

   export GOOGLE_GENAI_USE_VERTEXAI=TRUE
   

Model Garden Deployments

Supported in ADKPython v0.2.0

You can deploy various open and proprietary models from the Vertex AI Model Garden to an endpoint.

Example:

from google.adk.agents import LlmAgent
from google.genai import types # For config objects

# --- Example Agent using a Llama 3 model deployed from Model Garden ---

# Replace with your actual Vertex AI Endpoint resource name
llama3_endpoint = "projects/YOUR_PROJECT_ID/locations/us-central1/endpoints/YOUR_LLAMA3_ENDPOINT_ID"

agent_llama3_vertex = LlmAgent(
    model=llama3_endpoint,
    name="llama3_vertex_agent",
    instruction="You are a helpful assistant based on Llama 3, hosted on Vertex AI.",
    generate_content_config=types.GenerateContentConfig(max_output_tokens=2048),
    # ... other agent parameters
)

Fine-tuned Model Endpoints

Supported in ADKPython v0.2.0

Deploying your fine-tuned models (whether based on Gemini or other architectures supported by Vertex AI) results in an endpoint that can be used directly.

Example:

from google.adk.agents import LlmAgent

# --- Example Agent using a fine-tuned Gemini model endpoint ---

# Replace with your fine-tuned model's endpoint resource name
finetuned_gemini_endpoint = "projects/YOUR_PROJECT_ID/locations/us-central1/endpoints/YOUR_FINETUNED_ENDPOINT_ID"

agent_finetuned_gemini = LlmAgent(
    model=finetuned_gemini_endpoint,
    name="finetuned_gemini_agent",
    instruction="You are a specialized assistant trained on specific data.",
    # ... other agent parameters
)

Anthropic Claude on Vertex AI

Supported in ADKPython v0.2.0Java v0.1.0

Some providers, like Anthropic, make their models available directly through Vertex AI.

=== "Python"

**Integration Method:** Uses the direct model string (e.g.,
`"claude-3-sonnet@20240229"`), *but requires manual registration* within ADK.

**Why Registration?** ADK's registry automatically recognizes `gemini-*` strings
and standard Vertex AI endpoint strings (`projects/.../endpoints/...`) and
routes them via the `google-genai` library. For other model types used directly
via Vertex AI (like Claude), you must explicitly tell the ADK registry which
specific wrapper class (`Claude` in this case) knows how to handle that model
identifier string with the Vertex AI backend.

**Setup:**

1. **Vertex AI Environment:** Ensure the consolidated Vertex AI setup (ADC, Env
   Vars, `GOOGLE_GENAI_USE_VERTEXAI=TRUE`) is complete.

2. **Install Provider Library:** Install the necessary client library configured
   for Vertex AI.

    ```shell
    pip install "anthropic[vertex]"
    ```

3. **Register Model Class:** Add this code near the start of your application,
   *before* creating an agent using the Claude model string:

    ```python
    # Required for using Claude model strings directly via Vertex AI with LlmAgent
    from google.adk.models.anthropic_llm import Claude
    from google.adk.models.registry import LLMRegistry

    LLMRegistry.register(Claude)
    ```

   **Example:**

   ```python
   from google.adk.agents import LlmAgent
   from google.adk.models.anthropic_llm import Claude # Import needed for registration
   from google.adk.models.registry import LLMRegistry # Import needed for registration
   from google.genai import types

   # --- Register Claude class (do this once at startup) ---
   LLMRegistry.register(Claude)

   # --- Example Agent using Claude 3 Sonnet on Vertex AI ---

   # Standard model name for Claude 3 Sonnet on Vertex AI
   claude_model_vertexai = "claude-3-sonnet@20240229"

   agent_claude_vertexai = LlmAgent(
       model=claude_model_vertexai, # Pass the direct string after registration
       name="claude_vertexai_agent",
       instruction="You are an assistant powered by Claude 3 Sonnet on Vertex AI.",
       generate_content_config=types.GenerateContentConfig(max_output_tokens=4096),
       # ... other agent parameters
   )
   ```

=== "Java"

**Integration Method:** Directly instantiate the provider-specific model class (e.g., `com.google.adk.models.Claude`) and configure it with a Vertex AI backend.

**Why Direct Instantiation?** The Java ADK's `LlmRegistry` primarily handles Gemini models by default. For third-party models like Claude on Vertex AI, you directly provide an instance of the ADK's wrapper class (e.g., `Claude`) to the `LlmAgent`. This wrapper class is responsible for interacting with the model via its specific client library, configured for Vertex AI.

**Setup:**

1.  **Vertex AI Environment:**
    *   Ensure your Google Cloud project and region are correctly set up.
    *   **Application Default Credentials (ADC):** Make sure ADC is configured correctly in your environment. This is typically done by running `gcloud auth application-default login`. The Java client libraries use these credentials to authenticate with Vertex AI. Follow the [Google Cloud Java documentation on ADC](https://cloud.google.com/java/docs/reference/google-auth-library/latest/com.google.auth.oauth2.GoogleCredentials#com_google_auth_oauth2_GoogleCredentials_getApplicationDefault__) for detailed setup.

2.  **Provider Library Dependencies:**
    *   **Third-Party Client Libraries (Often Transitive):** The ADK core library often includes the necessary client libraries for common third-party models on Vertex AI (like Anthropic's required classes) as **transitive dependencies**. This means you might not need to explicitly add a separate dependency for the Anthropic Vertex SDK in your `pom.xml` or `build.gradle`.

3.  **Instantiate and Configure the Model:**
    When creating your `LlmAgent`, instantiate the `Claude` class (or the equivalent for another provider) and configure its `VertexBackend`.

**Example:**

```java
import com.anthropic.client.AnthropicClient;
import com.anthropic.client.okhttp.AnthropicOkHttpClient;
import com.anthropic.vertex.backends.VertexBackend;
import com.google.adk.agents.LlmAgent;
import com.google.adk.models.Claude; // ADK's wrapper for Claude
import com.google.auth.oauth2.GoogleCredentials;
import java.io.IOException;

// ... other imports

public class ClaudeVertexAiAgent {

    public static LlmAgent createAgent() throws IOException {
        // Model name for Claude 3 Sonnet on Vertex AI (or other versions)
        String claudeModelVertexAi = "claude-3-7-sonnet"; // Or any other Claude model

        // Configure the AnthropicOkHttpClient with the VertexBackend
        AnthropicClient anthropicClient = AnthropicOkHttpClient.builder()
            .backend(
                VertexBackend.builder()
                    .region("us-east5") // Specify your Vertex AI region
                    .project("your-gcp-project-id") // Specify your GCP Project ID
                    .googleCredentials(GoogleCredentials.getApplicationDefault())
                    .build())
            .build();

        // Instantiate LlmAgent with the ADK Claude wrapper
        LlmAgent agentClaudeVertexAi = LlmAgent.builder()
            .model(new Claude(claudeModelVertexAi, anthropicClient)) // Pass the Claude instance
            .name("claude_vertexai_agent")
            .instruction("You are an assistant powered by Claude 3 Sonnet on Vertex AI.")
            // .generateContentConfig(...) // Optional: Add generation config if needed
            // ... other agent parameters
            .build();

        return agentClaudeVertexAi;
    }

    public static void main(String[] args) {
        try {
            LlmAgent agent = createAgent();
            System.out.println("Successfully created agent: " + agent.name());
            // Here you would typically set up a Runner and Session to interact with the agent
        } catch (IOException e) {
            System.err.println("Failed to create agent: " + e.getMessage());
            e.printStackTrace();
        }
    }
}
```

Open Models on Vertex AI

Supported in ADKPython v0.1.0

Vertex AI offers a curated selection of open-source models, such as Meta Llama, through Model-as-a-Service (MaaS). These models are accessible via managed APIs, allowing you to deploy and scale without managing the underlying infrastructure. For a full list of available options, see the Vertex AI open models for MaaS documentation.

=== "Python"

You can use the [LiteLLM](https://docs.litellm.ai/) library to access open models like Meta's Llama on VertexAI MaaS

**Integration Method:** Use the `LiteLlm` wrapper class and set it
as the `model` parameter of `LlmAgent`. Make sure you go through the [LiteLLM model connector for ADK agents](/adk-docs/agents/models/litellm/#litellm-model-connector-for-adk-agents) documentation on how to use LiteLLM in ADK

**Setup:**

1. **Vertex AI Environment:** Ensure the consolidated Vertex AI setup (ADC, Env
   Vars, `GOOGLE_GENAI_USE_VERTEXAI=TRUE`) is complete.

2. **Install LiteLLM:**
        ```shell
        pip install litellm
        ```

**Example:**

```python
from google.adk.agents import LlmAgent
from google.adk.models.lite_llm import LiteLlm

# --- Example Agent using Meta's Llama 4 Scout ---
agent_llama_vertexai = LlmAgent(
    model=LiteLlm(model="vertex_ai/meta/llama-4-scout-17b-16e-instruct-maas"), # LiteLLM model string format
    name="llama4_agent",
    instruction="You are a helpful assistant powered by Llama 4 Scout.",
    # ... other agent parameters
)

```