Files
google__adk-docs/docs/live/get-started/streaming-python.md
Kaz Sato 03eccf55e0 docs(live): decompose the dev guide and fix staleness vs adk-python main (#2086)
* docs(live): decompose the development guide into capability pages

Split dev-guide/part1-5 into Sessions, Events, Tools, Workflows, Audio and
video, Configuration, Voice, Supported models, and Build a custom server.
Rewrite index.md as the section Overview with a streaming-type decision table.

Implements Phase 2 of the Live Interactions<>ADK documentation revamp.

* docs(live): drop half-cascade model coverage

Half-cascade models are no longer supported for live agents. Remove the
Native Audio vs Half-Cascade architecture framing from Supported models and
the half-cascade caveats from Voice configuration. The eight prebuilt Live
API voices are kept, relabeled as native-audio voices alongside the extended
Text-to-Speech list.

* docs(live): retire the five-part dev guide and rewire navigation

Delete live/dev-guide/ and live/streaming-tools.md now that their content
lives in the capability pages. Regroup the Live nav into Get started / Build /
Ship / Reference, repoint every partN.md cross-link at its new page and
anchor, and add direct redirects for the removed paths (mkdocs-redirects does
not chain, so streaming/* keys point at final destinations).

* docs(live): point at the API reference instead of pinned source

Swap the RunConfig, Event, SequentialAgent, LiveRequestQueue and
Runner.run_live source-reference notes for Python API reference links.
Implementation pointers with line ranges are left as source links, since they
document internals with no public reference equivalent.

* docs(live): fix docs against adk-python main and drop the bidi-demo links

The bidi-demo sample was removed from adk-samples, so all the source links in
docs/live/ were dead. The sample is not shipped here either, so remove every
reference to it instead of repointing the links.

The code snippets themselves are unchanged. What goes away is only the
scaffolding that pointed at the sample:

- 32 code fences lose their linked 'Demo implementation: file.py:NN-MM' title
  and become plain language-tagged fences.
- The 'Complete Demo Implementation' note in custom-server.md and the 'Demo
  Implementation' note in events.md are dropped; both existed only to link out.
- The 'Learn More' note in tools.md and the model setup step in models.md keep
  their guidance but no longer cite the sample's files.
- Prose that named the demo ('The bidi-demo demonstrates how to...') is
  rewritten to describe the pattern directly.
- The Bidi Demo card and its screenshot are removed from the Live demos section
  of index.md; LensMosaic remains.

Staleness fixes verified against adk-python main:

- StreamingMode.BIDI is inert. Only run_async() reads RunConfig.streaming_mode;
  run_live() never does. Remove it from every run_live()-facing sample and
  rewrite the 'StreamingMode: BIDI or SSE' section around the Runner method you
  call. Keeps the old anchor via attr_list.
- configuration.md: run_live(session=...) is gone; use user_id/session_id.
- tools.md: streaming tools are registered lazily on first model call, not
  scanned up front; the input_stream queue is created only for tools annotated
  with LiveRequestQueue, and stop_streaming resets it to None. The old
  runners.py / function_tool.py line references pointed at unrelated code.
- sessions.md: document DEFAULT_MAX_RECONNECT_ATTEMPTS = 5 and the go_away
  reconnect trigger; correct 'automatic closure in SSE mode', which really only
  happens for the internal queue under support_cfc.
- events.md: audio artifacts require RunConfig.save_live_blob=True;
  get_author_for_event() also keys off llm_response.input_transcription.
- configuration.md: document history_config and the
  initial_history_in_client_content=True that ADK sets when seeding history.

Not changed: get-started/streaming-java.md still sets StreamingMode.BIDI, which
could not be verified without an adk-java checkout.

* Refresh the Live API supported-model list

Checked against the Gemini Live API and Agent Platform model docs:

- models.md: replace the model list with a platform/model/stage table covering
  gemini-3.1-flash-live-preview (Preview, Gemini Live API only),
  gemini-2.5-flash-native-audio-preview-12-2025 (Preview), and
  gemini-live-2.5-flash-native-audio (now GA, not "public preview").
- Document what Gemini 3.1 Live does not support: proactivity, affective
  dialog, async function calling, thinking_budget (it uses thinking_level),
  plus multi-part server events and the turn-coverage default change.
- Note that no Gemini 3.x Live model exists on Agent Platform, and that Live
  API models are unavailable in the `global` location.
- voice.md: replace the Platform Compatibility text, which wrongly said
  proactivity and affective dialog are unavailable on Agent Platform, with a
  per-model support table.
- configuration.md: CFC's model check is a literal `gemini-2` prefix match, so
  it rejects Gemini 3.x; refresh the runners.py line anchor.
- bidi-demo: same model table in the README, the 3.1 option and the regional
  location requirement in .env.example, and an expanded model comment in
  agent.py. The default stays on 2.5 native audio because the demo exposes
  proactivity and affective dialog toggles. Re-anchored the agent.py line
  links in models.md, tools.md, and sessions.md.

* docs(live): align docs with current Live API model capabilities

Verified docs/live/ and docs/runtime/runconfig.md against the Gemini Live
API capabilities guide, the Agent Platform Live API docs, and ADK 2.6.3.

Model consistency:

- response_modalities=["TEXT"] was presented as a valid live configuration
  in configuration.md, events.md and sessions.md. Every Live API model ADK
  supports is a native audio model, and those accept AUDIO only. Reframed
  around AUDIO plus output audio transcription, and kept TEXT where it is
  actually correct: the run_async() / SSE path.
- docs/runtime/runconfig.md configured response_modalities=["AUDIO","TEXT"]
  in all three language samples. A session accepts exactly one modality.
- events.md snippets read event.content.parts[0], which drops content on
  gemini-3.1-flash-live-preview because it sends multiple parts per server
  event -- the failure models.md already warns about. All four snippets now
  iterate over parts.
- tools.md gave the streaming-tools root agent model="gemini-flash-latest",
  which has no Live API support, so the example could not run under
  run_live() on either platform. That alias is still used for the one-shot
  generate_content call inside the tool, where it is correct.
- configuration.md "Standard Gemini Models (1.5 Series) Accessed via SSE"
  described a retired model family and labelled gemini-pro-latest /
  gemini-flash-latest as 1.5 with 2M context.
- sessions.md: document that send_client_content is seeding-only on Gemini
  3.x Live, and that ADK reroutes single-part text to send_realtime_input.
- models.md: gemini-live-2.5-flash-native-audio is the only GA Live API
  model on Agent Platform, not the only one.

Coverage and links:

- configuration.md: document explicit_vad_signal, translation_config,
  avatar_config and model_input_context.
- voice.md: note that ADK picks the live API version (v1alpha / v1beta1),
  so proactivity and affective dialog need no http_options.
- Replace redirecting upstream URLs with their current targets:
  live-guide -> live-api/capabilities, live-session ->
  live-api/session-management, live -> live-api, and
  cloud.google.com/vertex-ai -> the Agent Platform equivalents.

Verified correct, left alone: session and context limits, audio and video
specs, the proactivity / affective dialog model matrix, thinking_level vs
thinking_budget, the support_cfc gemini-2 prefix check, and ADK's AUDIO
default in run_live().

* docs(live): trim the response-modality and SSE material

Every Live API model ADK supports is a native audio model, so a live
session's response modality is always AUDIO and there is nothing to
choose. Shrink the section to the one thing that still matters --
reading text off event.output_transcription.

StreamingMode is only read by run_async(); the SSE tutorial that grew
around it here (protocol diagrams, progressive-streaming walkthrough,
mode-selection table, 1.5-series model list) duplicates
runtime/runconfig.md and describes models that no longer exist. Keep
the inert-BIDI warning and the run_live()/run_async() split, drop the
rest.

Document explicit_vad_signal, translation_config, avatar_config and
model_input_context, which had no coverage at all.

* docs(live): cut duplicated and non-ADK material

Six sections carried weight that did not belong to them:

- sessions.md 'Best Practices for Live API Connection and Session
  Management' restated the Session Resumption and Context Window
  Compression sections verbatim, down to the RunConfig snippets.
  Deleted.
- sessions.md 'Concurrency and Thread Safety' + 'Message Ordering
  Guarantees' explained asyncio.Queue at length and reproduced the
  upstream task already in custom-server.md. Condensed to the three
  properties that actually affect calling code, with a pointer to
  the private _queue attribute dropped.
- sessions.md 'Architectural Patterns for Managing Quotas' was an
  ASCII decision tree and a comparison table for two patterns that
  reduce to one sentence each.
- index.md 'Real-world applications' spent five industry vignettes
  making one point.
- events.md 'Deserializing on the Client' pasted 80 lines of the
  bidi-demo's UI code, calling helpers that no longer exist anywhere
  in these docs. Reduced to the event-shape handling it was meant to
  show.
- audio-video.md 'Handling Image Input at the Client' was 130 lines
  of getUserMedia/canvas/FileReader boilerplate plus a seven-point
  recap of it.

Also fix two dead absolute links: /agents/multi-agents/#workflow-agents-as-orchestrators
(the page now redirects to workflows/index.md and the anchor is gone)
and /live/streaming-tools/ (no such page; the content is in tools.md).

* docs(live): restructure the live docs around ADK ownership

The live section had accumulated content it did not own: backend limits
restated on capability pages, Web Audio API implementation presented as
ADK guidance, and shared concepts re-explained rather than linked.

Applies one rule throughout: if a fact would still be true with the ADK
source deleted, it belongs on models.md or behind an upstream link, not
on a capability page.

- audio-video.md is now the format contract only (505 -> 121). The
  browser mic-capture, ring-buffer playback, and camera-frame code was
  Web Audio API with no ADK in it, had no counterpart in adk-python, and
  no test anywhere. Deleted rather than relocated. The twelve numbered
  'Key Implementation Details' lists restated the code comments directly
  above them; deleted. The streaming-tool lifecycle section duplicated
  tools.md; replaced with a link.
- custom-server.md gains 'Connect a client': what adk web handles
  (16 kHz capture, 24 kHz playback, 1 fps JPEG, transcripts, barge-in),
  where it stops, and the /run_live wire protocol, which was previously
  undocumented. Keeps the one JS snippet that shows ADK's event shape.
  Drops 'Client-side patterns'.
- sessions.md hands its platform-limits table and quota numbers to
  models.md, keeping the session-pool design guidance. The same figures
  had been stated in three places across two pages.
- models.md gains 'Platform limits and quotas' as the single source, and
  loses the 'Key characteristics' list that restated configuration.md.
- configuration.md drops the 'Platform Support' column, which read
  'Both' on 13 of 15 rows and labelled the two exceptions as platform
  constraints when they are model constraints.
- tools.md compresses 'Tool execution context' to the one fact that is
  live-specific: an InvocationContext spans the whole run_live() loop,
  not a single turn.
- workflows.md points at graphs/index.md, the ADK 2.0 graph workflow
  page, rather than the v0.1.0 multi-agent umbrella.
- Six internal links used absolute paths, which mkdocs does not
  validate, so --strict had been silently ignoring them. Now relative.
- Fixes class.="grid cards" in get-started/index.md, which was breaking
  the card grid.

* docs(live): standardize page leads and cut duplicated RunConfig prose

Every live page opened by narrating its own table of contents ("This page
covers X, Y, and Z"), which duplicates the rendered TOC, ages badly when
a heading changes, and spends a paragraph before the reader gets a fact.
evaluation.md already did the better thing: state the shared baseline,
link the canonical page, then cover only the delta. That is now the
convention across the section.

- sessions.md, events.md, configuration.md, audio-video.md,
  workflows.md, tools.md, models.md and get-started/index.md now name
  their non-live counterpart in the lead instead of listing their own
  headings. Three pages had no outbound link to the shared concept at
  all: tools.md to Custom Tools, models.md to Models for agents, and
  workflows.md pointed at the v0.1.0 umbrella rather than graph
  workflows.
- configuration.md drops the custom_metadata section (85 lines) for a
  pointer plus the one live-specific consequence: a run_live() call is a
  single invocation, so metadata is stamped on the whole session rather
  than one turn. runtime/runconfig.md already owns the field.
- configuration.md trims max_llm_calls and save_live_blob to the facts
  that are live-specific — max_llm_calls does not apply to run_live() at
  all, and save_live_blob writes ~1.92 MB per minute per session to two
  services — and drops the generic use-case and best-practice lists.
- custom-server.md replaces 'Key concepts', which re-pasted all three
  code blocks from the complete example directly above it, with prose
  explaining why the two tasks must run concurrently.

Live section: 2820 -> 2211 lines.

* docs(live): reframe pages around capabilities, fix eval config key

* Apply batched suggestions from code review

Co-authored-by: Joe Fernandez <931947+joefernandez@users.noreply.github.com>

* Apply suggestion from @joefernandez

* Apply batched suggestions from code review

Co-authored-by: Joe Fernandez <931947+joefernandez@users.noreply.github.com>

---------

Co-authored-by: Stephen Allen <stephenaallen@google.com>
Co-authored-by: Joe Fernandez <931947+joefernandez@users.noreply.github.com>
2026-09-01 17:15:50 -07:00

7.8 KiB

Build live streaming agent with Python

With this quickstart, you'll learn to create a simple agent and use ADK Streaming to enable voice and video communication with it that is low-latency and bidirectional. We will install ADK, set up a basic "Google Search" agent, try running the agent with Streaming with adk web tool, and then explain how to build a simple asynchronous web app by yourself using ADK Streaming and FastAPI.

Note: This guide assumes you have experience using a terminal in Windows, Mac, and Linux environments.

Supported models for voice/video streaming

Voice and video streaming requires a Gemini model that supports the Live API. You can find the model ID(s) that support it in the documentation:

1. Setup Environment & Install ADK

Create & Activate Virtual Environment (Recommended):

# Create
python3 -m venv .venv
# Activate (each new terminal)
# macOS/Linux: source .venv/bin/activate
# Windows CMD: .venv\Scripts\activate.bat
# Windows PowerShell: .venv\Scripts\Activate.ps1

Install ADK:

pip install google-adk

2. Project Structure

Create the following folder structure with empty files:

adk-streaming/  # Project folder
└── app/ # the web app folder
    ├── .env # Gemini API key
    └── google_search_agent/ # Agent folder
        ├── __init__.py # Python package
        └── agent.py # Agent definition

agent.py

Copy-paste the following code block into the agent.py file.

For model, please double-check the model ID as described earlier in the Models section.

from google.adk.agents import Agent
from google.adk.tools import google_search  # Import the tool

root_agent = Agent(
   # A unique name for the agent.
   name="basic_search_agent",
   # The Large Language Model (LLM) that agent will use.
   # Please fill in the latest model id that supports live from
   # https://adk.dev/live/get-started/streaming-python/#supported-models
   model="...",
   # A short description of the agent's purpose.
   description="Agent to answer questions using Google Search.",
   # Instructions to set the agent's behavior.
   instruction="You are an expert researcher. You always stick to the facts.",
   # Add google_search tool to perform grounding with Google search.
   tools=[google_search]
)

agent.py is where all your agent(s)' logic will be stored, and you must have a root_agent defined.

Notice how easily you integrated grounding with Google Search capabilities. The Agent class and the google_search tool handle the complex interactions with the LLM and grounding with the search API, allowing you to focus on the agent's purpose and behavior.

intro_components.png

Copy-paste the following code block to __init__.py file.

from . import agent

3. Set up the platform

To run the agent, choose a platform from either Google AI Studio or Google Cloud Agent Platform:

=== "Gemini - Google AI Studio" 1. Get an API key from Google AI Studio. 2. Open the .env file located inside (app/) and copy-paste the following code.

    ```env title=".env"
    GOOGLE_GENAI_USE_ENTERPRISE=FALSE
    GOOGLE_API_KEY=PASTE_YOUR_ACTUAL_API_KEY_HERE
    ```

3. Replace `PASTE_YOUR_ACTUAL_API_KEY_HERE` with your actual `API KEY`.

=== "Gemini - Google Cloud Agent Platform" 1. You need an existing Google Cloud account and a project. * Set up a Google Cloud project * Set up the gcloud CLI * Authenticate to Google Cloud, from the terminal by running gcloud auth login. * Enable the Agent Platform API. 2. Open the .env file located inside (app/). Copy-paste the following code and update the project ID and location.

    ```env title=".env"
    GOOGLE_GENAI_USE_ENTERPRISE=TRUE
    GOOGLE_CLOUD_PROJECT=PASTE_YOUR_ACTUAL_PROJECT_ID
    GOOGLE_CLOUD_LOCATION=us-central1
    ```

For more information on connecting to Google Cloud from ADK agents, see Connect to Google Cloud and Agent Platform.

4. Try the agent with adk web

Now it's ready to try the agent. Run the following command to launch the dev UI. First, make sure to set the current directory to app:

cd app

Also, set SSL_CERT_FILE variable with the following command. This is required for the voice and video tests later.

=== "OS X & Linux" bash export SSL_CERT_FILE=$(python3 -m certifi)

=== "Windows" powershell $env:SSL_CERT_FILE = (python3 -m certifi)

Then, run the dev UI:

adk web

!!!info "Note for Windows users"

When hitting the `_make_subprocess_transport NotImplementedError`, consider using `adk web --no-reload` instead.

!!! warning "Caution: ADK Web for development only"

ADK Web is ***not meant for use in production deployments***. You should
use ADK Web for development and debugging purposes only.

Open the URL provided (usually http://localhost:8000 or http://127.0.0.1:8000) directly in your browser. This connection stays entirely on your local machine. Select google_search_agent.

Try with voice and video

To try with voice, reload the web browser, click the microphone button to enable the voice input, and ask the the following questions in voice. The agent will use the google_search tool to get the latest information to answer those questions. You will hear the answer in voice in real-time.

  • What is the weather in New York?
  • What is the time in New York?
  • What is the weather in Paris?
  • What is the time in Paris?

To try with video, reload the web browser, click the camera button to enable the video input, and ask questions like "What do you see?". The agent will answer what they see in the video input.

Caveat

  • You can not use text chat with the native-audio models. You will see errors when entering text messages on adk web.

Stop the tool

Stop adk web by pressing Ctrl-C on the console.

Note on ADK Streaming

Model callbacks (before_model_callback and after_model_callback) are not invoked on the streaming path; ADK only runs them on the run_async path. Agent callbacks (before_agent_callback, after_agent_callback) and tool callbacks (before_tool_callback, after_tool_callback) do run while streaming, as do LongRunningFunctionTool and ExampleTool. Of the workflow agents, only SequentialAgent supports streaming: LoopAgent and ParallelAgent raise NotImplementedError.

Congratulations! You've successfully created and interacted with your first Streaming agent using ADK!

Next steps: build custom streaming app

Build a custom server walks through the server and client code for a custom asynchronous web app built with ADK, enabling real-time, bidirectional audio and text communication. From there, Sessions and the streaming loop covers the application lifecycle in depth, and Events covers everything run_live() hands back to you.