* Finalize draft for 0011-external-storage * Fix Python external storage examples * Add TypeScript external storage guidance * Address review findings on external storage references Python: - Import ClientConfig from temporalio.envconfig, not temporalio.client. load_client_connect_config() is a staticmethod on the envconfig class; the temporalio.client.ClientConfig TypedDict has no such member, so the snippet raised AttributeError. Follow main's env-config convention (setdefault target_host) from #261. - Register real Workflow/Activity placeholders. Worker() with empty workflows and activities raises "At least one activity, Nexus service, or workflow must be specified", and wrap the setup in async main(). Go: - Cover the GCS driver (contrib/gcp/gcsdriver + gcssdk), which the SDK ships and the docs install alongside S3. - Load client options with envconfig.MustLoadDefaultClientOptions() and note that Workers inherit External Storage from their Client. Align coverage across all three languages, each of which was missing something the others had: - 50 MiB MaxPayloadSize/max_payload_size ceiling and the matching anti-pattern (Go, Python). - Store/Retrieve are not retried within a Task attempt; the Task retries as a whole, so storage must be idempotent (Go, Python). - Multi-region durability with CRR + an MRAP ARN (Go, Python). - Distinct driver names when registering two drivers of the same kind (Go, Python). - Codec Server guidance (TypeScript), including that neither the TypeScript nor Python SDK ships a storage-aware handler. - Built-in driver behavior sections (concurrency, content-addressed keys, integrity checks, diagnostics) in Go and Python. - ctx.Context on the Go driver contexts, mirroring TypeScript's abortSignal guidance; optional type() override in Python. Also: standardize the TypeScript Public Preview admonition on the repo's wording, drop the transplanted `payloadSizeThreshold: 1` anti-pattern (TypeScript compares >=, so 1 behaves like 0), replace site-relative plugins-guide links with absolute URLs, refresh the index pointers, and revert an unrelated whitespace change in the Spring AI reference. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Harden external storage driver examples * Fix correctness bugs in external storage references Address code-review findings on the new external storage docs: - Go: add missing "context" and "log" imports to the S3 driver, GCS driver, and client/worker setup snippets, which presented complete import lists but failed to compile. - Go: add go.temporal.io/sdk/contrib/envconfig to both go get lines; it is a separate module and is imported by the setup snippet. - Go: give the local-disk worked example an import block, and introduce the commonpb alias at its first use in the selector example. - Go and Python: validate claim data in Retrieve/retrieve so a hand-crafted reference payload cannot read files outside the store directory, matching the hardening already applied to Store/store. - Python: the Worker inherits the Data Converter from its Client and takes no data_converter argument; the prose said to pass it to both. Verified by compiling every Go snippet against sdk-go and exercising both path guards. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Route large-payload triage to the external storage references The new external storage docs were only reachable from the language index files, so the paths an agent actually takes when a user hits a payload limit still sent it to hand-roll the claim-check pattern. - core/error-reference.md: TMPRL1103 recovery now points at built-in External Storage before manual reference passing. - core/gotchas.md: the payload-limit fix notes the SDK does this for you in Go, Python, and TypeScript. - core/patterns.md: Large Data Handling leads with the SDK-native option and scopes the manual pattern to the cases that need it. Also link the Go external storage sample from the Codec Server section, matching what the Python reference already does. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: skill-sync[bot] <skill-sync[bot]@users.noreply.github.com> Co-authored-by: Brian Strauch <brian@brianstrauch.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
8.1 KiB
Temporal Python SDK Reference
Overview
The Temporal Python SDK (temporalio) provides a fully async, type-safe approach to building durable workflows. Python 3.9+ required. Workflows run in a sandbox by default for determinism protection.
Quick Demo of Temporal
Add Dependency on Temporal: In the package management system of the Python project you are working on, add a dependency on temporalio.
activities/greet.py - Activity definitions (separate file for performance):
from temporalio import activity
@activity.defn
def greet(name: str) -> str:
return f"Hello, {name}!"
workflows/greeting.py - Workflow definition (import activities through sandbox):
from datetime import timedelta
from temporalio import workflow
with workflow.unsafe.imports_passed_through():
from activities.greet import greet
@workflow.defn
class GreetingWorkflow:
@workflow.run
async def run(self, name: str) -> str:
return await workflow.execute_activity(
greet, name, start_to_close_timeout=timedelta(seconds=30)
)
worker.py - Worker setup (registers activity and workflow, runs indefinitely and processes tasks):
import asyncio
import concurrent.futures
from temporalio.client import Client
from temporalio.envconfig import ClientConfig
from temporalio.worker import Worker
# Import the activity and workflow from our other files
from activities.greet import greet
from workflows.greeting import GreetingWorkflow
async def main():
connect_config = ClientConfig.load_client_connect_config()
connect_config.setdefault("target_host", "localhost:7233")
client = await Client.connect(**connect_config)
# Run the worker
with concurrent.futures.ThreadPoolExecutor(max_workers=100) as activity_executor:
worker = Worker(
client,
task_queue="my-task-queue",
workflows=[GreetingWorkflow],
activities=[greet],
activity_executor=activity_executor,
)
await worker.run()
if __name__ == "__main__":
asyncio.run(main())
Start the dev server: Start temporal server start-dev in the background.
Start the worker: Start python worker.py in the background (appropriately adjust command for your project, like uv run python worker.py)
starter.py - Start a workflow execution:
import asyncio
from temporalio.client import Client
from temporalio.envconfig import ClientConfig
import uuid
# Import the workflow from the previous code
from workflows.greeting import GreetingWorkflow
async def main():
connect_config = ClientConfig.load_client_connect_config()
connect_config.setdefault("target_host", "localhost:7233")
client = await Client.connect(**connect_config)
# Execute a workflow
result = await client.execute_workflow(GreetingWorkflow.run, "my name", id=str(uuid.uuid4()), task_queue="my-task-queue")
print(f"Result: {result}")
if __name__ == "__main__":
asyncio.run(main())
Run the workflow: Run python starter.py (or uv run, etc.). Should output: Result: Hello, my-name!.
Key Concepts
Workflow Definition
- Use
@workflow.defndecorator on class - Put any state initialization logic in the
__init__of your workflow class to guarantee that it happens before signals/updates arrive. If your state initialization logic requires the workflow parameters, then add the@workflow.initdecorator and parameters to your__init__. - Use
@workflow.runon the entry point method - Must be async (
async def) - Use
@workflow.signal,@workflow.query,@workflow.updatefor handlers
Activity Definition
- Use
@activity.defndecorator - Can be sync or async functions
- Default to sync activities - safer and easier to debug
- Sync activities need
activity_executor(ThreadPoolExecutor) - Async activities require async-safe libraries throughout (e.g.,
aiohttpnotrequests)
See sync-vs-async.md for detailed guidance on choosing between sync and async.
Worker Setup
- Load connection settings with
ClientConfig.load_client_connect_config(), connect the client, and create a Worker with workflows and activities - Run the worker
- Activities can specify custom executor
Determinism
Workflow code must be deterministic!. All sources of non-determinism should either use Temporal-provided actions or (primarily) be defined in Activities. Read references/core/determinism.md and references/python/determinism.md to understand more.
File Organization Best Practice
Keep Workflow definitions in separate files from Activity definitions. The Python SDK sandbox reloads Workflow definition files on every execution for determinism protection. Minimizing file contents improves Worker performance.
my_temporal_app/
├── workflows/
│ └── greeting.py # Only Workflow classes
├── activities/
│ └── translate.py # Only Activity functions/classes
├── worker.py # Worker setup, imports both
└── starter.py # Client code to start workflows
In the Workflow file, import Activities through the sandbox:
# workflows/greeting.py
from temporalio import workflow
with workflow.unsafe.imports_passed_through():
from activities.translate import TranslateActivities
Common Pitfalls
- Non-deterministic code in workflows - Use activities for all non-deterministic and/or fallible code
- Blocking in async activities - Use sync activities or async-safe libraries only
- Missing executor for sync activities - Add
activity_executor=ThreadPoolExecutor() - Forgetting to heartbeat - Long activities need
activity.heartbeat() - Using gevent - Incompatible with SDK
- Using
print()in workflows - Useworkflow.loggerinstead for replay-safe logging - Mixing Workflows and Activities in same file - Causes unnecessary reloads, hurts performance, bad structure
- Forgetting to wait on activity calls -
workflow.execute_activity()is async; you must eventually await it (directly or viaasyncio.gather()for parallel execution)
Writing Tests
See references/python/testing.md for info on writing tests.
Additional Resources
Reference Files
references/python/patterns.md- Signals, queries, child workflows, saga pattern, etc.references/python/determinism.md- Sandbox behavior, safe alternatives, pass-through pattern, history replayreferences/python/gotchas.md- Python-specific mistakes and anti-patternsreferences/python/error-handling.md- ApplicationError, retry policies, non-retryable errors, idempotencyreferences/python/observability.md- Logging, metrics, tracing, Search Attributesreferences/python/testing.md- WorkflowEnvironment, time-skipping, activity mockingreferences/python/sync-vs-async.md- Sync vs async activities, event loop blocking, executor configurationreferences/python/advanced-features.md- Schedules, worker tuning, and morereferences/python/data-handling.md- Data converters, Pydantic, payload encryptionreferences/python/external-storage.md- Claim-check pattern for large payloads (S3 driver, custom drivers, codec-server handling, multi-region durability)references/python/versioning.md- Patching API, workflow type versioning, Worker Versioningreferences/python/standalone-activities.md- Standalone Activities: run an Activity directly from a Client without a Workflow (Public Preview). Concept overview atreferences/core/standalone-activities.md.references/python/determinism-protection.md- Python sandbox specifics, forbidden operations, pass-through importsreferences/python/ai-patterns.md- LLM integration, Pydantic data converter, AI workflow patternsreferences/python/workflow-streams.md- Public-Previewtemporalio.contrib.workflow_streamslibrary: durable, offset-addressed event channel for streaming progress to subscribers.
Python Integrations
For Python-specific third-party integrations (OpenAI Agents SDK, Google ADK, etc.), see references/integrations.md and filter for Python. Reference files live under references/python/integrations/.