mirror of
https://github.com/temporalio/skill-temporal-developer.git
synced 2026-09-14 13:52:58 +08:00
5de78ea2a3
* Finalize draft for 0011-external-storage * Fix Python external storage examples * Add TypeScript external storage guidance * Address review findings on external storage references Python: - Import ClientConfig from temporalio.envconfig, not temporalio.client. load_client_connect_config() is a staticmethod on the envconfig class; the temporalio.client.ClientConfig TypedDict has no such member, so the snippet raised AttributeError. Follow main's env-config convention (setdefault target_host) from #261. - Register real Workflow/Activity placeholders. Worker() with empty workflows and activities raises "At least one activity, Nexus service, or workflow must be specified", and wrap the setup in async main(). Go: - Cover the GCS driver (contrib/gcp/gcsdriver + gcssdk), which the SDK ships and the docs install alongside S3. - Load client options with envconfig.MustLoadDefaultClientOptions() and note that Workers inherit External Storage from their Client. Align coverage across all three languages, each of which was missing something the others had: - 50 MiB MaxPayloadSize/max_payload_size ceiling and the matching anti-pattern (Go, Python). - Store/Retrieve are not retried within a Task attempt; the Task retries as a whole, so storage must be idempotent (Go, Python). - Multi-region durability with CRR + an MRAP ARN (Go, Python). - Distinct driver names when registering two drivers of the same kind (Go, Python). - Codec Server guidance (TypeScript), including that neither the TypeScript nor Python SDK ships a storage-aware handler. - Built-in driver behavior sections (concurrency, content-addressed keys, integrity checks, diagnostics) in Go and Python. - ctx.Context on the Go driver contexts, mirroring TypeScript's abortSignal guidance; optional type() override in Python. Also: standardize the TypeScript Public Preview admonition on the repo's wording, drop the transplanted `payloadSizeThreshold: 1` anti-pattern (TypeScript compares >=, so 1 behaves like 0), replace site-relative plugins-guide links with absolute URLs, refresh the index pointers, and revert an unrelated whitespace change in the Spring AI reference. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Harden external storage driver examples * Fix correctness bugs in external storage references Address code-review findings on the new external storage docs: - Go: add missing "context" and "log" imports to the S3 driver, GCS driver, and client/worker setup snippets, which presented complete import lists but failed to compile. - Go: add go.temporal.io/sdk/contrib/envconfig to both go get lines; it is a separate module and is imported by the setup snippet. - Go: give the local-disk worked example an import block, and introduce the commonpb alias at its first use in the selector example. - Go and Python: validate claim data in Retrieve/retrieve so a hand-crafted reference payload cannot read files outside the store directory, matching the hardening already applied to Store/store. - Python: the Worker inherits the Data Converter from its Client and takes no data_converter argument; the prose said to pass it to both. Verified by compiling every Go snippet against sdk-go and exercising both path guards. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Route large-payload triage to the external storage references The new external storage docs were only reachable from the language index files, so the paths an agent actually takes when a user hits a payload limit still sent it to hand-roll the claim-check pattern. - core/error-reference.md: TMPRL1103 recovery now points at built-in External Storage before manual reference passing. - core/gotchas.md: the payload-limit fix notes the SDK does this for you in Go, Python, and TypeScript. - core/patterns.md: Large Data Handling leads with the SDK-native option and scopes the manual pattern to the cases that need it. Also link the Go external storage sample from the Codec Server section, matching what the Python reference already does. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: skill-sync[bot] <skill-sync[bot]@users.noreply.github.com> Co-authored-by: Brian Strauch <brian@brianstrauch.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
195 lines
8.1 KiB
Markdown
195 lines
8.1 KiB
Markdown
# Temporal Python SDK Reference
|
|
|
|
## Overview
|
|
|
|
The Temporal Python SDK (`temporalio`) provides a fully async, type-safe approach to building durable workflows. Python 3.9+ required. Workflows run in a sandbox by default for determinism protection.
|
|
|
|
## Quick Demo of Temporal
|
|
|
|
**Add Dependency on Temporal:** In the package management system of the Python project you are working on, add a dependency on `temporalio`.
|
|
|
|
**activities/greet.py** - Activity definitions (separate file for performance):
|
|
|
|
```python
|
|
from temporalio import activity
|
|
|
|
@activity.defn
|
|
def greet(name: str) -> str:
|
|
return f"Hello, {name}!"
|
|
```
|
|
|
|
**workflows/greeting.py** - Workflow definition (import activities through sandbox):
|
|
|
|
```python
|
|
from datetime import timedelta
|
|
from temporalio import workflow
|
|
|
|
with workflow.unsafe.imports_passed_through():
|
|
from activities.greet import greet
|
|
|
|
@workflow.defn
|
|
class GreetingWorkflow:
|
|
@workflow.run
|
|
async def run(self, name: str) -> str:
|
|
return await workflow.execute_activity(
|
|
greet, name, start_to_close_timeout=timedelta(seconds=30)
|
|
)
|
|
```
|
|
|
|
**worker.py** - Worker setup (registers activity and workflow, runs indefinitely and processes tasks):
|
|
|
|
```python
|
|
import asyncio
|
|
import concurrent.futures
|
|
from temporalio.client import Client
|
|
from temporalio.envconfig import ClientConfig
|
|
from temporalio.worker import Worker
|
|
|
|
# Import the activity and workflow from our other files
|
|
from activities.greet import greet
|
|
from workflows.greeting import GreetingWorkflow
|
|
|
|
async def main():
|
|
connect_config = ClientConfig.load_client_connect_config()
|
|
connect_config.setdefault("target_host", "localhost:7233")
|
|
client = await Client.connect(**connect_config)
|
|
|
|
# Run the worker
|
|
with concurrent.futures.ThreadPoolExecutor(max_workers=100) as activity_executor:
|
|
worker = Worker(
|
|
client,
|
|
task_queue="my-task-queue",
|
|
workflows=[GreetingWorkflow],
|
|
activities=[greet],
|
|
activity_executor=activity_executor,
|
|
)
|
|
await worker.run()
|
|
|
|
if __name__ == "__main__":
|
|
asyncio.run(main())
|
|
```
|
|
|
|
**Start the dev server:** Start `temporal server start-dev` in the background.
|
|
|
|
**Start the worker:** Start `python worker.py` in the background (appropriately adjust command for your project, like `uv run python worker.py`)
|
|
|
|
**starter.py** - Start a workflow execution:
|
|
|
|
```python
|
|
import asyncio
|
|
from temporalio.client import Client
|
|
from temporalio.envconfig import ClientConfig
|
|
import uuid
|
|
|
|
# Import the workflow from the previous code
|
|
from workflows.greeting import GreetingWorkflow
|
|
|
|
async def main():
|
|
connect_config = ClientConfig.load_client_connect_config()
|
|
connect_config.setdefault("target_host", "localhost:7233")
|
|
client = await Client.connect(**connect_config)
|
|
|
|
# Execute a workflow
|
|
result = await client.execute_workflow(GreetingWorkflow.run, "my name", id=str(uuid.uuid4()), task_queue="my-task-queue")
|
|
|
|
print(f"Result: {result}")
|
|
|
|
if __name__ == "__main__":
|
|
asyncio.run(main())
|
|
```
|
|
|
|
**Run the workflow:** Run `python starter.py` (or uv run, etc.). Should output: `Result: Hello, my-name!`.
|
|
|
|
## Key Concepts
|
|
|
|
### Workflow Definition
|
|
|
|
- Use `@workflow.defn` decorator on class
|
|
- Put any state initialization logic in the `__init__` of your workflow class to guarantee that it happens before signals/updates arrive. If your state initialization logic requires the workflow parameters, then add the `@workflow.init` decorator and parameters to your `__init__`.
|
|
- Use `@workflow.run` on the entry point method
|
|
- Must be async (`async def`)
|
|
- Use `@workflow.signal`, `@workflow.query`, `@workflow.update` for handlers
|
|
|
|
### Activity Definition
|
|
|
|
- Use `@activity.defn` decorator
|
|
- Can be sync or async functions
|
|
- **Default to sync activities** - safer and easier to debug
|
|
- Sync activities need `activity_executor` (ThreadPoolExecutor)
|
|
- Async activities require async-safe libraries throughout (e.g., `aiohttp` not `requests`)
|
|
|
|
See `sync-vs-async.md` for detailed guidance on choosing between sync and async.
|
|
|
|
### Worker Setup
|
|
|
|
- Load connection settings with `ClientConfig.load_client_connect_config()`, connect the client, and create a Worker with workflows and activities
|
|
- Run the worker
|
|
- Activities can specify custom executor
|
|
|
|
### Determinism
|
|
|
|
**Workflow code must be deterministic!**. All sources of non-determinism should either use Temporal-provided actions or (primarily) be defined in Activities. Read `references/core/determinism.md` and `references/python/determinism.md` to understand more.
|
|
|
|
## File Organization Best Practice
|
|
|
|
**Keep Workflow definitions in separate files from Activity definitions.** The Python SDK sandbox reloads Workflow definition files on every execution for determinism protection. Minimizing file contents improves Worker performance.
|
|
|
|
```
|
|
my_temporal_app/
|
|
├── workflows/
|
|
│ └── greeting.py # Only Workflow classes
|
|
├── activities/
|
|
│ └── translate.py # Only Activity functions/classes
|
|
├── worker.py # Worker setup, imports both
|
|
└── starter.py # Client code to start workflows
|
|
```
|
|
|
|
**In the Workflow file, import Activities through the sandbox:**
|
|
|
|
```python
|
|
# workflows/greeting.py
|
|
from temporalio import workflow
|
|
|
|
with workflow.unsafe.imports_passed_through():
|
|
from activities.translate import TranslateActivities
|
|
```
|
|
|
|
## Common Pitfalls
|
|
|
|
1. **Non-deterministic code in workflows** - Use activities for all non-deterministic and/or fallible code
|
|
2. **Blocking in async activities** - Use sync activities or async-safe libraries only
|
|
3. **Missing executor for sync activities** - Add `activity_executor=ThreadPoolExecutor()`
|
|
4. **Forgetting to heartbeat** - Long activities need `activity.heartbeat()`
|
|
5. **Using gevent** - Incompatible with SDK
|
|
6. **Using `print()` in workflows** - Use `workflow.logger` instead for replay-safe logging
|
|
7. **Mixing Workflows and Activities in same file** - Causes unnecessary reloads, hurts performance, bad structure
|
|
8. **Forgetting to wait on activity calls** - `workflow.execute_activity()` is async; you must eventually await it (directly or via `asyncio.gather()` for parallel execution)
|
|
|
|
## Writing Tests
|
|
|
|
See `references/python/testing.md` for info on writing tests.
|
|
|
|
## Additional Resources
|
|
|
|
### Reference Files
|
|
|
|
- **`references/python/patterns.md`** - Signals, queries, child workflows, saga pattern, etc.
|
|
- **`references/python/determinism.md`** - Sandbox behavior, safe alternatives, pass-through pattern, history replay
|
|
- **`references/python/gotchas.md`** - Python-specific mistakes and anti-patterns
|
|
- **`references/python/error-handling.md`** - ApplicationError, retry policies, non-retryable errors, idempotency
|
|
- **`references/python/observability.md`** - Logging, metrics, tracing, Search Attributes
|
|
- **`references/python/testing.md`** - WorkflowEnvironment, time-skipping, activity mocking
|
|
- **`references/python/sync-vs-async.md`** - Sync vs async activities, event loop blocking, executor configuration
|
|
- **`references/python/advanced-features.md`** - Schedules, worker tuning, and more
|
|
- **`references/python/data-handling.md`** - Data converters, Pydantic, payload encryption
|
|
- **`references/python/external-storage.md`** - Claim-check pattern for large payloads (S3 driver, custom drivers, codec-server handling, multi-region durability)
|
|
- **`references/python/versioning.md`** - Patching API, workflow type versioning, Worker Versioning
|
|
- **`references/python/standalone-activities.md`** - Standalone Activities: run an Activity directly from a Client without a Workflow (Public Preview). Concept overview at `references/core/standalone-activities.md`.
|
|
- **`references/python/determinism-protection.md`** - Python sandbox specifics, forbidden operations, pass-through imports
|
|
- **`references/python/ai-patterns.md`** - LLM integration, Pydantic data converter, AI workflow patterns
|
|
- **`references/python/workflow-streams.md`** - Public-Preview `temporalio.contrib.workflow_streams` library: durable, offset-addressed event channel for streaming progress to subscribers.
|
|
|
|
### Python Integrations
|
|
|
|
For Python-specific third-party integrations (OpenAI Agents SDK, Google ADK, etc.), see `references/integrations.md` and filter for Python. Reference files live under `references/python/integrations/`.
|