design-patterns

826 notes in this subject, read out of this brain and free to use. This is page 6 of 14.

When to use infrequent polling

Infrequent polling (1 minute or slower) is a good fit for batch job completion checks, long-running external processes, rate-limited APIs, and operations that may take hours or days. It is not suitable for sub-minute polling requirements or operations requiring immediate response.

When to use frequent polling

Frequent polling (1 second or faster) is a good fit for real-time status checks, high-priority operations requiring fast response, and short-lived external operations lasting minutes. It is not suitable for long-running operations lasting hours or days, rate-limited APIs, or resource-constrained external services.

Periodic sequence with Child Workflows and Continue-As-New

For complex polling sequences with changing parameters or multiple Activities between attempts, use Child Workflows with Continue-As-New to prevent unbounded history. The Child Workflow loops up to a maximum number of attempts (e.g., 10), executing an Activity and sleeping between attempts. After reaching the maximum attempts, it calls Continue-As-New with the same parameters to start a fresh execution. The parent Workflow starts the Child Workflow with a specific workflow ID and waits for its result. The parent remains unaware of the child's Continue-As-New calls.

Three polling strategies for different frequencies

The Polling External Services pattern implements three distinct strategies optimized for different polling frequencies: frequent polling (1 second or faster) using Activity loops with heartbeats, infrequent polling (1 minute or slower) using Activity retries with fixed backoff, and periodic sequence (complex polling) using Child Workflows with Continue-As-New.

Infrequent polling implementation with Activity retries

For polling intervals of 1 minute or slower, implement a single-poll Activity that throws an exception when the external service is not ready. Temporal handles the retry scheduling based on the retry policy. Configure the retry policy with backoff_coefficient=1 (fixed interval) and initial_interval set to the desired polling frequency (e.g., 60 seconds). The start_to_close_timeout should be short (e.g., 2 seconds). Retries do not add events to the Workflow history, keeping it small.

Frequent polling implementation with heartbeats

For polling intervals of 1 second or faster, implement a loop inside an Activity that calls activity.heartbeat() on each iteration, checks the external service status, and sleeps for 1 second between iterations. The heartbeat reports progress and enables Temporal to detect stuck Activities. The Workflow configures the Activity with a heartbeat timeout shorter than the start-to-close timeout (e.g., heartbeat_timeout=2s, start_to_close_timeout=60s). If the Activity misses a heartbeat, Temporal detects the failure and retries the Activity on another Worker.

Fairness pattern

Fairness distributes Worker capacity evenly across tenants or users so a burst from one caller does not starve the others. Use this pattern when multiple tenants share the same Workers.

Priority Task Queues pattern

Priority Task Queues assigns a priority level to Workflows and Activities so time-sensitive work runs ahead of lower-priority work on the same Task Queue. Use this pattern when urgent work must not wait behind bulk work.

Pattern selection for downstream rate limits

When a downstream dependency has a fixed rate limit, use Downstream Rate Limiting to cap throughput at the Worker.

Downstream Rate Limiting pattern

Downstream Rate Limiting caps the Activity execution rate against a downstream service by routing throttled Activities to a dedicated Task Queue whose Workers enforce a throughput limit. Use this pattern when a downstream dependency has a fixed rate limit.

QoS and Throughput Patterns overview

Temporal provides patterns to control how fast work executes, protect downstream services from overload, and ensure fair capacity distribution across tenants or callers.

Pattern selection for tenant isolation

When multiple tenants share the same Workers, use Fairness to keep one tenant's burst from starving others.

Pattern selection for priority work

When urgent work must not wait behind bulk work, use Priority Task Queues.

When Priority Task Queues pattern is a good fit

This pattern is a good fit when your system mixes time-sensitive operations (payment processing, user-facing requests) with background or batch work (reporting, data imports, inventory management), and you want urgent tasks to proceed even during periods of high load. It also works well when you need to mark urgent tasks that should override normal processing, such as triggering immediate re-runs of failed critical tasks.

Set priority via CLI

```sh temporal workflow start \ --type ChargeCustomer \ --task-queue my-task-queue \ --workflow-id charge-customer-wf \ --input '{"customerId":"12345"}' \ --priority-key 1 ``` This example shows how to set priority when starting a Workflow using the Temporal CLI.

Priority Task Queues pitfall: assuming hard isolation between priority levels

Priority controls dispatch order, not Worker capacity allocation. A priority-5 task may still consume a Worker slot that is then unavailable for a priority-1 task arriving a moment later.

Priority enabled by default

Priority is enabled by default in Temporal Cloud and self-hosted Temporal.

Priority Task Queues vs Fairness pattern

Both Priority Task Queues and Fairness patterns provide soft isolation and support dynamic priority. Priority Task Queues have low complexity with 1–5 levels, while Fairness has low complexity with unlimited keys. Use Priority for strict ordering by urgency; use Fairness to distribute capacity proportionally across tenants.

Priority Task Queues pitfall: neglecting low-priority starvation

Under sustained high load, priority-5 tasks may wait indefinitely. Use ScheduleToStartTimeout on low-priority activities to surface starvation as a visible failure.

Priority Task Queues pitfall: assigning priority 1 to all work by default

When every caller sets the highest priority, the feature provides no ordering benefit. Establish an explicit policy for which work types qualify for each level.

Priority Task Queues best practice: monitor queue depth per priority level

Sustained backlog growth at a priority level signals that Worker capacity is insufficient for the submitted load at that level.

Priority Task Queues best practice: set PriorityKey at Workflow start

Set PriorityKey at Workflow start in the start options, not inside Workflow code. Workflow code cannot change its own priority after it starts.

Priority Task Queues best practice: reserve priority 1 for urgent work

Reserve priority 1 for genuinely urgent work. If high priority is the fallback when no priority is specified, the highest level fills with routine work and the feature provides no benefit. The default is 3 when no key is set.

Priority Task Queues best practice: use no more than five levels

Keep priority levels coarse—for example, 1 = urgent, 3 = normal, 5 = batch—rather than mapping fine-grained business importance to many values. The PriorityKey range is 1–5.

Priority Task Queues trade-offs

Lower-priority tasks are blocked until all higher-priority tasks have started. In an environment with a continuously replenished high-priority backlog, low-priority tasks may be significantly delayed. The built-in PriorityKey range is 1–5; if more than five distinct levels are needed, the feature cannot accommodate them.

Priority Task Queues benefits

Native priority requires no extra queues, routing logic, or additional Worker pools. A single pool of Workers serves all priority levels, so idle capacity at low-priority levels is automatically used by higher-priority work without any additional configuration.

When Priority Task Queues pattern is not a good fit

This pattern is not a good fit when all work is effectively equal in urgency, when a continuously replenished high-priority backlog could starve lower-priority work indefinitely, or when you need hard capacity isolation between tiers. If your concern is prioritizing work amongst tenants or customers, consider the Fairness pattern instead, which distributes capacity proportionally using weighted fairness keys rather than strict ordering.

Child Workflows and Activities inherit parent priority

Activities and Child Workflows inherit the parent Workflow's priority unless they explicitly set their own priority.

Priority sub-queue dispatch order

The Temporal matching service maintains a sub-queue for each priority level and exhausts all tasks at a given level before dispatching to the next. Workers receive all priority-1 tasks before any priority-2 task, and so on.

Priority Task Queues best practice: override Activity priority deliberately

Activities inherit the parent Workflow's priority by default. Override only when a specific Activity must run at a different level than its Workflow.

PriorityKey range and default value

PriorityKey is an integer from 1 to 5, where 1 is the highest priority and 5 is the lowest. Tasks default to priority 3 when no key is set.

Priority Task Queues pattern overview

The Priority Task Queues pattern assigns a PriorityKey to Workflows, Activities, and Child Workflows so that time-sensitive work executes ahead of lower-priority work within a single Task Queue, without requiring separate queues or routing logic.

Priority Task Queues pitfall: changing priority after scheduling

PriorityKey is evaluated when a task enters the queue and cannot be changed while it waits. To re-prioritize an already-queued task, cancel it and reschedule with the new priority.

Update is recorded in workflow history before response returned

The Update is recorded in Workflow history before the response is returned to the client, providing strong consistency.

Update validator runs before Update handler

In all SDKs, the validator runs before the Update handler. If the validator throws an exception, the Update is rejected and the client receives a typed error. If the validator passes, the Update handler modifies state and returns a typed result.

Updates vs Signals vs Queries comparison

| Approach | Use case | Response type | Latency | Consistency | | --- | --- | --- | --- | --- | | Update | Request-response | Sync typed value | Higher | Strong | | Signal | Fire-and-forget | None | Lower | Eventual | | Query | Read-only | Sync typed value | Lowest | Eventual |

Updates enable synchronous request-response with typed responses

Workflow Updates enable synchronous request-response interactions where clients receive immediate, typed responses while the Workflow continues processing. Updates modify Workflow state, validate inputs, and return results directly to the caller with strong consistency guarantees.

Update ID deduplication scope

Update ID deduplication by the Server is scoped to a single Workflow Run within a single Workflow Execution. You only need to track processed Update IDs in Workflow state when carrying them across a Continue-As-New boundary, because the deduplication does not persist across Continue-As-New.

Update common pitfalls

Common pitfalls with Updates: Performing long operations in the Update handler blocks Workflow Task execution; offload long-running work to Activities and use Workflow.await in the handler to wait for results. Exceeding the 2,000 total Updates limit adds events to history; use Continue-As-New before reaching the limit, and the server sets SuggestContinueAsNew at 90% of the limit. Not setting Update timeouts means the caller blocks indefinitely if the Worker is unavailable; always set a context timeout or deadline. Assuming you must set an Update ID for retry safety is incorrect; the SDK auto-generates a unique updateId when you do not provide one, and retried client calls are deduplicated automatically, so a transient retry will not run the handler twice. Set a stable, business-meaningful updateId when you want a retried Update-with-Start to attach to an existing in-flight Update instead of starting duplicate work. Using Updates for fire-and-forget operations is incorrect; Updates require a Worker to be online and responsive, so use Signals instead for fire-and-forget operations.

Update best practices

Best practices for Updates: Validate early by checking inputs at the start of the Update handler to fail fast. Handle errors by throwing typed exceptions for validation failures. Return quickly and do not perform long operations in the Update handler. Track Update IDs only across Continue-As-New; within a single Workflow Execution, the Server deduplicates retried Updates automatically by Update ID, so a retry does not run the handler twice. Only track processed Update IDs in Workflow state when carrying them across a Continue-As-New boundary, because Update ID deduplication is scoped to a single Workflow Run. Set timeouts by configuring appropriate Update timeouts. Maintain state consistency by ensuring state modifications are atomic within the handler.

Updates performance trade-offs

Updates are slower than Signals because they require a history write. The Update handler blocks Workflow Task execution and consumes Workflow Task execution time. For fire-and-forget messages that need no response, Updates require more machinery than Signals. Update arguments and return values are limited by the Workflow history event size, typically 2 MB per event. Each Update adds events to Workflow history, contributing to the 50K event limit. There is a maximum of 10 in-flight Updates per Workflow execution and a maximum of 2,000 total Updates in Workflow history.

When to use Updates pattern

The Update pattern is a good fit for request-response patterns requiring immediate confirmation, input validation before accepting work, synchronous state modifications with typed responses, operations requiring strong consistency guarantees, and entity Workflows that need external state Updates. It is not a good fit for fire-and-forget operations (use Signals), read-only operations (use Queries), high-throughput scenarios where latency matters (Updates are slower than Signals), or operations that do not need an immediate response.

Resumable Activity pitfall: not clearing correction state

After applying the correction, set corrected_account = None (or equivalent) before re-entering the loop. Otherwise, if the corrected activity also fails, the Workflow immediately re-uses the previous correction instead of waiting for a new one.

Resumable Activity pitfall: accepting corrections in wrong state

If a Signal arrives while the Activity is running (not parked), the correction should be queued and applied after the current attempt completes. The Signal handler always runs — the condition check (wait_condition) determines when the Workflow acts on it.

Resumable Activity pitfall: conflating with general retry loop

This pattern is for correcting input data. For retrying the same call against a temporarily unavailable system, use Fast/Slow Retries pattern instead. Do not confuse the correction loop with a general retry loop.

Resumable Activity best practice: proactive notifications

Notify the operator proactively. The AWAITING_CORRECTION transition is a good point to send an alert — an email, a Slack message, or a ticket — rather than waiting for the operator to notice in the Temporal UI.

Resumable Activity best practice: validate corrections

Validate the correction in the Signal handler. Check that the corrected account is non-empty and matches the expected format before setting the state. An invalid correction parks the Workflow again, but a clear error message helps operators.

Resumable Activity best practice: expose status via query

Expose status via a Query method. The getStatus Query gives operations tooling visibility into where the Workflow is parked without requiring access to the Workflow history.

Resumable Activity best practice: history growth management

Manage history growth in long-running correction loops. Each correction cycle — park, receive signal, re-execute Activity — adds events to the Workflow history (signal received, state transitions, Activity scheduled/completed). For workflows that may receive many corrections over time, use Continue-As-New to carry the current state into a fresh execution before the history grows too large, rather than relying solely on an arbitrary correction counter.

Resumable Activity: sending correction signals via CLI

Operators send correction and approval Signals using the Temporal CLI: ```bash # Correct the account number temporal workflow signal \ --workflow-id transfer-wf-001 \ --name retryWithCorrection \ --input '"account-123"' # Approve the transfer temporal workflow signal \ --workflow-id transfer-wf-001 \ --name approve \ --input 'true' ``` The Workflow wakes immediately when the Signal is delivered.

Resumable Activity best practice: bounded retries

Use a bounded MaximumAttempts before parking. Allow a few automatic retries to recover from transient failures. Parking immediately on the first failure forces operators to intervene for problems that would have resolved on their own.

Resumable Activity: Activity implementation with non-retryable errors

The Activity must distinguish between permanent failures (such as invalid account number) and transient failures. Throw a non-retryable ApplicationFailure for permanent input errors so the Workflow catches the ActivityError immediately and transitions to AWAITING_CORRECTION instead of exhausting all retry attempts first. Let all other exceptions propagate so the RetryPolicy handles transient failures.

Resumable Activity pattern overview

The Resumable Activity pattern (also called Pause On Failure) parks a Workflow in a durable waiting state after Activity retries are exhausted, waits for a corrective Signal from a human operator, then re-executes the Activity with corrected input. Use this pattern when failures are caused by bad input data that can be corrected externally — such as a wrong account number, invalid reference, or missing record — and abandoning the Workflow is worse than pausing it.

Resumable Activity state transitions

The Workflow transitions through these states: PENDING (initial) → TRANSFERRING (workflow starts) → AWAITING_CORRECTION (activity retries exhausted) → TRANSFERRING (retryWithCorrection signal received) → AWAITING_APPROVAL (activity succeeds) → COMPLETED (approve(true) signal) or REJECTED (approve(false) signal). From AWAITING_CORRECTION, the workflow can also transition to FAILED if correction attempts exceed a maximum (e.g., 5 attempts).

Resumable Activity solution steps

Use a bounded RetryPolicy to allow a few automatic retries (in case the failure is transient), then catch the exhausted ActivityError in the Workflow. Transition to an AWAITING_CORRECTION state and block on workflow.wait_condition or equivalent. Register a Signal handler that accepts the corrected input and unblocks the condition. When the Signal arrives, re-execute the Activity with the corrected input. Optionally use a second Signal to gate final approval before completing.

Resumable Activity vs polling or immediate failure

When an Activity fails due to bad input, retrying with the same input will never succeed. Standard options are: (1) Fail the Workflow immediately and require the client to restart and re-enter data, (2) Mark the error as non-retryable with the same result, or (3) Poll for the correction from inside the Workflow which wastes resources. The Resumable Activity pattern solves this by having the Workflow pause — consuming zero resources — until an authorized operator provides corrected data, then resume exactly where it left off.

Resumable Activity vs Approval pattern

Distinguish Resumable Activity from the Approval pattern. The Approval pattern gates forward progress on a human decision. Resumable Activity recovers from failure with a human-supplied data correction. Both use Signals and wait_condition, but serve different roles in a process.

Resumable Activity best practice: logging and state transitions

Log and record state at every transition. The AWAITING_CORRECTION and AWAITING_APPROVAL states can last hours or days. Structured log lines at each transition make the audit trail clear. For operational visibility, also update a Search Attribute at each transition (for example, a Keyword attribute storing the current status) so operators can filter and query workflows by state directly from the Temporal UI or CLI.

Resumable Activity pitfall: waiting without timeout

If operators never send the correction Signal, the Workflow waits indefinitely. Add a durable timer if the process must resolve within a time bound.

Signal with Start solves distributed system coordination

Without Signal with Start, clients must check if the Workflow exists before Signaling, start the Workflow if it does not exist and then Signal it, and handle race conditions when multiple clients try to start the same Workflow. Signal with Start eliminates this complexity by handling the check and start atomically on the platform.

Give your agent this brain