design-patterns

826 notes in this subject, read out of this brain and free to use. This is page 7 of 14.

Signal with Start solves distributed system coordination

Without Signal with Start, clients must check if the Workflow exists before Signaling, start the Workflow if it does not exist and then Signal it, and handle race conditions when multiple clients try to start the same Workflow. Signal with Start eliminates this complexity by handling the check and start atomically on the platform.

Signal idempotency in entity Workflows

Entity Workflows using Signal with Start must track processed operation IDs to prevent duplicate processing. Maintain a set of processed item IDs or operation IDs, and at the start of each signal handler, check if the operation was already processed and return early if it was. This handles client retries where the same signal may be delivered multiple times.

Entity Workflows run indefinitely

Entity Workflows created with Signal with Start block indefinitely using constructs like workflow.wait_condition(lambda: False) in Python, workflow.Await(ctx, func() bool { return false }) in Go, Workflow.await(() => false) in Java, and condition(() => false) in TypeScript. These represent long-lived entities that receive operations throughout their lifetime.

Signal with Start use cases

Signal with Start is a good fit for entity Workflows (accounts, shopping carts, user sessions, clusters), event-driven architectures (Kafka consumers, message queue processors), Workflows that receive multiple operations over their lifetime, lazy entity creation where you only create when the first operation arrives, and fire-and-forget operations where immediate response is not needed.

Signal with Start not suitable for one-time operations or request-response

Signal with Start is not a good fit for one-time operations (use REJECT_DUPLICATE policy instead), request-response patterns requiring synchronous confirmation (use Update with Start), or operations that need immediate return values. For these cases, other patterns are more appropriate.

Signal with Start benefits

Signal with Start provides an atomic operation where start and Signal happen with no race conditions. Workflows only exist when needed (lazy creation). The client does not need to check if the Workflow exists. The operation is safe to retry because duplicate starts are handled by the Workflow ID. The pattern is a natural fit for long-lived business entities.

Signal with Start trade-offs

Signal with Start trade-offs are: Signals are fire-and-forget with no immediate confirmation that the Signal was processed. You still need to track processed operation IDs in the Workflow for Signal idempotency. Workflows must handle unbounded execution using Continue-As-New. Signals do not return values — use Queries or Updates for that instead.

Workflow ID duplicate policies with Signal with Start

ALLOW_DUPLICATE (default): Allows a new Workflow Execution with the same ID after the previous one has closed (completed, failed, timed out, terminated, or cancelled). Does not affect a currently running Workflow — Signal with Start delivers the Signal to the running execution. ALLOW_DUPLICATE_FAILED_ONLY: Allows restart only if the previous run failed, preventing accidental restarts of running Workflows. Recommended for entity Workflows. REJECT_DUPLICATE: Prevents any duplicate starts, useful for one-time operations, not entity Workflows. TERMINATE_IF_RUNNING: Terminates the running Workflow and starts a new one — use with caution.

Signal with Start best practices

Best practices for Signal with Start: Derive Workflow ID from stable business identifiers (account ID, user ID). Implement Signal idempotency by tracking processed operation IDs. Use WorkflowInit to initialize state before Signals are delivered. Handle unbounded execution using Continue-As-New for long-running entity Workflows. Choose ALLOW_DUPLICATE_FAILED_ONLY for entity Workflows. Include a unique operation or reference ID in every Signal. Check for duplicates at the start of Signal handlers and return early.

Signal with Start pitfall: not implementing idempotency

A common pitfall is not implementing Signal idempotency. Signals can be delivered more than once (for example, client retries). Without tracking processed operation IDs, the Workflow processes duplicates. Always maintain a set of processed operation IDs and check for duplicates at the start of signal handlers.

Signal with Start pitfall: unbounded history growth

Entity Workflows that receive many Signals without calling Continue-As-New will hit the 50K event or 10K Signal limit. This causes unbounded history growth. Use isContinueAsNewSuggested() to trigger Continue-As-New when approaching limits.

Signal with Start pitfall: losing pending Signals on Continue-As-New

When calling Continue-As-New, drain all pending Signals before continuing, and pass any unprocessed signals as input to the new execution. Failing to do so loses pending Signals and the work they represent.

Signal with Start pitfall: race between SignalWithStart and Continue-As-New

A potential race can occur between SignalWithStart and Continue-As-New. However, Temporal prevents this race — if a Signal arrives while the Workflow is completing via Continue-As-New, the Workflow rewinds to process the Signal first.

Signal with Start compared to Update with Start and REJECT_DUPLICATE

Signal with Start is for entity Workflows with fire-and-forget responses and signal-level idempotency. Update with Start is for request-response patterns with sync return values and update-level idempotency. REJECT_DUPLICATE is for one-time operations with async workflow-level idempotency. Choose based on whether you need lazy entity creation, synchronous responses, or one-time operation guarantees.

Key SDK differences in Saga implementation

Go uses a slice of closures and iterates from the end on error (some samples use defer instead, both achieve LIFO with explicit rollback trigger). Python uses a list with reversed() to iterate compensations in LIFO order on error. TypeScript uses an array with unshift() to maintain LIFO order and manually iterates on error. Java uses the SDK's Saga helper to track compensations and trigger them with saga.compensate().

Saga pattern pitfall: permanently failing compensations

If a compensation Activity fails with a non-retryable error, the Saga cannot fully roll back. Design compensations with generous retry policies.

Saga pattern pitfall: forgetting to register compensation

If a step succeeds but its compensation was never registered, a later failure leaves that step's effects permanently in place.

Saga pattern pitfall: non-idempotent compensations

Compensations may run even when the forward Activity never executed (if registered before execution) or may run multiple times on retry. All compensations must be idempotent; non-idempotent compensations are a common pitfall.

Saga pattern best practice: idempotent compensations

All compensations must be idempotent. Compensations may run even when the forward Activity never executed (if registered before execution) or may run multiple times on retry. Use idempotency keys to ensure safe re-execution.

Saga pattern best practice: re-throw original error

Always re-throw the original exception after running compensations so the Workflow reports the correct failure reason.

Saga pattern pitfall: large payloads in compensation state

Passing large objects through the compensation chain can exceed the 2 MB payload limit. Use references (IDs, URLs) instead of full data.

Saga pattern definition and problem

The Saga pattern manages distributed transactions across multiple services by coordinating a sequence of local transactions, each with a compensating action that can undo its effects if subsequent steps fail. It solves the problem of maintaining data consistency across multiple services or databases without using traditional ACID transactions, avoiding the scalability and coupling issues of two-phase commit.

Saga pattern solution approach

Each step is implemented as a local transaction with a corresponding compensation transaction. If any step fails, compensation transactions execute in reverse order to undo the effects of all completed steps. Compensations are registered as each step completes, then automatically triggered when errors occur to ensure cleanup happens reliably.

When to register compensations: before vs after execution

Two approaches exist: (1) Register before Activity execution (recommended for safety) — ensures the compensation runs even if the Activity fails after partial completion; the compensation must be idempotent and handle cases where the forward Activity never executed. (2) Register after Activity execution (appropriate when safe) — only compensates Activities that completed successfully with simpler compensation logic, appropriate when Activities are truly atomic (all-or-nothing). Register compensations before execution and ensure idempotency when in doubt.

Saga pattern comparison with alternatives

Saga (orchestration) provides eventual consistency with compensating transactions as rollback mechanism, loose coupling, and high scalability. Two-phase commit provides strong ACID consistency with distributed lock/rollback, tight coupling, and low scalability. Saga (choreography) provides eventual consistency with event-driven compensations, very loose coupling, and high scalability. Local transaction provides strong ACID consistency with database rollback, no coupling, and single service scalability.

Saga pattern best practice: idempotency keys for forward Activities

Use idempotency keys for forward Activities by passing a unique identifier (such as a client ID or Workflow ID) to each Activity so retries do not create duplicate side effects.

Saga pattern best practice: timeouts for compensation Activities

Set StartToCloseTimeout on compensation Activities but avoid ScheduleToCloseTimeout on compensations. Do not set Workflow-level timeouts — let compensations retry until they succeed.

Saga pattern best practice: small compensation payloads

Keep compensation payloads small by passing references (IDs, URLs) instead of full data objects to avoid exceeding the 2 MB payload limit.

Saga pattern best practice: handle compensation failures

Log compensation failures but continue executing remaining compensations. If a compensation fails, log the error and continue. In production, alert for manual intervention on persistent compensation failures.

Saga pattern use cases

The Saga pattern is a good fit when you need to maintain consistency across multiple services or databases, traditional distributed transactions (two-phase commit) are too slow or unavailable, you can define compensating actions for each step, eventual consistency is acceptable, and you need to handle long-running transactions spanning hours or days. It is not a good fit for operations requiring strong ACID consistency, single-service transactions using local database transactions, processes without meaningful compensations, or operations that must appear atomic to external observers.

Saga pattern benefits

The Saga pattern maintains eventual consistency without distributed locks. Each service can use its own database and transaction model. Temporal's durable execution guarantees that compensations will execute even after Worker failures. The pattern scales better than two-phase commit protocols.

Saga pattern trade-offs

The trade-offs of the Saga pattern are that only eventual consistency is provided (intermediate states are visible to other processes), you must design idempotent compensation Activities, compensation logic must be maintained alongside forward logic, and some operations may not have meaningful compensations.

Pick First (Race) pattern purpose

Pick First (Race) starts multiple Activities in parallel, takes the first result to arrive, and cancels the rest. This pattern should be used when several approaches compete and you want the first to finish.

Task orchestration patterns overview

Task orchestration patterns compose and coordinate multiple units of work within a Workflow by decomposing large processes into reusable pieces, running work concurrently, and racing alternatives against each other.

Child Workflows pattern purpose

Child Workflows decompose a complex Workflow into smaller, reusable units. Each child has its own Workflow ID, history, and lifecycle. This pattern should be used when a process is large or reused across Workflows.

When to use Pick First pattern

Use Pick First (Race) when several approaches compete and you want the first to finish, as it cancels the losers after taking the first result.

When to use Child Workflows versus Parallel Execution

Use Child Workflows when a process is large or reused across Workflows to break it into independent histories. Use Parallel Execution when independent work can run at the same time to achieve higher throughput with concurrency bounds.

Parallel Execution pattern purpose

Parallel Execution runs multiple Activities concurrently for higher throughput, with error handling and a bound on how many run at once. This pattern should be used when independent work can run at the same time.

Sliding Window pitfall: losing signals across Continue-as-New

If a child signals before the parent's new run has registered the signal handler, the signal can be buffered and delivered correctly because Temporal buffers signals for existing Workflow IDs. However, ensure the signal handler is registered before any await, not conditionally.

Sliding Window pattern overview

The Sliding Window pattern maintains a fixed-size pool of concurrently running child Workflows. As each child completes, it signals the parent, which immediately starts a replacement, keeping the concurrency level constant. This provides bounded concurrency, maximum throughput within that bound, and protection against history bloat via Continue-as-New.

Sliding Window vs other patterns

The Sliding Window pattern differs from Batch Iterator (sequential processing, limited throughput) and Fan-Out (starts all children at once, can overwhelm downstream systems). Sliding Window is used when processing an arbitrarily large record set with bounded concurrency, maximum throughput within that bound, and protection against history bloat.

Sliding Window parent workflow logic

The parent Workflow keeps a live count of in-flight children called `active`. It runs a single loop that starts a child whenever a slot is free. The first windowSize slots are free, so those children start immediately; after that, a backpressure condition blocks each start until an in-flight child signals completion. Each child processes one record and when finished, signals the parent, which decrements `active` and starts the next record's child.

Sliding Window Continue-as-New behavior

Continue-as-New is called after the parent has started windowSize children. Because child Workflows have stable Workflow IDs and Continue-as-New preserves the parent's Workflow ID, children started by a previous run can still signal the current run. The parent carries `active` into the next run so it knows how many carried-over children will still signal it.

Sliding Window signal handling across runs

Children read the parent's Workflow ID from their own Workflow metadata rather than receiving it as an argument. They signal by Workflow ID with no run ID, so they always reach the current run even after the parent has continued as new.

Sliding Window best practice: preserve parent Workflow ID

The parent Workflow ID must be stable across Continue-as-New runs — do not generate a new one. Children read the parent's Workflow ID from their own Workflow metadata (workflowInfo().parent in TypeScript, workflow.info().parent in Python, workflow.GetInfo(ctx).ParentWorkflowExecution in Go, Workflow.getInfo().getParentWorkflowId() in Java) rather than receiving it as an argument, then signal by Workflow ID with no run ID so they always reach the current run.

Sliding Window best practice: use PARENT_CLOSE_POLICY_ABANDON

Use PARENT_CLOSE_POLICY_ABANDON on child Workflows. This lets children that were started by a previous run complete normally even after the parent has continued as new.

Sliding Window best practice: size the window conservatively

Size the window conservatively at first. Each in-flight child counts toward the 2,000 unfinished-actions limit for the parent. A window of 50–200 is a reasonable starting point depending on child duration and downstream capacity.

Sliding Window best practice: pass only IDs to child Workflows

Pass only IDs (not full records) to child Workflows. Workflow inputs are stored in event history. Keep them small.

Sliding Window best practice: carry minimal state into Continue-as-New

Carry minimal state into Continue-as-New. Pass windowSize, startIndex, the live in-flight count (active), a running totalProcessed, and the record ID list (or a reference to it). Do not accumulate results in the parent — collect them out-of-band if needed.

Sliding Window pitfall: race between Continue-as-New and signal draining

After Continue-as-New, the new run must handle signals from children started by the previous run. Pass startIndex (the next unstarted record) and active (the live in-flight count at the moment of CAN) to the new run so it knows how many carried-over children to expect signals from, without re-starting them. The new run folds active in with +=, so a completion that arrives before run() executes is still counted correctly.

Sliding Window pitfall: thundering herd on startup

Starting hundreds of children simultaneously causes a burst of Activity polls. Ramp up the window gradually or use the Batch Iterator pattern if rate limiting is more important than throughput.

Updatable Timer pitfall: not validating new deadlines

Accepting a deadline in the past causes the timer to expire immediately. Always check that the new deadline is in the future before updating.

Updatable Timer best practice: consider max extensions

Limit how many times or how far deadlines can be extended.

Updatable Timer best practice: log changes

Log each deadline update for observability.

Updatable Timer best practice: reuse timer helper

Extract to a helper class or function for use across Workflows.

Updatable Timer best practice: combine with conditions

Use a blocking wait with both time and business conditions.

Updatable Timer pitfall: time-based conditions without duration

A wait without a timeout does not create a timer. The condition is only re-evaluated on state changes (Signals, Activity completions). Always provide a timeout for time-based waits.

Updatable Timer pitfall: duration not re-evaluated

The timer duration is set once when the wait is called. Changing the duration variable afterward has no effect. This is why the timer helper loops and recalculates.

Updatable Timer pattern overview

The Updatable / Debounced Timer pattern implements a sleep operation that can be interrupted and dynamically adjusted via Signals. It enables Workflows to wait for deadlines that can be extended or shortened based on external events, making it suitable for approval processes, SLA management, and time-sensitive business operations.

Updatable Timer problem statement

Without an updatable timer, you must use fixed timeouts that cannot be adjusted, cancel and restart Workflows to change deadlines, poll frequently to check for deadline changes, or implement complex state machines to handle timing updates. Workflows often need to wait for deadlines that can be extended or shortened dynamically, react immediately when the deadline changes, and continue waiting with the new deadline without restarting.

Give your agent this brain