Task queue identification
A Task Queue is identified by its name, specified via the --task-queue flag. This is a required parameter when starting or executing an Activity. The Task Queue name is where Activity tasks are dispatched to Workers.
Temporal · Concepts · all subjects
task queues & workers
40 notes, read out of this brain and free to use. Each one was extracted from a source and is re-checked against its exam.
Activity fairness key and weight for task dispatch
The --fairness-key flag (max 64 bytes) and --fairness-weight flag (range 0.001-1000) are used for proportional task dispatch. These parameters enable fair distribution of activities across workers when specified.
WorkerDeploymentVersion is required for Lambda Workers
WorkerDeploymentVersion is required for serverless workers. Worker Deployment Versioning is always enabled for Serverless Workers. Each Workflow must have a versioning behavior, either AutoUpgrade or Pinned. Set it per-Workflow with the [Workflow] attribute, or set a worker-level default with DefaultVersioningBehavior in DeploymentOptions. The default versioning behavior is AutoUpgrade.
Serverless Workers definition and lifecycle
Serverless Workers run on ephemeral, on-demand compute rather than long-lived processes. Temporal invokes the Worker when Tasks arrive, and the Worker shuts down when the work is done.
.NET Serverless Workers AWS Lambda support
The .NET SDK supports AWS Lambda as a serverless provider for Temporal Workers through the Temporalio.Extensions.Aws.Lambda NuGet package. This package covers setup, configuration, Lambda-tuned defaults, observability, and the invocation lifecycle.
Worker connection and Task Queue polling
When a new Worker process is created, the Worker creates a long-lasting connection to the Temporal Service, polling a Task Queue for Tasks that are related to the code it is capable of executing.
Worker definition and responsibilities
Workers are responsible for executing Workflow and Activity code (application code). Workers are horizontally scalable, often run with systems like Kubernetes, and configured according to the application's needs. The Worker polls on the specified Task Queue, processing those Tasks, and reporting the results back to the Temporal Service. They execute both Workflows and Activities efficiently and reliably.
Task routing in Multi-Cluster Replication
When the Multi-Cluster Replication feature is enabled, Tasks are sent to the Parent Task Queue partition that matches that Namespace, if it exists.
Ramping Worker Deployment Version definition
The Ramping Worker Deployment Version is the version where a configurable percentage of Workflows are routed to unless they were previously pinned on a different version. The ramp percentage can be in the range [0, 100]. Workflows that don't go to the Ramping Version will go to the Current Version. If no Ramping Version is specified, 100% of new Workflows and Auto-Upgrade Workflows will go to the Current Version.
Target Worker Deployment Version definition
The Target Worker Deployment Version is the version your Workflow will upgrade to next. This could be the Deployment's Current Version or the Ramping Version. For example, if an Auto-Upgrade Workflow was running on Version A, the Current Version is B, and there is a 5% ramp to C, there is a 95% chance that its Target Version is B and 5% that it's C. Workflow ID determines whether the workflow falls into the 95% group or the 5% group.
Worker Deployment Version states
A Worker Deployment Version moves through the following states: 1) Inactive - the version exists because a Worker with that version has polled the server, and if this version never becomes Active, it will never be Draining or Drained. 2) Active - the version is either Current or Ramping, so it is accepting new Workflows and existing auto-upgrade Workflows. 3) Draining - the version has open pinned Workflows running on it, but stopped being Current or Ramping, usually because a newer version has been deployed; it is possible to be Draining and have no open pinned Workflows for a short time since the drainage status is updated only periodically. 4) Drained - the version was draining and now all the pinned Workflows that were running on it are closed; closed Workflows may still re-run some code paths if they are Queried within their Retention Period and Workers with that version are still polling.
Auto-upgrade Workflows never inherit versions
Auto-upgrade Workflows never inherit versions when starting new runs.
Pinned Workflows default version inheritance to children
By default, Pinned workflows will pass their version to any Pinned children.
Cron Jobs never inherit versioning behavior
Cron Jobs never inherit versioning behavior or version.
Versioning Override Inheritance rules
Children, crons, retries, and continue-as-new inherit the source run's override if: the override is pinned, AND the new Workflow's Task Queue belongs to the override version. Override inheritance is evaluated separately and takes precedence over inherited base version.
Worker Deployment definition and purpose
A Worker Deployment is a logical service that groups similar Workers together for unified management. Each Deployment has a name (such as your service name) and supports versioning through a series of Worker Deployment Versions.
Worker Deployment Version structure and reporting
A Worker Deployment Version represents an iteration of a Worker Deployment. Each Deployment Version is identified by a deployment name and a Build ID. The deployment name groups related Workers across versions, and the Build ID identifies a specific release of your Worker code. Each Deployment Version consists of Workers that share the same code build and environment. When a Worker starts polling for Workflow and Activity Tasks, it reports its Deployment Version to the Temporal Server.
Pinned Workflow behavior
A Pinned Workflow is guaranteed to complete on a single Worker Deployment Version. You can mark a Workflow Type as pinned when you register it by adding an additional Pinned parameter. If you need to move a pinned Workflow to a new version, use the temporal workflow update-options command.
Auto-Upgrade Workflow behavior
An Auto-Upgrade Workflow will move to the latest Worker Deployment Version automatically whenever you change the current version. Auto-upgrade Workflows are not restricted to a single Deployment Version and need to be kept replay-safe manually using patching.
Current Worker Deployment Version definition
The Current Worker Deployment Version is the version where Workflows are routed to unless they were previously pinned on a different version. Other versions can continue polling to allow pinned Workflows to finish executing or in case you need to roll back. If no current version is specified, the default is unversioned.
Child Workflow version inheritance when parent is Pinned
When Parent is Pinned: Child inherits the parent's version if the child's Task Queue belongs to that version. Child's first Workflow task executes in the same version as its parent. If child is also Pinned, child remains Pinned to the inherited version for its lifetime. If child is Auto-Upgrade, child's behavior changes to Auto-Upgrade after the first task completes. If child's Task Queue is not in the same Worker Deployment as parent, no inheritance occurs and child starts on Current Version of its task queue.
Child Workflow version inheritance when parent is Auto-Upgrade
When Parent is Auto-upgrade: Child inherits no initial Versioning Behavior. Child starts on the Current Version of its Worker Deployment like all new Workflow executions.
Continue-as-New version inheritance when original Workflow is Pinned
When Original Workflow is Pinned: The Pinned version is inherited across the Continue-As-New chain. If the new run's Task Queue is not in the same Worker Deployment as the original Workflow, no inheritance occurs and new run starts on Current Version of its task queue.
Continue-as-New version inheritance when original Workflow is Auto-Upgrade
When Original Workflow is Auto-upgrade: No version inheritance occurs.
Retry version inheritance conditions
Version inheritance occurs during retries only when all the following conditions are met: 1) The retried run is effectively pinned at the time of retry. 2) The retried run inherited a pinned version when it started (that is, it is a child of a pinned parent, or a Continue-As-New of a pinned run). 3) The retried run is running on a Task Queue in the inherited version. When these conditions are not met, no version inheritance occurs.
Nexus Endpoint routing to Task Queues
A Nexus Endpoint creates an entry point that separates callers from the underlying Nexus Task Queue. The Nexus callers only interact with the Nexus Endpoint. This endpoint routes Nexus Requests to a target Task Queue that's polled by a Nexus Worker.
Task Queues are dynamically allocated without explicit registration
Task Queues are lightweight components that don't require explicit registration. They're created on demand when a Workflow Execution, Activity, or Nexus Operation is invoked, and/or when a Worker Process subscribes to start polling. When a named Task Queue is created, individual Task Queues for Workflows, Activities, and Nexus are created using the same name. A Temporal Application can use, and the Temporal Service can maintain, an unlimited number of Task Queues.
Workers poll Task Queues via synchronous RPC
Workers poll for Tasks in Task Queues via synchronous RPC. A Worker Process polls for a message only when it has spare capacity, avoiding overloading itself. This implementation enables load balancing across many Worker Processes.
Benefits of Task Queues for communication and reliability
Task Queues enable Task Routing (routing of specific Tasks to specific Worker Processes or a specific process), activity server-side throttling (limiting Task dispatching rate to Worker Processes while supporting higher rates during spikes), and persistence for Workflow and Activity Tasks so they remain until Workers recover if a Worker Process goes down. Worker Processes do not need to advertise themselves through DNS or any other network discovery mechanism. Worker Processes connect directly to the Temporal Service for secure communication without needing to open exposed ports.
Nexus and Query Tasks use sync matching instead of persistence
Nexus and Query Tasks are not persisted. Instead, they are sync matched when, and only when, polled by a Worker. Sync matching immediately matches and delivers a Task to an available Worker without persisting a Task to the Service database. The caller is responsible to retry failed operations. Caller Workflows that invoke Nexus Operations will automatically retry Nexus Tasks until exceeding the Schedule-to-Close timeout.
Worker Task Queue registration requirements
Any Worker can pick up any Task on a given Task Queue. You must ensure that if a Worker accepts a Task that it can process that task using one of its registered Workflows, Activities, or Nexus Operation handlers. This means that all Workers listening to a Task Queue must register all Workflows, Activities, and Nexus Operations that live on that Queue. All Worker Entities listening to the same Task Queue name must be registered to handle the exact same Workflows Types, Activity Types, and Nexus Operations.
Exceptions to identical Task Queue Worker registrations
There are two exceptions to the 'Task Queue Workers with identical registrations' rule. First, Worker Versioning may be used during Worker upgrade binary rollouts so it's okay to have temporarily misaligned registrations. Second, dynamic Workflows or Activity components may be used—if a Task arrives with a recognized method signature, the Worker can use a pre-registered dynamic stand-in.
Not Found errors for missing Task handlers
When Workers don't have a registered Workflow, Activity, Nexus Operation, or dynamic Workflow or Activity component for a given Task, the Task will fail with a 'Not Found' error. 'Not Found' Workflow Tasks and Activity Tasks are treated as retryable errors. 'Not Found' Nexus Operation handlers are non-retryable and must be manually retried from the caller Workflow.
Places where Task Queue names are set
There are five places where the name of the Task Queue can be set by the developer: (1) when spawning a Workflow Execution (required), (2) when creating a Worker Entity and running a Worker Process (required), (3) when spawning an Activity Execution (optional; inherits from Workflow Execution if not provided), (4) when spawning a Child Workflow Execution (optional; inherits from Parent Workflow Execution if not provided), and (5) when creating a Nexus Endpoint (Nexus Endpoints route requests to the target Task Queue).
Activity Execution Task Queue inheritance
A Task Queue name can be provided when spawning an Activity Execution, but this is optional. An Activity Execution inherits the Task Queue name from its Workflow Execution if one is not provided.
Task Queue definition and types
A Task Queue is a lightweight, dynamically allocated queue that one or more Worker Entities poll for Tasks. There are three types of Task Queues: Activity Task Queues, Workflow Task Queues, and Nexus Task Queues.
Child Workflow Execution Task Queue inheritance
A Task Queue name can be provided when spawning a Child Workflow Execution, but this is optional. A Child Workflow Execution inherits the Task Queue name from its Parent Workflow Execution if one is not provided.
Task Queue partitioning and default partition count
Task Queues can be scaled by adding partitions. By default each Task Queue has 4 partitions.
Task ordering in single partition Task Queues
Task Queues with a single partition are almost always first-in, first-out, with rare edge case exceptions. However, using a single partition limits you to low- and medium-throughput use cases.
Task ordering in multi-partition Task Queues
In Task Queues with multiple partitions, each task is assigned to a random partition. Generally partitions will act as FIFO queues, so once a task queue builds up a backlog, the sync match rate (tasks that can be dispatched immediately) will drop to nearly zero because the task queue will instead dispatch tasks from the backlog (async matches) first.