Priority Key concept in task queues
Task queue priority can be controlled using a priority key mechanism that determines the relative priority of tasks in dispatch order.
task-queues
10 notes, read out of this brain and free to use. Each one was extracted from a source and is re-checked against its exam.
Fairness Key concept in task queues
Task queue fairness can be controlled using a fairness key mechanism to ensure fair distribution of work across different criteria.
Task Queue Priority and Fairness
Temporal provides mechanisms to control dispatch order of tasks in task queues using task queue priority and fairness features. These features allow you to specify priority keys and fairness keys to determine which tasks get picked next by workers.
Priority and Fairness interactive walkthrough available
An interactive walkthrough is available that allows you to step through scenarios to see which task gets picked next and understand why, with SDK code examples provided.
Task Queue
A Task Queue is a first-in, first-out queue that a Worker Process polls for Tasks.
Task
A Task is the context needed to make progress with a specific Workflow Execution or Activity Execution.
Dedicated Task Queue recommendation for production
For production deployments, host PermitSlotWorkflow on a dedicated Task Queue so application Worker backpressure does not slow down permit acquire and release operations. In development, a single-queue topology is acceptable, but production should separate the permit Workflow's Task Queue from the application Workflows' Task Queue to prevent contention.
Resource-based task queue routing fundamentals
Task Queues in Temporal are dynamically created when first referenced. Route Activities to appropriate Worker pools based on their resource requirements: GPU-intensive Activities route to GPU-equipped Workers with ML libraries (TensorFlow, PyTorch, CUDA); CPU-intensive Activities route to high-CPU instances; Memory-intensive Activities route to memory-optimized instances; Standard Activities route to cost-effective general-purpose instances.
Task Queue constants definition pattern
Define Task Queue name constants in a shared Python module: ```python # task_queues.py """Task Queue constants for resource-based routing.""" # Resource-specific task queues STANDARD_CPU_QUEUE = "cpu-standard" GPU_ML_QUEUE = "gpu-ml-inference" HIGH_MEMORY_QUEUE = "high-memory-analytics" VIDEO_ENCODING_QUEUE = "video-encoding-hardware" ``` Import these constants in both Workflow and Worker files to ensure consistent queue naming and prevent typos that would create separate queues.
Task Queue partitioning dynamic configuration keys
matching.numTaskqueueReadPartitions (Int, default 4) sets the number of read partitions for a Task Queue. matching.numTaskqueueWritePartitions (Int, default 4) sets the number of write partitions for a Task Queue. Both keys must be set together. Task Queues that need to support high traffic require higher number of partitions; values should be set in accordance with your poller count. When increasing partitions, the same count must be set for both read and write operations.