Server Frontend API reference location
The Temporal Server Frontend API gRPC reference is available at /self-hosted-guide/server-frontend-api-reference. This API is used by Client and Worker SDKs to communicate with Temporal Server.
reference
188 notes in this subject, read out of this brain and free to use. This is page 1 of 4.
SDK API reference documentation locations
Temporal provides complete API documentation for all supported SDKs at the following locations: .NET SDK at https://dotnet.temporal.io/api/, Go SDK at https://pkg.go.dev/go.temporal.io/sdk, Java SDK at https://www.javadoc.io/doc/io.temporal/temporal-sdk/latest/index.html, PHP SDK at https://php.temporal.io/namespaces/temporal.html, Python SDK at https://python.temporal.io/, Ruby SDK at https://ruby.temporal.io/, Rust SDK at https://docs.rs/temporalio-sdk/latest/temporalio_sdk/, and TypeScript SDK at https://typescript.temporal.io.
TEMPORAL_CLIENT_AUTHORITY environment variable
TEMPORAL_CLIENT_AUTHORITY overrides the :authority gRPC header. This is a Go SDK extension with no equivalent in other clients and no CLI flag. TOML key: profile.<name>.authority. CLI flag: none. Read by: Go.
TEMPORAL_ADDRESS environment variable
TEMPORAL_ADDRESS specifies the host and port of the Temporal Frontend Service, such as localhost:7233. TOML key: profile.<name>.address. CLI flag: --address. Read by: every client.
TEMPORAL_PROFILE environment variable
TEMPORAL_PROFILE specifies the name of the configuration profile to load. It defaults to 'default'. TOML key: none. CLI flag: --profile. Read by: every client.
CLI flag, environment variable, and TOML configuration precedence
When the same setting comes from more than one place, the precedence order from highest to lowest is: (1) a CLI flag passed explicitly on the command line, (2) an environment variable, (3) a value in a TOML configuration file.
TEMPORAL_CODEC_AUTH environment variable
TEMPORAL_CODEC_AUTH specifies the authorization header value sent to the remote Codec Server. TOML key: profile.<name>.codec.auth. CLI flag: --codec-auth. Read by: every client except Java.
TEMPORAL_NAMESPACE environment variable
TEMPORAL_NAMESPACE specifies the Temporal Namespace to connect to. TOML key: profile.<name>.namespace. CLI flag: --namespace. Read by: every client.
TEMPORAL_CODEC_ENDPOINT environment variable
TEMPORAL_CODEC_ENDPOINT specifies the endpoint for a remote Codec Server. SDKs that read this variable do not apply the codec by default, so it is intended mostly for CLI use. TOML key: profile.<name>.codec.endpoint. CLI flag: --codec-endpoint. Read by: every client except Java.
TEMPORAL_CONFIG_FILE environment variable
TEMPORAL_CONFIG_FILE specifies the path to the TOML configuration file. It defaults to temporal.toml in a platform-specific directory. TOML key: none. CLI flag: --config-file. Read by: every client.
TEMPORAL_GRPC_META_* environment variables
TEMPORAL_GRPC_META_* sets gRPC headers. The part after _META_ becomes the header key, so TEMPORAL_GRPC_META_SOME_KEY sets some-key. TOML key: profile.<name>.grpc_meta. CLI flag: --grpc-meta. Read by: every client.
ContinueAsNewWorkflowExecution command
ContinueAsNewWorkflowExecution is a Command triggered by a call to Continue-As-New from within the Workflow. The corresponding Event is WorkflowExecutionContinuedAsNew. This Command is not awaitable; a Workflow Execution cannot await on the action resulting from this Command. It is one of the few Events that will be the last in a Workflow Execution Event History.
RequestCancelExternalWorkflowExecution command
RequestCancelExternalWorkflowExecution is a Command triggered by a call to request cancellation of another Workflow Execution. The corresponding Event is RequestCancelExternalWorkflowExecutionInitiated. This Command is awaitable; a Workflow Execution can await on the action resulting from this Command. By default, you cannot have more than 2,000 pending Signals to other Workflows.
CompleteWorkflowExecution command
CompleteWorkflowExecution is a Command triggered when the Workflow Function Execution returns. It indicates to the Temporal Service that the Workflow Execution is complete. The corresponding Event is WorkflowExecutionCompleted. This Command is not awaitable; a Workflow Execution cannot await on the action resulting from this Command. It is one of the few Events that will be the last in a Workflow Execution Event History.
SignalExternalWorkflowExecution command
SignalExternalWorkflowExecution is a Command triggered by a call to Signal another Workflow Execution. The corresponding Event is SignalExternalWorkflowExecutionInitiated. This Command is awaitable; a Workflow Execution can await on the action resulting from this Command. By default, you cannot have more than 2,000 pending Signals to other Workflows.
RecordMarker command
RecordMarker is a Command triggered by the SDK. The corresponding Event is MarkerRecorded. This Command is not awaitable; a Workflow Execution cannot await on the action resulting from this Command.
CancelTimer command
CancelTimer is a Command triggered by a call to cancel a Timer. The corresponding Event is TimerCanceled. This Command is not awaitable; a Workflow Execution cannot await on the action resulting from this Command.
UpsertWorkflowSearchAttributes command
UpsertWorkflowSearchAttributes is a Command triggered by a call to upsert Workflow Search Attributes. The corresponding Event is UpsertWorkflowSearchAttributes. This Command is not awaitable; a Workflow Execution cannot await on the action resulting from this Command.
RequestCancelActivityTask command
RequestCancelActivityTask is a Command triggered by a call to request the cancellation of an Activity Task. The corresponding Event is ActivityTaskCancelRequested. This Command is not awaitable; a Workflow Execution cannot await on the action resulting from this Command.
ScheduleActivityTask command
ScheduleActivityTask is a Command triggered by a call to execute an Activity. The corresponding Event is ActivityTaskScheduled. This Command is awaitable; a Workflow Execution can await on the action resulting from this Command. By default, you cannot schedule more than 2,000 Activities concurrently.
ScheduleNexusOperation command
ScheduleNexusOperation is a Command triggered by a call to execute a Nexus Operation in the caller Workflow. The corresponding Event is NexusOperationScheduled. This Command is awaitable; a Workflow Execution can await on the action resulting from this Command. By default, you cannot schedule more than 30 Nexus Operations concurrently.
StartTimer command
StartTimer is a Command triggered by a call to start a Timer. The corresponding Event is TimerStarted. This Command is awaitable; a Workflow Execution can await on the action resulting from this Command.
CancelNexusOperation command
CancelNexusOperation is a Command triggered by a call to request the cancellation of a Nexus Operation. The corresponding Event is NexusOperationCancelRequested. This Command is not awaitable; a Workflow Execution cannot await on the action resulting from this Command.
ProtocolMessageCommand
ProtocolMessageCommand is a Command that helps guarantee ordering constraints for features such as Updates. This Command points at the message from which the Event is created. Therefore, just from the Command, you cannot predict the resulting Event type.
CancelWorkflowExecution command
CancelWorkflowExecution is a Command triggered when the Workflow has successfully cleaned up after receiving a Cancellation Request, which appears as WorkflowExecutionCancelRequestedEvent in the Event History. The corresponding Event is WorkflowExecutionCanceled. This Command is not awaitable; a Workflow Execution cannot await on the action resulting from this Command. It is one of the few Events that will be the last in a Workflow Execution Event History.
FailWorkflowExecution command
FailWorkflowExecution is a Command triggered when the Workflow Execution returns an error or an exception is thrown. The corresponding Event is WorkflowExecutionFailed. This Command is not awaitable; a Workflow Execution cannot await on the action resulting from this Command.
StartChildWorkflowExecution command
StartChildWorkflowExecution is a Command triggered by a call to spawn a Child Workflow Execution. The corresponding Event is ChildWorkflowExecutionStarted. This Command is awaitable; a Workflow Execution can await on the action resulting from this Command. By default, you cannot have more than 2,000 pending Child Workflows.
Command definition
A Command is a requested action issued by a Worker to the Temporal Service after a Workflow Task Execution completes.
workflow_success metric
The workflow_success metric shows the number of Workflows that successfully completed.
workflow_continued_as_new metric
The workflow_continued_as_new metric shows the number of Workflow Executions that were Continued-As-New from a past execution.
workflow_cancel metric
The workflow_cancel metric shows the number of Workflows canceled before completing execution.
schedule_action_success metric
The schedule_action_success metric measures the successful execution of Workflows as per their schedules or through manual triggers. This metric confirms that Workflows are running as expected without delays or errors. Example Prometheus query: sum(rate(schedule_action_success{namespace="$namespace"}[5m]))
Common metric tags for gRPC service requests
Temporal emits metrics for each gRPC service request with type, operation, and namespace tags. Use the operation tag to get request rates, error rates, or latencies per operation. Use the service_name tag with service role tag values to get details for a specific service. All common tags are defined in metric_defs.go.
schedule_missed_catchup_window metric
The schedule_missed_catchup_window metric tracks occurrences when the system fails to execute a Scheduled Action within the defined catchup window. Missed catchup windows can result from extended outages beyond the configured catchup period. Example Prometheus query: sum(rate(schedule_missed_catchup_window{namespace="$namespace"}[5m]))
schedule_buffer_overruns metric
The schedule_buffer_overruns metric indicates instances where the buffer for holding Scheduled Workflows exceeds its maximum capacity. This scenario typically occurs when schedules with a buffer_all overlap policy have their average run length exceeding the average schedule interval. Example Prometheus query: sum(rate(schedule_buffer_overruns{namespace="$namespace"}[5m]))
persistence_latency metric
The persistence_latency metric shows the latency on persistence operations. Example Prometheus query for latency by percentile: histogram_quantile(0.95, sum(rate(persistence_latency_bucket{service_name="history"}[1m])) by (operation, le))
persistence_errors metric
The persistence_errors metric shows all persistence errors and is a good indicator for connection issues between the Temporal Service and the persistence store. Example Prometheus query: sum (rate(persistence_errors{service_name="history"}[1m]))
persistence_requests metric
The persistence_requests metric is emitted on every persistence request. Example Prometheus queries: sum by (operation) (rate(persistence_requests{service_name="history"}[1m])) for History Service and sum by (operation) (rate(persistence_requests{service_name="matching"}[1m])) for Matching Service.
service_latency_userlatency metric (History Service)
The service_latency_userlatency metric shows the latency introduced because of Workflow logic. For example, if one Workflow schedules many Activities or Child Workflows at the same time, it can cause per-Workflow lock contention. The wait period for the per-Workflow lock is counted as userlatency.
Temporal Service metrics reference location
All metrics emitted by the Temporal Service are listed in metric_defs.go at https://github.com/temporalio/temporal/blob/main/common/metrics/metric_defs.go. For details on setting up metrics in the Temporal Service configuration, see the Temporal Service configuration reference at /references/configuration#global.
task_latency_schedule metric (History Service)
The task_latency_schedule metric is available only in v1.18.0+ and measures the duration from Task submission (to the Task scheduler) to processing (Task schedule to start latency for in-memory queue) in the History Service.
task_latency_queue metric (History Service)
The task_latency_queue metric measures the duration, end-to-end, from when the Task should be executed (from the time it was fired) to when the Task is done in the History Service.
task_latency metric (History Service)
The task_latency metric measures the in-memory latency across multiple attempts in the History Service.
task_latency_processing metric (History Service)
The task_latency_processing metric shows the processing latency per attempt in the History Service. Example Prometheus query: histogram_quantile(0.95, sum(rate(task_latency_processing_bucket{operation=~"TransferActive.*",service_name="history"}[1m])) by (operation, le))
task_attempt metric (History Service)
The task_attempt metric shows the number of attempts on each Task Execution. A Task is retried forever, and each retry increases the attempt count. Example Prometheus query: histogram_quantile(0.95, sum(rate(task_attempt_bucket{operation=~"TransferActive.*"}[1m])) by (operation, le))
queue_latency_schedule metric (History Service)
The queue_latency_schedule metric is available only in v1.18.0+ and measures the time to schedule 100 Tasks in one Task channel in the host-level Task scheduler in the History Service. If fewer than 100 Tasks are in the Task channel for 30 seconds, the latency is scaled to 100 Tasks upon emission. Note: This is still an experimental metric and is subject to change.
task_errors metric (History Service)
The task_errors metric is emitted on every Task process error in the History Service. Example Prometheus query: sum(rate(task_errors{operation=~"TransferActive.*"}[1m]))
task_requests metric (History Service)
The task_requests metric is emitted on every Task process request in the History Service. Example Prometheus query: sum(rate(task_requests{operation=~"TransferActive.*"}[1m]))
no_poller_tasks metric (Matching Service)
The no_poller_tasks metric is emitted whenever a task is added to a task queue that has no poller and is a counter metric. This is usually an indicator that either the Worker or the starter programs are using the wrong Task Queue.
persistence_error_with_type metric
The persistence_error_with_type metric shows all errors related to the persistence store with type and contains an error_type tag. Example Prometheus query: sum(rate(persistence_error_with_type{service_name="history"}[1m])) by (error_type)
asyncmatch_latency metric (Matching Service)
The asyncmatch_latency metric measures the time from creation to delivery for async matched Tasks. The larger this latency, the longer Tasks are sitting in the queue waiting for Workers to pick them up. Example Prometheus query: histogram_quantile(0.95, sum(rate(asyncmatch_latency_bucket{service_name="matching"}[5m])) by (operation, le))
poll_timeouts metric (Matching Service)
The poll_timeouts metric shows when no Tasks are available for the poller within the poll timeout. Example Prometheus query: sum(rate(poll_timeouts{}[5m]))
poll_success metric (Matching Service)
The poll_success metric shows Tasks that are successfully matched to a poller. Example Prometheus query: sum(rate(poll_success{}[5m]))
schedule_rate_limited metric
The schedule_rate_limited metric reflects instances where the creation of Workflows by a Schedule is throttled due to rate limiting policies within a Namespace. This metric is crucial for identifying scheduling patterns that frequently hit rate limits, potentially causing missed catchup windows. Example Prometheus query: sum(rate(schedule_rate_limited{namespace="$namespace"}[5m]))
client_errors metric
The client_errors metric is an indicator for connection issues between different Server roles. Example Prometheus query: sum(rate(client_errors{service_name="frontend",service_role="history"}[5m]))
service_error_with_type metric
The service_error_with_type metric is available only in v1.17.0+ and identifies errors encountered by the service. Example Prometheus query for service errors by type for the Frontend Service: sum(rate(service_error_with_type{service_name="frontend"}[5m])) by (error_type)
service_latency metric
The service_latency metric shows latencies for all Client request operations and is usually the starting point to investigate which operation is experiencing high-latency issues. Example Prometheus query for P95 service latency by operation for the Frontend Service: histogram_quantile(0.95, sum(rate(service_latency_bucket{service_name="frontend"}[5m])) by (operation, le))
workflow_failed metric
The workflow_failed metric shows the number of Workflows that failed before completion.
service_requests metric
The service_requests metric shows service requests received per Task Queue. Example Prometheus query: sum(rate(service_requests{operation="AddWorkflowTask"}[2m]))
workflow_timeout metric
The workflow_timeout metric shows the number of Workflows that timed out before completing execution.