observability/metrics

21 notes, read out of this brain and free to use. Each one was extracted from a source and is re-checked against its exam.

Metrics configuration via TemporalRuntime Telemetry

Metrics in .NET are configured on the Metrics property of the Telemetry property on the TemporalRuntime. The TemporalRuntime object should be created globally and used for all clients, so it must be configured before any other Temporal code.

Prometheus endpoint setup example

The following example exposes a Prometheus endpoint on port 9000: ```csharp using Temporalio.Client; using Temporalio.Runtime; var runtime = new TemporalRuntime(new() { Telemetry = new() { Metrics = new() { Prometheus = new("0.0.0.0:9000") } }, }); var client = await Temporalio.ConnectAsync(new("localhost:7233") { Runtime = runtime }); ```

Custom metric meter using DiagnosticSource extension

A custom metric meter can be set on the telemetry options to handle metrics programmatically. The Temporalio.Extensions.DiagnosticSource extension provides a custom metric meter implementation that sends all metrics to a System.Diagnostics.Metrics.Meter instance: ```csharp using System.Diagnostics.Metrics; using Temporalio.Client; using Temporalio.Extensions.DiagnosticSource; using Temporalio.Runtime; // Create .NET meter using var meter = new Meter("My.Meter"); // Can create MeterListener or OTel meter provider here... // Create Temporal runtime with a custom metric meter for that meter var runtime = new TemporalRuntime(new() { Telemetry = new() { Metrics = new() { CustomMetricMeter = new CustomMetricMeter(meter) }, }, }); var client = await Temporalio.ConnectAsync(new("localhost:7233") { Runtime = runtime }); ```

Java SDK Platform documentation

Platform documentation covers Observability and Enriching the UI.

Spring Boot metrics integration

To enable metrics in Spring Boot Temporal integration, set up Spring Boot Actuator with MeterRegistry. The Temporal Spring Boot integration will automatically pick up the MeterRegistry bean and use it to report Temporal metrics. Alternatively, define a custom io.micrometer.core.instrument.MeterRegistry bean.

MicrometerClientStatsReporter for metrics in Java SDK

Use the MicrometerClientStatsReporter class to integrate metrics with Micrometer MeterRegistry configured for your metrics backend. Create a PrometheusMeterRegistry, wrap it in a MicrometerClientStatsReporter, configure a scope with a RootScopeBuilder, and set it on WorkflowServiceStubsOptions using setMetricsScope().

Java SDK metrics setup with Prometheus example

PrometheusMeterRegistry registry = new PrometheusMeterRegistry(PrometheusConfig.DEFAULT); StatsReporter reporter = new MicrometerClientStatsReporter(registry); Scope scope = new RootScopeBuilder() .reporter(reporter) .reportEvery(com.uber.m3.util.Duration.ofSeconds(10)); WorkflowServiceStubsOptions stubOptions = WorkflowServiceStubsOptions.newBuilder().setMetricsScope(scope).build();

Python SDK Prometheus metrics example

from temporalio.runtime import Runtime, TelemetryConfig, PrometheusConfig new_runtime = Runtime(telemetry=TelemetryConfig(metrics=PrometheusConfig(bind_address="0.0.0.0:9000"))) my_client = await Client.connect("my.temporal.host:7233", runtime=new_runtime)

Emit metrics with Prometheus in Python SDK

Metrics in Python are configured globally and should be set before any other Temporal code. Use the Runtime class with TelemetryConfig and PrometheusConfig to expose a Prometheus endpoint. Example: Create a Runtime with telemetry enabled, specifying the bind address like '0.0.0.0:9000', then pass it to Client.connect().

Activity metrics interceptor example

This example demonstrates using an interceptor to measure Schedule-To-Start and Schedule-To-Close latency. The SimpleWorkerInterceptor.intercept_activity() returns an ActivityMetricsInterceptor that overrides execute_activity(). Before executing the activity, it records Schedule-To-Start latency using activity.info().started_time - activity.info().current_attempt_scheduled_time. After the activity completes, it records Schedule-To-Close latency using datetime.now(timezone.utc) - activity.info().current_attempt_scheduled_time. Both metrics are recorded using activity.metric_meter().create_histogram_timedelta().

Cloud Run Worker supports standard observability features

A Cloud Run Worker emits the same traces and metrics as a Worker anywhere else. Configure metrics export and OpenTelemetry tracing interceptors through the same Observability configuration used for other Python Workers.

Custom metric handling in Ruby with MetricBuffer

Instead of Prometheus or OpenTelemetry, provide an instance of `Temporalio::Runtime::MetricBuffer` as a `buffer` argument to the `MetricsOptions`. Call `retrieve_updates` periodically on the buffer to get metric updates.

Metrics configuration in Ruby via telemetry

Metrics in Ruby are configured on the `metrics` argument of the `telemetry` argument when creating a global `Temporalio::Runtime`. This object should be created globally and used for all clients, so it should be configured before any other Temporal code.

Prometheus endpoint setup in Ruby

To expose a Prometheus metrics endpoint in Ruby, create a global `Temporalio::Runtime` with telemetry configured. Pass a `Temporalio::Runtime::PrometheusMetricsOptions` with a `bind_address` parameter to specify the endpoint (for example, '0.0.0.0:9000'). Example: `Temporalio::Runtime.default = Temporalio::Runtime.new(telemetry: Temporalio::Runtime::TelemetryOptions.new(metrics: Temporalio::Runtime::MetricsOptions.new(prometheus: Temporalio::Runtime::PrometheusMetricsOptions.new(bind_address: '0.0.0.0:9000'))))`

sticky_cache_size metric

The sticky_cache_size metric reports the size of the Workflow cache, indicating how much memory is being used to store cached Workflow state.

activity_schedule_to_start_latency metric

The activity_schedule_to_start_latency metric is a timer that measures the time from when an Activity Task is scheduled (placed in a Queue) to when a Worker starts (picks up from the Task Queue) that Task. This metric helps ensure that Activities are being processed from the queue in a timely manner.

workflow_task_schedule_to_start_latency metric

The workflow_task_schedule_to_start_latency metric is a timer that measures the time from when a Workflow Task is scheduled (placed in a Queue) to when a Worker starts (picks up from the Task Queue) that Task. This metric helps ensure that Workflow Tasks are being processed from the queue in a timely manner.

worker_task_slots_used metric

The worker_task_slots_used gauge reports the number of executor slots currently in use for a Worker type. Unlike worker_task_slots_available, this metric works with both fixed size slot suppliers and resource-based slot suppliers.

worker_task_slots_available metric

The worker_task_slots_available gauge reports the number of available executor slots that are currently available and unoccupied for a Worker type. Tag it with worker_type=WorkflowWorker for Workflow Task Workers or worker_type=ActivityWorker for Activity Workers. This metric can only be used with fixed size slot suppliers and cannot be used with resource-based slot suppliers.

Metrics prefix: temporal_

All metrics emitted by Temporal SDKs are prepended with the temporal_ prefix. For example, worker_task_slots_available is actually temporal_worker_task_slots_available when used in monitoring systems.

workflow_active_thread_count metric

The workflow_active_thread_count metric reports the number of cached Workflow threads currently active in the Workflow cache.

Give your agent this brain