testing and debugging

32 notes, read out of this brain and free to use. Each one was extracted from a source and is re-checked against its exam.

Workflow show displays Event History

The show command displays a Workflow Execution's Event History. When using JSON output (--output json), the results can be passed to an SDK to perform a replay. Use temporal workflow show --workflow-id YourWorkflowId --output json

Workflow stack trace query for troubleshooting

The stack command performs a Query on a Workflow Execution using a __stack_trace-type Query to display a stack trace of the threads and routines currently in use by the Workflow for troubleshooting. Use temporal workflow stack --workflow-id YourWorkflowId

Workflow describe shows execution information including reset points

The describe command displays information about a specific Workflow Execution. Use temporal workflow describe --workflow-id YourWorkflowId to get details, or add --reset-points true to show the Workflow Execution's auto-reset points.

.NET SDK testing types: end-to-end, integration, and unit

In Temporal .NET SDK, you can create three types of automated tests: End-to-end tests run a Temporal Server and Worker with all Workflows and Activities, starting and interacting with Workflows from a Client. Integration tests fall between end-to-end and unit testing, including running Activities with mocked Context and SDK imports, running Workers with mock Activities using a Client to start Workflows, or running Workflows with mocked SDK imports. Unit tests run a piece of Workflow or Activity code and mock any code it calls. The majority of tests should generally be written as integration tests.

.NET testing framework compatibility

The .NET SDK is compatible with any testing framework and does not have a specific recommendation. Most .NET SDK samples use xUnit.

Testing Workflows with WorkflowEnvironment.StartLocalAsync

The non-time-skipping WorkflowEnvironment can be started via StartLocalAsync, which supports all standard Temporal features. It is the real Temporal dev server packaged in the Temporal CLI, lazily downloaded on first use, and run as a sub-process in the background. Assuming tests properly use separate Task Queues, the same server can and should be reused across tests.

.NET Workflow test example with standard server

Example test for a simple SayHelloWorkflow: ```csharp using Temporalio.Testing; using Temporalio.Worker; [Fact] public async Task SayHelloWorkflow_SimpleRun_Succeeds() { // Start local dev server await using var env = await WorkflowEnvironment.StartLocalAsync(); // Create a worker using var worker = new TemporalWorker( env.Client, new TemporalWorkerOptions($"task-queue-{Guid.NewGuid()}"). AddWorkflow<SayHelloWorkflow>()); // Run the worker only for the life of the code within await worker.ExecuteAsync(async () => { // Execute the workflow and confirm the result var result = await env.Client.ExecuteWorkflowAsync( (SayHelloWorkflow wf) => wf.RunAsync("Temporal"), new(id: $"wf-{Guid.NewGuid()}", taskQueue: worker.Options.TaskQueue!)); Assert.Equal("Hello, Temporal!", result); }); } ``` This example demonstrates how to start a local dev server, create a worker with a workflow, execute the workflow, and assert on the result.

Testing Workflows with time skipping using StartTimeSkippingAsync

The time-skipping WorkflowEnvironment can be started via StartTimeSkippingAsync, which is a reimplementation of the Temporal server with special time skipping capabilities. Like StartLocalAsync, this lazily downloads the process to run when first called. However, unlike StartLocalAsync, this class is not thread safe nor safe for use with independent tests. It can be technically reused, but only for one test at a time because time skipping is locked/unlocked at the environment level. Developers are encouraged to run it per test needed.

Automatic time skipping in .NET Workflow tests

When using WorkflowEnvironment.StartTimeSkippingAsync, the time-skipping server automatically skips to the next event when waiting on a workflow result. By calling ExecuteWorkflowAsync on the client, you are actually calling StartWorkflowAsync + GetResultAsync, and GetResultAsync automatically skips time as much as it can (basically until the end of the Workflow or until an Activity is run). To disable automatic time-skipping while waiting for a workflow result, run code as a lambda passed to env.WithAutoTimeSkippingDisabled or env.WithAutoTimeSkippingDisabledAsync.

.NET Workflow test example with time skipping

Example test for a Workflow that waits a day using time skipping: ```csharp using Temporalio.Testing; using Temporalio.Worker; [Fact] public async Task WaitADayWorkflow_SimpleRun_Succeeds() { // Start time-skipping test server await using var env = await WorkflowEnvironment.StartTimeSkippingAsync(); // Create a worker using var worker = new TemporalWorker( env.Client, new TemporalWorkerOptions($"task-queue-{Guid.NewGuid()}"). AddWorkflow<WaitADayWorkflow>()); // Run the worker only for the life of the code within await worker.ExecuteAsync(async () => { // Execute the workflow and confirm the result var result = await env.Client.ExecuteWorkflowAsync( (WaitADayWorkflow wf) => wf.RunAsync(), new(id: $"wf-{Guid.NewGuid()}", taskQueue: worker.Options.TaskQueue!)); Assert.Equal("all done", result); }); } ``` This test runs almost instantly because time is automatically skipped.

Manual time skipping with WorkflowEnvironment.DelayAsync

Until a Workflow is waited on, all time skipping in the time-skipping environment is done manually via WorkflowEnvironment.DelayAsync. This allows testing scenarios like Signal timeouts by manually advancing time without waiting on the workflow result.

.NET Workflow Signal test example with manual time skipping

Example test for testing a Signal timeout with manual time skipping: ```csharp using Temporalio.Testing; using Temporalio.Worker; [Fact] public async Task SignalWorkflow_SignalTimeout_HasExpectedResult() { await using var env = await WorkflowEnvironment.StartTimeSkippingAsync(); using var worker = new TemporalWorker( env.Client, new TemporalWorkerOptions($"task-queue-{Guid.NewGuid()}"). AddWorkflow<SignalWorkflow>()); await worker.ExecuteAsync(async () => { var handle = await env.Client.StartWorkflowAsync( (SignalWorkflow wf) => wf.RunAsync(), new(id: $"wf-{Guid.NewGuid()}", taskQueue: worker.Options.TaskQueue!)); await env.DelayAsync(TimeSpan.FromSeconds(50)); Assert.Equal("got timeout", await handle.GetResultAsync()); }); } ``` This demonstrates manually skipping time using env.DelayAsync to test timeout behavior.

Mocking Activities in .NET Workflow tests

When testing Workflows, you can mock Activities instead of actually running them. Activities are just methods with the [Activity] attribute. Simply write different/empty/fake/asserting Activities and pass those to the Worker to have different activities called during the test.

Testing Activities with ActivityEnvironment

Unit testing an Activity or any code that could run in an Activity is done via the Temporalio.Testing.ActivityEnvironment class. Simply instantiate the class, and any code inside RunAsync will be invoked inside the activity context.

ActivityEnvironment members for testing

ActivityEnvironment provides the following important members to affect the activity context: Info (Activity info, defaulted to a basic set of values), Logger (Activity logger, defaulted to a null logger), Cancel(CancelReason) (Helper to set the reason and cancel the source), CancelReason (Cancel reason), CancellationTokenSource (Token source for issuing cancellation), Heartbeater (Callback invoked each heartbeat), WorkerShutdownTokenSource (Token source for issuing Worker shutdown), and PayloadConverter (Defaulted to default payload converter).

.NET Workflow Replay test example from JSON

Example of replaying a Workflow history from JSON: ```csharp using Temporalio; using Temporalio.Worker; public static async Task ReplayFromJsonAsync(string historyJson) { var replayer = new WorkflowReplayer( new WorkflowReplayerOptions().AddWorkflow<MyWorkflow>()); await replayer.ReplayWorkflowAsync(WorkflowHistory.FromJson("my-workflow-id", historyJson)); } ``` If there is a non-determinism, this will throw an exception.

.NET Workflow Replay test example for multiple histories

Example of checking that all Workflow histories for a certain Workflow type are safe with current Workflow code: ```csharp using Temporalio; using Temporalio.Client; using Temporalio.Worker; public static async Task CheckPastHistoriesAsync(ITemporalClient client) { var replayer = new WorkflowReplayer( new WorkflowReplayerOptions().AddWorkflow<MyWorkflow>()); var listIter = client.ListWorkflowHistoriesAsync("WorkflowType = 'SayHello'"); await foreach (var result in replayer.ReplayWorkflowsAsync(listIter)) { if (result.ReplayFailure != null) { ExceptionDispatchInfo.Throw(result.ReplayFailure); } } } ``` This fetches multiple Workflow histories and replays them all to check for non-determinism.

Three types of automated tests in Temporal

Temporal applications can be tested with three types of automated tests: End-to-end (running a Temporal Server and Worker with all Workflows and Activities, and interacting with Workflows from a Client), Integration (anything between end-to-end and unit testing, such as running Activities with mocked Context, running Workers with mock Activities using a Client, or running Workflows with mocked SDK imports), and Unit (running a piece of Workflow or Activity code and mocking any code it calls). Integration tests are generally recommended as the majority of tests.

Ruby SDK compatible test frameworks

The Ruby SDK is compatible with any testing framework and does not have a specific recommendation. Most Ruby SDK samples use minitest.

WorkflowEnvironment.start_local for testing

A non-time-skipping Temporalio::Testing::WorkflowEnvironment can be started via start_local, which supports all standard Temporal features. It is the real Temporal dev server packaged in the Temporal CLI, lazily downloaded on first use and run as a sub-process in the background. The same server can and should be reused across tests when tests properly use separate Task Queues.

WorkflowEnvironment.start_time_skipping for time-based testing

A time-skipping Temporalio::Testing::WorkflowEnvironment can be started via start_time_skipping, which is a reimplementation of the Temporal server with special time skipping capabilities. Unlike start_local, this class is not thread safe nor safe for use with independent tests. It can be reused but only for one test at a time because time skipping is locked/unlocked at the environment level. Developers are encouraged to run it per test needed.

Automatic time skipping in time-skipping test environment

The time-skipping server automatically skips to the next event when waiting on a Workflow result. When calling execute_workflow on the client, it performs start_workflow + result, and result automatically skips time as much as it can, basically until the end of the workflow or until an activity is run. To disable automatic time-skipping while waiting for a workflow result, run code in a block passed to env.auto_time_skipping_disabled.

Manual time skipping in time-skipping environment

Until a Workflow is waited on, all time skipping in the time-skipping environment is done manually via WorkflowEnvironment#sleep. This method can be called to advance time in the test environment when testing scenarios like timeouts that depend on time advancement.

Mocking Activities in Workflow tests

When testing Workflows, Activities can be mocked by writing different, empty, fake, or asserting versions of them. Activities are just classes that extend Temporalio::Activity::Definition. Pass the mock Activity classes to the Worker to have different activities called during the test.

Unit testing Activities with ActivityEnvironment

Unit testing an Activity or any code that could run in an Activity is done via the Temporalio::Testing::ActivityEnvironment class. Instantiate the class and any code inside the block to run will be invoked inside the activity context. Several things about the activity environment can be customized via parameters when constructing the environment, including setting the info, providing a proc to call back on each heartbeat, and setting the cancellation to be used.

Replaying Workflow history to check for non-determinism

Given a Workflow's history, it can be replayed locally to check for non-determinism errors. Create a Temporalio::Worker::WorkflowReplayer with the workflow class, create a Temporalio::WorkflowHistory from the JSON history, and call replayer.replay_workflow(history). If there is a non-determinism, this will raise an exception. Event history can be loaded from JSON, fetched individually from a Workflow handle, or fetched in a list using replayer.replay_workflows().

Example: Simple Workflow test with start_local

```ruby def test_simple_workflow Temporalio::Testing::WorkflowEnvironment.start_local do |env| worker = Temporalio::Worker.new( env.client, task_queue: "tq-#{SecureRandom.uuid}", workflows: [SimpleWorkflow] ) worker.run do result = env.client.execute_workflow( SimpleWorkflow, 'some-name', id: "wf-#{SecureRandom.uuid}", task_queue: worker.task_queue ) assert_equal 'Hello, some-name!', result end end end ``` This demonstrates testing a simple workflow using a local test server with minitest.

Example: Workflow test with time skipping

```ruby def test_wait_a_day_workflow Temporalio::Testing::WorkflowEnvironment.start_time_skipping do |env| worker = Temporalio::Worker.new( env.client, task_queue: "tq-#{SecureRandom.uuid}", workflows: [WaitADayWorkflow] ) worker.run do result = env.client.execute_workflow( WaitADayWorkflow, id: "wf-#{SecureRandom.uuid}", task_queue: worker.task_queue ) assert_equal 'all done', result end end end ``` This demonstrates testing a workflow with a long sleep using automatic time skipping, which makes the test run almost instantly.

Example: Testing workflow signal with timeout using manual time skipping

```ruby def test_signal_workflow_timeout Temporalio::Testing::WorkflowEnvironment.start_time_skipping do |env| worker = Temporalio::Worker.new( env.client, task_queue: "tq-#{SecureRandom.uuid}", workflows: [SignalWorkflow] ) worker.run do handle = env.client.start_workflow( SignalWorkflow, id: "wf-#{SecureRandom.uuid}", task_queue: worker.task_queue ) env.sleep(50) assert_equal 'got timeout', handle.result end end end ``` This demonstrates testing the timeout path of a workflow by manually advancing time 50 seconds with env.sleep().

Example: Replaying Workflow history from JSON

```ruby def replay_from_json(history_json) replayer = Temporalio::Worker::WorkflowReplayer.new(workflows: [MyWorkflow]) history = Temporalio::WorkflowHistory.from_history_json(history_json) replayer.replay_workflow(history) end ``` This shows how to replay a single workflow history from JSON exported from the CLI or web UI to check for non-determinism errors.

Example: Replaying all Workflow histories of a type

```ruby replayer = Temporalio::Worker::WorkflowReplayer.new(workflows: [MyWorkflow]) replayer.replay_workflows(client.list_workflows("WorkflowType = 'MyWorkflow'")).each do |result| raise result.replay_failure if result.replay_failure end ``` This demonstrates checking that all Workflow histories for a certain Workflow type are safe with the current Workflow code by iterating over replay results.

Test server support for time skipping in Ruby SDK

The Ruby test server supports skipping time, so it should be used for both end-to-end and integration tests with Workers.

Give your agent this brain