What is Temporal

Temporal is an open source, distributed and scalable workflow orchestration engine capable of running millions of workflows. Workflows can hold state and describe which activities (workflow tasks) should be carried out. Activities are distributed using task queues and executed on worker nodes organized in a cluster. Failure handling is taken off the hands of application developers and handled by the engine.

Temporal Architectural Design

Illustration of a Temporal Cluster The Temporal “system” consists of a Temporal Server (either the Temporal Cloud service or self-hosted) orchestrating work with a fleet of Temporal workers (operated by application developers), and Temporal Clients (embedded in applications) over gRPC. 

These workflows are carried out by the Temporal (or self-hosted) server which consists of four independently scalable services: 

• Frontend gateway (rate limiting, routing, authorizing) 

• History subsystem to maintains data (mutable state, queues, and timers)

• Matching subsystem to host task queues for dispatching

• Worker service to handle the internal background workflows

Deployment of Temporal Server

We deployed Temporal (server) using a custom helm chart, see this for Dev and this for Stage environment. Refer to the README.md file to see the changes in comparison to the official helm chart from Temporal git repo.

Temporal Persistence Layer 

Temporal supports Cassandra, MySQL, and PostgreSQL schemas and can be used as the server's database. The database stores the following types of data:

• Tasks to be dispatched

• The state of Workflow Executions

• The mutable state of Workflow Executions

• Event History, which provides an append-only log of Workflow Execution History Events

• Namespace metadata for each Namespace in the Cluster

• Visibility data, which enables operations like "show all running Workflow Executions"

For us, we chose GCP Cloud SQL Postgres database as temporal persistence layer. You may refer to this terraform code for the deployment of the database.

Frontend Service URLs - WEB GUI

Values.yaml

Add and update the TEMPORAL_URL in respective values.yaml as follows:

Monitoring

We enabled the serviceMonitor in our Temporal’s helm chart values.yaml which should expose its metrics in our Prometheus servers Thanos and shall reflect in Grafana.

Temporal has some dashboards available in their dashboards repo. This repository contains community-driven Grafana dashboards that can be used for monitoring Temporal Server and SDK metrics.

References