Certificate management - Temporal Cloud feature guide

Temporal Cloud requires security certificates for secure access and communication.

Temporal Cloud access is secured by the mutual Transport Layer Security (mTLS) protocol, which requires a CA certificate from the user.

A Worker Process requires a CA certificate and private key to connect to Temporal Cloud. Temporal Cloud does not require an exchange of secrets; only the certificates produced by private keys are used for verification.

Don't let your certificates expire

An expired root CA certificate invalidates all downstream certificates.

An expired end-entity certificate prevents a Temporal Client from connecting to a Namespace or starting a Workflow Execution. If the client is on a Worker, any current Workflow Executions that are processed by that Worker either run indefinitely without making progress until the Worker resumes or fail because of timeouts.

To update certificates, see How to add, update, and remove certificates in a Temporal Cloud Namespace.

All certificates used by Temporal Cloud must meet the following requirements.

Requirements for CA certificates in Temporal Cloud

Certificates provided to Temporal for your Namespaces must meet the following requirements.

CA certificates

A CA certificate is a type of X.509v3 certificate used for secure communication and authentication. In Temporal Cloud, CA certificates are required for configuring mTLS.

CA certificates must meet the following criteria:

• The certificates must be X.509v3.
• Each certificate in the bundle must be either a root certificate or issued by another certificate in the bundle.
• Each certificate in the bundle must include CA: true.
• A certificate cannot be a well-known CA (such as DigiCert or Let's Encrypt) unless the user also specifies certificate filters.
• The signing algorithm must be either RSA or ECDSA and must include SHA-256 or stronger message authentication. SHA-1 and MD5 cannot be used.
• The certificates cannot be generated with a passphrase.

End-entity certificates

An end-entity certificate is a type of X.509v3 certificate used by clients to authenticate themselves. Temporal Cloud lets you limit access to specific end-entity certificates by using certificate filters.

An end-entity (leaf) certificate must meet the following criteria:

• The certificate must be X.509v3.
• Basic constraints must include CA: false.
• The key usage must include Digital Signature.
• The signing algorithm must be either RSA or ECDSA and must include SHA-256 or stronger message authentication. SHA-1 and MD5 cannot be used.

When a client presents an end-entity certificate, and the whole certificate chain is constructed, each certificate in the chain (from end-entity to the root) must have a unique Distinguished Name.

caution

Distinguished Names are not case sensitive; that is, uppercase letters (such as ABC) and lowercase letters (such as abc) are equivalent.

How to issue root CA and end-entity certificates

Temporal Cloud authenticates a client connection by validating the client certificate against one or more CA certificates that are configured for the specified Namespace.

Choose one of the following options to generate and manage the certificates:

Option 1: You already have certificate management infrastructure

If you have existing certificate management infrastructure that supports issuing CA and end-entity certificates, it satisfies the requirements. When you configure the client SDK, you need to present a complete certificate chain up to the CA certificate given to Temporal.

Option 2: You don't have certificate management infrastructure

If you don't have an existing certificate management infrastructure, issue the CA and end-entity certificates using tcld or open source tools such as certstrap.

info

A certificate bundle can contain up to 16 CA certificates. A certificate bundle can have a maximum payload size of 32 KB before base64 encoding.

Use tcld to generate certificates

You can generate CA and end-entity certificates by using tcld.

Although Temporal Cloud supports long-lived CA certificates, a CA certificate generated by tcld has a maximum duration of 1 year (-d 1y). You must set an end-entity certificate to expire before its root CA certificate, so specify its duration appropriately.

To create a new CA certificate, use tcld gen ca.

mkdir temporal-certs
cd temporal-certs
tcld gen ca --org temporal -d 1y --ca-cert ca.pem --ca-key ca.key

The contents of the generated ca.pem should be pasted into the "CA Certificates" section of your Namespace settings page.

To create a new end-entity certificate, use tcld gen leaf.

tcld gen leaf --org temporal -d 364d --ca-cert ca.pem --ca-key ca.key --cert client.pem --key client.key

Use certstrap

Temporal Cloud requires client certificates for authentication and secure communication. Certstrap is a popular and easy-to-use tool for issuing certificates.

Before you begin, ensure you have installed Certstrap by following the instructions in the Certstrap README.

A Certificate Authority (CA) is a trusted entity that issues digital certificates. These certificates certify the ownership of a public key by the named subject of the certificate. End-entity certificates are issued and signed by a CA, and they are used by clients to authenticate themselves to Temporal Cloud.

Create a self-signed CA certificate and use it to issue an end-entity certificate for your Temporal Cloud namespace.

1. Create a Certificate Authority (CA)

Create a new Certificate Authority (CA) using Certstrap:

./certstrap init --common-name "Cert"

This command creates a self-signed CA certificate named Cert.crt in the out folder within the Certstrap directory. This CA certificate will be used to sign and issue end-entity certificates.

2. Set the Namespace Name

Set the Namespace Name as the common name for the end-entity certificate:

macOs

For Linux or macOS:

export NAMESPACE_NAME=your-namespace

Windows

For Windows:

set NAMESPACE_NAME=your-namespace

Replace your-namespace with the name of your Temporal Cloud namespace.

3. Request an End-Entity Certificate

Next, request a certificate with a common name equal to the Namespace Name:

./certstrap request-cert --common-name ${NAMESPACE_NAME}

This command creates a Certificate Signing Request (CSR) for an end-entity certificate, but not the actual certificate itself.

4. Sign the Certificate Request

Sign the certificate request and generate the end-entity certificate:

./certstrap sign ${NAMESPACE_NAME} --CA "Cert"

This command takes the CSR from the previous step and signs it with your CA (Cert). The result is an end-entity certificate (your-namespace.crt) that is now a valid certificate signed by your CA.

5. (optional) Convert to PKCS8 Format for Java SDK

If you are using the Temporal Java SDK, you will need to convert the PKCS1 file format to PKCS8 file format. Export the end-entity's private key to a PKCS8 file:

openssl pkcs8 -topk8 -inform PEM -outform PEM -in <out/your-namespace.key> -out <out/your-namespace.pkcs8.key> -nocrypt

6. Use the Certificates with Temporal Cloud

You can now use the generated client certificate (your-namespace.crt) and the CA certificate (Cert.crt) with Temporal Cloud. You will upload the contents of the Cert.crt file to the CA Certificates section of your Namespace settings.

Follow the instructions to upload the CA certificate and configure your client with the end-entity certificate.

How to control authorization for Temporal Cloud Namespaces

Because Temporal Cloud uses mTLS for authorization, we recommend that an end-entity certificate be scoped to a specific Namespace. Temporal Cloud requires full CA chains, so you can achieve authorization in two ways.

Option 1: Issue a separate root certificate for each Namespace

Each certificate must belong to a chain up to the root CA certificate. Temporal uses the root CA certificate as the trusted authority for access to your Namespaces.

1. Ensure that your certificates meet the certificate requirements.
2. Add client CA certificates to a Cloud Namespace.

Option 2: Use the same root certificate for all Namespaces but create a separate certificate filter for each Namespace

How to manage certificate filters in Temporal Cloud

How to receive notifications about certificate expiration

To keep your Namespace secure and online, you must update the CA certificate for the Namespace before the certificate expires.

To help you remember to do so, Temporal Cloud sends email notifications to users who have the Account Owner or Global Admin roles or the Namespace Admin permission 15 days before expiration and, if necessary, 10 days before expiration.

If the certificate is not updated, 5 days before expiration Temporal Cloud creates a support ticket on behalf of the Account Owner, Global Admins, and Namespace Admins.

To ensure that you receive email notifications, configure your junk-email filters to permit email from noreply@temporal.io.

After a support ticket is created, admins should expect a follow-up from the Temporal Developer Success team.

To change who receives certificate-expiration notifications for a Namespace (or to provide feedback about such notifications), create a support ticket.

How to add, update, and remove certificates in a Temporal Cloud Namespace

note

To manage certificates for a Namespace, a user must have Namespace Admin permission for that Namespace.

To manage certificates for Temporal Cloud Namespaces, use the Namespaces page in Temporal Cloud UI or the tcld namespace accepted-client-ca commands.

Don't let your certificates expire! Add reminders to your calendar to issue new CA certificates well before the expiration dates of the existing ones. Temporal Cloud begins sending notifications 15 days before expiration. For details, see the previous section (How to receive notifications about certificate expiration).

When updating CA certificates, it's important to follow a rollover process (sometimes referred to as "certificate rotation"). Doing so enables your Namespace to serve both CA certificates for a period of time until traffic to your old CA certificate ceases. This prevents any service disruption during the rollover process.

Be aware that the subject of the existing certificate and the subject of the new certificate must not be identical. One way to meet this requirement is to add a version or a date to the common name (CN).

Update certificates using Temporal Cloud UI

Updating certificates using the following strategy allows for a zero-downtime rotation of certificates.

1. On the left side of the window, select Namespaces.

2. Select the name of the Namespace to update.

3. In the top-right portion of the page for the Namespace, select Edit.

4. On the Edit page, select the CA Certificates card to expand it.

5. In the certificates box, scroll to the end of the existing certificate (that is, past -----END CERTIFICATE-----).

6. On the following new line, paste the entire PEM block of the new certificate.

7. Select Save.

8. Wait until all Workers are using the new certificate.

9. Return to the Edit page of the Namespace and select the CA Certificates card.

10. In the certificates box, delete the old certificate, leaving the new one in place.

11. Select Save.

Update certificates using tcld

Updating certificates using the following strategy allows for a zero-downtime rotation of certificates.

1. Create a single file that contains both your old and new CA certificate PEM blocks. Just concatenate the PEM blocks on adjacent lines.

   -----BEGIN CERTIFICATE-----
   ... old CA cert ...
   -----END CERTIFICATE-----
   -----BEGIN CERTIFICATE-----
   ... new CA cert ...
   -----END CERTIFICATE-----

2. Run the tcld namespace accepted-client-ca set command with the CA certificate bundle file.

   tcld namespace accepted-client-ca set --ca-certificate-file <path>

3. Monitor traffic to your old certificate until it ceases.

4. Create another file that contains only the new CA certificate.

5. Run the tcld namespace accepted-client-ca set command again with the updated CA certificate bundle file.

How to manage certificate filters in Temporal Cloud

To limit access to specific end-entity certificates, create certificate filters. Each filter contains values for one or more of the following fields:

• commonName (CN)
• organization (O)
• organizationalUnit (OU)
• subjectAlternativeName (SAN)

Corresponding fields in the client certificate must match every specified value in the filter.

The values for the fields are case-insensitive. If no wildcard is used, each specified value must match its field exactly.

To match a substring, place a single * wildcard at the beginning or end (but not both) of a value. You cannot use a * wildcard by itself.

You can create a maximum of 25 certificate filters in a Namespace.

If you provide a well-known CA certificate, you cannot clear a certificate filter. A well-known CA certificate is one that is typically included in the certificate store of an operating system.

Examples

In the following example, only the CN field of the certificate's subject is checked, and it must be exactly code.example.com. The other fields are not checked.

AuthorizedClientCertificate {
  CN : "code.example.com"
}

In the following example, the CN field must be stage.example.com and the O field must be Example Code Inc.

AuthorizedClientCertificate {
  CN : "stage.example.com"
  O : "Example Code Inc."
}

When using a * wildcard, the following values are valid:

• *.example.com matches code.example.com and text.example.com.
• Example Code* matches Example code and Example Code Inc.

The following values are not valid:

• .example.*
• code.*.com
• *

Manage certificate filters using Temporal Cloud UI

To add or remove a certificate filter, follow these steps:

1. On the left side of the window, click Namespaces.
2. On the Namespaces page, click the name of the Namespace to manage.
3. On the right side of the page for the selected Namespace, click Edit.
4. On the Edit page, click Certificate Filters.
   • To add a certificate filter, click Add a Certificate Filter and enter values in one or more fields.
   • To remove a certificate filter, click the × in the upper-right corner of the filter details.
5. To cancel your changes, click Back to Namespace. To save your changes, click Save.

Manage certificate filters using tcld

To set or clear certificate filters, use the following tcld commands:

• tcld namespace certificate-filters import
• tcld namespace certificate-filters clear

To view the current certificate filters, use the tcld namespace certificate-filters export command.