Set up custom timetable support for Remote Execution

Airflow 3This feature is only available for Airflow 3.x Deployments.

When Dags use custom timetables that connect to external data sources, such as querying Snowflake for scheduling metadata, the scheduler must retrieve connection credentials from your secrets backend at schedule time. When you configure Customer Managed Identity for a Remote Execution Deployment, the setup only authorizes the apiserver to access your cloud resources for reading task logs. To support custom timetables, the scheduler also needs authorization to access your secrets backend. Without this configuration, the scheduler can’t retrieve connections from the secrets backend, causing errors like:

airflow.exceptions.AirflowNotFoundException: The conn_id `<connection-id>` isn't defined

Prerequisites

A Remote Execution Deployment with Customer Managed Identity configured.
A secrets backend configured for your Remote Execution Agent.

Authorize the scheduler

Extend your existing Customer Managed Identity configuration to include the scheduler service account. This is the same process used when you first configured workload identity for the apiserver.

AWS
GCP
Azure

No additional configuration is required. The default Customer Managed Identity setup for AWS uses a wildcard pattern in the IAM trust policy that authorizes all service accounts in the Deployment namespace, including the scheduler:

"<clusterOIDCIssuerUrl>:sub": "system:serviceaccount:<deployment-namespace>:*"

If you specified individual service accounts instead of using a wildcard, add the scheduler service account to your IAM trust policy:

"<clusterOIDCIssuerUrl>:sub": "system:serviceaccount:<deployment-namespace>:<deployment-namespace>-scheduler-serviceaccount"

The Customer Managed Identity setup for GCP authorizes only the apiserver by default. Run the following command to add the scheduler:

gcloud iam service-accounts add-iam-policy-binding \
  --role roles/iam.workloadIdentityUser \
  --member "serviceAccount:<gke-project-id>.svc.id.goog[<deployment-namespace>/<deployment-namespace>-scheduler-serviceaccount]" \
  <your-service-account>@<your-project>.iam.gserviceaccount.com \
  --project <your-project>

Replace the following values:

<gke-project-id>: The GCP project ID of the GKE cluster running your Remote Execution Agent
<deployment-namespace>: Your Deployment’s Kubernetes namespace
<your-service-account>: The GCP service account configured as your Deployment’s Customer Managed Identity
<your-project>: The GCP project containing your service account

Use the same values from the command you ran when configuring Customer Managed Identity for the apiserver.

The Customer Managed Identity setup for Azure creates federated identity credentials only for the apiserver by default. Run the following command to add the scheduler:

az identity federated-credential create \
  --name <deployment-namespace>-scheduler \
  --identity-name <managed-identity-name> \
  --resource-group <resource-group> \
  --issuer <aks-oidc-issuer-url> \
  --subject system:serviceaccount:<deployment-namespace>:<deployment-namespace>-scheduler-serviceaccount

Replace the following values:

<deployment-namespace>: Your Deployment’s Kubernetes namespace
<managed-identity-name>: Name of your user-assigned managed identity
<resource-group>: Resource group containing your managed identity
<aks-oidc-issuer-url>: The OIDC issuer URL for your AKS cluster, available in the Customer Managed Identity modal in the Astro UI

Use the same --identity-name, --resource-group, and --issuer values from the command you ran when configuring Customer Managed Identity for the apiserver.

Verify the configuration

After updating your workload identity configuration, verify that the scheduler can retrieve connections:

Check the scheduler logs for authentication errors. Cloud-specific errors such as AADSTS700213: No matching federated identity record found (Azure) or AccessDenied (AWS/GCP) should no longer appear.
Trigger a Dag that uses a custom timetable dependent on a connection from your secrets backend. The Dag should schedule without AirflowNotFoundException errors.

Overview

Get started

Develop DAGs

AI features

Deploy code

Manage Deployments

Automation & CI/CD

Platform observability

Administration

Release notes

Best practices

Reference

Airflow 3

Set up custom timetable support for Remote Execution

Prerequisites

Authorize the scheduler

Verify the configuration

See also

​Prerequisites

​Authorize the scheduler

​Verify the configuration

​See also

Prerequisites

Authorize the scheduler

Verify the configuration

See also