Set up custom timetable support for Remote Execution

Configure the scheduler to access your secrets backend when using custom timetables on Remote Execution.

Airflow 3

This feature is only available for Airflow 3.x Deployments.

When Dags use custom timetables that connect to external data sources, such as querying Snowflake for scheduling metadata, the scheduler must retrieve connection credentials from your secrets backend at schedule time.

When you configure Customer Managed Identity for a Remote Execution Deployment, the setup only authorizes the apiserver to access your cloud resources for reading task logs. To support custom timetables, the scheduler also needs authorization to access your secrets backend.

Without this configuration, the scheduler can’t retrieve connections from the secrets backend, causing errors like:

airflow.exceptions.AirflowNotFoundException: The conn_id `<connection-id>` isn't defined

Prerequisites

• A Remote Execution Deployment with Customer Managed Identity configured.
• A secrets backend configured for your Remote Execution Agent.

Authorize the scheduler

Extend your existing Customer Managed Identity configuration to include the scheduler service account. This is the same process used when you first configured workload identity for the apiserver.

AWS

No additional configuration is required. The default Customer Managed Identity setup for AWS uses a wildcard pattern in the IAM trust policy that authorizes all service accounts in the Deployment namespace, including the scheduler:

"<clusterOIDCIssuerUrl>:sub": "system:serviceaccount:<deployment-namespace>:*"

If you specified individual service accounts instead of using a wildcard, add the scheduler service account to your IAM trust policy:

"<clusterOIDCIssuerUrl>:sub": "system:serviceaccount:<deployment-namespace>:<deployment-namespace>-scheduler-serviceaccount"

GCP

The Customer Managed Identity setup for GCP authorizes only the apiserver by default. Run the following command to add the scheduler:

gcloud iam service-accounts add-iam-policy-binding \
  --role roles/iam.workloadIdentityUser \
  --member "serviceAccount:<gke-project-id>.svc.id.goog[<deployment-namespace>/<deployment-namespace>-scheduler-serviceaccount]" \
  <your-service-account>@<your-project>.iam.gserviceaccount.com \
  --project <your-project>

Replace the following values:

• <gke-project-id>: The GCP project ID of the GKE cluster running your Remote Execution Agent
• <deployment-namespace>: Your Deployment’s Kubernetes namespace
• <your-service-account>: The GCP service account configured as your Deployment’s Customer Managed Identity
• <your-project>: The GCP project containing your service account

Use the same values from the command you ran when configuring Customer Managed Identity for the apiserver.

Azure

The Customer Managed Identity setup for Azure creates federated identity credentials only for the apiserver by default. Run the following command to add the scheduler:

az identity federated-credential create \
  --name <deployment-namespace>-scheduler \
  --identity-name <managed-identity-name> \
  --resource-group <resource-group> \
  --issuer <aks-oidc-issuer-url> \
  --subject system:serviceaccount:<deployment-namespace>:<deployment-namespace>-scheduler-serviceaccount

Replace the following values:

• <deployment-namespace>: Your Deployment’s Kubernetes namespace
• <managed-identity-name>: Name of your user-assigned managed identity
• <resource-group>: Resource group containing your managed identity
• <aks-oidc-issuer-url>: The OIDC issuer URL for your AKS cluster, available in the Customer Managed Identity modal in the Astro UI

Use the same

--identity-name

,

--resource-group

, and

--issuer

values from the command you ran when configuring Customer Managed Identity for the apiserver.

Verify the configuration

After updating your workload identity configuration, verify that the scheduler can retrieve connections:

1. Check the scheduler logs for authentication errors. Cloud-specific errors such as AADSTS700213: No matching federated identity record found (Azure) or AccessDenied (AWS/GCP) should no longer appear.
2. Trigger a Dag that uses a custom timetable dependent on a connection from your secrets backend. The Dag should schedule without AirflowNotFoundException errors.

See also

• Authorize an Astro Deployment to cloud resources using workload identity
• Configure secrets backend for Remote Execution