Skip to main content

Documentation Index

Fetch the complete documentation index at: https://astronomer-preview.mintlify.app/llms.txt

Use this file to discover all available pages before exploring further.

This is feature is only available if you are on theEnterprisetier or above. SeeAstro Plans and Pricing.
Airflow 3This feature is only available for Airflow 3.x Deployments.
Remote Execution Deployments require two sets of container images:
  • Astro Runtime images: Astronomer’s managed distribution of Apache Airflow, powering all Deployments on Astro.
  • Remote Execution Agent images: Specifically built for Remote Execution Agents.
When using Remote Execution mode, the server (Deployment on Astro) and client (Remote Execution Agents) are deployed separately. You must maintain dedicated client images for the Remote Execution Agents. These client images are based on the Astro Remote Execution Agent image and must be pushed to a remote image registry accessible by your Agents. Because these client images are distinct from the usual Astro Runtime images, you need to use dedicated commands to build and deploy them. This guide explains how to initialize an Astro project for Remote Execution and how to build and push client images for Remote Execution Agents.
For details about Astro project image deploys with Remote Execution, see Deploy an Astro project.For best practices and procedures to upgrade your Astro Runtime version, including for Remote Execution Deployments, see Upgrade Astro Runtime.

​
Prerequisites

1
Create a Remote Execution project
2
Run:
3
astro dev init my-remote-project --remote-execution-enabled --remote-image-repository <remote-image-repository-url>
4
This generates the required files for building client images. If you omit --remote-image-repository, you can configure it later with:
5
astro config set remote.client_registry <remote-image-repository-url>
6
Review generated files
7
  • Dockerfile.client – Dockerfile for client images
  • requirements-client.txt – Python dependencies for client images
  • packages-client.txt – OS-level packages for client images
    • 8
    Configure dag access
    9
    Set up dag access for your Remote Execution Agents using either LocalDagBundle or GitDagBundle. Update your Agent’s Helm chart values.yaml accordingly. See Dag bundles.
    10
    Log in to your remote Docker registry
    11
    Before you can deploy a client image, you must be logged in to your remote Docker registry. The Astro CLI does not manage authentication for your registry.
    12
    Log in using:
    13
    docker login <remote-registry> --username <registry-username> --password-stdin
    14
    Replace <remote-registry> and <registry-username> with the details for your registry.
    15
    Build and deploy the Astro Deployment (server) image
    16
    From your project directory, deploy the Astro Runtime image to your Remote Execution Deployment:
    17
    astro deploy
    18
    This command builds your project and pushes the updated Astro Runtime image to Astro, updating the Airflow components in your managed Astro Deployment.
    19
    Monitor the Deployment to confirm that the new version is running in the Astro UI.
    20
    The astro deploy command updates the orchestration plane only. To update the execution plane, follow the Remote Execution Agent image steps below.
    21
    Build and deploy Remote Execution Agent (client) images
    22
    From your project directory, run:
    23
    astro remote deploy
    24
    You can use additional flags as needed, such as:
    25
    astro remote deploy --platform linux/amd64
astro remote deploy --platform linux/amd64,linux/arm64
astro remote deploy --image-name my-custom-image:tag
astro remote deploy --build-secrets id=mysecret,src=secrets.txt
    26
    Platform mismatch can cause build failuresBy default, astro remote deploy builds an image for your computer’s platform. For example, if you are on a Mac with Apple silicon, it will build for linux/arm64. If your Agents need linux/amd64, the image may not work.To avoid issues, always set the --platform flag: astro remote deploy --platform linux/amd64,linux/arm64
    27
    This command only pushes the client image to your remote registry. See astro remote for more information.
    28
    Update Remote Execution Agents to use the new image
    29
    After pushing the image, you must update your Agent configuration to actually use it:
    30
  • Edit your Agent Helm chart values.yaml.
  • Update the image field for each of the worker, dagProcessor, and triggerer component sections to the new client image reference (as just pushed).
  • Apply the changes by running:
    • 31
    helm upgrade <release-name> <chart-path> -f values.yaml
    32
    Agents will not use the new client image until this update is performed.
    33
    If you manage your Agent deployments via a CI/CD pipeline, update your pipeline to set the new client image in the Helm values.yaml file, and trigger the deployment process to roll out the change automatically. This ensures that updates to the client image are reliably applied across all agents without manual intervention.