Migrate to Airflow 3 from Airflow 2 in Astro Private Cloud

Migrate to Airflow 3.x from Airflow 2.x

This guide describes the steps required to migrate your Airflow deployments from Airflow 2 (AF2) to Airflow 3 (AF3) in Astro Private Cloud 1.0.

1. Enable Airflow 3 on the cluster

1. Navigate to the Cluster Configuration Override page in the Astro Private Cloud UI.
2. Enable the Airflow 3 flag:

astronomer:
    houston:
        deployments:
            airflowV3:
                enabled: true

3. Enable the mandatory Dag Processor component for AF3:

astronomer:
    houston:
        deployments:
            dagProcessorEnabled: true

global:
    loggingSidecar:
        enabled: true

Airflow 3 requires sidecar logging. Deployments without this component will not be able to view task logs.

2. Configure Registry access

1. Allowlist the Azure Container Registry (azurecr) repository and URL for pulling Airflow 3 images.
2. If using third-party registries, configure mirroring to ensure reliable image pulls.

AF3 images are hosted at

astrocrpublic.azurecr.io

.

3. Pre-migration Dag compatibility check

Before upgrading, follow Astronomer’s Airflow 2 → 3 upgrade guide to verify that your Dags are compatible with Airflow 3.

4. Update Deployment Dockerfile

1. Modify your Dockerfile to use the Airflow 3 image and updated registry URL:

FROM astrocrpublic.azurecr.io/runtime:3.1-2

2. Build and push your updated Docker image.
   AF2 images are available on Quay (quay.io/astronomer/airflow), while AF3 images are hosted on astrocr (astrocrpublic.azurecr.io).

5. Update Deployment

Perform the Deployment update using one of the following methods:

1. UI: Navigate to your Deployment and trigger an update with the new AF3 image.
2. API: Use the UpsertDeployment API to update the Deployment image.
3. CLI: Use the Astro CLI to deploy the updated image.

Ensure the updated Deployment reflects the Airflow 3 runtime version.

6. Post-migration validation

• Confirm that Dags are running as expected in Airflow 3.
• Monitor logs and metrics to ensure stability.
• Verify that the Dag Processor component is active and functioning.

This guide provides a safe and supported path to migrate from Airflow 2 to Airflow 3 in Astro Private Cloud 1.0.