Remote Execution Agent maintenance policy

Remote Execution Agent maintenance policy

Airflow 3

This feature is only available for Airflow 3.x Deployments.

Remote Execution Agent versioning

Astro Remote Execution Agent versions are released regularly and use semantic versioning. Astronomer ships major, minor, and patch releases of the Remote Execution Agent in the format of major.minor.patch Each Remote Execution Agent minor version is maintained for 6 months from the release.

• Major versions are released for significant feature additions. Major versions are not guaranteed to be backward compatible.
• Minor versions are released for functional changes. Minor releases are backward compatible.
• Patch versions are released for bug and security fixes that resolve unwanted behavior. Patch releases are backward compatible.

Bug fixes are delivered through new minor.patch versions. If you report an issue with a maintained Astro Runtime image that is not on the latest minor.patch version, Astronomer Support may ask you to upgrade to see if it resolves the issue.

You can find full information about releases in the Remote Execution Agent release notes.

Image tag naming conventions

The following table describes the naming conventions for the image tags, allowing you to specify particular versions or allow your environment to use the latest options. Astronomer recommends using a fixed tag, with the versions for the Runtime, Python, and Remote Execution Agent explicitly defined.

Tag Format	Description
<runtime_version>-python-<python_version>-astro-agent-x.y.z	Recommended configuration format. Fixed tag with specific Remote Execution Agent and Python versions
<runtime_version>	Floating tag pointing to latest Remote Execution Agent and Python versions
<runtime_version>-python-<python_version>	Floating tag pointing to latest Remote Execution Agent versions
<runtime_version>-base	Base image without ONBUILD support
<runtime_version>-python-<python_version>-base	Floating tag for base image without ONBUILD support
<runtime_version>-python-<python_version>-astro-agent-x.y.z-base	Fixed tag with specific Remote Execution Agent and Python versions for base image

Remote Execution Agent images are not currently compatible with the RHEL platform.

Helm Chart Versioning

Remote Execution Agent Helm chart versions follow the Remote Execution Agent versioning, which means that every Agent release has a corresponding reference Helm chart release, so that the versioning is coupled. This allows you to know which Remote Execution Agent Helm chart version to upgrade to for a particular version of the Agent.

Upgrade Considerations

The Remote Execution Agent is distributed as a Docker image through the Astro control plane registry, images.astronomer.cloud, and includes a Runtime image with multi-Python version support for the Agent and your Dag processor. This allows you to run an image in the Execution Plane with all requirements for running Dag code and the program used by the Agent. It also means that there are three foundational components in your execution plane that can be upgraded to ensure your Remote Execution Agent works with the most up-to-date versions of Airflow and Astro resources:

• Remote Execution Agent version
• Runtime Version, in your Orchestration Plane/Astro UI
• Runtime Version, in your Execution Plane/Agent image
• Python Version

The remote execution agents in the Execution Plane must always use an image with a Runtime Version that is less than or equal to the Astro Runtime version in the Orchestration Plane. In general, Astro Runtime versions have backwards compatibility for Remote Execution versions. If an incompatibility exists, you can find them listed in Version upgrade considerations.

Astronomer recommends upgrading your Orchestration plane, your Astro Runtime, and Execution plane, your Remote Execution Agent, separately.

For upgrading the Execution Plane, Astronomer also recommends upgrading the agent version, the Astro Runtime version, and the Python version individually.

Remote Execution Agent image upgrade process

1. Update your Astro project Dockerfile with the new version of Astro Remote Execution Agent image.

2. Build your image and publish to your image registry.

3. Update your Remote Execution Agent’s Helm values.yaml file with the location of your new image in your image registry for the following parameters. If you want to use the same image for all components, you can specify the image at the top level and it will be used for all components. If you want to use different images for different components, for example for each worker queue, you can specify the image for each component individually to override the top-level image.

   Single image

   image: <image-url>
   

   Per-component image overrides

   image: <image-url> # Default image
   dagProcessor:
     image: <image-url-override>
   triggerer:
     image: <image-url-override>
   workers:
     - name: ...
       image: <image-url-override>
     - name: ...
       image: <image-url-override>
   

4. Run the following helm commands to upgrade your installation:

   helm repo update
   helm upgrade astro-agent astronomer/astro-remote-execution-agent \
       --namespace astro-agent \
       --values values.yaml
   
   If you want to use a specific version of the Remote Execution Agent Helm chart, you can specify it with the --version flag in the helm upgrade command. If you don’t specify a version, it will upgrade using the latest available chart version after running helm repo update.

Version upgrade considerations

The following sections include upgrade considerations for specific Astro Remote Execution Agent versions. This includes breaking changes, database migrations, incompatibilities, and other considerations.

If a version isn’t included in this section, then there are no specific upgrade considerations for that version.

Remote Execution Agent 1.0.0

The Remote Execution Agent based on Airflow 3.0-1 has a known incompatibility with the Astro Runtime 3.0-2. Do not upgrade to or create Remote Deployments that use the combination of an Orchestration Plane version 3.0-2 and Remote Execution Agent version using Runtime 3.0-1.

• Roll back to previous deploys
• Roll back Deployments after a broken upgrade