Create an Azure Blob Storage connection in Airflow

Learn how to create an Azure Blob Storage connection in Airflow.

Info

This page has not yet been updated for Airflow 3. The concepts shown are relevant, but some code may need to be updated. If you run any examples, take care to update import statements and watch for any other breaking changes.

Azure Blob Storage provides the storage for all of your Azure Storage data objects, including blobs, files, queues, and tables. Integrating your Azure storage account with Airflow lets you perform different kind of operations on blob objects stored in the cloud. For example, you can create or delete a container, upload or read a blob, or download blobs using Airflow.

This guide explains how to set up an Azure Blob Storage connection using the Azure Blob Storage connection type. Astronomer recommends using this connection type because it utilizes the wasb protocol, which means you can connect with any Azure Storage account including Azure Data Lake Gen 1 and Azure Data Lake Gen 2.

Prerequisites

• The Astro CLI.
• A locally running Astro project.
• An Azure storage account.
• Permissions to access blob data from your local Airflow environment.

Get connection details

To create an Azure Blob Storage connection in Airflow, you can use any of the following methods:

Create your connection

Info

Astro users can also create connections using the Astro Environment Manager, which stores connections in an Astro-managed secrets backend. These connections can be shared across multiple deployed and local Airflow environments. See Create Airflow connections in the Astro UI.

How it works

Airflow uses the Azure SDK for Python to connect to Azure services through the WasbHook.

See also

• Apache Airflow Microsoft Azure provider package documentation.
• Azure blob storage modules in the Airflow Registry.
• Import and export Airflow connections using Astro CLI.