dbt deploy action for deploying dbt code to Astro

Use pre-built Astronomer CI/CD templates to automate deploying dbt code to Astro using GitHub Actions.

There is a hard limit of 10 dbt bundles per Astro Deployment

If you have a dbt project that you want to use with the deploy action for a GitHub action, you can choose to use either the dbt deploy on its own, or you can create GitHub actions that combine the infer action with a dbt deploy.

Prerequisites

• An Astro project hosted in a GitHub repository.
• An Astro Deployment.
• A Deployment API token, Workspace API token, or Organization API token.
• Access to GitHub Actions.

Each CI/CD template implementation might have additional requirements.

If you use a self-hosted runner to execute jobs from GitHub Actions, the Astro CLI’s config.yaml file, which stores default deploy details, might be shared across your organization and hence multiple CI/CD pipelines. To reduce the risk of accidentally deploying to the wrong Deployment, ensure the following:

• Add ASTRO_API_TOKEN to your repository and include a check in your GitHub workflow to verify that it exists.
• Use Deployment API tokens, which are scoped only to one Deployment, instead of Workspace or Organization API tokens.
• Specify deployment-id or deployment-name in your action. For example, astro deploy <deployment-id> or astro deploy -n <deployment-name>.
• Add the command astro logout at the end of your workflow to ensure that your authentication token is cleared from the config.yaml file

Setup

Single branch

Prerequisites

• The root folder name for the directory that contains your dbt project.

Implementation

To automate code deploys to a single Deployment using GitHub Actions for a dbt project, complete the following setup in a Git-based repository that hosts an Astro project:

1. Set the following as a GitHub secret:

• ASTRO_API_TOKEN: The value for your Workspace or Organization API token.

2. In your project repository, create a new YAML file in .github/workflows that includes the following configuration:

name: Astronomer CI - Deploy dbt code

on:
  push:
    branches:
      - main

env:
  ## Sets Deployment API credentials as environment variables
  ASTRO_API_TOKEN: ${{ secrets.ASTRO_API_TOKEN }}

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
    - name: Deploy to Astro
      uses: astronomer/deploy-action@v0.13.0
      with:
            deployment-id: <deployment id>
            deploy-type: dbt
            root-folder: <dbt-root-folder>

3. (Optional) You can add optional configurations to customize your workflow. For example, if you add wake-on-deploy to your configuration, the Deploy Action wakes a hibernating Deployment before deploying code to it.

Using

wake-on-deploy

takes precedence over any existing Deployment hibernation overrides that you configured through the Astro UI or

config.yaml

file.

Multiple branch

The following template can be used to create a multiple branch CI/CD pipeline using GitHub Actions. A multiple branch pipeline can be used to test dags in a development Deployment and promote them to a production Deployment.

Configuration requirements

• You have both a dev and main branch of an Astro project hosted in a single GitHub repository.
• You have respective dev and prod Deployments on Astro where you deploy your GitHub branches to.
• You have at least one API token with access to both of your Deployments.
• The root folder name for the directory that contains your dbt project.

Implementation

1. Set the following as GitHub secrets:

• PROD_ASTRO_API_TOKEN: The value for your production Workspace or Organization API token.
• DEV_ASTRO_API_TOKEN: The value for your development Workspace or Organization API token.

2. In your project repository, create a new YAML file in .github/workflows that includes the following configuration:

name: Astronomer CI - Deploy dbt code (Multiple Branches)

on:
  push:
    branches: [dev]
  pull_request:
    types:
      - closed
    branches: [main]

jobs:
  dev-push:
    if: github.ref == 'refs/heads/dev'
    env:
      ## Sets DEV Deployment API credential as an environment variable
      ASTRO_API_TOKEN: ${{ secrets.DEV_ASTRO_API_TOKEN }}
    runs-on: ubuntu-latest
    steps:
    - name: Deploy to Astro
      uses: astronomer/deploy-action@v0.13.0
      with:
        deployment-id: <dev-deployment-id>
        deploy-type: dbt
        root-folder: <dbt-root-folder>
  prod-push:
    if: github.event.action == 'closed' && github.event.pull_request.merged == true
    env:
      ## Sets Prod Deployment API credential as an environment variable
      ASTRO_API_TOKEN: ${{ secrets.PROD_ASTRO_API_TOKEN }}
    runs-on: ubuntu-latest
    steps:
    - name: Deploy to Astro
      uses: astronomer/deploy-action@v0.13.0
      with:
        deployment-id: <prod-deployment-id>
        deploy-type: dbt
        root-folder: <dbt-root-folder>

3. (Optional) You can add optional configurations to customize your workflow. For example, if you add wake-on-deploy to your configuration, the Deploy Action wakes a hibernating Deployment before deploying code to it.

Using

wake-on-deploy

takes precedence over any existing Deployment hibernation overrides that you configured through the Astro UI or

config.yaml

file.

Combine dbt and Astro project

In addition to configuring the deploy action to deploy just your dbt changes, you can also configure the action to do a dbt deploy and an infer deploy.

Prerequisites

• The root folder name for the directory that contct.ains your dbt project

Implementation

To automate code deploys to a single Deployment using GitHub Actions for a dbt project, complete the following setup in a Git-based repository that hosts both an Astro project and a dbt project:

1. Set the following as a GitHub secret:

• ASTRO_API_TOKEN: The value for your Workspace or Organization API token.

2. In your project repository, create a new YAML file in .github/workflows that includes the following configuration:

name: Astronomer CI - Deploy dbt and Astro project code

on:
  push:
    branches:
      - main

env:
  ## Sets Deployment API credentials as environment variables
  ASTRO_API_TOKEN: ${{ secrets.ASTRO_API_TOKEN }}

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
    - name: DBT Deploy to Astro
      uses: astronomer/deploy-action@v0.13.0
      with:
        deployment-id: <deployment id>
        deploy-type: dbt
        root-folder: <dbt-root-folder>
    - name: DAGs/Image Deploy to Astro
      uses: astronomer/deploy-action@v0.13.0
      with:
        deployment-id: <deployment id>
        root-folder: <astro-project-root-folder>
        parse: true

3. (Optional) You can add optional configurations to customize your workflow. For example, if you add wake-on-deploy to your configuration, the Deploy Action wakes a hibernating Deployment before deploying code to it.

Using

wake-on-deploy

takes precedence over any existing Deployment hibernation overrides that you configured through the Astro UI or

config.yaml

file.