dbt deploy action for deploying dbt code to Astro
If you have a dbt project that you want to use with the deploy action for a GitHub action, you can choose to use either the dbt deploy on its own, or you can create GitHub actions that combine the infer
action with a dbt deploy.
Prerequisites
- An Astro project hosted in a GitHub repository.
- An Astro Deployment.
- A Deployment API token, Workspace API token, or Organization API token.
- Access to GitHub Actions.
Each CI/CD template implementation might have additional requirements.
If you use a self-hosted runner to execute jobs from GitHub Actions, the Astro CLI's config.yaml
file, which stores default deploy details, might be shared across your organization and hence multiple CI/CD pipelines. To reduce the risk of accidentally deploying to the wrong Deployment, ensure the following:
- Add
ASTRO_API_TOKEN
to your repository and include a check in your GitHub workflow to verify that it exists. - Use Deployment API tokens, which are scoped only to one Deployment, instead of Workspace or Organization API tokens.
- Specify
deployment-id
ordeployment-name
in your action. For example,astro deploy <deployment-id>
orastro deploy -n <deployment-name>
. - Add the command
astro logout
at the end of your workflow to ensure that your authentication token is cleared from theconfig.yaml
file.
Setup
- Single branch
- Multiple branch
- Combine dbt and Astro project
Prerequisites
- The root folder name for the directory that contains your dbt project.
Implementation
To automate code deploys to a single Deployment using GitHub Actions for a dbt project, complete the following setup in a Git-based repository that hosts an Astro project:
-
Set the following as a GitHub secret:
ASTRO_API_TOKEN
: The value for your Workspace or Organization API token.
-
In your project repository, create a new YAML file in
.github/workflows
that includes the following configuration:name: Astronomer CI - Deploy dbt code
on:
push:
branches:
- main
env:
## Sets Deployment API credentials as environment variables
ASTRO_API_TOKEN: ${{ secrets.ASTRO_API_TOKEN }}
jobs:
build:
runs-on: ubuntu-latest
steps:
- name: Deploy to Astro
uses: astronomer/deploy-action@v0.9.0
with:
deployment-id: <deployment id>
deploy-type: dbt
root-folder: <dbt-root-folder> -
(Optional) You can add optional configurations to customize your workflow. For example, if you add
wake-on-deploy
to your configuration, the Deploy Action wakes a hibernating Deployment before deploying code to it.
Using wake-on-deploy
takes precedence over any existing Deployment hibernation overrides that you configured through the Astro UI or config.yaml
file.
The following template can be used to create a multiple branch CI/CD pipeline using GitHub Actions. A multiple branch pipeline can be used to test DAGs in a development Deployment and promote them to a production Deployment.
Configuration requirements
- You have both a
dev
andmain
branch of an Astro project hosted in a single GitHub repository. - You have respective
dev
andprod
Deployments on Astro where you deploy your GitHub branches to. - You have at least one API token with access to both of your Deployments.
- The root folder name for the directory that contains your dbt project.
Implementation
-
Set the following as GitHub secrets:
PROD_ASTRO_API_TOKEN
: The value for your production Workspace or Organization API token.DEV_ASTRO_API_TOKEN
: The value for your development Workspace or Organization API token.
-
In your project repository, create a new YAML file in
.github/workflows
that includes the following configuration:name: Astronomer CI - Deploy dbt code (Multiple Branches)
on:
push:
branches: [dev]
pull_request:
types:
- closed
branches: [main]
jobs:
dev-push:
if: github.ref == 'refs/heads/dev'
env:
## Sets DEV Deployment API credential as an environment variable
ASTRO_API_TOKEN: ${{ secrets.DEV_ASTRO_API_TOKEN }}
runs-on: ubuntu-latest
steps:
- name: Deploy to Astro
uses: astronomer/deploy-action@v0.9.0
with:
deployment-id: <dev-deployment-id>
deploy-type: dbt
root-folder: <dbt-root-folder>
prod-push:
if: github.event.action == 'closed' && github.event.pull_request.merged == true
env:
## Sets Prod Deployment API credential as an environment variable
ASTRO_API_TOKEN: ${{ secrets.PROD_ASTRO_API_TOKEN }}
runs-on: ubuntu-latest
steps:
- name: Deploy to Astro
uses: astronomer/deploy-action@v0.9.0
with:
deployment-id: <prod-deployment-id>
deploy-type: dbt
root-folder: <dbt-root-folder> -
(Optional) You can add optional configurations to customize your workflow. For example, if you add
wake-on-deploy
to your configuration, the Deploy Action wakes a hibernating Deployment before deploying code to it.
Using wake-on-deploy
takes precedence over any existing Deployment hibernation overrides that you configured through the Astro UI or config.yaml
file.
In addition to configuring the deploy action to deploy just your dbt changes, you can also configure the action to do a dbt deploy
and an infer
deploy.
Prerequisites
- The root folder name for the directory that contct.ains your dbt project
Implementation
To automate code deploys to a single Deployment using GitHub Actions for a dbt project, complete the following setup in a Git-based repository that hosts both an Astro project and a dbt project:
-
Set the following as a GitHub secret:
ASTRO_API_TOKEN
: The value for your Workspace or Organization API token.
-
In your project repository, create a new YAML file in
.github/workflows
that includes the following configuration:name: Astronomer CI - Deploy dbt and Astro project code
on:
push:
branches:
- main
env:
## Sets Deployment API credentials as environment variables
ASTRO_API_TOKEN: ${{ secrets.ASTRO_API_TOKEN }}
jobs:
build:
runs-on: ubuntu-latest
steps:
- name: DBT Deploy to Astro
uses: astronomer/deploy-action@v0.9.0
with:
deployment-id: <deployment id>
deploy-type: dbt
root-folder: <dbt-root-folder>
- name: DAGs/Image Deploy to Astro
uses: astronomer/deploy-action@v0.9.0
with:
deployment-id: <deployment id>
root-folder: <astro-project-root-folder>
parse: true -
(Optional) You can add optional configurations to customize your workflow. For example, if you add
wake-on-deploy
to your configuration, the Deploy Action wakes a hibernating Deployment before deploying code to it.
Using wake-on-deploy
takes precedence over any existing Deployment hibernation overrides that you configured through the Astro UI or config.yaml
file.