Default deploy action for deploying code to Astro

By default, the deploy action uses the infer deploy type, which enables the action to determine whether to use either a dags-only deploy or an image-and-dags deploy, depending on the files you change.

Prerequisites

An Astro project hosted in a GitHub repository.
An Astro Deployment.
A Deployment API token, Workspace API token, or Organization API token.
Access to GitHub Actions.

Each CI/CD template implementation might have additional requirements.

If you use a self-hosted runner to execute jobs from GitHub Actions, the Astro CLI’s config.yaml file, which stores default deploy details, might be shared across your organization and hence multiple CI/CD pipelines. To reduce the risk of accidentally deploying to the wrong Deployment, ensure the following:

Add ASTRO_API_TOKEN to your repository and include a check in your GitHub workflow to verify that it exists.
Use Deployment API tokens, which are scoped only to one Deployment, instead of Workspace or Organization API tokens.
Specify deployment-id or deployment-name in your action. For example, astro deploy <deployment-id> or astro deploy -n <deployment-name>.
Add the command astro logout at the end of your workflow to ensure that your authentication token is cleared from the config.yaml file.

If you stage multiple commits to dag files and push them all at once to your remote branch, the template only deploys dag code changes from the most recent commit. It will miss any code changes made in previous commits.To avoid this, either push commits individually or configure your repository to Squash commits for pull requests that merge multiple commits simultaneously.

Setup

Each template installs the Astro CLI with setup-astro-cli before deploy-action runs. Astronomer recommends installing the CLI in a separate step so that you can pin or upgrade the CLI version independently of the action, including with Dependabot, and reuse a single installation across multiple deploy steps. To pin a version, set the version input, for example version: "1.40.1". If you omit this step, deploy-action installs the latest version of the Astro CLI itself.

Single branch
Multiple branch
Custom Image

To automate code deploys to a single Deployment using GitHub Actions, complete the following setup in a Git-based repository that hosts an Astro project:

Set the following as a GitHub secret:

ASTRO_API_TOKEN: The value for your Workspace or Organization API token.

In your project repository, create a new YAML file in .github/workflows that includes the following configuration:

name: Astronomer CI - Deploy code

on:
  push:
    branches:
      - main

env:
  ## Sets Deployment API credentials as environment variables
  ASTRO_API_TOKEN: ${{ secrets.ASTRO_API_TOKEN }}

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
    - name: Install Astro CLI
      uses: astronomer/setup-astro-cli@v0.0.1
    - name: Deploy to Astro
      uses: astronomer/deploy-action@v0.14.0
      with:
        deployment-id: <your-deployment-id>

(Optional) You can add optional configurations to customize your workflow. For example, if you add wake-on-deploy to your configuration, the Deploy Action wakes a hibernating Deployment before deploying code to it.

Using wake-on-deploy takes precedence over any existing Deployment hibernation overrides that you configured through the Astro UI or config.yaml file.

The following template can be used to create a multiple branch CI/CD pipeline using GitHub Actions. A multiple branch pipeline can be used to test dags in a development Deployment and promote them to a production Deployment.

Configuration requirements

You have both a dev and main branch of an Astro project hosted in a single GitHub repository.
You have respective dev and prod Deployments on Astro where you deploy your GitHub branches to.
You have at least one API token with access to both of your Deployments.

Implementation

Set the following as GitHub secrets:

PROD_ASTRO_API_TOKEN: The value for your production Workspace or Organization API token.
DEV_ASTRO_API_TOKEN: The value for your development Workspace or Organization API token.

In your project repository, create a new YAML file in .github/workflows that includes the following configuration:

name: Astronomer CI - Deploy code (Multiple Branches)

on:
  push:
    branches: [dev]
  pull_request:
    types:
      - closed
    branches: [main]

jobs:
  dev-push:
    if: github.ref == 'refs/heads/dev'
    env:
      ## Sets DEV Deployment API credential as an environment variable
      ASTRO_API_TOKEN: ${{ secrets.DEV_ASTRO_API_TOKEN }}
    runs-on: ubuntu-latest
    steps:
    - name: Install Astro CLI
      uses: astronomer/setup-astro-cli@v0.0.1
    - name: Deploy to Astro
      uses: astronomer/deploy-action@v0.14.0
      with:
        deployment-id: <dev-deployment-id>
  prod-push:
    if: github.event.action == 'closed' && github.event.pull_request.merged == true
    env:
      ## Sets Prod Deployment API credential as an environment variable
      ASTRO_API_TOKEN: ${{ secrets.PROD_ASTRO_API_TOKEN }}
    runs-on: ubuntu-latest
    steps:
    - name: Install Astro CLI
      uses: astronomer/setup-astro-cli@v0.0.1
    - name: Deploy to Astro
      uses: astronomer/deploy-action@v0.14.0
      with:
        deployment-id: <prod-deployment-id>

(Optional) You can add optional configurations to customize your workflow. For example, if you add wake-on-deploy to your configuration, the Deploy Action wakes a hibernating Deployment before deploying code to it.

Using wake-on-deploy takes precedence over any existing Deployment hibernation overrides that you configured through the Astro UI or config.yaml file.

If your Astro project requires additional build-time arguments to build an image, you need to define these build arguments using Docker’s build-push-action.

Prerequisites

An Astro project that requires additional build-time arguments to build the Runtime image.

Implementation

Set the following as a GitHub secret:

ASTRO_API_TOKEN: The value for your Workspace or Organization API token.

In your project repository, create a new YAML file in .github/workflows that includes the following configuration:

name: Astronomer CI - Additional build-time args

on:
  push:
    branches:
      - main

jobs:
  build:
    runs-on: ubuntu-latest
    env:
      ASTRO_API_TOKEN: ${{ secrets.ASTRO_API_TOKEN }}
    steps:
    - name: Check out the repo
      uses: actions/checkout@v3
    - name: Create image tag
      id: image_tag
      run: echo ::set-output name=image_tag::astro-$(date +%Y%m%d%H%M%S)
    - name: Build image
      uses: docker/build-push-action@v2
      with:
        deployment-id: <your-deployment-id>
        tags: ${{ steps.image_tag.outputs.image_tag }}
        load: true
        # Define your custom image's build arguments, contexts, and connections here using
        # the available GitHub Action settings:
        # https://github.com/docker/build-push-action#customizing .
        # This example uses `build-args` , but your use case might require configuring
        # different values.
        build-args: |
          <your-build-arguments>
    - name: Install Astro CLI
      uses: astronomer/setup-astro-cli@v0.0.1
    - name: Deploy to Astro
      uses: astronomer/deploy-action@v0.14.0
      with:
        deployment-id: <your-deployment-id>
        image-name: ${{ steps.image_tag.outputs.image_tag }}

For example, to create a CI/CD pipeline that deploys a project which installs Python packages from a private GitHub repository, you would use the following configuration:

name: Astronomer CI - Custom base image

on:
  push:
    branches:
      - main

jobs:
  build:
    runs-on: ubuntu-latest
    env:
      ASTRO_API_TOKEN: ${{ secrets.ASTRO_API_TOKEN }}
    steps:
    - name: Check out the repo
      uses: actions/checkout@v3
    - name: Create image tag
      id: image_tag
      run: echo ::set-output name=image_tag::astro-$(date +%Y%m%d%H%M%S)
    - name: Create SSH Socket
      uses: webfactory/ssh-agent@v0.5.4
      with:
        # GITHUB_SSH_KEY must be defined as a GitHub secret.
        ssh-private-key: ${{ secrets.GITHUB_SSH_KEY }}
    - name: (Optional) Test SSH Connection - Should print hello message.
      run: (ssh git@github.com) || true
    - name: Build image
      uses: docker/build-push-action@v2
      with:
        tags: ${{ steps.image_tag.outputs.image_tag }}
        load: true
        ssh: |
          github=${{ env.SSH_AUTH_SOCK }}
    - name: Install Astro CLI
      uses: astronomer/setup-astro-cli@v0.0.1
    - name: Deploy to Astro
      uses: astronomer/deploy-action@v0.14.0
      with:
        deployment-id: <your-deployment-id>
        image-name: ${{ steps.image_tag.outputs.image_tag }}

If you need guidance configuring a CI/CD pipeline for a more complex use case involving custom Runtime images, reach out to Astronomer support.

(Optional) You can add optional configurations to customize your workflow. For example, if you add wake-on-deploy to your configuration, the Deploy Action wakes a hibernating Deployment before deploying code to it.

Using wake-on-deploy takes precedence over any existing Deployment hibernation overrides that you configured through the Astro UI or config.yaml file.

Overview

Get started

Develop DAGs

AI features

Deploy code

Manage Deployments

Automation & CI/CD

Platform observability

Administration

Release notes

Best practices

Reference

Airflow 3

Default deploy action for deploying code to Astro

Prerequisites

Setup

Configuration requirements

Implementation

Prerequisites

Implementation

​Prerequisites

​Setup

​Configuration requirements

​Implementation

​Prerequisites

​Implementation

Prerequisites

Setup

Configuration requirements

Implementation

Prerequisites

Implementation