Automation & CI/CDCI/CDCI/CD templatesGitHub Actions

GitHub Actions templates for deploying to Astro from private networks

The Astro GitHub integration can automatically deploy code from a GitHub repository to Astro without you needing to configure a GitHub action. In addition, the Astro UI shows Git metadata for each deploy on your Deployment information screen. See Deploy code with the Astro GitHub integration for setup steps.

If you don’t have access to the Astronomer deploy action because you can’t access the public internet from your GitHub repository, use one of the following private network templates to deploy to Astro.

Read the following sections to choose the right template for your use case. If you have one Deployment and one environment on Astro, use the single branch implementation. If you have multiple Deployments that support development and production environments, use the multiple branch implementation. If your team builds custom Docker images, use the custom image implementation.

You can configure your CI/CD pipelines to deploy a full project image or your dags directory. To learn more about CI/CD on Astro, see Choose a CI/CD strategy.

If you use a self-hosted runner to execute jobs from GitHub Actions, the Astro CLI’s config.yaml file, which stores default deploy details, might be shared across your organization and hence multiple CI/CD pipelines. To reduce the risk of accidentally deploying to the wrong Deployment, ensure the following:

  • Add ASTRO_API_TOKEN to your repository and include a check in your GitHub workflow to verify that it exists.
  • Use Deployment API tokens, which are scoped only to one Deployment, instead of Workspace or Organization API tokens.
  • Specify deployment-id or deployment-name in your action. For example, astro deploy <deployment-id> or astro deploy -n <deployment-name>.
  • Add the command astro logout at the end of your workflow to ensure that your authentication token is cleared from the config.yaml file.

Prerequisites

Setup

To automate code deploys to a Deployment using GitHub Actions, complete the following setup in a Git-based repository that hosts an Astro project:

  1. Set the following as GitHub secrets:
  • ASTRO_API_TOKEN: The value for your Workspace or Organization API token.
  1. In your project repository, create a new YAML file in .github/workflows that includes the following configuration. When you make a commit to a specified branch, this workflow sets your Deployment API credentials as environment variables, installs the latest version of the Astro CLI, checks to see if your dags folder has changes, and then either completes a full code deploy or a dag-only code deploy.
1name: Astronomer CI - Deploy code
2
3on:
4  push:
5    branches:
6      - main
7
8env:
9  ## Sets Deployment API credentials as environment variables
10  ASTRO_API_TOKEN: ${{ secrets.ASTRO_API_TOKEN }}
11
12jobs:
13  build:
14    runs-on: ubuntu-latest # add the appropriate image
15    steps:
16        # Install the Astro CLI (current version)
17        - name: checkout repo
18          uses: actions/checkout@v3
19          with:
20            fetch-depth: 2
21            clean: false
22        - name: Install the CLI
23          run: curl -sSL install.astronomer.io | sudo bash -s
24        # Determine if only dag files have changes
25        - name: Deploy to Astronomer
26          run: |
27            files=$(git diff --name-only $(git rev-parse HEAD~1) -- .)
28            dags_only=1
29            for file in $files; do
30            if [[ $file != dags/* ]]; then
31                echo "$file is not a dag, triggering a full image build"
32                dags_only=0
33                break
34            fi
35            done
36            ### If only dags changed deploy only the dags in your 'dags' folder to your Deployment
37            if [ $dags_only == 1 ]
38            then
39                astro deploy --dags
40            fi
41            ### If any other files changed build your Astro project into a Docker image, push the image to your Deployment, and then push and dag changes
42            if [ $dags_only == 0 ]
43            then
44                astro deploy
45            fi