Upgrade to Airflow 3

This guide provides steps to upgrade your Astro project from Apache Airflow 2 to Apache Airflow 3.

Prerequisites​

• An Astro account with user permissions to create a new Deployment, such as Workspace Operator or Deployment Admin
• An existing Astro Deployment running Airflow 2.x
• Astro CLI >=1.34.0 installed on your local machine
• Your Astro project directory

Step 1: Prepare Your Local Project for Airflow 3​

Before upgrading to Airflow 3, you must test your DAGs and dependencies locally to identify and resolve any compatibility issues. Follow these steps to prepare your local project:

Upgrade Test Your Local Project​

Test your project locally to identify any compatibility issues:

astro dev upgrade-test

Use the results of this test to identify compatibility issues. Address any compatibility issues by updating your DAGs, custom operators, and dependencies. Common changes include:

• Updating import paths that have changed in Airflow 3
• Replacing deprecated features with their recommended alternatives
• Updating Airflow providers to versions compatible with Airflow 3 see list of minimum versions in Airflow here

Run Unit Tests​

In your local Astro project, update the Dockerfile to use an Airflow 3 image:

FROM astrocrpublic.azurecr.io/runtime:<astro-runtime-version>

Next, run the following to execute any unit tests:

astro dev pytest

To learn more, refer to full docs on testing.

Run Your Project Locally​

For a final check and to test DAGs manually, start Airflow locally:

astro dev start

Refer to the docs on local Airflow development to make and test any changes.

Step 2: Choose Your Upgrade Strategy​

Strategy A (Recommended)​

There are two main strategies to upgrade to Airflow 3. The recommended strategy (Strategy A) for upgrading to Airflow 3 is to:

1. Create a new Airflow 3 Deployment in Astro
2. Update your local project to use an Airflow 3 image
3. Deploy your updated project to the new Deployment

This approach allows you to:

• Test your DAGs with Airflow 3 without affecting your production environment
• Easily roll back if you encounter compatibility issues
• Gradually migrate your workflows to the new environment

To continue with this strategy, proceed to Step 3A.

Strategy B (Advanced, not recommended)​

Alternatively, you can try a more advanced strategy (Strategy B) of doing an in-place upgrade to Airflow 3. In this strategy:

1. Prepare and test your DAGs on Airflow 3 locally
2. Make an Airflow 3 deploy against your Airflow 2 Deployment, initiating an upgrade

This approach allows you to:

• Make changes on Deployments you're already running, without needing new infrastructure
• One-shot moving all workflows to Airflow 3

To continue with this strategy, move to Step 3B.

Step 3A: Upgrade to Airflow 3 with a New Deployment​

1. Create a New Airflow 3 Deployment​

1. In the Astro UI, go to the Deployments tab and click New Deployment
2. Make sure the latest Airflow 3 Runtime is selected
3. Configure your Deployment settings and complete the creation process
4. Note your new Deployment ID for later use

For more details, check out the docs to Create a new Deployment.

2. Deploy to Your New Airflow 3 Deployment​

Deploy your updated project:

astro deploy <your-new-deployment-id>

For additional customization, refer to docs on deploying Astro project images.

Finally, verify that your DAGs are running correctly in the new environment.

3. Migrate Your Workflows​

1. Update any CI/CD pipelines to deploy to your new Airflow 3 Deployment
2. Redirect any integrations or external systems to use your new Deployment
3. Once you've confirmed everything is working correctly, you can decommission your Airflow 2 Deployment

Step 3B: Upgrade an existing Deployment to Airflow 3​

Upgrade your Airflow 2 Deployment to Airflow 3​

Deploy your updated project:

astro deploy --force-upgrade-to-af3