Best practices for upgrading Astro Runtime on Astro

Astro includes features that make upgrading to the latest version of Astro Runtime easy and, more importantly, safe. When leveraging these features, follow these best practices to ensure that your Deployments remain stable after you upgrade:

• Monitor for upgrade opportunities using Organization dashboards, as well as by following the Astro Runtime maintenance and lifecycle policy.
• Check Upgrade Astro Runtime for advisories on the specific version you're upgrading to.
• Prepare to upgrade safely using upgrade tests.
• In the case of a malfunction after upgrading, revert to your original Astro Runtime version using deploy rollbacks.

Use this document to learn more about each of these best practices for upgrading Astro Runtime.

Best practice guidance​

Astronomer allows you to control the version of Astro Runtime that you run your Astro deployments on ¹, but we do also have an opinionated best practice that we believe most Astro users would benefit from following.

Astronomer recommends:

• Using the slim version of Astro Runtime
• Staying on the very latest release of Astro Runtime, i.e., always upgrading as soon as possible
• Controlling your provider dependencies explicitly, by specifying their versions in your requirements.txt file

In your Dockerfile:

# Upgrade this as often as you can - it's safe
# Use astro-runtime-slim instead of astro-runtime

In your requirements.txt file:

# Use ~= to allow upgrading minor and patch version, but not breaking changes
# Only upgrade major versions explicitly, and only after reviewing release notes

apache-airflow-providers-google ~= 14.0  # release notes: https://airflow.apache.org/docs/apache-airflow-providers-google/stable/changelog.html
apache-airflow-providers-snowflake ~= 6.1  # release notes: https://airflow.apache.org/docs/apache-airflow-providers-snowflake/stable/changelog.html

Rationale​

Astronomer thinks about the Airflow code in two major categories:

The main concept underpinning Astronomer's upgrade recommendations is that you should always strictly prefer the latest version of core, but you may not necessarily want the latest provider versions for the operators that you use. Great pains are taken in core to prevent backwards incompatibilities and bugs that could be disruptive to your Airflow pipelines. Because the slim version of Runtime contains only the core parts of Airflow and no optional providers, you never need to change your DAG code to upgrade to the latest Runtime slim version², unless you are upgrading from Airflow 2 to Airflow 3, which rarely necessitates code level changes.

Fortunately, the optional providers can be both upgraded and downgraded freely, without requiring rollbacks. Which means that the risk introduced by upgrading pre-maturely is low. In many cases it can be a rational choice to simply upgrade your optional providers to the latest version, run your DAGs and see if anything breaks. If you get an unexpected dag import error or task failure, evaluate if that failure could have been caused by the provider upgrade.

However, for greater safety when working with production deployments, it is prudent to check the provider release notes for the major version you are upgrading to and assess them for any breaking changes that might impact your DAGs.

• The optional providers are packages like apache-airflow-providers-google that extend Airflow by providing operators that help you interface with services or perform common tasks without having to write a lot of new code. You only need to install a optional provider packages for the specific services and tasks that you actually use in your Astro deployment.

The main concept underpinning Astronomer's upgrade recommendations is that you should always strictly prefer the latest version of core, but you may not necessarily want the latest provider versions for the Operators that you use. Great pains are taken in core to prevent backwards incompatibilities and bugs can be disruptive to your expected workflows. Because the slim version of Runtime has only the core components of Airflow and no optional providers, you never need to change your DAG code to upgrade to the latest Runtime slim version². Providers, on the other hand, do introduce breaking changes with some frequency, so you might need to change DAG code to upgrade and therefore you might not want to upgrade your providers right away.

Fortunately, the optional providers can be both upgraded and downgraded freely, without requiring rollback. So the risk of upgrading pre-maturely is actually quite low. It can be, in many cases, a rational choice to simply upgrade your optional providers to the latest version, see if anything breaks, and if you get an unexpected dag import error or task failure, evaluate if that failure could have been caused by the provider upgrade. However, for greater safety, it is prudent to check the provider release notes for the major version you are upgrading to.

Performing an upgrade​

The Astro CLI has built-in functionality to identify major version upgrades in packages which could introduce breaking changes to your DAGs. Astronomer recommends making use of this feature prior to every upgrade.

Use the process described in Upgrade Astro Runtime as the basis for all upgrades. It gives step-by-step guidance on completing an upgrade, as well as version-specific advice. In particular, the following steps should be followed for a smooth upgrade:

• Run upgrade tests to anticipate and address problems. This involves a single command that:

  • Compares dependency versions between the current and upgraded environments.
  • Checks DAGs in the project against the new Airflow version for errors.
  • Runs a DAG parse test with the new Airflow version.
  • Produces in new project directory upgrade-test-<current_version>--<upgraded_version>:
    • An upgraded Dockerfile.
    • pip freeze output for both versions.
    • A report with environment metadata and a Results table including any errors logged.

  

  • A Dependency Compare report listing the upgrades, additions and removals coming in the new Astro runtime version and underlying Airflow package. For example, a "Major Updates" section lists the packages receiving new major versions, and a "Minor Updates" lists the packages receiving new minor versions:

  Major Updates:
  azure-mgmt-datafactory 6.1.0 >> 7.0.0
  
  Minor Updates:
  Flask-Caching 2.1.0 >> 2.2.0
  

For more details about the upgrade-test command, see Testing your Astro project locally.

Monitoring for upgrades​

Astro users on the Enterprise plan can use Organization dashboards to check whether their presently running Astro Runtime versions are currently maintained, as well as how much time is left before they become unmaintained. This is especially helpful in cases where your Organization has many Deployments to track. See Organization dashboards for steps to access these dashboards.

If you are not on the Astro Enterprise plan, you can still reference the Astro Runtime maintenance and lifecycle policy to see when you'll need to upgrade your Deployments to a new version of Astro Runtime. A Deployment Health Alert will also be displayed if a Deployment is on an unmaintained Astro Runtime version.

Roll back Deployments after a broken upgrade​

In the case of an emergency, you can roll back to your previous deploy to downgrade the Astro Runtime version. Astro's rollback functionality is similar to the Git revert command. Rollbacks revert project code only, keeping environment variables and other configuration unchanged.

Some rollbacks require reverting the underlying database schema used by Airflow. Sometimes, data cannot be represented in the older schema and so will be dropped by the rollback process. For example, if you upgrade to Runtime 12, new tasks will have the try number associated with Event Logs, but if you roll back to a previous version, these try numbers will be lost. This data is always data that would not have been present to begin with if you had not upgraded your Runtime, so there is not risk of data loss from the overall process of upgrading and subsequently rolling back.

See also​

• Upgrade Astro Runtime.