Skip to main content
Use this document to upgrade an Astro Private Cloud (APC) 2.x installation to a newer 2.x version, for example from 2.0.1 to 2.1.0. This is a standard Helm chart upgrade. You change the chart version, you add any values that the new version requires, and you run helm upgrade.
This document covers 2.x to 2.x upgrades only. For a major upgrade, use one of the following guides instead:
A few notes before you start:
  • Astronomer recommends that you upgrade through minor versions in order. Include every minor version between your current version and your target version.
  • The upgrade doesn’t restart running Airflow tasks, as long as astronomer.houston.upgradeDeployments.enabled=false stays set in your upgrade command.
  • A 2.x to 2.x upgrade needs no values.yaml migration script. The migration scripts in the bin/ folder of the Astronomer Helm chart apply only to the 1.x and 0.37 upgrade paths.
  • This document assumes a single Helm release. If global.plane.mode is unified, that’s your whole installation. If you run a split control plane and one or more data planes (control and data mode), each is a separate Helm release with its own namespace and values.yaml, and Steps 7 through 11 apply once per release. Upgrade the control plane first, then each data plane. See Migrate from unified mode to a split control plane and data plane for what these modes mean.

Step 1: Read the release notes and breaking changes

Read every release note between your current version and your target version, not only the note for the target version. Astronomer maintains each minor version separately, so the same change can appear in more than one minor version. A minor release can change default behavior. For example, APC 2.1.0 includes the following breaking changes for an installation that upgrades from 2.0.x:
  • Soft delete is removed. When you delete a Deployment, APC now removes the Deployment immediately, along with its Airflow database if APC created that database. A database that APC didn’t provision is left in place.
  • The default USER role no longer includes the system.workspace.create permission. Authenticated users can no longer create a Workspace without an explicit grant. If your Teams create their own Workspaces, grant system.workspace.create through a custom role or a system role assignment before you upgrade. See Start using custom roles after you upgrade to 2.1.
  • Default container security contexts follow the Kubernetes Pod Security Standards Restricted profile. If you set a securityContext for a component, provide the whole block. APC doesn’t deep-merge your values with the defaults. See Configure security contexts.
  • global.acme is removed. Remove it from values.yaml before you upgrade, because the strict schema validator rejects unknown keys.
  • Cluster Admin permissions are folded into System Admin. Review any automation that asserts an exact permission set.
  • The Airflow Operator sub-chart condition changed. The sub-chart is gated on airflow-operator.enabled first and global.airflowOperator.enabled second. If your cluster already runs a standalone Airflow Kubernetes Operator, set airflow-operator.enabled: false so that APC doesn’t install a second one.
For the full list, see Breaking changes in the release notes.

Step 2: Check permissions

You need System Admin permissions in APC to upgrade the platform. To confirm that you are a System Admin, check that you can open the System menu in the APC UI. You also need permissions to create Kubernetes resources. To confirm that you have them, run the following commands:
If every command returns yes, you have the required Kubernetes permissions.

Step 3: Check version compatibility

Confirm that your target APC version supports your Kubernetes version, your database version, and the Astro CLI versions that your users run. See Version compatibility reference and Kubernetes version support. Check this now, not only in Step 13, because Step 11 and Step 12 both use astro deploy before you get there. If your organization is already on APC 2.x, your CLI is usually already compatible, so treat this as a confirmation rather than a blocker.
To avoid a long service disruption, upgrade APC to a compatible version before you upgrade Kubernetes.

Step 4: Get the platform Helm chart

How you get the chart depends on whether the computer that runs helm upgrade can reach https://helm.astronomer.io.
Update your local copy of the Astronomer Helm repository:

Step 5: Mirror the platform container images

Complete this step if your installation pulls platform images from a registry that you control. Skip it if your installation pulls platform images from Astronomer’s public image repositories. The image list includes every platform image, such as the APC API (quay.io/astronomer/ap-houston-api), the APC UI (quay.io/astronomer/ap-astro-ui), and Commander (quay.io/astronomer/ap-commander). Each APC version pins new image tags, so repeat this step for every upgrade.
  1. Gather the list of platform images for your target version with one of the following methods:
Mac and Linux users with jq installed can set CHART_VERSION in the following snippet and run it to produce a list of images.
  1. Copy the images into your registry with the same naming scheme that you configured at install time. See Configure the Docker registry used for platform images and Configure a custom registry for Deployment images.
Air gapped installationsIf you want the new Astro Runtime versions to appear in APC, refresh the Astro Runtime release metadata ConfigMap at the same time. This is the ConfigMap named in astronomer.houston.runtimeReleasesConfigMapName, which is astro-runtime-base-images by default. For the commands, see Fetch Astro Runtime updates.

Step 6: Back up the platform database

Back up your platform database before you upgrade. At minimum, create a snapshot or backup of your PostgreSQL database (RDS snapshot, Azure backup, Cloud SQL backup, or pg_dump).

Step 7: Check that the platform is healthy

Confirm that the platform is healthy before you change anything. Run the following commands:
If you run a split control plane and one or more data planes, repeat the platform-namespace check for the control plane namespace and for every data plane namespace — <your-platform-namespace> here means whichever release’s namespace you’re checking. <your-deployment-namespace> is a separate, per-Deployment Airflow namespace and isn’t the same thing as a data plane’s platform namespace. Every Pod must be in the Running or Completed state. If a Pod is in CrashLoopBackOff or another unhealthy state, confirm that this is expected before you continue.

Step 8: Prepare values.yaml

If you run a split control plane and one or more data planes, repeat this step for every release, using that release’s own namespace and release name. Name the files so you can tell them apart later, for example values-cp.yaml and values-dp-01.yaml, since Step 10 and the rollback at the end of this document both depend on matching the right file to the right release.
  1. Retrieve the live platform configuration:
  2. Open the file. If it contains the line USER-SUPPLIED VALUES:, delete that line.
  3. Copy the file to old_values.yaml. Keep this copy in case you need to roll back.
  4. Add or change the values that the release notes for your target version call for. For an upgrade to 2.1, this includes the following changes:
    • Remove global.acme.
    • Set airflow-operator.enabled: false if your cluster already runs a standalone Airflow Kubernetes Operator.
    • Set astronomer.houston.config.customRBAC.enabled: true on the control plane if you want to use custom roles. This is an APC API setting, so it belongs only in the control plane’s values.yaml — there’s no equivalent data plane change. See Start using custom roles after you upgrade to 2.1.
The APC API validates your configuration against a strict schema, which astronomer.houston.strictSchemaCheck controls. The validator rejects unknown keys, so a key that a new version removed fails the upgrade instead of being ignored. Remove deprecated keys before you upgrade.

Step 9: Validate the upgrade (dry run)

Run the upgrade with --dry-run to check that it succeeds before you change your cluster:
If you have more than one release to upgrade, run the dry run separately for each, pointing -f at that release’s values file and its namespace and release name.
Don’t continue until the dry run completes successfully. If it fails, fix the reported errors first.

Step 10: Run the upgrade

Create a file named upgrade.sh with the following script, then run it:
Set the following variables:
  • CHART_NAME: astronomer/astronomer if you pull the chart from a Helm repository. If you install from a file, use the filename of the chart, for example astronomer-2.1.0.tgz.
  • CHART_VERSION: your target APC version, including the patch number and a v prefix, for example v2.1.0.
If you have more than one release to upgrade, run this script once per release: change RELEASE_NAME, NAMESPACE, and the values.yaml path in the -f flag to match that release, upgrading the control plane first and then each data plane in turn. The data plane uses the same astronomer/astronomer chart, so CHART_NAME and CHART_VERSION stay the same across releases. The script sets two Helm options that matter for an upgrade:
  • astronomer.houston.upgradeDeployments.enabled=false stops APC from upgrading the Airflow charts, so running Airflow tasks aren’t disturbed. You upgrade your Deployments separately in Step 12.
  • --reset-values makes values.yaml the single source of truth for the release. This is why Step 8 retrieves the live values first. Without the live values in the file, --reset-values drops your current configuration.
Control plane reliability groupsIf you run two or more control planes in a control plane reliability group, roll the chart version across them with a different procedure. See Upgrade the chart version.

Step 11: Verify the upgrade

Confirm that every platform Pod is in the Running or Completed state:
If you have more than one release, run this check for each one — control plane and every data plane — before you move on. Once every release is upgraded, confirm that the platform works as a whole. You should be able to:
  • Sign in to the APC UI.
  • See your Workspaces and Deployments.
  • See metrics for a Deployment.
  • Run astro deploy for an existing Deployment.
  • Open the Airflow UI for a Deployment.
  • Read task logs in the Airflow UI.
  • Create a new Deployment that becomes healthy.
If a Pod is unhealthy or one of these checks fails, see Debug upgrade.

Step 12: Upgrade Airflow Deployments

Existing Deployments keep running after the platform upgrade. Astronomer still recommends that you upgrade them, so that they are fully compatible with the new platform version. To upgrade Deployments, use one of the following options:
  • APC UI: Open each Deployment and trigger an upgrade.
  • APC API: Use the upsertDeployment mutation for programmatic or bulk upgrades.
  • Astro CLI: Use the astro deploy command.

Step 13: Upgrade the Astro CLI

Each APC version works with specific versions of the Astro CLI. You confirmed compatibility in Step 3; now make sure everyone in your organization actually runs a compatible version. See Version compatibility reference and Upgrade the CLI.

Start using custom roles after you upgrade to 2.1

Custom roles are new in APC 2.1. They’re additive, so there’s nothing to migrate. The built-in System, Workspace, and Deployment roles (Viewer, Editor, and Admin) work exactly as they did before the upgrade, and every existing user, Team, and service account assignment survives the upgrade unchanged.
There is one exception. In 2.1, the default USER role no longer includes the system.workspace.create permission. If your Teams create their own Workspaces today, they can’t after the upgrade. To give the permission back, create a custom role that grants system.workspace.create and assign it, or assign a system role that includes it. This is the one role change that a 2.0 to 2.1 upgrade can require.

Enable custom roles

Custom roles are turned off by default. Whoever manages the platform Helm release turns them on with the following value:
This is a values.yaml change on the control plane, so you can set it during the upgrade itself, in Step 8. To turn custom roles on later, add the value and apply the change on its own. See Apply a config change. After the change takes effect, a Roles and Permissions page appears in the navigation for System Admins.

Move Teams onto custom roles

Astronomer recommends that you start from a built-in role instead of from a blank role. A role that you build from blank rarely grants everything a Team uses today, and the missing permissions appear as failures for your users.
  1. Create the role from an existing one. Click Create Role, then use Start from an existing role to populate the permission picker from any built-in or custom role at the same scope. Alternatively, open a built-in role and use Clone Role to make an independent copy. Both options give you the access your Teams have today.
  2. Remove the permissions that the Team doesn’t need, and add any permissions the built-in role doesn’t already grant — for example system.workspace.create, which the default USER role loses in 2.1 (see the warning above).
  3. Assign the new role to one Team.
  4. Confirm the result on the Permission audit page, which shows what that Team can do and where the access comes from.
  5. Move the remaining Teams after the first one works as expected.
Watch for two behaviors when you move Teams onto custom roles:
  • A Workspace-scoped role that includes Deployment-level permissions grants those permissions on every Deployment in the Workspace, including Deployments created later.
  • APC stops you from revoking a role assignment that leaves a scope without an admin. This guard doesn’t apply when you edit a role’s permissions. If you narrow a role that grants admin access, you can lock your organization out of System administration. Confirm that another System Admin exists first.
On Airflow 2 Deployments, there is currently no way to confirm that a custom role is enforced. If enforcement fails, the person silently gets the broader default access level instead. After you assign a custom role on an Airflow 2 Deployment, confirm that the person has only the access that the role grants. See Known issues and limitations.
For the rest of the custom role workflow, see:

Roll back

If the upgrade fails and you need your previous version back, roll the release back to the previous revision and restore your previous configuration:
If you upgraded more than one release, roll back each one that you upgraded, using that release’s own old_values.yaml.
If you run a control plane reliability group on 2.1, don’t use helm rollback. After a rollback, the chart version shown in the APC UI is stale, and that stale value feeds the version-eligibility gate. Revert with helm upgrade to the target older version instead. See Upgrade the chart version.