Adopt Astro Runtime Operator managed Deployments (Preview)

• Stage 1: you run Airflow with the Astro Runtime Operator on your own Kubernetes cluster. Each Airflow Deployment is defined by one Airflow custom resource, and the operator turns that resource into the running Kubernetes workloads.
• Stage 2: you register that cluster as an APC data plane and adopt its Airflow Deployments. APC takes ownership of a defined set of settings, and the operator continues to own the rest.

• The operator’s Custom Resource Definition (CRD) is installed once on the cluster. It only defines the shape of an Airflow resource; it doesn’t hold any Deployment’s configuration.
• Each of your Airflow Deployments is one Airflow custom resource, created against that definition. It holds that Deployment’s configuration.
• The operator watches those custom resources and builds the real Kubernetes objects from them: schedulers, workers, services, and the rest.

​

What adoption changes

• Component resources are brought into your platform’s supported range. If a component in your custom resource requests less than your platform’s minimum or more than its maximum, APC adjusts it to the nearest supported value at adoption. See Configure component size limits.
• Adoption is not a migration of history. Existing task logs stay wherever they are today. If you switch task logging to APC, only logs written after the switch are readable from the Airflow UI; older ones remain in your own store but the Airflow UI no longer resolves them.

​

Prerequisites

• Airflow Deployments running under the Astro Runtime Operator, on a cluster where operator support is enabled. See Airflow Operator mode.
• The operator’s cluster registered as an APC data plane. See Install a data plane cluster and Register a data plane cluster.
• Operator support and adoption enabled on your platform. See Enable adoption on your platform.
• Permission to adopt. Two permissions are involved: workspace.deployments.adopt to adopt a Deployment into a Workspace, and system.deployments.adopt to browse adoption candidates, which is separate because listing candidates scans a whole cluster. Among the built-in roles, Workspace Admin carries the first and System Admin the second. Cluster Admin does not carry either, because it governs cluster configuration rather than Deployments. If your platform uses custom roles, both permissions can be granted to one. See Manage permissions and the role and permission reference.
• No existing APC Deployment using the custom resource’s name, or the namespace it runs in. Adoption is rejected if either is already taken.
• The custom resource references its image by tag, not by digest. See the warning under What adoption changes.
• To use the Astro CLI instead of the UI, APC 2.1.0 or later and a matching Astro CLI. See Install the Astro CLI.

​

Enable adoption on your platform

​

Don’t install a second operator

global:
  airflowOperator:
    enabled: true
    adoption:
      enabled: true

airflow-operator:
  # Skip installing APC's own operator; this cluster already runs one.
  enabled: false

​

Confirm it’s enabled

​

Step 1: Decide how to handle logging, images, and metrics

​

Task logging

• Route logs to APC. APC configures Airflow to write task logs to APC’s configured log store, and the Airflow UI reads them back from there. This overrides your Deployment’s existing remote logging. If your tasks currently log to Amazon S3, Google Cloud Storage, or your own Elasticsearch, they log to APC instead from then on. Logs written before the switch stay where they are, and the Airflow UI no longer resolves them.
• Keep your own logging. APC changes nothing about logging. Your tasks keep logging where they do prior to adoption, not in APC’s configured log store.

​

Image registry

• Use APC’s registry (--use-apc-registry). APC takes over the Deployment’s image and provisions the pull credential its pods need. You don’t have to move the image yourself first: adoption leaves the Deployment on the image it already runs, and the switch to your platform’s registry happens on your first astro deploy, which moves the image and the pull credential together. Deploy code with astro deploy or a CI/CD pipeline as normal. See Deploy code overview and CI/CD.
• Keep your own (the default). APC leaves the Deployment’s image and pull Secret exactly as they are and never manages them. Use this when something outside APC builds and pushes the image.

​

Deploy code when you keep your own image

astro deploy --remote --image-name=<your-registry>/<repository>:<tag> --runtime-version=<runtime-version> <deployment-id>

​

Metrics

​

Settings APC can’t fully represent

​

Step 2: Review adoption candidates

• Astro UI
• APC API

1. In the control plane, go to System > Adoption Candidates (Preview).
2. Select the data plane cluster from the dropdown. Candidates appear only after you pick a cluster.
3. Review the candidates. Each shows the custom resource’s name, its Kubernetes namespace, and the Astro Runtime and Airflow versions read from the resource.

query {
  adoptionCandidates(clusterId: "<data-plane-cluster-id>") {
    crName
    crNamespace
    runtimeVersion
    airflowVersion
  }
}

​

Step 3: Adopt the Deployment

• Astro UI
• APC API
• Astro CLI

1. Go to System > Adoption Candidates (Preview).
2. Select the data plane cluster from the dropdown. The candidates on that cluster then appear. Until you pick one, the page prompts you to select a cluster and shows no candidates.
3. Select Adopt on the candidate you want. The adoption drawer opens, showing the custom resource and the namespace and cluster it runs on.
4. Choose the Workspace to adopt the Deployment into. This is the only required field.
5. Optionally set a Label and Description. The label defaults to the custom resource name when left blank.
6. Set the options you decided on in Step 1:
   • Route logs to APC (Elasticsearch / Vector)
   • Use APC’s image registry
   • Adopt even if some settings can’t be mapped to APC, which is selected by default. Clear it to require a clean match.
7. Select Adopt Deployment.

mutation {
  adoptDeployment(
    workspaceUuid: "<workspace-id>"
    clusterId: "<data-plane-cluster-id>"
    crNamespace: "<airflow-cr-namespace>"
    crName: "<airflow-cr-name>"
    label: "My adopted deployment"
    useApcLogging: false
    useApcRegistry: false
    acceptIncompatibilities: true
  ) {
    id
    releaseName
    namespace
    isAdopted
    adoptedAt
  }
}

workspaceUuid, clusterId, crNamespace, and crName are required. label defaults to the custom resource name, useApcLogging and useApcRegistry default to false, and acceptIncompatibilities defaults to true.

The Deployment is adopted into your currently selected Workspace. Switch Workspaces first if you need a different one:

astro workspace switch

Then adopt:

astro deployment adopt --cluster-id=<cluster-id> --name=<cr-name> --namespace=<cr-namespace>

--cluster-id, --name, and --namespace are required. Add any of the following:

Flag	Description
--label, -l	Display label. Defaults to the custom resource name.
--description	Longer description for the Deployment.
--use-apc-logging	Route task logs to APC. Off by default.
--use-apc-registry	Pull images from APC’s registry. Off by default.
--accept-incompatibilities	Adopt even when some fields have no APC representation. On by default; pass --accept-incompatibilities=false to require a clean match.

​

What happens when you adopt

1. Reads the live Airflow custom resource from the cluster and maps it onto an APC Deployment.
2. Creates the Deployment in your Workspace and grants you Deployment Admin on it.
3. Applies its managed configuration to the running custom resource. The Deployment’s pods restart once during this step.

​

Step 4: Give the Deployment’s users APC accounts

• Flask-AppBuilder (FAB), which is the Airflow 2 default. Airflow keeps its own user table, so APC reads it and offers you the roster. Follow the rest of this step.
• Airflow 3. APC sets Airflow 3’s auth manager to its own at adoption, and there is no FAB user table behind it, so there is no roster to read. APC tells you this explicitly rather than showing an empty list. Invite the users to APC instead, and skip the rest of this step.

​

Import users from Airflow 2

• Astro UI
• APC API

1. Open the adopted Deployment and go to the Import Users tab. It appears only on adopted Deployments.
2. Review the discovered users. For each, APC suggests a Workspace role and a Deployment role based on the user’s Airflow role.
3. Adjust the roles and clear the checkbox next to anyone you don’t want to import.
4. If your platform uses an identity provider, select Skip invite email. Those users sign in through your IdP, so they never need to set a password.
5. Select Import selected users.

Discover the users:

query {
  adoptedAirflowUsers(deploymentId: "<deployment-id>") {
    username
    email
    fullName
    active
    fabRoles
    suggestedWorkspaceRole
    suggestedDeploymentRole
    alreadyImported
  }
}

Import the ones you want:

mutation {
  deploymentUserBulkImport(
    deploymentId: "<deployment-id>"
    bypassInvite: false
    users: [
      { email: "user@example.com", fullName: "Example User", workspaceRole: WORKSPACE_VIEWER, deploymentRole: DEPLOYMENT_EDITOR }
    ]
  ) {
    email
    status
    created
    rolesAssigned
    inviteToken
    error
  }
}

Each row succeeds or fails independently, so a single bad address doesn’t fail the whole import.Set bypassInvite: true for an IdP-backed platform to skip the invitation email, the same as Skip invite email in the Astro UI.

​

How roles are mapped

​

Step 5: Verify the adoption

1. The Deployment appears in your Workspace and reports healthy.
2. Your Airflow is still serving, and its Dags and history are intact.
3. Sign in to the Airflow UI as an imported user and confirm the expected role.
4. Change something in APC and confirm it reaches the running Airflow. Adding an environment variable is the simplest check. See Environment variables.
5. If you routed logs to APC, run a task and confirm its logs appear in the Airflow UI.
6. Confirm the Deployment’s metrics are populating. See Deployment metrics.

​

Manage an adopted Deployment

• Deploy code: Deploy code overview, Deploy Dags, CI/CD. If the Deployment kept its own image, deploy with --remote --image-name instead: see Deploy code when you keep your own image.
• Environment variables: Environment variables. Variables that were set per-component in your custom resource are shown read-only, because APC applies variables to all components uniformly. Variables that reference your own Secrets or ConfigMaps are not shown and keep working untouched.
• Executor: Kubernetes executor. APC applies the change and the operator adjusts the supporting components.
• Resources: Scale Deployment resources, Configure component size limits. Webserver or API server sizing is set through APC; scheduler, worker, and triggerer sizing stays with your custom resource. Worker settings in particular are not applicable, see Worker queues and autoscaling.
• Runtime upgrades: Migrate to Airflow 3.
• Secrets backends: Secrets backend. Unchanged by adoption; these are ordinary environment variables to APC.

​

Worker queues and autoscaling

• Your worker queues keep running unchanged, including every queue beyond the first. APC does not collapse them into one, rename them, or add a queue of its own.
• Your KEDA autoscaling keeps running unchanged. APC does not disable it, even though APC’s own operator-based Deployments don’t use it.
• Extra worker queues are recorded as partially represented at adoption. APC’s own view of the Deployment shows the first queue; the others are stored but not surfaced as editable.

​

Data plane failover

​

Keep your own pipeline and APC from fighting

​

Use server-side apply, and remove APC-owned fields from your manifest

1. Apply with server-side apply so your write only touches the fields your manifest actually declares:
   kubectl apply --server-side -f airflow-cr.yaml
   
   In Argo CD, set ServerSideApply=true. Flux’s kustomize-controller already uses server-side apply.
2. Delete the APC-owned fields from the manifest you apply. Your manifest is usually the custom resource as it looked before adoption, so it still declares fields APC now owns, such as spec.image, spec.runtimeVersion, spec.executor, and the whole spec.webserver or spec.apiserver block.

​

Trigger a Deployment update after any full-object write

mutation {
  upsertDeployment(deploymentUuid: "<deployment-id>") {
    id
  }
}

​

Never let your pipeline prune APC’s Secrets

​

Check which fields APC owns

kubectl get airflow <cr-name> -n <cr-namespace> --show-managed-fields -o yaml

​

Pause management with cordon

​

Release a Deployment

• Astro UI
• APC API
• Astro CLI

In the Deployments list, open the Deployment’s actions menu, select Unadopt Deployment, and confirm. Adopted Deployments carry an Adopted badge in the Adopted column.

mutation {
  unadoptDeployment(deploymentUuid: "<deployment-id>") {
    id
    releaseName
  }
}

astro deployment unadopt --deployment-id=<deployment-id>

The CLI asks for confirmation before releasing.

​

What happens when you release

​

Known limitations

• The cluster must already be an APC data plane. You cannot adopt Deployments from a cluster the control plane doesn’t know about. Register it first. See Register a data plane cluster.
• Airflow web authentication takeover is one-way. Once APC applies its authentication to an adopted Deployment, there is no supported path back to your original configuration while the Deployment remains adopted.
• Releasing a Deployment interrupts Airflow web access until you restore its original web authentication configuration. See Release a Deployment.
• A platform upgrade can restart adopted Deployments. Cordon any adopted Deployment you don’t want APC to act on during an upgrade, and uncordon it afterwards.
• Worker settings in APC don’t reach an adopted Deployment. Worker count, worker resources, and autoscaling are accepted and stored but never applied, and changing the executor discards the Deployment’s worker queues and KEDA configuration. See Worker queues and autoscaling.
• Data plane failover does not cover adopted Deployments, and nothing blocks you from initiating failover on a data plane that has them. Plan their recovery separately. See Data plane failover.
• Images pinned by digest are converted to tag references at adoption. The digest pin is not preserved, and the substituted tag may not exist. Re-tag before adopting. See the warning under What adoption changes.
• Plain astro deploy breaks a Deployment that kept its own image. It pushes to APC’s built-in registry and repoints the Deployment there, but no pull credential was provisioned for it. Use --remote --image-name. See Deploy code when you keep your own image.
• The registry and logging choices are fixed at adoption. Changing either means releasing the Deployment and adopting it again.
• Users can’t be imported from an adopted Airflow 3 Deployment. APC takes over its auth manager at adoption, so there are no Airflow-local users to read. Invite them to APC instead.
• Existing task logs are not migrated when you route logging to APC. Only logs written after the switch are readable from the Airflow UI; older logs stay in your own store and the Airflow UI no longer resolves them.
• Deleted APC Secrets can’t be restored by a Deployment update. If <cr-name>-registry, <cr-name>-elasticsearch, or <cr-name>-env is deleted or pruned, reissuing it requires Astronomer support. See Never let your pipeline prune APC’s Secrets.
• Clusters using the authentication sidecar don’t get an ingress for adopted Deployments, so the Airflow UI link is not reachable from APC on those clusters.

​

Reference

​

Adoption fields on a Deployment

​

API operations

​

Related documentation

• Install a data plane cluster
• Register a data plane cluster
• Airflow Operator mode
• Move the operator under APC
• Data plane architecture
• Data plane failover
• Configure a Deployment
• Cordon and uncordon a Deployment
• Environment variables
• Deploy code overview
• Configure logging
• Deployment metrics
• Manage permissions
• Manage platform users
• Invite users to a Workspace or Deployment