Configure External Secrets Operator security

This document is the configuration reference and setup runbooks for External Secrets Operator (ESO) security in Astro Private Cloud (APC). Use it after you’ve chosen a mode in External Secrets Operator security. For the full Kubernetes manifests these runbooks apply, see External Secrets Operator security manifests reference.

Mode 1 (the default ClusterSecretStore setup) is available in APC 2.0 and later. Modes 2 and 3 and the other options in this document require APC 2.1 or later.

The three modes are:

Mode 1 — Default shared identity: unchanged from previous releases. ESO uses its standard chart RBAC. No extra setup.
Mode 2 — Hardened shared identity: you set external-secrets.rbac.create: false to disable the ESO sub-chart’s default RBAC, then apply Astronomer’s minimal ClusterRole and ClusterRoleBinding.
Mode 3 — Customer-managed isolated identity: namespace pools, with per-namespace RBAC and a per-namespace secret store that you provision.

All examples use AWS Secrets Manager. GCP Secret Manager and Hashicorp Vault (Kubernetes auth) follow the same shape with a different provider block. Throughout, <release-name> is the Helm release name you install the Astronomer chart with (for example astronomer).

The manifests and chart values in this document assume the platform is installed in the astronomer namespace. If you installed it into a different namespace, replace astronomer accordingly throughout.

Prerequisites

An APC data plane cluster with data plane failover enabled (global.dataPlaneFailover.enabled: true). Mode 1 is available in APC 2.0 and later; Modes 2 and 3 require APC 2.1 or later.
ESO custom resource definitions installed on the cluster — either by the chart (default) or by you. See Install the ESO CRDs yourself.
Backend authentication ready — either a workload identity (AWS IRSA, GKE Workload Identity, or Vault Kubernetes auth) or a cloud credential pair.
kubectl access to the cluster. For Mode 3, this can be a limited-privilege user.

Common setup (Modes 1 and 2)

The shared-identity modes need a platform namespace, backend credentials, and one secret store. Full manifests are in the manifests reference.

Create the platform namespace

kubectl create namespace astronomer

Create the credentials secret

Create the backend credentials in the astronomer namespace. Skip this if you use workload identity.

apiVersion: v1
kind: Secret
metadata:
  name: secrets-backend-credentials
  namespace: astronomer
type: Opaque
data:
  access-key: <base64-aws-access-key-id>
  secret-access-key: <base64-aws-secret-access-key>

Create the secret store

Create a cluster-scoped ClusterSecretStore. In Mode 2 you can instead create a namespaced SecretStore annotated with astronomer.io/commander-sync that the platform syncs to all Deployment namespaces. See Secret store manifests.

If you use custom release names, set forceDeleteWithoutRecovery: true on the AWS provider in your store. AWS Secrets Manager soft-deletes secrets by default, with a 30-day recovery window, so reusing a release name for a new Deployment can collide with the still-recoverable secret from the one you deleted. Hard-deleting avoids the collision, at the cost of no recovery window. See Force-delete secrets in AWS Secrets Manager.

Runbook: Mode 1 — Default shared identity

No ESO-specific configuration is required beyond the common setup for most backends. Install the platform as usual. ESO uses its standard chart-managed RBAC — with ServiceAccount token creation disabled by default — and Deployments authenticate as the shared identity through the secret store you created. If your backend is Hashicorp Vault, re-enable token creation (unpinned) by setting external-secrets.rbac.serviceAccountTokenCreate: true, because Vault’s Kubernetes auth needs ESO to mint a ServiceAccount token.

global:
  dataPlaneFailover:
    enabled: true
    externalSecretManagerName: astronomer-secret-store
  plane:
    mode: "data"
external-secrets:
  enabled: true

Runbook: Mode 2 — Hardened shared identity

Start from Mode 1, then create ESO’s ServiceAccount and RBAC and disable the chart’s ESO RBAC and ServiceAccount creation, so ESO uses the objects you provisioned. Provision the ServiceAccount and RBAC before you install the platform, so ESO starts with them already in place.

Complete the common setup

Create the credentials secret and a shared secret store.

Create the ESO ServiceAccount

Create the ESO controller ServiceAccount in the astronomer namespace, so it exists before ESO starts. See ESO controller ServiceAccount.

apiVersion: v1
kind: ServiceAccount
metadata:
  name: <release-name>-external-secrets
  namespace: astronomer

Apply the provided ESO RBAC manifests

Apply the minimal ClusterRole and ClusterRoleBinding Astronomer provides, before you install the platform. They scope ESO’s cluster access and pin ServiceAccount token creation to the ServiceAccount you created in the astronomer namespace. See Mode 2 hardened ESO cluster RBAC.

Set the platform values

Disable the ESO sub-chart’s default RBAC and ServiceAccount creation, point ESO at the ServiceAccount you created, and turn off the cluster-scoped secret processing the platform doesn’t use.

global:
  dataPlaneFailover:
    enabled: true
    externalSecretManagerName: astronomer-secret-store
  plane:
    mode: "data"
external-secrets:
  enabled: true
  processClusterExternalSecret: false
  processClusterPushSecret: false
  rbac:
    create: false
  serviceAccount:
    create: false
    name: <release-name>-external-secrets

With external-secrets.rbac.create: false, the chart skips its ESO RBAC entirely — including the serviceAccountTokenCreate token rule — so the pinned token creation comes from the manifests you applied in the previous step. With serviceAccount.create: false, the chart uses the ServiceAccount you created earlier instead of creating its own.

Install or upgrade the platform and verify

helm upgrade <release-name> astronomer-internal/astronomer \
  --version 2.1.0 -f <platform-values>.yaml

Confirm the chart’s default ESO RBAC is gone and the applied minimal role is in place. See Verify the RBAC state.

Runbook: Mode 3 — Customer-managed isolated identity

The chart creates no cluster roles. You pre-provision the namespaces, per-namespace roles, and a per-namespace secret store, and install with a limited-privilege user. Use this for per-Deployment identity isolation, or when your organization requires you to provision all RBAC yourself. You can substitute your own role definitions for the generated ones. Mode 3 builds on the namespace pools feature. Complete the standard namespace pools setup first. The ESO objects described here are in addition to it.

Pre-provision namespaces and the installer identity

Create the pool namespaces, then create a limited-privilege installer identity. Generate it with the provided script, or apply the reference installer Role from the manifests reference:

python bin/generate-namespace-pools-rbac.py \
  --installer-user <installer> \
  --namespaces astronomer | kubectl apply -f -

Create the ESO controller ServiceAccount

Create the ESO controller ServiceAccount <release-name>-external-secrets in the astronomer namespace, so it exists before ESO starts. The chart doesn’t create it, because you set serviceAccount.create: false in the platform values (the next step). See ESO controller ServiceAccount.

apiVersion: v1
kind: ServiceAccount
metadata:
  name: <release-name>-external-secrets
  namespace: astronomer

Pre-provision the per-namespace ESO chain

In each pool namespace, pre-provision the per-namespace ServiceAccount, namespaced SecretStore, ESO reconcile Role and RoleBinding, and the resourceNames-pinned token-creation Role and RoleBinding. You must also provision the RBAC for the other platform components — Commander, kube-state-metrics, the Houston DB bootstrapper hook, Prometheus, and NGINX — not just ESO. See Component RBAC for restricted mode. Every namespace’s SecretStore must be named the same value as global.dataPlaneFailover.externalSecretManagerName, which you set per cluster when you enable data plane failover; only the name must match across namespaces, and each store’s contents can differ.Apply the full set from the manifests reference, or generate the platform RBAC with:

python bin/generate-namespace-pools-rbac.py \
  --release-name <release-name> \
  --release-namespace astronomer \
  --namespaces astronomer,airflow-pool-001,airflow-pool-002,airflow-pool-003 | kubectl apply -f -

For failover, pre-provision the same chain, with matching namespace names, on every cluster a Deployment can fail over to, and make sure your backend trust spans those clusters.

Set the platform values

With the ServiceAccount and RBAC in place, install with cluster roles disabled, namespace pools enabled, and ESO pointed at the ServiceAccount you created.

global:
  clusterRoles: false
  namespaceManagement:
    namespacePools:
      enabled: true
      createRbac: false
      namespaces:
        create: false
        names:
          - airflow-pool-001
          - airflow-pool-002
          - airflow-pool-003
  dataPlaneFailover:
    enabled: true
    externalSecretManagerName: astronomer-secret-store
  plane:
    mode: "data"
external-secrets:
  enabled: true
  processClusterExternalSecret: false
  processClusterPushSecret: false
  crd:
    create: true
  serviceAccount:
    create: false
    name: <release-name>-external-secrets

Install and verify

Run the helm upgrade with these values, then confirm the RBAC state. See Verify the RBAC state.

Install the ESO CRDs yourself

By default the APC chart installs the ESO CRDs. To install them yourself — for example, when a separate infrastructure team owns CRD installation — set the chart value that disables CRD creation and apply the CRDs before installing the platform. This is optional and independent of the mode, and it’s most relevant to Modes 2 and 3.

external-secrets:
  enabled: true
  crd:
    create: false

For the published CRD bundle URL and the kubectl apply command, see Install the ESO CRDs yourself.

Use Vault as your backend

If your secret backend is Hashicorp Vault, ESO authenticates to Vault with the Kubernetes auth method — ESO presents its ServiceAccount token, and Vault validates it and issues a Vault token. AppRole and token auth aren’t supported in 2.1, because they would require storing a static credential in the cluster. Because ESO must mint its own ServiceAccount token to authenticate, Vault backends require ServiceAccount token creation to be enabled: in Mode 1, set external-secrets.rbac.serviceAccountTokenCreate: true (unpinned); Modes 2 and 3 grant it through their applied manifests, pinned to the relevant ServiceAccount. For the full setup, see Configure Hashicorp Vault for data plane failover: create a Vault policy, enable a Kubernetes auth mount per data plane cluster, create a Vault role, and reference it from your SecretStore or ClusterSecretStore with a provider.vault block that uses auth.kubernetes. The Vault role maps a Kubernetes ServiceAccount to the policy that grants the secret paths: map the shared ESO ServiceAccount in Modes 1 and 2, or the per-namespace ServiceAccount in Mode 3.

Run ESO with multiple replicas

For extra resiliency, run the ESO controller with more than one replica and enable leader election, so a single replica reconciles secrets at a time while the others stand by. This helps ESO survive Pod failures and restarts, which matters for disaster recovery. It’s optional and works with any mode.

external-secrets:
  replicaCount: 2
  leaderElect: true

In Modes 2 and 3, where you manage ESO’s RBAC, also create the leader-election Role and RoleBinding in the astronomer namespace. See Leader-election RBAC. In Mode 1, the chart creates this RBAC for you.

Verify the RBAC state

The expected state depends on the mode.

Mode 1: ESO’s default chart RBAC is present — a cluster role and binding for external-secrets.
Mode 2: the chart’s default ESO RBAC isn’t created, and the minimal ClusterRole and ClusterRoleBinding you applied are present, with token creation pinned to ESO’s ServiceAccount.
Mode 3: no cluster-scoped ESO roles exist; only per-namespace roles and bindings.

# ESO cluster-scoped roles: present in Modes 1 and 2, absent (0) in Mode 3
kubectl get clusterrole -o json | jq '[.items[].metadata.name | select(contains("external-secrets"))] | length'
kubectl get clusterrolebinding -o json | jq '[.items[].metadata.name | select(contains("external-secrets"))] | length'

# Per-namespace ESO roles: present in Mode 3
kubectl get role -o json | jq '[.items[].metadata.name | select(contains("external-secrets"))] | length'
kubectl get rolebinding -o json | jq '[.items[].metadata.name | select(contains("external-secrets"))] | length'

Confirm the secret store is valid and secrets are syncing:

kubectl get secretstore,pushsecret -n airflow-pool-001
# secretstore ... Valid   ReadWrite   True
# pushsecret  ... Synced

Upgrade from 2.0 to 2.1

Upgrading to 2.1 requires no changes to keep your current configuration — Mode 1 is unchanged from previous releases.
Modes 2 and 3 are opt-in. Adopt Mode 2 by setting the ESO chart flags and applying the provided manifests; adopt Mode 3 per cluster through namespace pools.
If a later ESO version adds a new custom resource type the operator depends on, the Mode 2 minimal ClusterRole needs a corresponding update as a step in the data plane upgrade runbook.

Configuration reference

Setting	Default	Effect
`external-secrets.enabled`	`false`	Installs the ESO controller through the APC chart.
`external-secrets.rbac.create`	`true`	Whether the ESO sub-chart creates its default RBAC. Set `false` for Mode 2, then apply the provided minimal manifests.
`external-secrets.rbac.serviceAccountTokenCreate`	`false`	Whether the ESO sub-chart grants the ServiceAccount token-create rule. Defaults off in 2.1 to close the gap from previous releases. Set `true` (unpinned) for Vault backends in Mode 1; in Mode 2 it’s moot because `rbac.create: false` skips the chart’s ESO RBAC entirely.
`external-secrets.crd.create`	`true`	Whether the chart installs the ESO CRDs. Set `false` to install them yourself.
`external-secrets.serviceAccount.create`	`true`	Whether the ESO sub-chart creates its ServiceAccount. Set `false` in Modes 2 and 3 and create the ServiceAccount yourself.
`external-secrets.serviceAccount.name`	—	The name of the ServiceAccount ESO runs as. Set to `<release-name>-external-secrets` in Modes 2 and 3.
`external-secrets.processClusterExternalSecret`	`true`	Whether ESO reconciles cluster-scoped `ClusterExternalSecret` objects. Set `false` in Modes 2 and 3, which don’t use them.
`external-secrets.processClusterPushSecret`	`true`	Whether ESO reconciles cluster-scoped `ClusterPushSecret` objects. Set `false` in Modes 2 and 3, which don’t use them.
`external-secrets.replicaCount`	`1`	The number of ESO controller replicas. Set higher for resiliency, together with `leaderElect`.
`external-secrets.leaderElect`	`false`	Enables leader election so only one replica reconciles at a time. Enable it when running multiple replicas.
`global.clusterRoles`	`true`	When `false` (Mode 3), the chart creates no cluster-scoped roles and you provision per-namespace roles yourself.
`global.namespaceManagement.namespacePools.enabled`	`false`	Enables namespace pools. Required for Mode 3.
`global.namespaceManagement.namespacePools.createRbac`	`true`	Whether the chart generates the per-namespace platform RBAC. Set `false` in Mode 3 to provision it yourself.
`global.namespaceManagement.namespacePools.namespaces.create`	`true`	Whether the chart creates the pool namespaces. Set `false` when you pre-create them.
`global.namespaceManagement.namespacePools.namespaces.names`	—	The list of pool namespace names.
`global.dataPlaneFailover.enabled`	`false`	Enables data plane failover, which ESO secret sync supports. Required for these features.
`global.dataPlaneFailover.externalSecretManagerName`	—	The name of the secret store the platform uses when creating `ExternalSecret` and `PushSecret` objects. In Mode 3, every per-namespace `SecretStore` must use this exact name.

Overview

Architecture

Upgrade to Astro Private Cloud 2.0

Install Astro Private Cloud

Develop Dags

Airflow administration

Airflow 3

Deploy Dags

Manage Deployments

Platform administration

Manage Platform resources

Security and compliance

APC API

Platform Observability

Release notes

Reference

Configure External Secrets Operator security

Prerequisites

Common setup (Modes 1 and 2)

Runbook: Mode 1 — Default shared identity

Runbook: Mode 2 — Hardened shared identity

Runbook: Mode 3 — Customer-managed isolated identity

Install the ESO CRDs yourself

Use Vault as your backend

Run ESO with multiple replicas

Verify the RBAC state

Upgrade from 2.0 to 2.1

Configuration reference

​Prerequisites

​Common setup (Modes 1 and 2)

​Runbook: Mode 1 — Default shared identity

​Runbook: Mode 2 — Hardened shared identity

​Runbook: Mode 3 — Customer-managed isolated identity

​Install the ESO CRDs yourself

​Use Vault as your backend

​Run ESO with multiple replicas

​Verify the RBAC state

​Upgrade from 2.0 to 2.1

​Configuration reference

​Related documentation

Prerequisites

Common setup (Modes 1 and 2)

Runbook: Mode 1 — Default shared identity

Runbook: Mode 2 — Hardened shared identity

Runbook: Mode 3 — Customer-managed isolated identity

Install the ESO CRDs yourself

Use Vault as your backend

Run ESO with multiple replicas

Verify the RBAC state

Upgrade from 2.0 to 2.1

Configuration reference

Related documentation