Register and configure agents

Airflow 3This feature is only available for Airflow 3.x Deployments.

Remote Execution Agents execute Airflow tasks in your Kubernetes infrastructure. This guide covers registering agents with your Astro Deployment and installing the Helm chart.

Prerequisites

Astro Deployment configured for Remote Execution mode. See Create a Deployment.
Kubernetes 1.30 or later

Recommended Kubernetes configurationConfigure singleProcessOOMKill: true in your kubelet configuration. With this setting, Kubernetes kills only the process that runs out of memory instead of the entire Pod. Without it, one task’s out-of-memory error kills all tasks on the worker and loses all task logs.See Kubernetes documentation for details.

Helm 3 or later
Deployment API token with Deployment Admin role to pull the base Astro Remote Execution Agent Image

Step 1: Create agent token

The agent token authenticates your agent to the Astro orchestration plane. Create this token before installing the Helm chart.

Save the token value in a secure location immediately after creation. You cannot retrieve it again. The limit is 50 agent tokens per Deployment.

Astro UI
Astro API

Navigate to Deployment

In the Astro UI, click Deployments, then select your Remote Execution Deployment (in the legacy UI, select a Workspace first).

Open tokens view

Select the Remote Agents tab and toggle to the Tokens view.

Create token

Click +Agent Token
Enter a Name and Expiration period
Optionally add a Description
Click Create

Save token

Copy the agent token and save it securely. You will use this token in the Helm chart configuration.

Get bearer token

Retrieve your Deployment API token. This authenticates API requests.

Get organization ID

Make a GET request to the organizations endpoint:

curl https://api.astronomer.io/platform/v1beta1/organizations \
  -H "Authorization: Bearer <token>"

Locate the id field in the response.

Get Deployment ID

Make a GET request to the deployments endpoint using your organization ID:

curl https://api.astronomer.io/platform/v1beta1/organizations/<organizationId>/deployments \
  -H "Authorization: Bearer <token>"

Locate the id field for your Remote Execution Deployment in the response.

Create agent token

Make a POST request to create the token. Replace the organizationId, deploymentId, and your-API-Deployment-token placeholders with your own values. You can optionally set a description and expiration period for the token.This example sets a 30-day expiration:

curl -X POST https://api.astronomer.io/iam/v1beta1/organizations/<organizationId>/deployments/<deploymentId>/agent-tokens \
  -H "Authorization: Bearer <your-API-Deployment-token>" \
  -H "Content-Type: application/json" \
  -d '{
  "description": "Production agent token",
  "name": "prod-agent",
  "tokenExpiryPeriodInDays": 30
}'

Save token

Copy the token field from the response and save it securely.

Step 2: Install Helm chart

Astronomer recommends pulling both the Remote Execution Agent image and the Sentinel image and storing them in your private registry. Sentinel provides advanced monitoring and reporting for Remote Execution Agents, starting from version 1.2.0. The Agent base images are minimal, so you might need to add packages for your pipelines to function properly. Use either an Organization API token with the Org Owner role or a Deployment API token with the Deployment Admin role to authenticate.

Download values file

In the Astro UI, go to the Remote Agents tab
Toggle to the Agents view
Click Register a Remote Agent
Click Download to get the values.yaml file

Configure required values

Update the following values in values.yaml. All other values have working defaults.You must configure these values before installing the Helm chart:

agentToken, agentTokenSecretName, or agentTokenFile - See Agent token configuration
imagePullSecretName or imagePullSecretData - See Image pull secret configuration
namespace - Kubernetes namespace for agent deployment
resourceNamePrefix - Name prefix for Kubernetes resources
secretBackend - Must be configured before agents can execute tasks. See Configure secrets backend
xcomBackend - Must be configured before agents can execute tasks. See Configure XCom backend
stateStoreBackend - Required on Astro Runtime 3.3 and later, where agents don’t start without it. Helm chart 2.3.0 and later set a working default that isn’t suitable for production. See Configure state store backend

See the Helm chart comments and Helm chart configuration reference for descriptions of values.

Pull agent image for private registries

If self-hosting the image, log in to the image registry with your Deployment API token:

docker login images.astronomer.cloud -u cli -p <your-token>

Sentinel image available with 1.2.0 and laterStarting with Remote Execution Agent 1.2.0, a Sentinel image is published alongside the agent images to provide monitoring for Remote Execution Agents. The Sentinel image must be pulled separately. Astronomer recommends enabling Sentinel for all deployments. To enable Sentinel, configure the service in your values.yaml file. See Sentinel for Remote Execution Agents.

After you log in, you can pull the Remote Execution Agent and Sentinel images directly. To find the latest version and image path, refer to the Remote Execution Agent release notes for all currently hosted images and Remote Execution Agent image reference for their full URLs. For example:

docker pull images.astronomer.cloud/baseimages/astro-remote-execution-agent:3.1-3-python-3.12-astro-agent-1.2.0

docker pull images.astronomer.cloud/baseimages/astro-remote-execution-sentinel:1.2.0

Configure scope for registry proxiesIf you use JFrog Artifactory or a similar registry management tool to mirror or proxy images.astronomer.cloud, you need to configure specific include patterns instead of using the default **/* pattern.The Deployment API token has limited scope and cannot fetch manifests for all repositories. Configure your remote registry to include only these specific paths:

baseimages/astro-remote-execution-agent
baseimages/astro-remote-execution-sentinel

Without these specific patterns, you might encounter 403 Forbidden errors when JFrog attempts to crawl all repositories in the registry.

Pull the Remote Execution Agent image, apply customizations that your dags require, and push it to your private registry. Then update the values.yaml file to reference your customized image.

Install Helm chart

You must configure secretBackend in your values.yaml before running the Helm install. The installation fails if secretBackend has no value. See Configure secrets backend.

Run the following commands to install the agent:

helm repo add astronomer https://helm.astronomer.io
helm repo update
helm install astro-agent astronomer/astro-remote-execution-agent -f values.yaml

Step 3: Optionally set allowed IP ranges

Restrict Deployment access to specific IP address ranges for additional security or network isolation between environments.

Open Deployment settings

In the Astro UI, click the options menu for your Deployment and select Edit.

Add IP ranges

In the Advanced section, click +Add IP.
Enter an IP address range in CIDR format.
Click Add.
Repeat to add multiple ranges.

Step 4: Verify agent heartbeat

Confirm the agent is connected and healthy.

Check agent status

In the Astro UI, go to the Remote Agents tab. A healthy agent shows:

Health status: Healthy
Last heartbeat: Within the past minute

You can also verify locally that all agent client deployment Pods are running with kubectl get pods -n <namespace>. For more in-depth validation, check pod logs for heartbeat activity.To verify that your agents can communicate with your Astro Orchestration plane:

Connect to a host or Pod within your VPC that has your Remote Execution Agent running.
Run a DNS lookup to confirm the hostname resolves successfully:
nslookup <AstroClusterId>.external.astronomer.run
The response should show the Astro cluster’s public load balancer’s public IP addresses, or the private IP addresses assigned to your VPC Endpoint if you configured AWS PrivateLink.
Test connectivity to the endpoint:
curl -v https://<AstroClusterId>.external.astronomer.run
The expected response is 404 page not found. A successful connection confirms your Remote Execution Agents are able to communicate with the Astro orchestration plane over a public connection or via your private VPC endpoint.

Temporarily remove any configured allowed IP ranges if the agent is not starting up and reporting Healthy. If connecting using a public connection, your network team may need to allowlist the Astro cluster’s public load balancer’s public IP addresses (step 2) for outbound access from your VPC.

Configure dag bundles

After verifying agent health, configure how agents access DAG code. See Configure DAG sources.

Run test dag

Trigger a test DAG run to verify the agent executes tasks successfully.

If you expect tasks to run longer than the default grace period of 10 minutes, update the terminationGracePeriodSeconds parameter for your workers in values.yaml. This ensures that worker Pods have enough time to finish existing tasks before terminating. See Worker resource configuration.

HTTP/HTTPS proxy server supportStarting with Remote Execution Agent 1.3.2, the agents support running behind an HTTP(S) proxy server. Configure proxy settings using the HTTP_PROXY, HTTPS_PROXY, and NO_PROXY environment variables.For Remote Execution Agent versions earlier than 1.3.2, proxy servers are not supported. If your Kubernetes environment automatically adds a proxy configuration to Pods, the agents will fail to establish an outbound connection to the orchestration plane. You might see errors similar to these in worker logs:

"exc_type":"ReadError","exc_value":"[Errno 104] Connection reset by peer"
"exc_type":"HTTPStatusError","exc_value":"Client error '400 Bad Request' for url ...

Workaround: Remove the proxy configuration from the agent Pods, or upgrade to Agent 1.3.2 or later.

Agent token configuration

Provide the agent token using one of these methods:

agentToken

Store the token directly in values.yaml:

values.yaml

agentToken: "<your-token-value>"

Storing tokens directly in values files exposes them in version control. Use agentTokenSecretName or agentTokenFile for better security.

agentTokenSecretName

Reference an existing Kubernetes secret containing the token:

kubectl -n <namespace> create secret generic agent-token \
  --from-file=token=token.txt

In values.yaml:

values.yaml

agentTokenSecretName: "agent-token"

agentTokenFile

Mount a file containing the token. The agent reads the token at runtime:

values.yaml

agentTokenFile: "/path/to/token/file"

Image pull secret configuration

Configure image pull secrets to authenticate with your container registry. The configuration differs depending on whether you pull images directly from Astronomer’s registry or from a self-hosted registry.

The image pull secret requires an Astro API token, not an agent token. Use either an Organization API token with the Org Owner role or a Deployment API token with the Deployment Admin role. The agent token created in Step 1 authenticates the agent to the Astro orchestration plane and cannot be used for pulling images.

Astronomer registry
Self-hosted registry

Use this configuration when pulling images directly from images.astronomer.cloud.

imagePullSecretName

Reference an existing Kubernetes secret in your namespace:

kubectl create secret docker-registry -n <namespace> <secretName> \
  --docker-server=images.astronomer.cloud \
  --docker-username=cli \
  --docker-password=<your-astro-api-token>

In values.yaml:

values.yaml

imagePullSecretName: "<secretName>"

imagePullSecretData

Alternatively, provide Docker config JSON directly. The Helm chart creates a secret named image-pull-secret:

values.yaml

imagePullSecretData: |
  {
    "auths": {
      "images.astronomer.cloud": {
        "auth": "<base64-encoded-credentials>",
        "email": "<email>"
      }
    }
  }

Use this configuration when pulling images from a self-hosted registry, proxy, or mirror.

If you use a proxy or mirror for images.astronomer.cloud, you still need an Astro API token to authenticate with the upstream Astronomer registry. Configure this in your proxy settings. See Pull agent image for private registries.

imagePullSecretName

Reference an existing Kubernetes secret in your namespace:

kubectl create secret docker-registry -n <namespace> <secretName> \
  --docker-server=<your-registry-endpoint> \
  --docker-username=<your-registry-username> \
  --docker-password=<your-registry-password>

In values.yaml:

values.yaml

imagePullSecretName: "<secretName>"

imagePullSecretData

Alternatively, provide Docker config JSON directly. The Helm chart creates a secret named image-pull-secret:

values.yaml

imagePullSecretData: |
  {
    "auths": {
      "<your-registry-endpoint>": {
        "auth": "<base64-encoded-username:password>",
        "email": "<email>"
      }
    }
  }

Manage Remote Execution Agents

You can take the following actions on your registered Remote Execution Agents:

Cordon: Cordoning a Remote Execution Agent marks it as unavailable for scheduling new tasks, while allowing it to continue running and complete any tasks already in progress.

This allows you to gracefully remove the Agent from service without interrupting current workloads. For example, you can cordon an Agent to delete or perform maintenance, such as an upgrade, on the Agent or underlying infrastructure. A cordoned Agent will not receive new work, but it remains active until all running tasks have finished. Once ready to reintroduce the Agent to the task pool, it can be uncordoned to resume normal operation.

Uncordon: Uncordoning a Remote Execution Agent re-enables it to receive new tasks and resume normal scheduling.
Delete: Deletes the Remote Execution Agent from the Deployment.

Remote Execution Agent maintenance policy

Each Remote Execution Agent minor version is maintained for 6 months from the release month. See Agent maintenance policy for more details about versioning, support, and upgrade recommendations.

Next steps

After registering agents, configure the required components:

Configure secrets backend - Required for agent operation
Configure XCom backend - Required for passing data between tasks
Configure DAG sources - Required for accessing DAG code

Overview

Get started

Develop DAGs

AI features

Deploy code

Manage Deployments

Automation & CI/CD

Platform observability

Administration

Release notes

Best practices

Reference

Airflow 3

Register and configure agents

Prerequisites

Step 1: Create agent token

Step 2: Install Helm chart

Step 3: Optionally set allowed IP ranges

Step 4: Verify agent heartbeat

Agent token configuration

agentToken

agentTokenSecretName

agentTokenFile

Image pull secret configuration

imagePullSecretName

imagePullSecretData

imagePullSecretName

imagePullSecretData

Manage Remote Execution Agents

Remote Execution Agent maintenance policy

Next steps

​Prerequisites

​Step 1: Create agent token

​Step 2: Install Helm chart

​Step 3: Optionally set allowed IP ranges

​Step 4: Verify agent heartbeat

​Agent token configuration

​agentToken

​agentTokenSecretName

​agentTokenFile

​Image pull secret configuration

​imagePullSecretName

​imagePullSecretData

​imagePullSecretName

​imagePullSecretData

​Manage Remote Execution Agents

​Remote Execution Agent maintenance policy

​Next steps

​Related documentation

Prerequisites

Step 1: Create agent token

Step 2: Install Helm chart

Step 3: Optionally set allowed IP ranges

Step 4: Verify agent heartbeat

Agent token configuration

agentToken

agentTokenSecretName

agentTokenFile

Image pull secret configuration

imagePullSecretName

imagePullSecretData

imagePullSecretName

imagePullSecretData

Manage Remote Execution Agents

Remote Execution Agent maintenance policy

Next steps

Related documentation