This feature is only available for Airflow 3.x Deployments.
Remote Execution Agents execute Airflow tasks in your Kubernetes infrastructure. This guide covers registering agents with your Astro Deployment and installing the Helm chart.
The agent token authenticates your agent to the Astro orchestration plane. Create this token before installing the Helm chart.
Save the token value in a secure location immediately after creation. You cannot retrieve it again. The limit is 50 agent tokens per Deployment.
Astronomer recommends pulling both the Remote Execution Agent image and the Sentinel image and storing them in your private registry. Sentinel provides advanced monitoring and reporting for Remote Execution Agents, starting from version 1.2.0. The Agent base images are minimal, so you might need to add packages for your pipelines to function properly. Use either an Organization API token with the Org Owner role or a Deployment API token with the Deployment Admin role to authenticate.
values.yaml fileUpdate the following values in values.yaml. All other values have working defaults.
You must configure these values before installing the Helm chart:
agentToken, agentTokenSecretName, or agentTokenFile - See Agent token configurationimagePullSecretName or imagePullSecretData - See Image pull secret configurationnamespace - Kubernetes namespace for agent deploymentresourceNamePrefix - Name prefix for Kubernetes resourcessecretBackend - Must be configured before agents can execute tasks. See Configure secrets backendxcomBackend - Must be configured before agents can execute tasks. See Configure XCom backendSee the Helm chart comments and Helm chart configuration reference for descriptions of values.
If self-hosting the image, log in to the image registry with your Deployment API token:
Starting with Remote Execution Agent 1.2.0, a Sentinel image is published alongside the agent images to provide monitoring for Remote Execution Agents. The Sentinel image must be pulled separately. Astronomer recommends enabling Sentinel for all deployments. To enable Sentinel, configure the service in your values.yaml file. See Sentinel for Remote Execution Agents.
After you log in, you can pull the Remote Execution Agent and Sentinel images directly. To find the latest version and image path, refer to the Remote Execution Agent release notes for all currently hosted images and Remote Execution Agent image reference for their full URLs. For example:
If you use JFrog Artifactory or a similar registry management tool to mirror or proxy images.astronomer.cloud, you need to configure specific include patterns instead of using the default **/* pattern.
The Deployment API token has limited scope and cannot fetch manifests for all repositories. Configure your remote registry to include only these specific paths:
baseimages/astro-remote-execution-agentbaseimages/astro-remote-execution-sentinelWithout these specific patterns, you might encounter 403 Forbidden errors when JFrog attempts to crawl all repositories in the registry.
Pull the Remote Execution Agent image, apply customizations that your dags require, and push it to your private registry. Then update the values.yaml file to reference your customized image.
You must configure secretBackend in your values.yaml before running the Helm install. The installation fails if secretBackend has no value. See Configure secrets backend.
Run the following commands to install the agent:
Restrict Deployment access to specific IP address ranges for additional security or network isolation between environments.
Confirm the agent is connected and healthy.
In the Astro UI, go to the Remote Agents tab. A healthy agent shows:
You can also verify locally that all agent client deployment Pods are running with kubectl get pods -n <namespace>. For more in-depth validation, check pod logs for heartbeat activity.
To verify that your agents can communicate with your Astro Orchestration plane:
Connect to a host or Pod within your VPC that has your Remote Execution Agent running.
Run a DNS lookup to confirm the hostname resolves successfully:
The response should show the Astro cluster’s public load balancer’s public IP addresses, or the private IP addresses assigned to your VPC Endpoint if you configured AWS PrivateLink.
Test connectivity to the endpoint:
The expected response is 404 page not found. A successful connection confirms your Remote Execution Agents are able to communicate with the Astro orchestration plane over a public connection or via your private VPC endpoint.
Temporarily remove any configured allowed IP ranges if the agent is not starting up and reporting Healthy. If connecting using a public connection, your network team may need to allowlist the Astro cluster’s public load balancer’s public IP addresses (step 2) for outbound access from your VPC.
After verifying agent health, configure how agents access DAG code. See Configure DAG sources.
Trigger a test DAG run to verify the agent executes tasks successfully.
If you expect tasks to run longer than the default grace period of 10 minutes, update the terminationGracePeriodSeconds parameter for your workers in values.yaml. This ensures that worker Pods have enough time to finish existing tasks before terminating. See Worker resource configuration.
Starting with Remote Execution Agent 1.3.2, the agents support running behind an HTTP(S) proxy server. Configure proxy settings using the HTTP_PROXY, HTTPS_PROXY, and NO_PROXY environment variables.
For Remote Execution Agent versions earlier than 1.3.2, proxy servers are not supported. If your Kubernetes environment automatically adds a proxy configuration to Pods, the agents will fail to establish an outbound connection to the orchestration plane. You might see errors similar to these in worker logs:
"exc_type":"ReadError","exc_value":"[Errno 104] Connection reset by peer""exc_type":"HTTPStatusError","exc_value":"Client error '400 Bad Request' for url ...Workaround: Remove the proxy configuration from the agent Pods, or upgrade to Agent 1.3.2 or later.
Provide the agent token using one of these methods:
Store the token directly in values.yaml:
Storing tokens directly in values files exposes them in version control. Use agentTokenSecretName or agentTokenFile for better security.
Reference an existing Kubernetes secret containing the token:
In values.yaml:
Mount a file containing the token. The agent reads the token at runtime:
Configure image pull secrets to authenticate with your container registry. The configuration differs depending on whether you pull images directly from Astronomer’s registry or from a self-hosted registry.
The image pull secret requires an Astro API token, not an agent token. Use either an Organization API token with the Org Owner role or a Deployment API token with the Deployment Admin role. The agent token created in Step 1 authenticates the agent to the Astro orchestration plane and cannot be used for pulling images.
Use this configuration when pulling images directly from images.astronomer.cloud.
Reference an existing Kubernetes secret in your namespace:
In values.yaml:
Alternatively, provide Docker config JSON directly. The Helm chart creates a secret named image-pull-secret:
You can take the following actions on your registered Remote Execution Agents:
This allows you to gracefully remove the Agent from service without interrupting current workloads. For example, you can cordon an Agent to delete or perform maintenance, such as an upgrade, on the Agent or underlying infrastructure.
A cordoned Agent will not receive new work, but it remains active until all running tasks have finished. Once ready to reintroduce the Agent to the task pool, it can be uncordoned to resume normal operation.
Uncordon: Uncordoning a Remote Execution Agent re-enables it to receive new tasks and resume normal scheduling.
Delete: Deletes the Remote Execution Agent from the Deployment.
Each Remote Execution Agent minor version is maintained for 6 months from the release month.
See Agent maintenance policy for more details about versioning, support, and upgrade recommendations.
After registering agents, configure the required components: