Skip to main content
Airflow 3This feature is only available for Airflow 3.x Deployments.
AWS PrivateLink enables private connectivity between your Remote Execution Agents and the Astro orchestration plane without exposing traffic to the public internet. This guide explains how to set up a VPC Endpoint in your AWS environment to establish secure communication with Astro.

​
Overview

By default, Remote Execution Agents communicate with the Astro orchestration plane over the public internet. With AWS PrivateLink, you can route this traffic through a private connection within AWS, which provides enhanced security and can simplify network configurations for organizations with strict security requirements. The setup involves creating a VPC Endpoint in your AWS account that connects to Astronomer’s VPC Endpoint Service. Once configured, your Remote Execution Agents can communicate with Astro through this private connection.

​
Prerequisites

  • An Astro Deployment configured for Remote Execution.
  • Remote Execution Agents installed in an AWS environment.
  • Access to the AWS Console with permissions to create VPC Endpoints and modify Route53 configurations.

​
Astro-side configuration

Before you can create a VPC Endpoint, Astronomer must configure the VPC Endpoint Service on the Astro side. Contact Astronomer Support with the following information:
  • Your Astro Cluster ID.
  • The AWS Account ID where your Remote Execution Agents are running.
  • The AWS Region where your Remote Execution Agents are running.
Astronomer Support will provide you with the VPC Endpoint Service name, Service region, and supported Availability Zones required to create your VPC Endpoint.
If your Remote Execution Agents run in a different AWS region than the Astro orchestration plane, inform Astronomer Support. Additional configuration may be required on the Astro side, such as adding your region to the VPC Endpoint Service cross-region configuration or adding your AWS account to the allowed principals list.

​
Create a VPC Endpoint

After receiving the VPC Endpoint Service name from Astronomer Support, create a VPC Endpoint in your AWS account.
1

Navigate to VPC Endpoints

In the AWS Console, go to VPC > Endpoints.
2

Start the endpoint creation wizard

Click Create endpoint to begin the configuration.
3

Configure the endpoint

Set the following values:
  • Name tag: Enter a descriptive name, such as astro-privatelink.
  • Type: Select Endpoint services that use NLBs and GWLBs.
  • Service name: Enter the VPC Endpoint Service name provided by Astronomer Support, and click Verify service to confirm the service name is valid.
  • Cross-Region: Enable if required (optional).
  • VPC: Select the VPC where your Remote Execution Agents are running.
  • Subnets: Select at least one subnet. For high availability, select subnets in multiple Availability Zones.
  • Security group: Select or create a security group that allows inbound traffic on HTTPS port 443.
Subnet selection shows your subnets in the Availability zones supported by the VPCe Service. If there is a mismatch, you must create subnet(s) in the zones provided by Astronomer Support.

​
Configure DNS resolution

After creating the VPC Endpoint, configure DNS so that your Remote Execution Agents resolve the Astro orchestration plane hostname to the private endpoint IP addresses.

​
Configure Route53 private hosted zone

1

Create a private hosted zone

  1. In the AWS Console, go to Route 53 > Hosted zones.
  2. Click Create hosted zone.
  3. Enter external.astronomer.run as the domain name.
  4. Select Private hosted zone.
  5. Associate the hosted zone with the VPC where your VPC Endpoint was created.
2

Create an alias record

  1. In the hosted zone, click Create record.
  2. For Record name, enter your Astro Cluster ID.
  3. Select Alias.
  4. For Route traffic to, select Alias to VPC endpoint.
  5. Select your region and the VPC Endpoint you created.
  6. Click Create records.

​
Verify the connection

After completing the configuration, verify that your Remote Execution Agents can communicate with Astro through the private endpoint. Validate in the Astro UI that the agents are heart beating and reporting a Healthy status. You can also verify from within your network using the below instructions.
  1. Connect to a host within your VPC that has network access to the VPC Endpoint.
  2. Run a DNS lookup to confirm the hostname resolves to a private IP address:
The response should show the private IP addresses assigned to your VPC Endpoint rather than public IP addresses.
  1. Test connectivity to the endpoint:
The expected response is 404 page not found. If the connection is successful, your Remote Execution Agents will use the private endpoint for all communication with the Astro orchestration plane.

​
Multiple Remote Execution Agents

If you have multiple Remote Execution Agents across different VPCs, you can either create a VPC Endpoint in each VPC, or use a single VPC Endpoint and configure network routing between VPCs. The following table summarizes the actions required based on your configuration:
If you previously created a Route53 private hosted zone, you can associate additional VPCs with the same hosted zone rather than creating new zones for each VPC.

​
Restrict traffic to the private endpoint

After verifying that the private endpoint works correctly, you can optionally configure your Remote Execution API (Orchestration plane) to only allow traffic originating from your VPC Endpoint. This ensures that all communication with Astro uses the private connection. To restrict traffic:
  1. Take note of your Astro Cluster VPC Subnet Range, under Settings > Clusters > Cluster details (in the legacy UI, Organization Settings > Clusters > Cluster details).
  2. In the Astro UI, navigate to your Deployment and go to Settings.
  3. In your Deployment Advanced settings, add the cluster CIDR range to the Allowed IP address ranges list.
This configuration ensures that only traffic coming through the VPC Endpoint Service can reach the Deployment.

​
Troubleshooting

​
VPC Endpoint shows “pending acceptance”

The VPC Endpoint Service may require manual acceptance of endpoint connections, if still in pending state after 5 minutes. Contact Astronomer Support to approve your endpoint connection request.

​
DNS resolution returns public IP addresses

Verify that your Route53 private hosted zone is correctly configured and associated with the VPC where you are testing.

​
Connection timeouts

Check that the security group attached to the VPC Endpoint allows inbound traffic on port 443 from the subnets where your Remote Execution Agents are running.