AWS Networking: AWS PrivateLink

Create a PrivateLink connection to AWS.

On Astro standard clusters, only the following AWS PrivateLink endpoints are supported:

• Amazon S3 - Gateway Endpoint
• Amazon Elastic Compute Cloud (Amazon EC2) Autoscaling - Interface Endpoint
• Amazon Elastic Container Registry (ECR) - Interface Endpoints for ECR API and Docker Registry API
• Elastic Load Balancing (ELB) - Interface Endpoint
• AWS Security Token Service (AWS STS) - Interface Endpoint

Astro automatically supports cross-region connectivity for dedicated clusters that use AWS PrivateLink connections. Standard cross-region data transfer charges apply.

Use AWS PrivateLink to create private connections from Astro to your AWS services without exposing your data to the public internet.

All Astro clusters are pre-configured with the following AWS PrivateLink endpoints:

• Amazon S3 - Gateway Endpoint
• Amazon Elastic Compute Cloud (Amazon EC2) Autoscaling - Interface Endpoint
• Amazon Elastic Container Registry (ECR) - Interface Endpoints for ECR API and Docker Registry API
• Elastic Load Balancing (ELB) - Interface Endpoint
• AWS Security Token Service (AWS STS) - Interface Endpoint

To request additional endpoints, or assistance connecting to other AWS services, complete the following steps:

AWS Service Endpoint

• Prepare a list of your AWS Services that require Endpoints, such as SQS, Lambda, or DynamoDB.
• Contact Astronomer support and provide this information for next steps.

By default, Astronomer support activates the Enable DNS Name option on supported AWS PrivateLink endpoint services. With this option enabled, you can make requests to the default public DNS service name instead of the public DNS name that is automatically generated by the VPC endpoint service. For example, *.notebook.us-east-1.sagemaker.aws instead of vpce-xxx.notebook.us-east-1.vpce.sagemaker.aws. For more information about AWS DNS hostnames, see DNS hostnames.

Custom VPC Endpoint

• To retrieve your cluster’s AWS account ID on Astro, contact Astronomer support.
• Go to your VPCe Service configuration and add arn:aws:iam::<AWS account ID>:role/astronomer-remote-management into a list of the Allowed principals.
• Contact Astronomer support and provide a VPCe Service name for the custom Service.
• (Optional) If Route53 alias is required for the proper connectivity by your service such as Snowflake or EKS, add a preferable DNS name for the Endpoint to your request. Otherwise, you can skip this step.

You’ll incur additional AWS infrastructure costs for every AWS PrivateLink endpoint service that you use. See AWS PrivateLink pricing.