Documentation Index
Fetch the complete documentation index at: https://astronomer-preview.mintlify.app/llms.txt
Use this file to discover all available pages before exploring further.
A dedicated cluster exclusively runs Deployments from your Organization within a single-tenant environment on Astronomer’s cloud. Dedicated clusters provide more configuration options for regions, connectivity, and security than standard clusters. You might want to create a dedicated cluster if:
- You need to connect to data services using private networking.
- You want more options for the region your cluster is hosted in.
- You otherwise want to keep your Deployments as isolated as possible.
Dedicated clusters offer the self-service convenience of a fully managed service while respecting the need to keep data private, secure, and within a single-tenant environment. If you don’t need the aforementioned features, you can use one of the standard clusters when you Create a Deployment.
Setup
-
In the Astro UI, click Organization Settings, then click Clusters.
-
Click + Cluster.
-
Configure the following details about your cluster:
- Cloud Provider: Select AWS.
- Name: The name for your cluster.
- Region: Select the region that you want your cluster to run in.
- VPC Subnet Range: Provide a subnet range for Astro to connect to your existing AWS resources through VPC peering. The default is
172.20.0.0/20, and the minimum size you can use is /22.
-
(Optional) Expand Advanced Configuration to further define Kubernetes Pod Networking:
Astro allocates a secondary VPC CIDR range specifically for Kubernetes Pod IPs within your cluster. Pods use source network address translation (SNAT) for connections outside the VPC, so your data sources see connections from the primary VPC subnet, not the Pod CIDR. Only traffic internal to the VPC, such as to RDS databases, appears from Pod IPs in the secondary CIDR.
- Pod CIDR Range: Specify the range for Pod IP addresses. This range must not overlap with your VPC Subnet Range or any connected networks.
The Pod CIDR range isolates Pod networking and mitigates Pod IP exhaustion. SNAT ensures that all external connections use the main VPC subnet, simplifying integration with external services.
-
(Optional) In the Disaster Recovery section, configure cross-region disaster recovery:
Cross-region disaster recovery requires the Enterprise Business Critical tier.
- Cross-Region Disaster Recovery: Enable to create a secondary DR cluster in a different AWS region.
- Failover Region: Select the AWS region for the secondary cluster.
- DR VPC Subnet Range: Specify a different VPC subnet range for the secondary cluster, or leave blank to use the same range as the primary cluster.
- DR Pod CIDR Range: Specify a different Pod CIDR range for the secondary cluster, or leave blank to use the same range as the primary cluster.
- Task Logs Replication SLA: Enable to guarantee a 15-minute recovery point objective (RPO) for task logs. Additional charges apply.
See Disaster recovery for details on how these options affect your DR configuration.
-
Click Create cluster. After Astro finishes creating the cluster, users in your Organization can select the cluster when they create a Deployment.
-
In the Astro UI, click Organization Settings, then click Clusters.
-
Click + Cluster.
-
Configure the following details about your cluster:
- Cloud Provider: Select GCP.
- Name: The name for your cluster.
- Region: Select the region that your cluster runs in.
- VPC Subnet Range: Specify the range used by nodes in your GKE cluster (Default:
172.20.0.0/22). Astro uses this range to make private connections to your target data sources. The minimum size you can use is /24.
-
(Optional) Configure the following Advanced Configuration details about your cluster for Private Networking.
Astro uses source network address translation (SNAT) that performs many-to-one IP address translations for connections to your data sources and defaults secondary ranges to RFC 6598 address space (non-standard Private IP addresses), to minimize the risk and concern with IP overlap and exhaustion. When using private networking, like VPN or VPC Peering, between Astro and your target data sources, your target data sources see connections from the default VPC Subnet Range. If you’re using private connections, confirm that the following Advanced Configuration network ranges do not overlap with your target data source networks.
- Pod Subnet Range: Specify the range used by GKE Pods (Default:
100.64.0.0/16).
- Service Subnet Range: Specify the range used by Services in your GKE cluster (Default:
100.65.0.0/22).
- Service Peering Range: Specify the range used by Private Service connections (Default:
100.66.0.0/21)
If there is an overlap between the Advanced Configurations and your target data source networks, you can use the following alternative ranges:
- RFC 1918:
- 10.0.0.0/8, 10.0.0.0 – 10.255.255.255
- 172.16.0.0/12, 172.16.0.0 – 172.31.255.255
- 192.168.0.0/16, 192.168.0.0 – 192.168.255.255
- RFC 6598: 100.64.0.0/10, specifically IP addresses from 100.64.0.0 to 100.127.255.255
-
Click Create cluster. After Astro finishes creating the cluster, users in your Organization can select the cluster when they create a Deployment.
Configure cluster maintenance windowsAll GCP dedicated clusters are subscribed to the GKE regular release channel, meaning that Google automatically upgrades the cluster and its nodes whenever an upgrade is available.After you create a GCP cluster, you can control when these upgrades happen by requesting a maintenance window for the cluster. Maintenance windows determine when and how Google updates your cluster. You can use maintenance windows to ensure that upgrades don’t happen while critical dags are running on your cluster.To set a maintenance window, first choose a maintenance window time and read through the maintenance window considerations to make sure that the time is optimized for your cluster. Then, contact Astronomer Support and provide your cluster ID and desired maintenance window.
-
In the Astro UI, click Organization Settings, then click Clusters.
-
Click + Cluster.
-
Configure the following details about your cluster:
- Cloud Provider: Select Azure.
- Name: The name for your cluster.
- Region: Select the region that you want your cluster to run in.
- VPC Subnet Range: Provide a subnet range for Astro to connect to your existing Azure resources through a VNet connection. The default is
172.20.0.0/19, and the minimum size you can use is /20.
-
Click Create cluster. After Astro finishes creating the cluster, users in your Organization can select the cluster when they create a Deployment.