Skip to main content

Documentation Index

Fetch the complete documentation index at: https://astronomer-preview.mintlify.app/llms.txt

Use this file to discover all available pages before exploring further.

Passwordless setup is available only on Astro dedicated clusters. For Astro standard clusters, follow the steps inPrivate Registryto create a Kubernetes secret containing your registry credentials.
By default, the KubernetesPodOperator expects to pull container images that are hosted publicly. If your images are hosted on the container registry native to your cloud provider, you can grant access to the images directly.

​
Prerequisites

​
Setup

If your container image is hosted in Google Artifact Registry repository, add a permissions policy to the repository to allow the KubernetesPodOperator to pull the Docker image. You don’t need to create a Kubernetes secret or specify the Kubernetes secret in your dag. Docker images hosted in Google Artifact Registry repositories can be pulled only to Deployments hosted on GCP clusters.
1
Request the Compute Engine default service account ID
2
Contact Astronomer support to request the Compute Engine default service account ID for your cluster.
3
Add Google Artifact Registry repository permissions
4
  • Log in to Google Artifact Registry.
  • Click the checkbox next to the repository that you want to use.
  • In the Properties pane that appears, click ADD PRINCIPAL in the PERMISSIONS tab.
  • In the Add Principals text box, paste the Compute Engine default service account ID that was provided to you by Astronomer Support.
  • In the Assign Roles selector, search for Artifact Registry Reader and select the role that appears.
  • Click Save to grant read access for the registry to Astro.
    • 5
    Set up the KubernetesPodOperator
    6
    The following snippet is the minimum configuration you’ll need to create a KubernetesPodOperator task on Astro:
    7
    from airflow.configuration import conf
from airflow.providers.cncf.kubernetes.operators.pod import KubernetesPodOperator

namespace = conf.get("kubernetes", "NAMESPACE")

KubernetesPodOperator(
    namespace=namespace,
    image="<your-docker-image>",
    cmds=["<commands-for-image>"],
    arguments=["<arguments-for-image>"],
    labels={"<pod-label>": "<label-name>"},
    name="<pod-name>",
    task_id="<task-name>",
    get_logs=True,
    in_cluster=True,
)
    8
    For each instantiation of the KubernetesPodOperator, you must specify the following values:
    9
  • namespace = conf.get("kubernetes", "NAMESPACE"): Every Deployment runs on its own Kubernetes namespace within a cluster. Information about this namespace can be programmatically imported as long as you set this variable.
  • image: This is the Docker image that the operator will use to run its defined task, commands, and arguments. Astro assumes that this value is an image tag that’s publicly available on Docker Hub. To pull an image from a private registry, see Pull images from a Private Registry.
  • in_cluster: If a Connection object is not passed to the KubernetesPodOperator’s kubernetes_conn_id parameter, specify in_cluster=True to run the task in the Deployment’s Astro cluster.
    • 10
    Add the Google Artifact Registry URI
    11
    When you configure an instantiation of the KubernetesPodOperator, replace <your-docker-image> with the Google Artifact Registry image URI. To retrieve the URI:
    12
  • In the Google Artifact Registry, click the registry containing the image.
  • Click the image you want to use.
  • Click the copy icon next to the image in the top corner. The string you copy should be in the format <GCP Region>-docker.pkg.dev/<Project Name>/<Registry Name>/<Image Name>.