Get started with Observe

High-level technical prerequisites and setup steps for Astro Observe.

Astro Observe delivers pipeline-aware data observability purpose-built for Airflow, giving your team visibility into the health, quality, and performance of your business critical data products across the modern data stack. This guide outlines the fundamental requirements and steps to onboard.

Prerequisites

• Astro Deployment running Astro Runtime 9 or later (apache-airflow>=2.7.0)
• OpenLineage Airflow provider (apache-airflow-providers-openlineage>=1.12.1) and OpenLineage client (openlineage-python>=1.38.0) specified in project dependencies (for example, in requirements.txt)
• At least one asset running in Airflow
• Observe Member, Billing Admin, or higher role required to create data products (see user permissions reference)
• OpenLineage enabled for Remote Execution Agents if using Remote Execution

Astronomer recommends using the latest possible OpenLineage provider version and the latest OpenLineage client version. The client can be updated regardless of your Airflow version. See Configure OpenLineage on Astro for more information and upgrade instructions.

Set up Astro Observe

1. Enable and verify OpenLineage versions
2. Confirm your Deployments are running the recommended OpenLineage client and provider versions by checking your Astro project’s requirements.txt. See Configure OpenLineage on Astro for instructions on upgrading to the recommended client and provider versions.
3. Review assets and lineage
5. Create your first data product
6. A data product is a composition of assets that, taken together, deliver a result with business relevance.

   For guidance on identifying data products in your organization, see how to identify data products in your organization.

7. In the Astro UI, navigate to Observe > Data Products. Select + Data Product and follow the prompts to define your product and choose assets.
8. See Create a data product for more information.
9. Set a Service Level Agreement (SLA)
10. Add an SLA to your data product to monitor freshness or timeliness of your data product. See Create an SLA.
11. Configure alerts
12. Set up alerts to be notified of SLA violations or potential failures. To monitor the SLA you just created, click the SLAs tab in your data product, and select the SLA you want to monitor. Click + Alert to set up a Data Product SLA Violation Alert. See Create an alert for details on alert configuration settings.
13. Grant permissions to team members
14. To allow others to create data products, SLAs, and monitors in Observe, grant the Observe Member or Observe Admin role to any teammates who need to set up and manage these features. See user permissions reference for instructions on managing roles.

Next steps and resources

• For in-depth guides, advanced configuration, and best practices, see the full Create and use data products documentation.
• Explore how to work with assets, monitor data products, and set up notifications.
• Review permissions to ensure your team has the right access.