How to Improve Data Quality with Airflow's Great Expectations Operator

Watch On Demand

Hosted By

  • Benji Lampel Benji Lampel Ecosystem Engineer
  • Kenten Danas Kenten Danas Lead Developer Advocate

This webinar provides an overview of how to use the new Great Expectations (GX) operator to implement data quality checks in your Airflow DAGs. We cover new features of the GX operator, including a default Checkpoint feature, which makes Great Expectations more Airflow-centric and simpler to use. Questions covered in this session include:

What improvements have been made to the GX operator in the latest version? How can I implement the GX operator in my Airflow DAGs? How can I scale my data quality checks to large datasets using the GX operator and Airflow? What kinds of data quality checks can I implement with the GX operator?

Learn more about the Great Expectations provider in the public repo. The example code covered in this webinar can be found on the Astronomer Registry.

Astronomer Apache Airflow Fundamentals Certification badge

Get Apache Airflow Certified

If you want to learn more about how to get started with Airflow, you can join the thousands of other data engineers who have received the Astronomer Certification for Apache Airflow Fundamentals. This exam assesses an understanding of the basics of the Airflow architecture and the ability to create simple data pipelines for scheduling and monitoring tasks.

Learn More About Certification