In Apache Airflow®, data pipelines are defined as Directed Acyclic Graphs (DAGs) which represent dependencies between individual tasks in a workflow and can be scheduled to run automatically based on conditions such as at certain times and updates to datasets. With Airflow being the open-source standard for workflow orchestration, knowing how to write Airflow DAGs has become an essential skill for every data engineer.
This eBook provides a comprehensive overview of DAG writing features with plenty of example code. You’ll learn how to:
- Understand the building blocks DAGs, combine them in complex pipelines, and schedule your DAG to run exactly when you want it to
- Write DAGs that adapt to your data at runtime and set up alerts and notifications
- Scale your Airflow environment
- Systematically test and debug Airflow DAGs
By the end of this guide, you’ll know how to create and manage reliable, complex DAGs using advanced Airflow features.
Get Your Copy Today
By proceeding you agree to our Privacy Policy,
our Website Terms and to receive emails from Astronomer.
