The Astronomer Blog: Data Orchestration Insights & Guides

November 26, 2024

Airflow in Action: Infrastructure Management Insights from 1 Million Monthly Deploys at LinkedIn

Discover how LinkedIn uses Airflow to orchestrate 12k pipelines managing 1m deploys for 7k services. And Airflow 3.0 will make provisioning even easier!

November 21, 2024

Airflow in Action: Data Engineering Insights from Processing PBs of Data Every Day at Stripe

Learn how Stripe processes petabytes of data daily with Apache Airflow, ensuring compliance while accelerating developer workflows with its dev/test tooling

November 19, 2024

Airflow in Action: Scaling Insights from Bosch and 1.2 Million Pipeline Runs Per Day

Go on an Airflow journey with Bosch as they scale from 1k to 50k DAG runs per hour and 1.2 million pipeline runs per day with an average 1 second latency.

November 15, 2024

Customer Story: Autodesk’s Data Engineering Transformation with Astronomer and Apache Airflow

Autodesk scales with Astro! 🚀 Learn how they migrated from Oozie to Airflow, boosting efficiency, scalability, and data-driven decision making.

November 15, 2024

It Pays to be Picky: Leveraging Key Insights with Airflow and Astro

Here are four key observability insights to consider focusing on when setting up an observability solution: data freshness, on-time delivery, data dependencies tracking, and data quality.

November 14, 2024

Airflow in Action: Supercharging Data Science and ML Workflows At Apple

Learn how Apple uses Airflow to streamline experimentation with Jupyter Notebooks, boost productivity, and support large-scale, distributed ML workflows.

November 7, 2024

Airflow in Action: ETL Insights from Bloomberg — Slashing Runtimes by 50%

Learn why Bloomberg selected Airflow over Dagster and Prefect, reducing the ETL of 50 million loans and 5 billion data points by 50%

November 7, 2024

Airflow in Action: Data Engineering Insights from Burns & McDonnell

Discover how Burns & McDonnell scaled data delivery from ad-hoc workflows to a unified platform using Apache Airflow, providing reliable data in under 24 hours.

November 5, 2024

Update: Astro Observe is Now In Public Preview

Today, we’re excited to announce Astro Observe, which brings new and more robust data observability capabilities to users on Astro, OSS Airflow, Amazon Managed Workflows for Apache Airflow (MWAA), and Google Cloud Composer (GCC).

Astronomer Blog

Build, run, & observe your data workflows. All in one place.

Build, run, & observe
your data workflows.
All in one place.