Introducing Cohort 2 of the Astronomer Champions Program for Apache Airflow®!

Siva is a seasoned data engineering professional and data architect with extensive experience in Apache Airflow®, having worked on all Astronomer Airflow features. With expertise in designing and developing DAGs, Siva has successfully integrated secret backends (Azure Key Vault) and streamlined ETL and ELT (Snowflake) processes to optimize data flow and task orchestration. His work encompasses the creation of custom hooks, operators, and deferrable operators, as well as implementing XComs and datasets to ensure efficient data exchange and pipeline reliability.

Shivani, a skilled Data Engineer at Shyftlabs, embarked on her Airflow journey in 2021 and has since crafted sophisticated DAGs for multiple organizations. With meticulous attention to detail, she seamlessly integrates Airflow's core and advanced features, including cross-communication, conditional execution, complex scheduling techniques, dynamic DAG generation capabilities, and more, into data workflows while adhering to best practices.

Shivani plays a pivotal role in providing reliable and simplified solutions, ensuring extensibility to meet the dynamic demands of stakeholders. She is enthusiastic about pushing the boundaries of Airflow and contributing to its growth and innovation within the community.

Matt, currently employed at Disney, boasts seven years of experience in the data engineering domain. Introduced to Airflow in 2017, he utilized it to transition legacy processes into a modern orchestration framework and continues to employ it for defining jobs and managing datasets with a keen focus on observability and lineage.

Aside from Airflow, Matt possesses extensive expertise in other data engineering platforms like Databricks and Snowflake. He ardently shares insights with the data engineering community and eagerly anticipates utilizing this platform for further contributions.

Ajith, serving as a Senior Data Engineer at Miniclip, has utilized Airflow for the past 5 years. He led the development of an Airflow orchestration platform from its inception, meticulously designing it to accommodate future growth and evolving requirements. Ajith adeptly leverages features like Dynamic DAG creation and cluster policies to optimize development time and streamline data processing operations.

With a strategic approach to infrastructure management, Ajith ensures scalability, reliability, and cost-effectiveness in data pipelines. His proficiency in crafting reusable code simplifies DAG maintenance, contributing to seamless operations. Committed to driving innovation, Ajith actively shares insights and expertise through Medium blogs, fostering continuous improvement within the data engineering community.

As a Data Engineer, Amandeep specializes in designing and implementing data pipelines to ensure seamless data flow and processing within the organization. A significant part of his daily work revolves around leveraging Apache Airflow®.

Apache Airflow®'s flexible DAG definition allows Amandeep to express complex dependencies between tasks using Python code, enabling him to implement sophisticated data transformations, model training jobs, and batch processing tasks with ease.

Israel Amangoua exudes a profound passion for data. His affinity for Airflow emerged in 2021, igniting a deep appreciation for its promising capabilities in building robust, secure, and optimized pipelines. Leveraging its diverse providers, he extended its functionalities, notably harnessing features like automatic scalability, optimization, and enhanced error management through Dynamic Task Mapping and Sensor Operators in his ETL workflows, all supervised via Airflow's REST API.

Whether at BeOpenIT, Orange CI, or Bnp Paribas, Airflow remains an indispensable tool in his arsenal. Moving forward, alongside writing articles and sharing insights, Israel actively fosters community engagement.

Cody, a Field Engineer at Astronomer, assists users across various industries in learning and adopting Airflow. With his entire career dedicated to data, he has worked with a range of technologies and observed the need for a central system to connect the individual components that comprise the platform. Firmly believing in the Airflow mission, Cody founded the DC Apache Airflow® meetup and remains actively engaged in the community, authoring blogs about Apache Airflow® and other relevant technologies.

Daniel has been using Airflow since 2020 since he joined Hopper as a data scientist, and has been loving Airflow ever since. Since then, he has pivoted into data engineering, and has set up Airflow instances at two additional companies. Daniel is also a frequent open source contributor to Apache Airflow® and various Airflow-related packages.

He is a major contributor to the Astronomer Cosmos package, using his experience building a custom Airflow dbt integration at his previous employer to share insights to the broader Airflow community about how to effectively and idiomatically run dbt in Airflow. You can usually find him helping out in the #airflow-dbt Slack channel.

Deeptansh Aggarwal is a dynamic data engineer who embarked on a transformative journey with Apache Airflow® to revolutionize data orchestration processes. Intrigued by Airflow's potential, Deeptansh immersed himself in its ecosystem, mastering its intricacies to orchestrate complex data workflows efficiently.

Through hands-on experimentation and continuous learning, he honed his skills, becoming a proficient Airflow practitioner capable of architecting scalable solutions for diverse data challenges. Today, Deeptansh is recognized for his expertise in leveraging Airflow to optimize data pipelines, driving actionable insights and innovation in his projects.

In 2022, Fabio undertook the 'Data Orchestration Mission', transforming Python + SQL pipelines into a standardized and efficient system at DAZN. His passion for Software & Data Engineering led him to join lastminute.com, where he orchestrated ETL pipelines with Airflow, developed custom Airflow Operators for data quality checks, and actively manages a Python Community, advocating for Airflow among local Data Engineers.

Federico Benazzo has served as a lead in data engineering and analytics across multiple startups in Europe and the Middle East. He's adept at constructing data infrastructures from the ground up, utilizing Airflow's orchestration capabilities to schedule and execute intricate data pipelines.

Federico's expertise extends to the TaskFlow API, leading various integration projects to enhance Airflow's functionalities by combining it with other systems such as email backends, tracking systems, secrets managers, and logging services. A fervent advocate of the modern data stack, Federico is also deeply passionate about dbt and Tableau development.

Giorgos, currently serving as a Senior Data Engineer at Plum, exhibits a fervent dedication to constructing efficient, scalable, and cost-effective data and MLOps workflows utilizing Apache Airflow®. Throughout recent years, he has made significant contributions to the open-source community, notably creating dbt-airflow, a Python package that seamlessly integrates dbt and Airflow for scalable orchestration and execution of dbt projects.

In addition to his technical prowess, Giorgos actively promotes best practices in the online sphere through platforms like Medium and StackOverflow.

Hari embarked on his Airflow journey in 2019 when his team began exploring options for a modern data orchestration tool. Hari helped set up Airflow on various infrastructure options, including Airflow on an EC2 LocalExecutor, ECS with CeleryExecutor, and AWS MWAA. His team also utilizes Airflow to automate and schedule all of its ETL processes. He looks forward to contributing to the Airflow community.

Jana has been immersed in Airflow since 2022 and plays a pivotal role in a project heavily reliant on Cloud Composer, Google's managed Airflow service. Alongside maintaining and troubleshooting DAGs, she leads the migration of multiple projects from Composer 1/Airflow 1 to Composer 2/Airflow 2, ensuring a seamless transition for the production environment.

Holding the Google Cloud ACE certification, Jana is committed to continuous learning and actively contributes to the tech community by writing technical articles and sharing her data engineering projects on Medium within the Google Cloud community.

Jano Camacho, an experienced data engineer, possesses a robust background in orchestrating ETL processes using Apache Airflow®. Spanning various enterprises including EasyHealth, Tripadvisor, Belcorp, and Rappi, Jano commenced his Airflow journey in 2020 at Belcorp, orchestrating numerous critical pipelines, and continues this trajectory at Strata Analytics where he currently operates.

At Strata Analytics, he plays a pivotal role in enhancing the scheduling logic and reliability of DAGs, aiming for 100% orchestration of ETL processes through Airflow. Demonstrating expertise in leveraging Airflow features such as TaskFlow, XComs, and dynamic DAG generation, Jano simplifies and extends the functionality of data workflows with finesse.

Luan Moreno serves as a Data Engineer at Pythian, contributing to the development of a managed solution that utilizes Apache Airflow® as its orchestration layer. His initial encounter with Airflow dates back to 2018, where he tackled the challenge of building, maintaining, and orchestrating multiple ingestion jobs leveraging Spark as the processing engine.

Beyond his professional endeavors, Luan actively contributes to the community and played a pivotal role in the Astro Python SDK project. He collaborated closely with the team to integrate Azure and Airflow, streamlining the ingestion process of files from the Data Lake. Additionally, he possesses expertise in various technologies including Spark, Databricks, Snowflake, Trino, and dbt.

Manas Bhardwaj, a valuable member of the Data Engineering team at CRED, immersed himself in Apache Airflow® in 2021, bringing his expertise to the innovative data engineering endeavors. With a fervent dedication to constructing open-source data platforms honed throughout his professional journey from Cleartrip to CRED, Manas has spearheaded the development of a comprehensive Airflow framework, enhancing alerting, monitoring, and scaling capabilities.

Additionally, he has crafted a reconciliation platform, designed various ingestion frameworks, and flawlessly migrated approximately 1500 Airflow pipelines. Beyond his organizational contributions, Manas generously shares his knowledge, having delivered a talk at the Airflow Meetup in Bengaluru.

Péter Szécsi, an expert in Apache Airflow® solutions with over 5 years of experience, has assisted both large enterprises and startups in implementing Airflow, enabling them to efficiently streamline their data workflows. Recognizing the critical importance of DataOps practices, Péter is deeply involved in daily operational tasks.

As the CEO and co-founder of DataRock Labs, Péter leverages extensive Airflow expertise to drive the utilization of Airflow and dbt solutions for various projects, evolving DataRock Labs into a premier destination for cutting-edge data engineering solutions. His contributions extend beyond his work, actively participating in multiple open-source projects, including the Astronomer-Cosmos project related to Airflow. Péter also regularly shares his insights and expertise as a speaker at meetups and conferences, contributing to the growth and innovation of the data engineering community.

Priyadarshan Mohanty has been extensively utilizing Airflow since 2020, initially at Fanatics, Inc. and currently at Petco. In both roles, Airflow has played a pivotal role in orchestrating complex ETL processes crucial for business intelligence and analytics. Notably, he has improved operational efficiency by implementing Dataset Aware scheduling, effectively managing intricate upstream and downstream dependencies.

Furthermore, Priyadarshan has enhanced the automation and scalability of these processes through strategic deployment of Dynamic DAG Generation, streamlining workflows and increasing adaptability to evolving business requirements.

Rajesh, a seasoned lead data engineer, boasts a robust background in constructing and maintaining data pipelines tailored for data warehousing and analytical requirements. Proficient in a plethora of ETL tools, data warehousing platforms, APIs, and Python libraries, he harnesses the power of Airflow to seamlessly integrate DAGs into his data orchestration arsenal, ensuring the delivery of robust processes for clients.

Additionally, Rajesh actively contributes to the community through insightful blogs covering various topics such as Data-Aware Scheduling, Dynamic DAG generation, Alerts and Monitoring, Security, among others. His expertise extends to serving as the Subject Matter Expert (SME) for Airflow at Kipi.bi, where he imparts knowledge to data engineers on effectively integrating Airflow with platforms like Snowflake and other applications.

Santosh demonstrates a seasoned proficiency in leveraging Python operators within Airflow for seamless workflow automation. His adeptness extends to proficiently managing Google Cloud Composer, Google's managed Airflow solution, orchestrating efficiency in cloud operations.

Additionally, Santosh's passion for empowering diverse voices in tech shines through his ambassadorship of communities like Women Techmakers, Open UK, and Calico Big Cat.

Vasudev is a seasoned data engineer with extensive experience in Airflow. Introduced to the platform in 2018, he has honed his skills by utilizing various Airflow variants across on-premises and cloud-managed environments. Playing a pivotal role in crafting centralized custom Airflow building blocks, he has seamlessly integrated them into data engineering workflows, showcasing his ability to manage over 750 DAGs in production, thus highlighting Airflow's scalability and operational efficiency.

As a recognized Subject Matter Expert (SME) for Airflow, Vasudev has been instrumental in guiding numerous customers through adopting and implementing Airflow for their data engineering needs. Additionally, he actively engages with the tech community as a Google Developer Expert (GDE) in the cloud, sharing valuable insights and providing mentorship to aspiring data engineers.