How Atmosphere.tv Scaled Data Pipelines and Saved $10,000 Annually with Astronomer

Company Background: Transforming Public Venues with Engaging, Revenue-Driving Content

Atmosphere.tv is a leading digital out-of-home media company that transforms idle TV screens in public venues into engaging content channels that entertain, advertise, and generate revenue. Atmosphere’s unique business model supports establishments like bars, restaurants, and retail locations by providing engaging, audio-free content through Apple TV-based applications and custom Android devices. This curated content keeps customers engaged without the need for sound, increasing dwell time, which directly impacts ad revenue for the business.

As a data-driven company that relies heavily on advertisers, Atmosphere needs a robust data infrastructure to manage ad revenue, track streaming metrics for clients, and identify high-potential leads for device placements. In the words of Chris Brundage, Atmosphere’s Data Engineer, “Our growth meant we needed something scalable, fast, and efficient. We were at a place where our existing tools just weren’t keeping up.”

Initial Situation: Domo’s Limitations Stifled Growth and Productivity

Before using Astronomer and Airflow, Atmosphere relied on Domo for data orchestration, often supplemented by VM-based cron jobs. Domo had served its purpose early on, but as Atmosphere scaled, Domo’s limitations became glaring. Its data transformation speeds were frustratingly slow, frequently hitting 24-hour processing timeouts and making it impossible to keep up with Atmosphere’s growing data needs. “Domo’s data transformations were not good,” Chris said. “To the point where I’d make a change, test it, and then have hours to kill while waiting for Domo to process. Eventually, it was costing us productivity and performance.”

Another challenge with Domo was its lack of a staging environment and version control system, which hindered collaboration and made every change immediately live in production. Chris explained: “If two developers wanted to work on the same thing, it was just tough luck. Every change was live, and we had no way to safely roll back or revise. There was no real dev workflow.” As Atmosphere’s data grew, it became clear that a modern, scalable data stack was critical to the company’s future.

Instead of running each model one at a time, Cosmos lets me run everything in parallel where possible, which saves us hours.

Chris Brundage
Data Engineer
Atmosphere TV

Decision Process: Choosing Astronomer and Airflow to Build a Modern Data Platform

Recognizing the need for a more robust data solution, Atmosphere began evaluating alternatives that could handle complex data workflows, scale efficiently, and work well with the modern data tools they wanted to adopt. Airflow quickly emerged as the preferred orchestration tool for its flexibility, wide support community, and reputation as an industry standard for data pipeline automation. “Airflow was the obvious choice,” Chris explained. “It’s widely adopted, so learning and support would be easier, and it integrates seamlessly with our existing tools, like Snowflake and dbt.”

To manage Airflow without dedicating resources to infrastructure, Atmosphere chose Astro (Astronomer’s fully managed Airflow service), which eliminated the need for in-house DevOps. “I’d done DevOps before, but it wasn’t something I wanted to go back to,” Chris shared, explaining, “Given the extensive time and resources needed to manage infrastructure, I knew we’d benefit from offloading that responsibility. Astronomer made that part easy, allowing us to focus on data engineering, knowing the infrastructure was in expert hands.”

Alongside Airflow, Atmosphere implemented Snowflake as their data warehouse, Sigma for business intelligence, and dbt for data transformations. This combination gave them a flexible, powerful data stack that could scale with their needs. “Astronomer is the backbone of the operation, keeping everything in Airflow moving seamlessly,” Chris explained.

Implementation: A Seamless Six-Month Migration to Astronomer’s Managed Airflow

The migration from Domo to Astronomer took around six months from contract to completion. With Astronomer’s managed environment and robust documentation, the transition was straightforward. Astronomer’s support team provided hands-on guidance, with an Astronomer team member available in a shared Slack channel to answer questions, troubleshoot, and guide Atmosphere through the best practices. Chris appreciated this real-time access, saying, “I had a ton of questions, and they were patient with every single one. They helped me think of Airflow as more than just a Python tool—it was a fully capable orchestration platform.”

Astronomer’s Docker setup for local development further eased the transition. By offering pre-configured environments, Astronomer allowed Chris to test locally without setting up or managing multiple connections and variables. “I didn’t have to figure out Docker Compose or Airflow containers from scratch,” Chris shared. “Everything was ready for me, which meant I could get up and running faster.” Atmosphere’s deployment process benefited from Astronomer’s pre-built CI/CD pipeline documentation, which Chris customized slightly to fit the team’s needs.

To ensure efficient data transformations, Atmosphere introduced Cosmos at this stage, which became a pivotal tool for enhancing workflow performance and cost efficiency.

Cosmos: Empowering Flexible, Efficient Data Transformations

One of the most impactful aspects of Atmosphere’s new data setup has been Cosmos, which Atmosphere implemented to handle dbt transformations within Airflow. Cosmos empowered Atmosphere to leave behind dbt Cloud’s web-based limitations and create a truly integrated data transformation experience. “Switching to Cosmos was a no-brainer. It was cost-effective, easy to set up, and didn’t even require us to rework much of our dbt code.” Cosmos allowed Atmosphere to manage dbt models as individual Airflow nodes, making it easier to visualize and manage workflows while retaining version control in a Git-based system.

Cosmos also optimized Atmosphere’s data processes by enabling parallel processing of dbt models, which has reduced bottlenecks on data-intensive jobs. Cosmos identifies models that can be run in parallel by analyzing dependencies, allowing jobs to run concurrently where possible. Chris noted, “Instead of running each model one at a time, Cosmos lets me run everything in parallel where possible, which saves us hours.” Previously, running all dbt models sequentially would take up to two hours on local machines, whereas Cosmos’s parallel processing has improved this runtime significantly.

Cosmos’s post-deployment management has been equally beneficial. Chris shared, “With the ‘after-deploy’ DAG, we’re able to target only the modified dbt models and their dependencies, which has cut our post-deployment time down from hours to as little as five minutes.” By enabling selective refreshes, Cosmos avoids the need for full pipeline runs, optimizing both time and resource usage. “With Cosmos, we can target specific models for refreshes instead of running everything, which saves us time and reduces our Snowflake bill,” Chris noted. This capability has allowed Atmosphere to save on operational costs without compromising on speed or accuracy.

The introduction of Cosmos not only streamlined Atmosphere’s workflows but also delivered substantial cost savings. “We saved about $10,000 a year by switching to Cosmos, and it fit right in without even needing us to change our dbt code significantly,” Chris shared. This transition enabled better integration between transformations and orchestration in Airflow.

Cosmos provided the flexibility to move dbt models into Airflow DAGs as individual nodes, making complex transformation workflows more manageable. Chris elaborated, “Cosmos lets me map out each dbt model as a node in Airflow, which is a massive improvement. Running models in parallel saves hours, especially on high-volume jobs.”

We saved about $10,000 a year by switching to Cosmos, and it fit right in without even needing us to change our dbt code significantly.

Chris Brundage
Data Engineer
Atmosphere TV

Post-Implementation: How Astronomer Transformed Atmosphere’s Data Pipelines

Implementing Astronomer and Airflow delivered substantial gains in operational efficiency, development scalability, and cost savings, transforming Atmosphere’s data workflows. Processing speeds improved significantly, addressing one of the most critical pain points from their Domo days. Chris explained, “With Astronomer, what used to take hours now takes minutes. I’m able to actually do my job without sitting around waiting for things to process.”

This speed increase provided valuable data to decision-makers more quickly and accurately than ever before, allowing Atmosphere to scale its data pipelines alongside its ad revenue and client service initiatives. Astronomer’s robust infrastructure management enabled Atmosphere’s data engineering team to focus on building efficient workflows without worrying about server maintenance.

Additionally, the system’s flexible DAG management and real-time monitoring capabilities enhanced Atmosphere’s operational transparency and accountability. “I have about 34 DAGs running, and the flexibility to set them up exactly the way we need has been fantastic,” Chris added.

Looking Ahead: Scaling with Astronomer and Cosmos for Future Growth

Looking ahead, Atmosphere’s goals with Astronomer and Cosmos are focused on increasing efficiency, maintaining low operational costs, and refining data processes to optimize ad performance tracking. Chris emphasized that their primary focus is on refining ad revenue tracking and integrating data from Supply-Side Platforms to reconcile ad spend. Cosmos has proven to be a long-term investment in managed Airflow that allows Atmosphere to fine-tune data workflows, ensuring that they can meet both client and advertiser demands.

Atmosphere’s migration from Domo to Astronomer, enhanced by Cosmos, transformed its data stack from limited and costly to fast, flexible, and scalable. “Astronomer took care of the heavy lifting so we could focus on what matters,” said Chris. With Cosmos supporting efficient transformations, Atmosphere is positioned to manage its data growth sustainably, serve advertisers and clients with high-quality insights, and innovate within the competitive digital out-of-home market.