How easyJet Holidays Achieved 15x Faster Data Pipelines with Dagster

easyJet Holidays is the package holiday division of easyJet, offering bundled flight and hotel packages to destinations across Europe and beyond. By implementing Dagster as their modern data orchestration platform, they created a unified control plane that helps their engineering team confidently build, scale, and observe their data pipelines. Adopting Dagster enabled the easyJet Holidays team to make pipeline execution 15x faster, improve data freshness, and automate cumbersome manual processes that previously required dedicated support resources.

Currently, Dagster is the heart of the analytics platform and it’s doing a fantastic job. Even if we only ever used it for analytics we’re already getting loads of value, but next we're starting to look at Dagster for a kind of operational data platform.

Key Results

• Pipeline execution time: 2.5 hours → 10 minutes
• Cost savings: Dedicated support resources for troubleshooting → fully re-allocated spend
• Data freshness: Self-healing pipelines enhance reliability with more consistent and timely data

A complex and inefficient data pipeline — and an opportunity

easyJet Holidays began as a subsidiary of easyJet five years ago, operating as a relatively independent startup within the larger airline organization. Initially, Tom Cronin the Head of Data says, his team was “kind of borrowing the airline data platform” and using some of their infrastructure.

The company had already built its own independent data platform using basic AWS native services, Redshift, Glue, and Airflow, but the Holidays subsidiary had very different data needs. “The airline has all of these big, heavy, expensive requirements, and we don’t…It was like using a sledgehammer to drive a thumbtack,” Tom explains. Beyond being designed for vastly larger-scale workloads, the airline’s infrastructure came with a necessary amount of beauracracy that slowed down the smaller, more agile holiday business.

While functional, this vanilla setup created significant challenges that perfectly matched the pain points that Dagster was designed to solve:

• No unified control plane for observability. The existing “vanilla” system lacked end-to-end visibility across the data platform. There was no way to track lineage between data assets and it was impossible to see the full impact of failures or changes.
  
  
• Poor reliability and manual intervention. Pipeline execution times were lengthy, averaging 2.5 hours. There were also frequent pipeline failures that required dedicated support resources to manage them.
• Siloed teams without collaboration. In the existing system, analysts couldn't contribute to the data pipeline. There were no standardized workflows across teams, and changes required specialized knowledge of multiple systems

“There were constantly questions like, ‘Oh, I'd like to make this change. Now, is that a change to the glue job? Is it a change to the SQL script? Or is it a change in my Airflow config?’ And, by the way, the person asking that question doesn't know what any of those things are because they're an analyst whose job is to create dashboards,” Tom explains. “So immediately we had this huge problem of no one really understanding the end-to-end.”

A “mish-mash” system

When Tom arrived at easyJet Holidays as the new Head of Data, he quickly discovered that there was also a need for some good ETL tooling. “We built a load of essentially custom hacky Python code in Glue to do our data extraction. It was a mish-mash that worked — pretty much, anyway — but also it lacked some basic hygiene stuff,” Tom says.  

But he also recognized they needed more than just another ETL tool — they needed a modern data orchestration platform. Failed jobs required frequent manual intervention, fragmented tooling led to extended troubleshooting times, and making changes involved a data engineer parsing through 2000 lines of SQL. In the midst of all these issues, Tom saw more than the need to retool: he saw the opportunity to convert from a legacy stack to a powerful platform built on best practices:

I want anyone who writes SQL to be able to contribute changes. Data engineers should be able to write code for pipelines and follow software development practices. At the same time, analysts need the right access and visibility to make changes without having to open a ticket.

Why easyJet Holidays chose Dagster

The easyJet Holidays team initially explored a number of solutions on the market. While a number of the tools solved parts of the problem, he quickly realized they needed a more comprehensive solution—a true modern data orchestration platform that could serve as the unified control plane for their entire data platform.

After researching options, Tom discovered Dagster through the data engineering community on Reddit, where engineers consistently praised Dagster’s developer-centric approach and powerful capabilities. As he explored the platform, Tom found that Dagster aligned with his priorities for easyJet Holidays’ new data platform:

Democratized workflows: Dagster provided a single pane of glass for easyJet Holidays' entire data estate, with a unified catalog and lineage tracking across all data assets. This enabled the team to standardize workflows while preserving autonomy for different teams, supporting both data engineers and analysts with varying skill levels through one cohesive platform.

End-to-end visibility: The platform offered end-to-end visibility for faster debugging and better observability, with built-in monitoring and alerts to prevent data outages. Asset checks and testing ensured data quality throughout the pipeline, while branch deployments allowed the team to test changes safely before deploying to production.

Code-first approach: Dagster's code-first approach aligned perfectly with software engineering best practices, providing a clean, Python-based development framework. The local development experience allowed for faster iteration, with full Git integration for version control and declarative automation that significantly reduced manual work for the engineering team.

Flexibility and scalability: With native integrations for modern tools like dbt and the flexibility to adapt as their platform evolves, Dagster offered easyJet Holidays the ability to scale without increasing complexity. The platform's design reduced operational overhead while ensuring they could continue to evolve their data capabilities.

We needed a single pane of glass for our entire data estate, with everything in Python and Git so our team could follow software development best practices. Dagster did all of this and enabled our analysts—and anyone writing SQL—to contribute directly through pull requests instead of opening support tickets for the data team.

A rebuilt data platform with Dagster at the center

The easyJet Holidays Dagster implementation serendipitously coincided with a planned AWS migration, timing that Tom called "quite fortunate" for rebuilding their platform. "We were about to do a big migration moving between AWS accounts. Normally, I would see it as a headache, but I thought, ‘Hey, this is actually a great opportunity to not lift and shift anything – let’s instead rebuild the platform in the new environment.’

The team took an ambitious “no mess” approach of bringing zero code from their previous platform, essentially starting fresh with Dagster as the unified control plane for orchestration, observability, and collaboration. They designed a data-centric architecture from the ground up to solve the fundamental issues in their original system:

Data-centric architecture with assets as first-class citizens

• Shifted from task-oriented Airflow DAGs to assets that represent actual data
• Implemented a medallion architecture pattern with bronze (raw), silver (cleaned), and gold (business) layers
• Created modular, maintainable SQL transformations with dbt integration
• Organized work around data assets rather than tasks for better maintainability

End-to-end lineage and observability

• Mapped all data assets and their dependencies in Dagster's unified catalog
• Gained complete visibility into data flow across the entire platform
• Set up freshness policies to monitor data timeliness
• Enabled faster debugging and impact analysis with comprehensive lineage tracking

Modern developer experience

• Created development environments with branch deployments for safe testing
• Set up individual developer databases for local testing and iteration
• Implemented software engineering best practices with Git integration
• Enabled unit testing for data transformations

Built-in data quality and reliability:

• Integrated dbt tests as Dagster asset checks
• Implemented retry policies with appropriate backoff
• Added automation conditions to recover from common failures
• Built monitoring to alert only on genuine issues

A smooth and straight-forward onboarding process

Tom approached the onboarding process methodically, aiming for a process that could adapt as needed for each data team member’s needs and capabilities. He personally handled the initial setup and built the initial version “with confidence” due to Dagster's intuitive Python framework. Tom tested his proof of concept with real data flows before moving to team-wide onboarding:

Step 1: Demonstrating end-to-end visibility. Tom first introduced using Dagster's UI as the single pane of glass for the entire platform. He demonstrated how lineage tracking makes dependencies visible and troubleshooting easier, allowing team members to see exactly how data flows through their systems. He also showed how data assets were now connected across the entire platform, making it possible to draw data insights that their original fragmented toolset never allowed.

Step 2: One platform, many teams. Tom began with introducing analysts to simple dbt SQL models in the familiar IDE of their choice. Meanwhile, data engineers learned the more advanced Dagster Python framework for orchestration. Each team got to work with their preferred tools while still contributing to the same platform, using a workflow that enabled collaboration through Git and pull requests rather than tickets. 

Step 3: Showcasing reliability by design. Next, Tom used actual failure scenarios to demonstrate Dagster’s resilience. He showed the team how self-healing pipelines automatically recover without manual intervention — and, in the process, demonstrated the value of pipeline visibility. “I basically got it wrong on my first try and made a test downstream fail,” Tom says. “And I think that was a real light bulb moment for a lot of people on my team — ‘Oh, because it failed, that's how he found the error.’

The Dagster and dbt integration created a clean separation where our data engineers work with Dagster code while analysts just interact with SQL—which means any SQL developer in the organization can contribute to our pipeline without needing to understand the underlying details.

Reliability and efficiency, plus developer experience

Since implementing Dagster as a critical part of their data platform, the easyJet Holidays team has experienced a dramatic improvement in data pipeline reliability, operational efficiency, and developer experience. 

• Dramatically reduced pipeline execution times. The shift from Airflow and Glue to Dagster and dbt enabled the team to move transformations outside of source systems, expediting execution times from multiple hours to minutes.
• Faster failure identification and resolution. Dagster’s advanced dependency mapping, traceability, and asset-based model have drastically reduced the time spent identifying and resolving pipeline failures. When issues occur, the team can now do partial runs of the pipeline for specific assets, greatly reducing troubleshooting time.
• Modular data platform. Monolithic SQL scripts have been replaced by modular, maintainable code, structured around assets.
• Increased collaboration. With the adoption of Dagster and dbt, anyone who writes SQL can now contribute through pull requests.
• Improved testing practices. The introduction of testing environments through branch deployments further encourages contributions and collaboration from analysts and anyone writing SQL.
• End-to-end visibility: Dagster acts as a single pane-of-glass for their data estate with system insights and failure tracking that drastically improve troubleshooting and decision-making for the team.

Future Dagster-driven feature opportunities

Having established Dagster as their unified control plane for batch analytics, the easyJet Holidays team is now exploring how to leverage Dagster's composable architecture for operational use cases around real-time and streaming data: 

• Expanded workload support:
  
  • Operational data platform with real-time and streaming capabilities
  • AI/ML workflow integration within the same platform
  • More complex data transformations and analytics
• Deeper integration of platform components:
  
  • Closer alignment between analytical and operational data systems
  • Additional data quality tooling and monitoring
  • Enhanced metadata management across the platform

The team values Dagster's horizontal integration capabilities, which provide confidence that they can incorporate whatever new tools emerge in the data ecosystem without being locked into vendor-specific solutions.

As Tom explains, having Dagster as their unified control plane means "whatever new testing framework appears, the worst-case scenario is you just have a Dagster asset that says 'I ran that thing,' but it can do it." This flexibility and future-proofing are key to their long-term data strategy as they continue to scale across the organization without added complexity.