Flo Energy's Data Platform for Critical Energy Data

Flo Energy’s aim is to help businesses access affordable, renewable electricity solutions. In Singapore, Flo positions itself as a leading independent electricity retailer, offering commercial electricity plans, sustainability support, and tools like the FlowSmart dashboard for usage, billing, and contract management. In Australia, Flo extends that model with commercial electricity plans and renewable certificate offerings, including products that match customers' electricity consumption with Large-scale Generation Certificates (LGCs).

Supporting our business is a data operation that turns fragmented meter, weather, market, and strategy data into reliable insights. To accomplish this, Flo adopted Dagster as its centralized orchestration layer. What began as a platform for the data team has since evolved into an important platform supporting the business, enabling engineers and adjacent teams to build, monitor, and debug workflows across the organization.

Turning energy data into business decisions

Energy companies rely on data that is both technically reliable and operationally valuable. At Flo, Nils Terpstra, who leads the data and analytics team, estimates that 80% of the company's business is data-centric, requiring the synthesis of complex meter, weather, market, and strategy data into actionable insights.

This challenge is common across the energy industry, where providers operate in fragmented, time-sensitive environments. Pipeline failures are more than engineering headaches; they can disrupt pricing, forecasting, reporting, and overall operational confidence.

Flo required a platform that could unify scheduled batch workflows, event-driven processes, dbt transformations, Snowflake analytics, and downstream BI, while remaining accessible enough for broad cross-functional collaboration.

A history with Dagster

When Nils joined Flo, he was not approaching Dagster for the first time. One year earlier, he had worked at another electricity retailer facing similar problems. That team had been using an alternative orchestration tool to run data jobs.

This alternative tool could run scheduled jobs and trigger work, but it did not provide the asset-aware visibility, dependency management, or data-specific debugging experience that energy data workflows require. Realizing the shortcomings, the data team successfully migrated the company's data workflows to Dagster as a dedicated data orchestration platform.

That earlier experience shaped the decision when Nils joined Flo. Rather than start with generic job scheduling and grow into orchestration later, Nils chose Dagster at Flo when establishing the data team.

When I joined Flo and saw a similar need, it was clear that Dagster was the right foundation to start with and it was great to see how much the platform had matured, from the features and integrations to the documentation.

That maturity mattered. Flo was not just looking for a tool that could run jobs. They needed a platform that could support the way an energy data team actually works: ingesting data from multiple sources, transforming it, validating it, and exposing the results to both technical and business users.

Scaling with Dagster

Dagster integrates Flo’s analytics stack: Snowflake for storage, dbt for transformations, and Holistics for reporting all into a unified orchestration layer. This is ideal for a platform that requires both scheduled batch jobs and event-driven updates.

Event-driven workflows are handled by Dagster’s sensor-based automation coupled with schedules to handle orchestration. Rather than feeling like disconnected jobs, everything works together to produce the final forecasts necessary for the business. As the platform grew, Dagster’s code locations enabled decentralized ownership, allowing teams to manage their own workflows while maintaining visibility and operational control across the organization.

Today, the platform serves as a centralized hub, successfully orchestrating more than a thousand assets to support Flo's commercial operations.

By choosing this platform approach rather than the reactive "maintenance approach”, Flo positioned the platform to scale alongside their business needs.

Observable energy workflows

With Dagster, Flo’s workflows became easier to trace from source data to downstream outputs. When something failed, the team could see where it failed, what depended on it, and what needed to be rerun. Instead of asking whether a scheduled job had completed somewhere, the team could inspect the state of the data assets themselves.

That shift is particularly important in energy, where data freshness and completeness matter. If meter data is missing, weather data is stale, or a transformation fails silently, downstream pricing and forecasting workflows can become less trustworthy. Dagster helped make those risks visible.

The result was a platform that supported rapid troubleshooting and more confident operations. Failures became easier to investigate. Alerts became easier to connect to the workflows they represented. Data quality checks became part of the pipeline itself, rather than a separate manual process.

With this new observability, the team has achieved an average materialisation success rate of approximately 99.9% across its monitored production workloads. 

Building quality into the pipeline

Data quality is fundamental for an energy provider. Flo’s data needs to support operational decisions, commercial reporting, sustainability programs, and customer-facing insights. That means the platform needs to identify issues before they affect downstream reporting and analytics.

Dagster’s asset checks gave Flo a way to bring validation closer to the assets being produced. Asset checks are designed to test specific properties of data such as whether a column contains null values, whether a table matches an expected schema, or whether an asset needs refreshing. Dagster also supports modeling existing dbt tests as asset checks.

That matters because Flo already uses dbt for transformation. Instead of duplicating business logic in the orchestrator, Dagster can let dbt remain the place where transformation logic lives, while representing dbt models, dependencies, failures, logs, run history, and checks in the Dagster asset graph. Dagster’s dbt integration represents dbt models as assets and lets teams track failures, logs, run history, and dependencies at the level of individual dbt models.

For Flo, that separation of concerns is important. Dagster orchestrates the system. dbt owns much of the transformation logic. Snowflake stores the analytical data. Holistics exposes the outputs. Each layer has a clear role, and Dagster makes the whole system observable.

Why Dagster works for energy data teams

Energy data platforms have a specific shape. They are multi-source, time-sensitive, and decision-oriented. They combine historical usage, external market signals, operational constraints, customer information, and forecasts. They often require both scheduled processing and event-driven automation.

Dagster fit Flo’s needs because it gave the team a practical way to organize that complexity.

Python and YAML support made the platform approachable and flexible. Flo could use Python where workflows required full programming flexibility, while Dagster Components and YAML-based definitions made common patterns easier to read and maintain. Dagster Components are designed to provide project structure, common integration types, and a YAML DSL.

Native dbt support kept transformation logic where it belonged. Flo did not need to move business logic out of other systems and into the orchestration layer. Instead, Dagster could represent dbt models as assets, connect them to upstream and downstream dependencies, and surface failures and checks in the orchestration layer.

Sensors enabled event-driven workflows. Not every energy workflow has a consistent schedule. Some workflows should run when external data arrives or when an upstream asset is ready. Dagster sensors gave Flo a way to respond to those events and build more natural automation around real-world data availability.

Code locations helped teams scale ownership. As adoption grows across engineering and adjacent teams, Flo could separate workflows into different code locations. That will give teams autonomy without creating one tangled project. Data workflows can remain cleanly separated from other internal workflows, while the organization still benefited from a shared orchestration layer.

Debugging became a platform capability, not a tribal skill. For Nils, one of the clearest benefits of Dagster was the ability to understand what happened when something went wrong. Having previously witnessed a migration to Dagster at his former company, Nils specifically valued how Dagster replaced the tribal, manual debugging required in the old tool with a platform capability. Instead of chasing logs across disconnected systems, teams could use Dagster to inspect runs, assets, dependencies, and failures in context.

Advice for teams building with Dagster

When asked what advice he would give to other teams evaluating Dagster, Nils kept it simple:

Just do it.

For those unsure where to start, consider picking one core, high-impact data pipeline such as a critical forecasting or billing workflow and migrate it to Dagster. This provides immediate visibility and proof of value without requiring a full infrastructure overhaul.

That advice reflects Flo’s experience. Starting with Dagster gave Flo a foundation that could support that growth. It allowed the team to begin with core data workflows, then expand into more event-driven automation, more quality checks, more dbt integration, and more team-owned code locations over time.

For energy companies facing similar challenges, Flo’s story offers a useful pattern: do not wait until orchestration becomes painful to invest in it. Build the platform before the complexity becomes unmanageable.