Published on

    Good Data at Good Eggs: Using Dagster to manage the data platform

    Running pipelines is only part of the operational burden of running a data platform. We also need to manage the platform itself and control associated technical debt. We found that Dagster was a very natural place to do that work, with the advantage that our entire operational view of the platform is consolidated in a single tool.
    Published on

    Good Data at Good Eggs: Data observability with the asset catalog

    What we’re aiming for with Dagster is a completely horizontal view of our data assets. Our analysts will be able to look up when a raw data ingest from Stitch occurred, when a dbt model ran, or when a plot was generated by a Jupyter notebook and posted in Slack, through a single portal — a single "pane of glass."
    Published on

    Good Data at Good Eggs

    Adopting Dagster was a catalyst for the transformation of our data platform team. We hope our experience is encouraging to other teams facing similar challenges and opportunities.
    Published on

    Dagster: The Data Orchestrator

    Dagster is a new type of workflow engine: a data orchestrator. Moving beyond just managing the ordering and physical execution of data computations, Dagster introduces a new primitive: a data-aware, typed, self-describing, logical orchestration graph.
    Published on

    Dagster 0.6.0: Impossible Princess

    With 0.6.0, Dagster comes “batteries-included” — but still with pluggable options — for everything you need to execute, monitor, schedule, deploy, and debug your data applications.
    Published on

    Introducing Dagster

    Today the team at Elementl is proud to announce an early release of Dagster, an open-source library for building systems like ETL processes and ML pipelines. We believe they are, in reality, a single class of software system. We call them data applications.