Meet Compass — Dagster’s new AI data analyst for Slack. Turn questions into trusted insights, instantly. → Try Compass now.
Blog
November 17, 2025
When (and When Not) to Optimize Data Pipelines
Engineers often optimize the wrong parts of their pipelines, here's a profiling-first framework to identify real bottlenecks and avoid the premature optimization trap.
Oops! Something went wrong while submitting the form.
November 7, 2022
Adding Types to a Large Python Codebase
What we learned when we introduced dynamically typed code to a large Python codebase, bringing Dagster's public API to 100% type coverage.
Sean Mackesey
Engineering
October 31, 2022
Orchestrating Machine Learning Pipelines with Dagster
How to use Dagster’s open source data orchestrator to build machine learning pipelines and train ML models.
Sandy Ryza
Engineering
October 25, 2022
Build a poor man’s data lake from scratch with DuckDB
DuckDB is so hot right now. Learn how to build a data lake from dbt using DuckDB for SQL transformations, along with Python, Dagster, and Parquet files.
Pete Hunt
Engineering
October 19, 2022
The Unreasonable Effectiveness of Data Pipeline Smoke Tests
Data practitioners waste time writing unit tests to catch bugs they could have caught with smoke tests.
Sandy Ryza
Engineering
October 17, 2022
Web Workers are not the Answer
A tale of overstretched logs, counterintuitive web worker behavior, and ultimately a troublesome cursor issue.
Jordan Sanders
Engineering
October 16, 2022
Dagster at all 5 Steps of the Development Lifecycle
Dagster facilitates a data engineers work across all five steps in the development lifecycle.
No items found.
Product
October 6, 2022
A Dagster Crash Course
If you are looking to get up and running with Dagster in 10 minutes or less, this is a good place to start. Buckle up.
Pete Hunt
Product
October 4, 2022
Postgres: a Better Message Queue than Kafka?
When lots of event logs must be stored and indexed, Kafka is the obvious choice. Naturally, our queue runs on Postgres.
Pete Hunt
Engineering
August 17, 2022
Spend Less Time Debugging with Dagster
It’s not uncommon for a data engineer to devote 80% of their day to debugging. Dagster radically improves on this.
Sandy Ryza
Engineering
August 9, 2022
Launching Dagster Cloud to GA
The enterprise orchestration platform that puts developer experience first: hybrid or serverless deployments, native branching, and out-of-the-box CI/CD.
Nick Schrock
Product
August 5, 2022
Introducing Dagster 1.0: Hello
Announcing Dagster 1.0. - a stable foundation for building the orchestration layer for modern data platforms.
Sandy Ryza
Releases
August 3, 2022
The Open Core Business Model
The relationship between Dagster, the open-source project, and Dagster Cloud, our hosted SaaS platform.
