Introducing Dagster 1.0: Hello

We are extremely excited to announce the release of Dagster 1.0.

1.0 doesn’t include any seismic changes - rather, it’s a marker that indicates we’ve put the finishing touches on Dagster’s core abstractions. Data teams who want to access what’s unique about Dagster now have a stable foundation to build on.

Dagster is different from other data orchestrators. It’s the first orchestrator built to be used at every stage of the data development lifecycle - local development, unit tests, integration tests, staging environments, all the way up to production. And it’s the first orchestrator that includes software-defined assets - it frees up teams to think about critical data assets they’re trying to build and let the orchestrator manage the tasks.

The Dagster journey started four years ago, when Elementl’s founder and CEO Nick Schrock laid down the first line of code. Since then, Dagster has had 463 releases, and it’s grown to include a production-grade scheduler, schematized config system, rich logging, asset catalog, lightweight sensors, thirty three integration libraries, best-in-class web UI, and a whole lot more.

Through all this, we held off on declaring Dagster 1.0, because we believe that iteration results in better software - if you stick with the first approach that occurs to you, it often means you’re not exploring the full space of possibilities, and you’re likely to end up far away from the global optimum. Over that time, we made a number of tweaks to Dagster’s core APIs that substantially improved their ergonomics and utility. Eventually, we found that we were no longer making breaking changes.

Last year, we arrived at stable versions of Dagster’s core computational abstractions - ops, graphs, jobs, schedules, and sensors. This year, we did the same with the asset layer - software-defined assets, materializations, and asset partitions. With all of Dagster’s core abstractions now stable, it was time to make things official and declare 1.0.

1.0 represents the work of over 200 people who contributed code and hundreds more who contributed feedback, bug reports, and encouragement. We’re incredibly appreciative of everybody in the community who has helped Dagster get here.

Of course, there’s still tons left to do. Our core engineering team is already mapping out what we’ll build next on top of this foundation. We’re excited to lean deeper into declarative asset orchestration, to expand and harden Dagster’s integrations, and keep widening the capabilities of Dagster’s orchestration layer.

For more details on this release, check out the change log, release notes, and migration guide.

We're always happy to hear your feedback, so please reach out to us! If you have any questions, ask them in the Dagster community Slack (join here!) or start a Github discussion. If you run into any bugs, let us know with a Github issue. And if you're interested in working with us, check out our open roles!

Follow us:

Sign up for our newsletter

Subscribe to the RSS feed

Star us on GitHub

Follow us on Twitter

Subscribe to our YouTube Channel