Announcing Dagster 1.4: Material Girl

We are pleased to announce that Dagster 1.4.0 —  “Material Girl” —   is now available on Dagster Cloud and in Open Source.  Full details on the release can be found in the release notes, but here are the highlights:

Supercharge dbt™

The headlining act on this release is the dagster-dbt enhancements.  While Dagster already provided a dagster-dbt integration (as well as a dbt Cloud integration), we have gathered input from Dagster users who rely heavily on dbt and introduced brand new capabilities:

• A new @dbt-assets decorator allows much more control over how Dagster runs your dbt project.
• A new scaffolding command line interface makes it easy to create files and directories for a Dagster project that wraps an existing dbt project.
• Improved APIs for defining asset dependencies between upstream dbt assets and downstream non-dbt assets. It is now much easier to specify dependencies between dbt models and upstream non-dbt assets by specifying Dagster asset keys in the dbt metadata for dbt sources.

‍

Evolving asset materialization, improving the UI, and more

We continue to evolve asset materialization, giving you more fine-grained control and observability over when, why, and how your computations run.  These include auto-materialization history, auto-materialization performance, asset backfill cancellation, and a new group-level asset status UI.

We have introduced two new experimental features:  op-level concurrency limits and the ability to observe source assets without defining jobs.

We have also focused on improving some of the framework ergonomics, making it more logical and accessible.  Improvements here include:

• Deprecation of non_argument_deps in favor of a new deps parameter (makes it clear that this is a first-class way of defining dependencies, makes code more concise, while still accepting previous parameters)
• Renaming Dagit to Dagster UI (reducing the number of terms that new users need to learn)
• Auto-populating the launchpad with default field values (reducing cognitive load).

We have moved some features out of the “experimental” stage: DynamicPartitionDefinition and SensorResult.

Finally, we have a new UI layout change that is flagged as experimental and open to your feedback in the GitHub discussion found here.  The new layout renders the graph horizontally left-to-right instead of vertically. You can try the new horizontal asset graph layout by enabling the Experimental horizontal asset DAGs feature flag in your user settings. Give it a try, and let us know what you think.

1.4 Contributors:

We would like to thank all of the contributors to the 1.4 Dagster Core release.

askvinni | yuvalgimmunai  | nigelainscoe | plaflamme  | danielecappuccio  | armandobelardo  | boenshao  | loerinczy  | akan72  | out-running-27  | guy-rvvup  | thijs-blom  | jrouly  | adeboyed  | snikch  | chrishiste  | renzhe-brian  | chodera  | joel-olazagasti  | danielgafni  | KMontag42  | srggrs  | Jinior  | ldnicolasmay  | Elliot2718  | PeterJCLaw  | Tanguy-LeFloch  | leehuwuj  | weberdavid  | keyz  | pdstrnadJC  | nina-j  | wkeifenheim  | rubikplanet