Dagster 1.7: Core Updates Preview

A major set of updates to Dagster Core ahead of our Dagster+ launch.

The Dagster 1.7 release brings some key enhancements to Dagster Core. These will immediately benefit open-source Dagster users and set the stage for the upcoming Dagster+ launch on April 17th. Here is a recap of enhancements in the fourteen sub-releases and 1066 commits between 1.6.0 and 1.7.0.

Asset Tags
Asset Checks
And thanks to all the community contributors since 1.6.1

Organize your assets

Both Dagster Core (open-source) and the upcoming Dagster+ provide data asset cataloging. Here are some of the new features in Dagster Core, available in 1.7.0, that help you to organize, search, and share your critical assets:

New Asset Details page

We have released a new version of the Asset Details page. It includes an “Overview” tab that centralizes the most important information about the asset – such as current status, description, and columns – in a single place.

Dagster's updated Asset Details page. (click to zoom in)

Asset Tags

As your project grows, it helps to organize assets into logical groups. Using tags to label assets, ops, jobs, and job runs makes them easier to find. In addition to arbitrary tags, you can now assign assets to specific users, which in turn supports user-specific alerting.

These tags are surfaced in the asset catalog and details pages. You can filter your asset catalog based on these. For example, in the screenshot above, we filter down to just assets that contain PII. We'll also be announcing some exciting functionality to take action with asset tags during our upcoming Dagster+ launch on April 17th.

Asset Checks

As of Dagster 1.7.0 Asset Checks are no longer flagged as experimental, so you can expect the API to remain stable.

Furthermore, Asset Checks continue to evolve:

Checks can now be marked blocking, which causes downstream assets in the same run to be skipped if the check fails. This is set using the blocking parameter and defaults to False.
The new @multi_asset_check decorator lets you define a single function that executes multiple asset checks.
A freshness check is a type of asset check that allows you identify Dagster assets that are overdue for a data refresh. In 1.7 we introduce two new APIs — build_last_updated_freshness_checks and build_time_partition_freshness_checks — which allow you to define checks for when an asset is overdue for an update.
The new build_column_schema_change_checks API allows defining asset checks that warn when an asset’s columns have changed since its latest materialization.

Integrations Updates

dbt

Dagster’s dbt integration gets a number of updates in this release:* This integration can now be configured to automatically collect metadata about column schema and column lineage.* dbt tests are now pulled in as Dagster asset checks by default.* dbt resource tags are now automatically pulled in as Dagster asset tags.* dbt owners from dbt groups are now automatically pulled in as Dagster owners.

Data Freshness for Snowflake and GCP

The dagster-snowflake and dagster-gcp packages now both expose a fetch_last_updated_timestamps API, which makes it straightforward to collect data freshness information in source asset observation functions.

In addition to all of the above, we have made many quality-of-life improvements to the UI and made a ton of updates to the documentation.

And if you haven't already, do check out the courses in Dagster University, including the new module on dbt.

Looking for a more granular list of enhancements?
Check out the full Dagster Changelog.