Back to integrations
Dagster + Weights & Biases

Dagster Integration:
Dagster + Weights & Biases

Orchestrate your MLOps pipelines and maintain ML assets.

About this integration

Use Dagster and Weights & Biases (W&B) to orchestrate your MLOps pipelines and maintain ML assets. The integration with W&B makes it easy within Dagster to:

  • use and create W&B Artifacts
  • use and create Registered Models in W&B Model Registry
  • run training jobs on dedicated compute using W&B Launch
  • use the wandb client in ops and assets

The W&B Dagster integration provides a W&B-specific Dagster resource and IO Manager:

  • wandb_resource: a Dagster resource used to authenticate and communicate to the W&B API.
  • wandb_artifacts_io_manager: a Dagster I/O Manager used to consume W&B Artifacts.

Installation

To use this integration you will need a Weights and Biases account. Then you will need an W&B API Key, a W&B entity (user or team), and a W&B project. Full installation details can be found on the Weights and Biases website here.

Note that Weights & Biases do offer a free cloud account for personal (non-corporate) use. Check out their pricing page for details.

Example

A complete tutorial can be found on the Weights and Biases website here.

About Weights & Biases

Weights & Biases makes it easy to track your experiments, manage & version your data, and collaborate with your team so you can focus on building the best machine learning models.


Community / Partner integration:

This integration was built and is maintained by a community user or a technology partner from outside of Dagster Labs.