Engineering

We Tried ty for Performance. It Found Real Bugs

May 21, 2026

We Tried ty for Performance. It Found Real Bugs

We adopted Astral’s new Python type checker, ty, to speed up type checking in the Dagster monorepo. The performance gains were dramatic, but the bigger surprise was that ty caught real runtime bugs Pyright missed. Here’s what we learned migrating a large Python codebase incrementally to ty.

Dennis Hume

Thank you! Your submission has been received!

Oops! Something went wrong while submitting the form.

May 19, 2026

The Dagster Almanack: From Complexity to Composability

A complete guide with all the insights, tips, and some predictions for the data platform engineer, just like an Almanack provides, with practical information for daily life.

Simon Späti

April 20, 2026

How to Orchestrate Across Multiple Databricks Workspaces Without Losing Your Mind

Once your pipelines span multiple Databricks workspaces, you're no longer orchestrating a single system you're coordinating a distributed one.

Eric Thomas

Monorepos, the hub-and-spoke model, and Copybara

April 3, 2026

Monorepos, the hub-and-spoke model, and Copybara

How we configure Copybara for bi-directional syncing to enable a hub-and-spoke model for Git repositories

Colton Padden

March 17, 2026

DataOps with Dagster: A Practical Guide to Building a Reliable Data Platform

DataOps is about building a system that provides visibility into what's happening and control over how it behaves

Alex Noonan

March 5, 2026

Announcing AI Driven Data Engineering

AI coding agents are changing how data engineers work. This Dagster University course shows how to build a production-ready ELT pipeline from prompts while learning practical patterns for reliable AI-assisted development.

Dennis Hume

Sample-Level Versioning for ML Pipelines with Dagster and Metaxy

February 13, 2026

Sample-Level Versioning for ML Pipelines with Dagster and Metaxy

Learn how Metaxy can be used to build multimodal data pipelines with sample-level granularity on Dagster

Daniel Gafni

February 6, 2026

Evaluating Skills

We built a light weight evaluation framework to quantiatively measure the effectiveness of the Dagster Skills, and these are our findings.

Colton Padden

Pytest for Agent-Generated Code: Concrete Testing Strategies to Put Into Practice

January 26, 2026

Pytest for Agent-Generated Code: Concrete Testing Strategies to Put Into Practice

When agents write tests, intent matters as much as correctness. By defining clear testing levels, preferred patterns, and explicit anti-patterns, we give agents the structure they need to produce fast, reliable Pytest suites that scale with automation.

Dennis Hume

January 21, 2026

Dagster + Snowflake: Building Production AI Pipelines with Cortex

Snowflake handles AI compute while Dagster handles orchestration, observability, and the operational patterns that turn AI experiments into reliable production pipelines.

Alex Noonan

Dignified Python: 10 Rules to Improve your LLM Agents

January 9, 2026

Dignified Python: 10 Rules to Improve your LLM Agents

Modern LLMs generate patterns, not principles. Dignified Python gives agents the intent they lack, ensuring code is explicit, consistent, and engineered with care. Here are ten rules from our Claude prompt.

Colton Padden

January 7, 2026

Evaluating Model Behavior Through Chess

Benchmarks measure outcomes, not behavior. By letting AI models play chess in repeatable tournaments, we can observe how they handle risk, repetition, and long-term objectives, revealing patterns that static evals hide.

Dennis Hume

January 6, 2026

How to Enforce Data Quality at Every Stage: A Practical Guide to Catching Issues Before They Cost You

This post gives you a framework for enforcing data quality at every stage so you catch issues early, maintain trust, and build platforms that actually work in production.

Alex Noonan

Dagster Newsletter

Get updates delivered to your inbox