Rapidly developing ELT pipelines with dltHub and Dagster

Discover how to orchestrate dlt pipelines using Dagster’s robust scheduling, observability, and deployment features.

Join us for a technical deep dive co-hosted by Dagster Labs and our partners at dlthub, where we explore how to rapidly build and scale ELT pipelines using the power of open-source tooling. This session is perfect for data engineers, platform teams, and analytics engineers interested in streamlining data ingestion workflows from prototype to production. We’ll cover:

• Foundations of dlthub and Dagster

• Learn how these two modern data tools work together to create a seamless developer experience. We’ll introduce core concepts, complementary strengths, and the principles behind their design.

• Running dlt Pipelines with Dagster in Production

• Discover how to orchestrate dlt pipelines using Dagster’s robust scheduling, observability, and deployment features.

• Exploring Dagster Embedded ELT

• Dive into dagster_embedded_elt.dlt, an integration layer that brings dlt pipelines into Dagster with minimal boilerplate and maximum flexibility.

• A simple dlt pipeline integrated into Dagster to demonstrate basics like configuration, resources, and ops.

• A more advanced use case showcasing how to run a dlt pipeline on multiple nodes in parallel using Dagster's powerful partitioning and distributed execution model.

Whether you're exploring your first data ingestion project or scaling existing pipelines, this session will equip you with the tools and best practices to iterate faster, ship confidently, and operate reliably in production.