Automatically sync asset materialization events and lineage from Dagster Cloud to Atlan
If you're managing data pipelines in Dagster and cataloging assets in Atlan, you've probably experienced the friction of keeping these systems in sync. Your Atlan catalog shows what data should exist, but knowing whether assets actually materialized successfully requires jumping back to Dagster.
The new integration between Dagster Cloud & Atlan eliminates this context-switching by streaming asset events directly from your orchestration layer into your data catalog in real-time.
Live Pipeline Events in Your Data Catalog
The Dagster-Atlan integration creates a live connection between your Dagster Cloud instance and Atlan, automatically syncing:
✅ Asset materializations and failures as they occur in near real-time
✅ Run status events (success, failure, cancellation)
✅ Asset lineage relationships from your Dagster dependency graph
✅ Rich event metadata including run IDs, timestamps, partition keys, descriptions, and custom tags
This means your Atlan catalog becomes a living reflection of your data pipelines. It shows what actually ran, when it ran, and whether it succeeded.
How It Works
The integration operates through a dedicated event consumer that monitors your Dagster Cloud event log and forwards relevant events to Atlan via their API.
When you connect your Dagster Cloud organization to Atlan:
- Asset graph synchronization: Dagster sends your complete asset dependency graph to Atlan, ensuring lineage visualization matches your actual pipeline structure
- Event streaming begins: As assets materialize (or fail to materialize), events flow automatically to Atlan
- Atlan enriches the catalog: Asset status updates appear in real-time, enriching your catalog with operational metadata from the orchestration layer
What's Next?
This release focuses on streaming real-time events from Dagster to Atlan, ensuring your catalog always reflects pipeline reality.
If you have feedback on additional metadata you'd like to see synced to Atlan, or ideas for how to make the integration more useful, we'd love to hear from you.
Get Connected
The Dagster-Atlan integration is available now for all Dagster Cloud and Atlan customers.
To enable the integration:
- Visit the Atlan connectors marketplace and find the Dagster integration
- Follow the setup flow to get your authentication credentials
- Use the dagster-cloud CLI to configure the connection to your Dagster Cloud organization
Questions or feedback? Just reach out to your Dagster account team.
Built with 💙 by the Dagster team in collaboration with Atlan
.png)
