Less code.Better engineering.
Your data platform shouldn't feel like rocket science. Onboard teams to your data platform while enforcing standards with declarative data pipelines.
A New Way to Build Data Platforms
An opinionated approach focused on rapid development and operability from day one.
Build data and AI pipelines declaratively
Dagster Components let any stakeholder write a few lines of YAML to create production ready data pipelines.
Standardization without handholding
Build custom components in Python that abstract away glue code, so teams can reuse patterns, enforce best practices, and onboard quickly.
Skip boilerplate with the integration marketplace
We include many out-of-the-box components for common data technologies like dbt.
The next generation of data platform tooling
A powerful but approachable CLI, full IDE autocompletion support, and rich error reporting should be table stakes. Components ship with all that and support for Model Context Protocol so that you can fully leverage AI code generation.
The next evolution of data engineering is here
A developer experience that actually delivers on the promise of productivity.
Engineering time is a precious resource, and constantly chasing every new data request limits the team's capabilities to deliver more foundational work. Authoring pipelines in YAML empowers analysts and other teams to build and manage data pipelines all on their own
Dagster enables data platform teams to standardize best practices by authoring reusable Components. Teams using these Components can rest assured that they're building with best-practices in mind.
Building new pipelines is pain free with self-documenting components, IDE autocompletion, and component validation with detailed errors. A library of common components gives you everything you need to tackle common use cases right out of the box.
Components make data pipeline code so simple, even AI could write it. With defined schemas, your components are easy to build with modern AI developer tooling.
Dagster Components was designed with modern data teams in mind, supporting software development best practices such as infrastructure-as-code, GitOps, CI/CD, local development, and branch deployments.
How it works
Great data engineering starts with the right tools, built by experts who understand modern data pipelines.
Install dg and configure your editor
Get the best of both worlds with YAML for simple configurations and Python when complex use cases demand it.
Scaffold your pipeline with AI
Get right to coding with instant project scaffolding powered by AI.
Customize your pipeline
Edit your YAML file to ensure that all the details are correct and that the pipeline does exactly what you want it to.
Validate and Deploy
Instantly validate that your YAML is correct, manage your secrets, and deploy with a single command.