Integrations
Extend Dagster with our integration guides and libraries.
Airbyte
Orchestrate Airbyte connections and schedule syncs alongside upstream or downstream dependencies.
Amazon Web Services
Utilities for interfacing with AWS: S3, ECS, EMR, Cloudwatch, SecretsManager and Redshift.
Azure
Get utilities for ADLS2 and Blob Storage.
Census
Trigger Census synchs from within your Dagster pipelines.
Community / Partner supportedCube
Push changes from upstream data sources to Cube's semantic layer.
Community / Partner supportedDatabricks
Launch a Databricks job as a Dagster op.
Datadog
Publish metrics to Datadog from within Dagster ops and entralize your monitoring metrics.
dbt™
Put your dbt transformations to work, directly from within Dagster.
dbt Cloud™
Run dbt Cloud™ jobs as part of your data pipeline.
Delta Lake
Integrate your pipelines into Delta Lake.
Community / Partner supportedDocker
Launch runs or steps in a Docker container.
DuckDB
Read and write natively to DuckDB from Software Defined Assets.
DuckDB + Pandas
Translate between DuckDB tables and Pandas DataFrames.
DuckDB + Polars
Read inputs from and write Polars DataFrames to DuckDB
DuckDB + PySpark
Translate between DuckDB tables and PySpark DataFrames.
Fivetran
Orchestrate Fivetran connectors and schedule syncs with upstream or downstream dependencies.
Google Cloud Platform
Integrate with GCPs cloud capabilities: BigQuery, Dataproc, GCS, File Manager.
GitHub
Integrate with GitHub Apps and automate operations within your github repositories.
Great Expectations
Yield an expectation and its output with all relevant metadata.
Hashicorp Vault
Centrally manage credentials and certificates, then use them in your pipelines.
Community / Partner supportedHightouch
Trigger syncs and monitor them until they complete.
Community / Partner supportedJupyter Notebooks
Dagstermill eliminates the tedious "productionization" of Jupyter notebooks.
Kubernetes
Launch runs as Kubernetes Jobs. Use a Helm chart to deploy Dagster on a K8s cluster.
LakeFS
lakeFS provides version control and complete lineage over the data lake.
Community / Partner supportedMeltano
Tap into open source configurable ETL+ and the Singer integration library.
Community / Partner supportedMicrosoft Teams
Keep your team up to speed with Teams messages.
Open Metadata
Configure and schedule Dagster metadata and profiler workflows from the OpenMetadata UI.
Community / Partner supportedOpenAI
Integrate OpenAI calls into your Dagster pipelines, without breaking the bank.
PagerDuty
Centralize your monitoring with the dagster-pagerduty integration.
Pandas
Implement validation on pandas DataFrames.
Pandera
Generate Dagster Types from Pandera dataframe schemas.
Plural
Easily deploy Dagster open-source or just the Dagster agent to Kubernetes.
Community / Partner supportedPrometheus
Integrate with Prometheus via the prometheus_client library.
PySpark
Scale up data processing by executing PySpark code within Dagster.
Secoda
Help your team understand metadata from Dagster by adding context in Secoda.
Community / Partner supportedShell
Execute a Bash/shell command, directly or as a read from a script file.
Slack
Up your notification game and keep stakeholders in the loop.
Snowflake
An integration with the Snowflake data warehouse. Read and write natively to Snowflake from Software Defined Assets.
Snowflake + Pandas
Translate between slices of Snowflake tables and Pandas DataFrames.
Spark
Configure and run Spark jobs.
SSH/SFTP
Establish encrypted connections to networked resources.
Twilio
Integrate Twilio tasks into your data pipeline runs.
Weights & Biases
Orchestrate your MLOps pipelines and maintain ML assets.
Community / Partner supportedLooking for deployment options?
Check out hands-on guides to explore various deployment options.