Integrations

Extend Dagster with our integration guides and libraries.
Airbyte

Airbyte

Orchestrate Airbyte connections and schedule syncs alongside upstream or downstream dependencies.
Airflow

Airflow

Looking to move off Apache Airflow? Looking to run both platforms and incrementally adopt Dagster? We have you covered.
Amazon Web Services

Amazon Web Services

Utilities for interfacing with AWS: S3, ECS, EMR, Cloudwatch, SecretsManager and Redshift.
Azure

Azure

Get utilities for ADLS2 and Blob Storage.
Celery

Celery

Scale up the execution of Dagster-managed tasks on multiple machines.
Celery + Docker

Celery + Docker

Launches Celery-based tasks in docker containers.
Dagstermill

Dagstermill

Dagstermill eliminates the tedious "productionization" of notebooks.
Dask

Dask

Dask-based executor for Dagster.
Databricks

Databricks

Launch a Databricks job as a Dagster op.
Datadog

Datadog

Publish metrics to Datadog from within Dagster ops and entralize your monitoring metrics.
dbt

dbt

Put your dbt transformations to work, directly from within Dagster.
dbt Cloud

dbt Cloud

Run dbt Cloud jobs as part of your data pipeline.
Docker

Docker

Launch runs or steps in a Docker container.
DuckDB

DuckDB

Read and write natively to DuckDB from Software Defined Assets.
DuckDB + Pandas

DuckDB + Pandas

Translate between DuckDB tables and Pandas DataFrames.
DuckDB + PySpark

DuckDB + PySpark

Translate between DuckDB tables and PySpark DataFrames.
Fivetran

Fivetran

Orchestrate Fivetran connectors and schedule syncs with upstream or downstream dependencies.
Google Cloud Platform

Google Cloud Platform

Integrate with GCPs cloud capabilities: BigQuery, Dataproc, GCS, File Manager.
GitHub

GitHub

Integrate with GitHub Apps and automate operations within your github repositories.
Great Expectations

Great Expectations

Yield an expectation and its output with all relevant metadata.
Hashicorp Vault

Hashicorp Vault

Centrally manage credentials and certificates, then use them in your pipelines.
Community / Partner supported
Hex

Hex

Work in Hex, then pull Hex apps in to your pipeline as Software Defined Assets.
Community / Partner supported
Hightouch

Hightouch

Trigger syncs and monitor them until they complete.
Community / Partner supported
Jupyter Notebooks

Jupyter Notebooks

Dagstermill eliminates the tedious "productionization" of Jupyter notebooks.
Kubernetes

Kubernetes

Launch runs as Kubernetes Jobs. Use a Helm chart to deploy Dagster on a K8s cluster.
Microsoft Teams

Microsoft Teams

Keep your team up to speed with Teams messages.
MLflow

MLflow

Streamline the process of productionizing, maintaining and monitoring machine learning models.
MySQL

MySQL

MySQL-backed event log, run and schedule storage.
Noteable

Noteable

If orchestrating notebooks is on your roadmap, the Noteable team has made this much easier.
Community / Partner supported
Open Metadata

Open Metadata

Configure and schedule Dagster metadata and profiler workflows from the OpenMetadata UI.
Community / Partner supported
PagerDuty

PagerDuty

Centralize your monitoring with the dagster-pagerduty integration.
Pandas

Pandas

Implement validation on pandas DataFrames.
Pandera

Pandera

Generate Dagster Types from Pandera dataframe schemas.
Papermill

Papermill

Orchestrate Jupyter notebooks from Dagster.
Papertrail

Papertrail

Log Dagster job events to Papertrail.
PostgreSQL

PostgreSQL

Enable PostgreSQL-backed storage for event log, run and scheduling.
Prometheus

Prometheus

Integrate with Prometheus via the prometheus_client library.
PySpark

PySpark

Scale up data processing by executing PySpark code within Dagster.
Shell

Shell

Execute a Bash/shell command, directly or as a read from a script file.
Slack

Slack

Up your notification game and keep stakeholders in the loop.
Snowflake

Snowflake

An integration with the Snowflake data warehouse. Read and write natively to Snowflake from Software Defined Assets.
Snowflake + Pandas

Snowflake + Pandas

Translate between slices of Snowflake tables and Pandas DataFrames.
Spark

Spark

Configure and run Spark jobs.
SSH/SFTP

SSH/SFTP

Establish encrypted connections to networked resources.
Twilio

Twilio

Integrate Twilio tasks into your data pipeline runs.