Integrations
Extend Dagster with our integration guides and libraries.
Airbyte
Orchestrate Airbyte connections and schedule syncs alongside upstream or downstream dependencies.
AWS Athena
This integration allows you to connect to AWS Athena and analyze data in Amazon S3 using standard SQL within your Dagster pipelines.
AWS CloudWatch
This integration allows you to send Dagster logs to AWS CloudWatch, enabling centralized logging and monitoring of your Dagster jobs.
AWS ECR
This integration allows you to connect to AWS Elastic Container Registry (ECR), enabling you to manage your container images more ...
AWS EMR
The AWS EMR integration allows you to seamlessly integrate AWS EMR into your Dagster pipelines for petabyte-scale data processing using ...
AWS Glue
The AWS Glue integration enables you to initiate AWS Glue jobs directly from Dagster, seamlessly pass parameters to your code, and stream ...
AWS Lambda
Using the AWS Lambda integration with Dagster, you can leverage serverless functions to execute external code in your pipelines.
AWS Redshift
Using this integration, you can seamlessly integrate AWS Redshift into your Dagster workflows, leveraging Redshift’s data warehousing ...
AWS S3
The AWS S3 integration allows data engineers to easily read and write objects to the durable AWS S3 storage, enabling engineers to have a ...
AWS Secrets Manager
This integration allows you to manage, retrieve, and rotate credentials, API keys, and other secrets using AWS Secrets Manager.
AWS Systems Parameter Store
The excerpt for the document is: "The Dagster AWS Systems Manager (SSM) Parameter Store integration allows you to manage and retrieve ...
Azure Data Lake Storage Gen 2 (ADLS2)
Get utilities for ADLS2 and Blob Storage.
Census
Trigger Census synchs from within your Dagster pipelines.
Community / Partner supportedCube
Push changes from upstream data sources to Cube's semantic layer.
Community / Partner supportedDatabricks
The Databricks integration enables you to initiate Databricks jobs directly from Dagster, seamlessly pass parameters to your code, and ...
Datadog
Publish metrics to Datadog from within Dagster ops and entralize your monitoring metrics.
dbt™
Put your dbt transformations to work, directly from within Dagster.
dbt Cloud™
Run dbt Cloud™ jobs as part of your data pipeline.
Delta Lake
Integrate your pipelines into Delta Lake.
Community / Partner supporteddlt
Easily ingest and replicate data between systems with dlt through Dagster.
Docker
Run runs external processes in docker containers directly from Dagster.
DuckDB
Read and write natively to DuckDB from Software Defined Assets.
Fivetran
Orchestrate Fivetran connectors and schedule syncs with upstream or downstream dependencies.
GCP BigQuery
Integrate with GCP BigQuery.
GCP Dataproc
Integrate with GCP Dataproc.
GCP GCS
Integrate with GCP GCS.
GitHub
Integrate with GitHub Apps and automate operations within your github repositories.
Hashicorp Vault
Centrally manage credentials and certificates, then use them in your pipelines.
Community / Partner supportedHightouch
Trigger syncs and monitor them until they complete.
Community / Partner supportedJupyter Notebooks
Dagstermill eliminates the tedious "productionization" of Jupyter notebooks.
Kubernetes
Launch Kubernetes pods and execute external code directly from Dagster.
LakeFS
lakeFS provides version control and complete lineage over the data lake.
Community / Partner supportedLooker
The Looker integration allows you to monitor your Looker instance as assets in Dagster, along with other data assets.
Meltano
Tap into open source configurable ETL+ and the Singer integration library.
Community / Partner supportedMicrosoft Teams
Keep your team up to speed with Teams messages.
Open Metadata
Configure and schedule Dagster metadata and profiler workflows from the OpenMetadata UI.
Community / Partner supportedOpenAI
Integrate OpenAI calls into your Dagster pipelines, without breaking the bank.
PagerDuty
Centralize your monitoring with the dagster-pagerduty integration.
Pandas
Implement validation on pandas DataFrames.
Pandera
Generate Dagster Types from Pandera dataframe schemas.
Power BI
The Power BI integration allows you to monitor your Power BI workspace as assets in Dagster, along with other data assets.
Prometheus
Integrate with Prometheus via the prometheus_client library.
SDF
Put your SDF transformations to work, directly from within Dagster.
Community / Partner supportedSecoda
Help your team understand metadata from Dagster by adding context in Secoda.
Community / Partner supportedBash / Shell
Execute a Bash/shell command, directly or as a read from a script file.
Sigma
The Sigma integration allows you to monitor your Sigma organization as assets in Dagster, along with other data assets.
Slack
Up your notification game and keep stakeholders in the loop.
Sling
Extract and load data from popular data sources to destinations with Sling through Dagster.
Snowflake
An integration with the Snowflake data warehouse. Read and write natively to Snowflake from Software Defined Assets.
Spark
Configure and run Spark jobs.
SSH/SFTP
Establish encrypted connections to networked resources.
Tableau
The Tableau integration allows you to monitor your Tableau workspace as assets in Dagster, along with other data assets.
Twilio
Integrate Twilio tasks into your data pipeline runs.
Weights & Biases
Orchestrate your MLOps pipelines and maintain ML assets.
Community / Partner supportedLooking for deployment options?
Check out hands-on guides to explore various deployment options.