Orchestrate Airbyte connections and schedule syncs alongside upstream or downstream dependencies.

Looking to move off Apache Airflow? Looking to run both platforms and incrementally adopt Dagster? We have you covered.

Utilities for interfacing with AWS: S3, ECS, EMR, Cloudwatch, SecretsManager and Redshift.

Get utilities for ADLS2 and Blob Storage.

Scale up the execution of Dagster-managed tasks on multiple machines.

Launches Celery-based tasks in docker containers.

Trigger Census synchs from within your Dagster pipelines.

Push changes from upstream data sources to Cube's semantic layer.

Dagstermill eliminates the tedious "productionization" of notebooks.

Dask-based executor for Dagster.

Launch a Databricks job as a Dagster op.

Publish metrics to Datadog from within Dagster ops and entralize your monitoring metrics.

Put your dbt transformations to work, directly from within Dagster.

Run dbt Cloud™ jobs as part of your data pipeline.

Integrate your pipelines into Delta Lake.

Launch runs or steps in a Docker container.

Read and write natively to DuckDB from Software Defined Assets.

Translate between DuckDB tables and Pandas DataFrames.

Read inputs from and write Polars DataFrames to DuckDB

Translate between DuckDB tables and PySpark DataFrames.

Orchestrate Fivetran connectors and schedule syncs with upstream or downstream dependencies.

Integrate with GCPs cloud capabilities: BigQuery, Dataproc, GCS, File Manager.

Integrate with GitHub Apps and automate operations within your github repositories.

Yield an expectation and its output with all relevant metadata.

Centrally manage credentials and certificates, then use them in your pipelines.

Work in Hex, then pull Hex apps in to your pipeline as Software Defined Assets.

Trigger syncs and monitor them until they complete.

Dagstermill eliminates the tedious "productionization" of Jupyter notebooks.

Launch runs as Kubernetes Jobs. Use a Helm chart to deploy Dagster on a K8s cluster.

lakeFS provides version control and complete lineage over the data lake.

Tap into open source configurable ETL+ and the Singer integration library.

Keep your team up to speed with Teams messages.

Streamline the process of productionizing, maintaining and monitoring machine learning models.

MySQL-backed event log, run and schedule storage.

Configure and schedule Dagster metadata and profiler workflows from the OpenMetadata UI.

Integrate OpenAI calls into your Dagster pipelines, without breaking the bank.

Centralize your monitoring with the dagster-pagerduty integration.

Implement validation on pandas DataFrames.

Generate Dagster Types from Pandera dataframe schemas.

Orchestrate Jupyter notebooks from Dagster.

Log Dagster job events to Papertrail.

Easily deploy Dagster open-source or just the Dagster agent to Kubernetes.

Enable PostgreSQL-backed storage for event log, run and scheduling.

Integrate with Prometheus via the prometheus_client library.

Scale up data processing by executing PySpark code within Dagster.

Help your team understand metadata from Dagster by adding context in Secoda.

Execute a Bash/shell command, directly or as a read from a script file.

Up your notification game and keep stakeholders in the loop.

An integration with the Snowflake data warehouse. Read and write natively to Snowflake from Software Defined Assets.

Translate between slices of Snowflake tables and Pandas DataFrames.

Configure and run Spark jobs.

Establish encrypted connections to networked resources.

Integrate Twilio tasks into your data pipeline runs.

Orchestrate your MLOps pipelines and maintain ML assets.