Dagster + Modal

Run serverless Python functions at scale with Modal directly from your Dagster pipelines.

Dagster & Modal

About this integration

Modal is a serverless cloud platform that makes it easy to run Python code at scale without managing infrastructure. The Dagster-Modal integration enables you to execute compute-intensive workloads, ML training jobs, and data processing tasks on Modal's auto-scaling platform directly from your Dagster pipelines.

With this integration, you can:

  • Run serverless functions that automatically scale from zero to thousands of containers
  • Access GPU resources for machine learning training and inference workloads
  • Execute long-running jobs without worrying about infrastructure management
  • Optimize costs with pay-per-use pricing and automatic scaling
  • Deploy instantly with Modal's container orchestration
  • Monitor execution through Dagster's observability tools

Modal is particularly powerful for ML workflows, data processing at scale, and any compute-intensive tasks that benefit from elastic infrastructure.

Installation

pip install dagster-modal modal

Key Features

Serverless Execution

Modal automatically provisions and scales compute resources based on demand, eliminating the need to manage servers or containers.

GPU Support

Access powerful GPU instances (A10G, A100, H100) for machine learning training and inference without long-term commitments.

Instant Deployment

Deploy functions and containers instantly with Modal's fast cold-start times and efficient resource allocation.

Cost Optimization

Pay only for actual compute time with automatic scaling to zero when idle, making it cost-effective for variable workloads.

Container Orchestration

Run complex applications with full control over the runtime environment, dependencies, and resource requirements.