About this integration

Using this integration, you can trigger Airbyte syncs and orchestrate your Airbyte connections from within Dagster, making it easy to chain an Airbyte sync with upstream or downstream steps in your workflow.

The guide focuses on how to work with Airbyte connections using Dagster's software-defined asset (SDA) framework.

Installation

pip install dagster-airbyte

Example

from dagster import EnvVar
from dagster_airbyte import AirbyteResource, load_assets_from_airbyte_instance
import os

# Connect to your OSS Airbyte instance
airbyte_instance = AirbyteResource(
    host="localhost",
    port="8000",
    # If using basic auth, include username and password:
    username="airbyte",
    password=EnvVar("AIRBYTE_PASSWORD")
)

# Load all assets from your Airbyte instance
airbyte_assets = load_assets_from_airbyte_instance(airbyte_instance)

About Airbyte

Airbyte is an open-source data integration engine that helps you consolidate your SaaS application and database data into your data warehouses, lakes and databases.