Dagster University Presents: Testing with Dagster

We are happy to announce a new addition to Dagster University with our new course: Testing with Dagster!

Learn

Testing is often overlooked in data engineering. However, the only way to properly scale a data platform is to move beyond constant maintenance and troubleshooting issues in production.

In order to build with confidence, you need assurances that your code works as expected before it ships. That means having tests in place to validate new features or ensure changes do not have unintended consequences.

Testing with Dagster is a six-lesson course, each focused on a different aspect of testing. If you've never written tests before, this course provides a structured introduction to test design and an overview of testing in Python. If you're an experienced Python and Dagster user, you’ll find best practices and techniques to streamline your testing suite.

Testing in Dagster

At Dagster, we believe strongly in the power of testing. The only way we can release a new version of Dagster every week is by ensuring everything works as we develop. We want our users to have that same level of confidence in the code they build.

This module covers:

• The fundamentals of unit testing and writing asset tests in Dagster.
• Strategies for handling external dependencies in your Dagster deployment while maintaining full control in a testing environment.
• Best practices for integration testing to ensure your tests mirror real-world production scenarios.
• Proven Dagster-specific testing tips to help you maintain and optimize your project.

Example: Mocking API calls

@patch("requests.get")
def test_state_population_api_assets_config(mock_get, example_response, api_output):
    mock_response = Mock()
    mock_response.json.return_value = example_response
    mock_response.raise_for_status.return_value = None
    mock_get.return_value = mock_response

    result = dg.materialize(

        assets=[
            lesson_4.state_population_api_resource_config,
            lesson_4.total_population_resource_config,
        ],
        resources={"state_population_resource": lesson_4.StatePopulation()},
        run_config=dg.RunConfig(
            {"state_population_api_resource_config": lesson_4.StateConfig(name="ny")}
        ),
    )
    assert result.success

    assert result.output_for_node("state_population_api_resource_config") == api_output
    assert result.output_for_node("total_population_resource_config") == 9082539

Enroll Today

Like all Dagster University courses, Testing with Dagster is free and available to everyone. Simply sign up at Dagster University to get started. Once enrolled, you can track your progress and learn at your own pace.