Engineering

January 9, 2023

Build a GitHub Support Bot with GPT3, LangChain, and Python

In this tutorial, we tap into the power of OpenAI's ChatGPT to build a GitHub support bot using GPT3, LangChain, and Python.

November 30, 2022

Getting Stuff Done: a Guide to Productive Software Engineering

To be a more productive software engineer you need to master changes, how these affect the program and others on the team.

November 11, 2022

Pushing REST-API data to Google Sheets with Dagster

A total beginners tutorial in which we store REST API data in Google Sheets and learn some key abstractions.

November 7, 2022

Adding Types to a Large Python Codebase

What we learned when we introduced dynamically typed code to a large Python codebase, bringing Dagster's public API to 100% type coverage.

October 31, 2022

Orchestrating Machine Learning Pipelines with Dagster

How to use Dagster’s open source data orchestrator to build machine learning pipelines and train ML models.

October 25, 2022

Build a poor man’s data lake from scratch with DuckDB

DuckDB is so hot right now. Learn how to build a data lake from dbt using DuckDB for SQL transformations, along with Python, Dagster, and Parquet files.

October 19, 2022

The Unreasonable Effectiveness of Data Pipeline Smoke Tests

Data practitioners waste time writing unit tests to catch bugs they could have caught with smoke tests.

October 17, 2022

Web Workers are not the Answer

A tale of overstretched logs, counterintuitive web worker behavior, and ultimately a troublesome cursor issue.

October 4, 2022

Postgres: a Better Message Queue than Kafka?

When lots of event logs must be stored and indexed, Kafka is the obvious choice. Naturally, our queue runs on Postgres.

August 17, 2022

Spend Less Time Debugging with Dagster

It’s not uncommon for a data engineer to devote 80% of their day to debugging. Dagster radically improves on this.

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.

Dagster Newsletter

Get updates delivered to your inbox