The Orchestration Layer as the Data Platform Control Plane
Nick joined Eric and Kostas of The Data Stack Show to discuss the mismatch between the complexity and available tools in the data infrastructure space, and how this lead him to founding Dagster Labs. The group also talks about the challenges and fragmentation in the data engineering industry, the need for better abstraction layers, the role of orchestrators, comparing it to GraphQL's role in product engineering, the importance of data orchestration in the future of data infrastructure and engineering, and more.
We're always happy to hear your feedback, so please reach out to us! If you have any questions, ask them in the Dagster community Slack (join here!) or start a Github discussion. If you run into any bugs, let us know with a Github issue. And if you're interested in working with us, check out our open roles!