February 7, 2024 • 1 minute read • 
Learning and Sharing in Public
- Name
- Pedram Navid
- Handle
- @pdrmnvd
Pedram Navid, Head of Data Engineering at Dagster Labs makes an appearance on the Partially Redacted Podcast with Sean Falconer. They discuss the unique challenges and opportunities in the realm of data engineering, particularly the culture of learning and sharing within the field.
Pedram discusses the traditionally guarded nature of data engineering, contrasting it with the more open-source approach in software engineering. He highlights the potential downsides of this secrecy, such as the difficulty in learning best practices and innovating. The discussion also touches on the balance companies must strike between contributing to communal knowledge and protecting valuable data and intellectual property.
Pedram shares insights from his experiences at Dagster Labs, including the development of the Dagster Open Platform and its impact on fostering a culture of openness in data engineering. Additionally, they explore the future of collaboration in the field, considering emerging technologies and methodologies that could further encourage sharing and innovation over the next 5-10 years.
We're always happy to hear your feedback, so please reach out to us! If you have any questions, ask them in the Dagster community Slack (join here!) or start a Github discussion. If you run into any bugs, let us know with a Github issue. And if you're interested in working with us, check out our open roles!
Follow us:
Podcast
Tech Talks Daily: Data, Decisions, and Dagster
A Geek Leader: Interview with Nick Schrock
Scaling Data Pipelines
- Name
- Nick Schrock
- Handle
- @schrockn