November 12, 2024 • 3 minute read •
Case Study: From Disconnected Data to a Unified Platform
- Name
- Alex Noonan
- Handle
- @noonan
Founded in 1919, Servco has earned a distinguished reputation for service, innovation, and a steadfast commitment to customer-focused excellence. Based in Hawaiʻi, Servco, originally founded as a two-car garage on the North Shore of Oʻahu, has now diversified into various industries, including automotive, musical instruments, industrial equipment, and venture capital. Servco has grown to become the largest Toyota Group in Australia and also owns Fender Musical Instruments Corporation.
Tausif Islam and Sam Ikemoto are data engineers within Servco’s Hawaiʻi Automotive business. They shared insights into their journey from traditional BI to an evolved, code-first data platform that opens access to reliable data for all data professionals across the organization.
Their work showcases the challenges and triumphs of moving from on-premise, outdated systems to a fully integrated cloud-based environment. Below, we capture some key takeaways for data practitioners.
The Challenge: Legacy Systems and Limited Data Accessibility
Servco’s Hawaiʻi retail and distribution business spans all the islands. Until recently, it was grappling with challenges common to many traditional organizations: disconnected data, limited accessibility, and tribal knowledge concentrated in the hands of a few. Their systems, built on an on-premise SQL server, were difficult to access, with poorly documented tables and outdated data dictionaries.
This legacy setup created bottlenecks in delivering insights, and the lack of accessible documentation meant that analysts frequently had to rely on the engineering team for data interpretation and access.
"It was always just us doing the reporting work," Tausif noted, highlighting the pressure on the data team and analysts' difficulty in using data autonomously.
The Solution: Moving to a Cloud-Native, Self-Service Data Model
The transformation began with Servco's decision to move to a cloud-native platform. Tausif and Sam opted for Google Cloud's BigQuery and implemented Dagster, a modern orchestration tool, to replace their legacy SQL server processes. The choice of Dagster was driven by its developer-friendly, Python-first approach and its focus on assets and data lineage—crucial features that would help make data more transparent and accessible.
Dagster's data catalog capabilities were pivotal to their new approach. "We’re not just providing data anymore; we’re also providing context," Tausif explained. The data team started building semantic models, documenting tables and data lineage to provide analysts with a clearer view of what the data represented, how it was derived, and how it could be used.
This allowed Servco to create a self-service culture where analysts could pull data directly into Excel, Google Sheets, or Tableau without needing to go through the engineering team. As Tausif put it, "We want to provide clean, high-quality data in the most usable and secure format. We’re less focused on what tool our analysts use."
The Impact: Enabling True Self-Service
The new platform had a significant impact. Instead of gatekeepers, the data team became enablers, helping analysts access well-documented, trusted data directly. This change improved efficiency and enhanced data usage across the organization.
Sam added, "I’m interested to see how things pan out as the models grow. But right now, having lineage and everything in one spot makes my life much easier."
Integrating data cataloging features into Dagster also significantly helped analysts trust and leverage data. "We’re not just a vault of tribal knowledge anymore. People are trusting our semantic models because they’re verified by different departments like finance and auto retail," Tausif shared. The data team also trained 30+ users on using Dagster effectively, aiming for wide adoption among their finance and accounting teams.
Lessons for Fellow Data Practitioners
1. Choose the Right Orchestrator: Servco evaluated other tools but opted for Dagster due to its focus on assets and comprehensive data catalog features. If the orchestrator can also support documentation and lineage, it becomes far more powerful for business users.
2. Semantic Models Are Key: Building reusable, well-documented semantic models saves time and reduces redundancy. The data team took a customer-led approach and built models that could serve multiple business units, increasing trust and quality.
3. Unified Tools for Efficiency: Having everything—from documentation to orchestration—in one tool significantly reduced context-switching for the team. This approach allowed Sam and Tausif to focus more on adding value than firefighting.
4. From Orchestration to Governance: Tausif expressed his vision for Dagster to become a one-stop platform for orchestration, governance, and security, including features like data masking and policy tagging. While not yet fully realized, this vision underlines the growing importance of governance in data workflows.
The Road Ahead for Servco
Servco’s transformation is ongoing. The team continues to migrate data sources to the cloud, slowly deprecating their on-premise systems. They’re also preparing for an increase in the number of users accessing their data platform and monitoring how their models scale.
The biggest value? "Now, we’re sharing the data," Tausif said. "Our internal customers see the value in accessing the data themselves. And Dagster has helped us make that possible by demystifying the data and making it accessible in a usable way."
We're always happy to hear your feedback, so please reach out to us! If you have any questions, ask them in the Dagster community Slack (join here!) or start a Github discussion. If you run into any bugs, let us know with a Github issue. And if you're interested in working with us, check out our open roles!
Follow us:
Case Study: Analytiks - Fast-Track AI Projects With Managed Dagster+
- Name
- Pedram Navid
- Handle
- @pdrmnvd
Case Study: KIPP - Building a Resilient Data Platform with Dagster
- Name
- Fraser Marlow
- Handle
- @frasermarlow
Case Study: Mejuri - Building an eCommerce Data Platform
- Name
- Fraser Marlow
- Handle
- @frasermarlow