Three Weeks, 140 Iterations: How Group 1001 redefined their data platform

With Dagster® powering the data platform, the Group 1001 Innovations team is transforming the business at remarkable speed.

For decades the insurance industry has sought to eliminate the high volume of paperwork and bureaucracy related to the specialized processing of annuities transactions. Yet most companies in the industry are only making slow progress. Group 1001 is bucking the trend, having cracked the code on how to digitize one of the most resource-intensive parts of the business. This transformation is due, in large part, to the work of Gu Xie and the Group 1001 Innovations team, who built a core component of their data platform around Dagster®.

About Gu Xie

Gu Xie is the Head of Data Engineering at Group 1001 focusing on innovations - data enablement that accelerates the insurance organization's ability to execute its strategy. He shares his perspective on Data Engineering on Idea to Insight. Gu's passion is building nimble yet highly reliable data platforms to accelerate an organization's path from vision to execution.

Building toward the Future

Today, data is empowering and accelerating the business at Group 1001. But this was no accident. This forward-looking organization accelerated its pursuit of modern technology to handle vast amounts of information, scale more effectively, and develop more reliable and dynamic data.

Fast forward to today. With the adoption of Dagster® and other key technologies, the Company has evolved exponentially. The data engineering team has an even stronger handle on managing the data and has progressed to leveraging human-centered AI and machine learning to assist its employees who power the business.

Group 1001 adopted the Dagster® platform at an opportune time. In 2023, Group 1001’s business increased significantly, and concurrently, the company’s workload increased. With an efficient data platform in place, they were able to maintain optimal efficiency.

Growth: A Nice Problem to Have?

“You wouldn’t believe the number of PDF files that come into a large insurance organization,” says Gu, “The insurance business is still highly dependent on PDFs. Typically, someone would have a PDF on one screen and a data entry system on another screen, and manually enter the data. We automated all of that.”

Today, Group 1001 is using human-centered AI to read, perform Optical Character Recognition (OCR), and define entities in these documents with processes orchestrating the entire end-to-end workflow. The company is using a combination of human-centered AI and ML models to support Robotic Process Automation (RPAs) for thousands of documents every day.

Gu provides a specific example. A 1035 Exchange is a standard insurance industry process for moving funds from one organization to another. Despite advances in OCR, handling a 1035 Exchange typically requires manual intervention and verification. At Group 1001, it is automated. To achieve this, Group 1001 built a custom Robotic Process Automation (RPA) system into their data platform in record time.

“We kicked off a new process in early December of 2023. Just three weeks later, using Dagster®—after 140 iterations—we deployed our custom RPA for the 1035 transfers. Three weeks!”

Within hours, the team processed thousands of transfers.

"We built all this including, using Dagster®, and our engineering resources.

The Power of a Platform Approach

The data analytics team also delivered the analytics needs of the business in record time. They mapped key business processes, set up ingestion and transformation, and deployed new dashboards to the business in just two days.

“That’s when we realized the speed we could achieve. It was a real ‘Ah-ha’ moment for the business. What used to take months could now be done in days. That opened the door to projects like custom RPAs, human-centered AI-driven workflows, and even migrating our legacy systems. The team took on the challenge to migrate off large legacy systems in just four months, with two developers.”

“The velocity we’ve achieved is insane. And it’s not just about speed—it’s about making sure everything we build is maintainable.”

“We replicated over hundreds of SQL Server tables, migrated tens of millions of records, and built observability around the migration pipeline—all with minimal issues and just two developers,” says Gu, “The velocity we’ve achieved is insane. And it’s not just about speed—it’s about making sure everything we build is maintainable. Crushing technical debt is a priority for us, so we don’t get weighed down by it later.”

Org Design Matters

The data team at Group 1001 doesn’t just manage individual pipelines or stand-alone disconnected processes; they have established a true platform approach. There is another key to Group 1001’s success: organizational and team design.

Large enterprises typically aren’t designed for velocity because they lack ownership and accountability, says Gu. At Group 1001, teams own the outcome. Employees deliver reliable data products while prioritizing long-term maintainability and automation to keep that velocity going.

Gu’s team has grown from four employees to twelve - six data engineers, six data analysts. Across the entire organization, there are several data teams working on different projects. Dagster® will be the heart of collaboration across these data teams.

“Data is a multiplayer experience. Everything you produce—whether it’s a report, a table, or a model—is going to be consumed by someone else. Dagster® helps us collaborate by providing a technical interface, while other technologies will handle the functional interface for data discovery and trust.”

In organizations where there’s separation between data engineers and analysts, Gu typically sees significant friction. Problems get lobbed back and forth between teams, which slows everything down. At Group 1001, data engineering and BI teams work collaboratively to drive innovation.

“We pair data engineers and analysts in pods, so they collaborate directly. That removes a lot of friction. The engineer provides the high-quality data, and the analyst models it. It’s outcome-driven, and it works.”

The secret to moving as fast as Group 1001 does comes down to organizational design, says Gu.

“We align teams around outcomes, crush technical debt, and focus on automation. It’s about removing barriers and prioritizing speed without sacrificing quality.”

Achieving High Development Velocity

“I’ve built data platforms before, so I had known the importance of creating a platform for scale. You have to design for velocity from the start, automate concerns like governance and deployment, and remove friction for development teams. That's how you enable a culture of speed.”

The key is to remove friction and allow teams to focus on delivering value. When you do that, the whole organization benefits.

The opinions expressed herein are those of Gu Xie and not necessarily those of his employer.
©2024 Group 1001 IP Properties, LLC | Group 1001. All Rights Reserved

‍