March 22, 2024 • 5 minute read •

How Dagster Cloud Supports BCBS 239 Compliance

BCBS 239 establishes standards for banking risk management worldwide. Dagster helps data engineers meet these demanding standards.

Name: Fraser Marlow
Handle: @frasermarlow

Basel Committee on Banking Supervision's Standard Number 239, commonly called BCBS 239, focuses on the principles for effective risk data aggregation and reporting. It was established in response to the global financial crisis of 2007-2008, which revealed weaknesses in banks' ability to aggregate risk data and produce timely, accurate, and comprehensive risk reports.

The scope of BCBS 239 can be summarized as follows:

Risk Data Aggregation: BCBS 239 emphasizes the importance of banks having robust systems in place to aggregate risk data from across the organization. This includes data on credit, market, liquidity, and operational risks, etc. The standard aims to ensure that banks have accurate and complete data to support decision-making processes and risk management activities.

Risk Reporting: The standard also addresses the need for effective risk reporting mechanisms within banks. It outlines principles for the design and implementation of reporting frameworks, including reports' frequency, format, and content. The goal is to give senior staff and regulators timely access to relevant risk information to support oversight and decision-making.

Governance and Controls: BCBS 239 emphasizes the importance of strong governance and internal controls surrounding risk data aggregation and reporting processes. This includes clear accountability, resourcing, and independent validation of data and reporting systems.

Compliance and Implementation: The standard is not just about setting principles but also about ensuring compliance and effective implementation. Banks are expected to assess their current practices against the principles outlined in BCBS 239 and take necessary steps to address any gaps or deficiencies. Regulators play a crucial role in monitoring banks' compliance with the standard and may impose significant penalties for non-compliance.

Overall, the scope of BCBS 239 is broad and encompasses various aspects of risk data aggregation and reporting within banks. The aims of BCBS 239 are to push banks to strengthen their risk management practices and enhance the overall resilience of the financial system.

In terms of geographic scope, BCBS 239 applies to banks and financial institutions across the globe, regardless of their location or jurisdiction.

BCBS 239 and Data Engineering

Data engineering teams in large banks play a critical role in implementing and maintaining the data management systems necessary to comply with BCBS 239. By considering the eight factors outlined below, data engineering teams can effectively meet the demands of BCBS 239 and contribute to the bank’s overall success of risk data aggregation and reporting initiatives.

Dagster is designed to help data engineering teams build and maintain robust data management systems, and it offers several features that can support compliance with regulations like BCBS 239.

DE Dimension	Data Engineering Considerations	How Dagster Supports
Quality	Ensuring data quality is paramount. Data engineering teams should implement processes for data validation, cleansing, and enrichment to maintain accurate and reliable data. This includes monitoring data integrity throughout its lifecycle and implementing controls to identify and rectify errors promptly.	Dagster provides features for data quality checks within your pipelines. You can define expectations and validations for your data, ensuring that it meets the necessary standards before it's used in downstream tasks or reported for risk assessment.
Aggregation	Implement robust data aggregation mechanisms to collect and consolidate data from various sources across the organization. This involves designing data pipelines and workflows that efficiently gather data while maintaining its integrity and granularity.	With Dagster, you can create pipelines that aggregate data from various sources. Dagster's execution engine ensures that these aggregations are performed reliably, handling errors and retries as needed.
Traceability	Enable traceability of data by implementing mechanisms to track the lineage of data elements from their source systems through to their use in risk reporting. This helps ensure transparency and accountability in the data aggregation process and facilitates auditing and validation.	Dagster's asset catalog and lineage tracking features enable you to trace data from its origins through all transformations to its final form. This helps in understanding how data is transformed and used across your pipelines, which is essential for compliance.
Governance	Establish strong data governance frameworks to define policies, procedures, and standards for data management. Data engineering teams should work closely with data governance teams to ensure compliance with regulatory requirements and internal policies related to data management and risk reporting.	Dagster Cloud allows you to define and enforce data governance policies within your data pipelines. You can manage access, set permissions, and ensure that only authorized users can make changes to critical data processes.
Scalability and Performance	Design data management systems that can scale to handle large volumes of data efficiently. This includes optimizing data storage, processing, and retrieval to meet the performance requirements of risk reporting processes, which often involve complex analytics and computations.	Dagster is built to handle large volumes of data efficiently. It can scale your data pipelines horizontally, allowing you to process large datasets in a distributed manner.
Security and Privacy	Implement robust data security measures to protect sensitive information from unauthorized access, disclosure, or tampering. Data engineering teams should ensure compliance with relevant data protection regulations and industry standards while designing and maintaining data management systems.	While Dagster itself does not directly handle data security, it can be integrated with your existing security infrastructure. You can use Dagster to enforce security policies and ensure that sensitive data is handled appropriately within your pipelines.
Documentation and Metadata Management	Maintain comprehensive documentation and metadata for all data elements used in risk reporting. This includes documenting data definitions, transformations, and business rules to ensure clarity and consistency in data interpretation and usage.	Dagster automatically generates documentation for your pipelines.Dagster's ability to track and log metadata and lineage information for each data asset is crucial for understanding the data's lifecycle and for providing the necessary documentation to auditors or regulators.
Change Management	Implement effective change management processes to manage updates and modifications to data management systems and processes. Data engineering teams should ensure that changes are appropriately documented, tested, and validated to minimize the risk of disruptions to risk-reporting operations.	Dagster supports change management through its software development lifecycle features, such as code reviews, testing, and deployment pipelines. This ensures that any updates or modifications to data management systems and processes are tracked, reviewed, and deployed in a controlled manner.

In addition to the above, Dagster Cloud provides features for sharing reports on data asset metadata with non-technical users, which can be used to facilitate self-service by the auditing or risk-management teams.

It's important to note that while Dagster provides tools and features that can support compliance with BCBS 239, the actual compliance will depend on how these tools are implemented within your organization. Dagster can be a part of a larger compliance strategy, but it is not a one-stop solution for regulatory compliance. Compliance with BCBS 239 will require a combination of the appropriate technology, processes, and organizational policies.

In summary, Dagster provides a solid framework and set of tools that can help data engineering teams at large banks build data platforms that align with the principles of BCBS 239. It offers capabilities for ensuring data quality, traceability, governance, and documentation, as well as supporting robust data aggregation and change management processes. However, it's important to integrate Dagster effectively with your existing systems and processes to achieve full compliance with regulatory requirements.

If you would like to discuss how Dagster Labs can support your Data Engineering team’s goals related to BCBS 239, please get in touch.

We're always happy to hear your feedback, so please reach out to us! If you have any questions, ask them in the Dagster community Slack (join here!) or start a Github discussion. If you run into any bugs, let us know with a Github issue. And if you're interested in working with us, check out our open roles!

Follow us: