Dagster: general security
Elementl is the corporate sponsor of Dagster, an open-source data infrastructure project that lets customers develop, run, and monitor data pipelines. Elementl also runs Dagster Cloud, a hosted SaaS product built on top of Dagster. Elementl designs its processes and procedures related to its platform to meet its objectives for its data orchestration products and services.
While Dagster pipelines process customer data, Dagster Cloud’s hybrid architecture ensures that both the code that customers write to run pipelines, as well as the customer data upon which those pipelines operate, stay fully within the customer’s environment and are never stored or accessed by Dagster Cloud’s hosted services. Dagster Cloud stores the following types of data:
- Customer Metadata: Metadata about the pipelines that run in Dagster Cloud.
- Log Data: Logs, traces, and samples produced by Dagster Cloud while running pipelines.
Customer Metadata is treated as sensitive by Elementl. It is stored persistently, so that users can view information about past runs and understand the current state of their Dagster Cloud deployments. Elementl employees may access metadata to troubleshoot customer issues or to gather feedback for improving the Dagster Cloud product.
Log Data is produced by Dagster Cloud system components to make it easier for Elementl operators to monitor the health of the system and track down any issues. Log data is a trace of the actions performed by the system when serving web and API requests, backend services scheduling, launching, and monitoring jobs. Log data will include snapshots of Customer Metadata when the logs were captured, so that operators can understand the state of the system when the logs were produced. Log data also includes stack traces and samples of running code, as well as stack traces of errors that the agent encountered. Log data will only include the Customer Metadata listed above - since Dagster Cloud does not process customer data, customer data will not appear in logs.
All inbound network requests to Dagster Cloud require HTTPS, and all data stored persistently in our Postgres Database, Redis Message Queue, and S3 buckets are encrypted at rest.
Risk Management and Security Controls
Elementl maintains a Risk Management Policy with the goal of assessing and managing Elementl’s information security risks in order to achieve the company’s business and information security objectives. As part of this process, Elementl maintains a risk register to track all systems and procedures that could present risks to meeting the company’s objectives. The risk register is reevaluated annually, and tasks are incorporated into the regular Elementl product development process so they can be dealt with predictably and iteratively.
Risks are assessed and ranked according to their impact and their likelihood of occurrence. A formal IT risk assessment, network penetration tests, and Dagster Cloud penetration test are performed at least annually.
Incident Response Management
Elementl maintains an Incident Response Policy that gives any Elementl employee the ability to initiate a response to a potential security incident by notifying the internal security team through several channels and assists in classifying the severity of the incident.
Elementl’s management conducts quality assurance monitoring on a regular basis and additional training is provided based upon results of monitoring procedures. Management’s close involvement in Elementl’s operations helps to identify significant variances from expectations regarding internal controls. Upper management evaluates the facts and circumstances related to any suspected control breakdown. A decision for addressing any control’s weakness is made based on whether the incident was isolated or requires a change in the company’s procedures or personnel.
All Dagster Cloud data is hosted by Amazon Web Services (AWS). AWS data centers do not allow Elementl employees physical access.
Elementl’s physical office locations do not have access to operational or developmental environments, and do not house any customer information.
SOC 2 Type 2:
Our SOC 2 Type II report proves our commitment to security as we aim to keep our customer data secure.
In order to handle the most sensitive types of customer information, Elementl (the company behind Dagster) is HIPAA compliant to secure Protected Health Information.
Elementl’s services are designed with the assumption that certain controls will be implemented by user entities. Such controls are called complementary user entity controls.
The following complementary user entity controls should be implemented by user entities to provide additional assurance that the Trust Services Criteria described within this report are met. As these items represent only a part of the control considerations that might be pertinent at the user entities’ locations, user entities’ auditors should exercise judgment in selecting and reviewing these complementary user entity controls.
Dagster Cloud’s hybrid architecture ensures that all customer data stays in the customer’s cloud.
- User entities are responsible for understanding and complying with their contractual obligations to Elementl.
- User entities are responsible for ensuring that the access control levels that they set in Elementl tools match the desired access control level for use of Dagster Cloud.
- User entities are responsible for notifying Elementl of changes made to technical or administrative contact information.
- User entities are responsible for ensuring the supervision, management, and control of the use of Elementl services by their personnel.
- User entities are responsible for developing their own disaster recovery and business continuity plans that address the inability to access or utilize Elementl services.
- User entities are responsible for immediately notifying Elementl of any actual or suspected information security breaches, including compromised user accounts.
Found a bug or issue? Have questions or concerns? Contact us at email@example.com.