How AstrumU Maximizes Data Privacy & Utility with Immuta & Databricks

Kaj Pedersen, Chief Technical Officer at AstrumU, joined Immuta and Databricks at a recent Amazon Web Services Machine Learning Dev Day. This blog recaps his presentation on building a data governance and compliance strategy.   

Before founding AstrumU, Adam Wray identified a crucial — and costly — problem plaguing the American employment and education systems: there was no way to translate educational experiences into economic opportunity. Systems of higher education struggle to provide educational credentials that represent skills needed in the labor market. Meanwhile, employers remain weary of education systems’ ability to provide the talent they need. With no verified, longitudinal data available between education and industry, schools, employers, and students are unable to make sound decisions about their futures.

AstrumU works with organizations and individual learners to solve this problem. With AstrumU’s help optimizing hiring, employers are able to find better candidates for open roles, reduce turnover, and improve employee satisfaction. AstrumU also provides individual learners with the information they need to make decisions about their education that will maximize economic opportunities. For students, schools, and organizations, AstrumU standardizes education and employment data using an AI translation engine, and shares the information with learners, education professionals, and employers.

The Challenge

To quantify the return on education investment for learners, educators, and employers, AstrumU needs to maintain access to a very large database of employer and employee data, including private and sensitive information. And, to provide customers with the most up-to-date and accurate analytics and predictions, AstrumU regularly collects data and adds to its existing database.

Collecting, storing, and using personal information in this manner comes with inherent risk and introduces many challenges to the organization’s ability to derive value from its data. Without the right tools in place, AstumU’s database of sensitive information had the potential to jeopardize its reputation, customers, and product due to a variety of risks:

• Compliance Risk: AstrumU must comply with several data regulations, including FERPA, CCPA, GDPR, and SOC-2 guidelines, as well as contractual obligations to customers. Failing to observe any of these regulations can result in reputation tarnishment and legal fees up to $11 million. 
• Mission Risk: While modern data privacy calls for the strongest privacy and security capabilities available, if implemented incorrectly, these controls can become overly restrictive data access control policies and block AstrumU’s ability to use its data. To avoid this scenario, the AstrumU team needed to avoid short-term solutions that could hinder execution. Instead, they needed a scalable solution that would enable safe data access and use.
• Reputation Risk: News of data breaches can significantly tarnish – if not destroy – a company’s reputation. Research conducted by Ponemon Institute found that reputational damage from a data breach could equal a loss of up to 32% of the organization’s annual gross revenue. Respectful data use and strong protections can help prevent breaches, misuse, and data leakage, and maintain AstrumU’s reputation as a trustworthy, data-driven company.
• Customer Risk: Costly legal fees and business reputation are not the only consequences of a data breach. Security threats to AstumU’s customer data could result in leaked or stolen personal information. Maintaining modern governance and strong security was deemed necessary to protecting customer data, strengthening customer relationships, and retaining business. 

In order for AstrumU’s relatively small team to effectively mitigate risks and implement a long-term data governance solution, they needed additional help. Forming technology partnerships was the only way to achieve data security and compliance goals without introducing additional risks. With a partner, the AstrumU team could effectively implement a governance framework that supports SOC-2 requirements; enhances their understanding of data lineage; and builds a strategy for protecting personally identifiable information (PII) without sacrificing utility.

The Solution

o accomplish its Databricks access control goals and protect its most valuable asset — sensitive customer data — AstrumU integrated Databricks with Immuta. Together, they implemented protections at every stage of data use — from capture and modelling to storage and query. AstrumU uses Databricks’ cloud-based open-source platform to support its complex AI models. Immuta’s seamless integration with Databricks provides an additional layer of security that satisfies AstrumU’s data governance, access, and compliance needs.

Immuta and Databricks have improved AstrumU’s data processes in four key ways:

Data Unification: 

Since AstrumU caters to a variety of customers, it uses many types of data, stored in different places. With Immuta and Databricks, AstrumU avoided the manual process of physically combining or relocating its disparate data sources and instead created a single point of access, making data access significantly easier for data analysts and simplifying the data management and governance processes for data engineers.

Data Policies:

Given the sensitive nature of AstrumU’s data, its data team needed a strong data privacy and governance strategy. Immuta’s fine-grained access control and policy orchestration allow the AstrumU team to set policies based on dynamic user attributes. Immuta’s policies automatically hide, mask, redact, and anonymize data within the control plan on the row-, column-, or cell-level. This reduces time to data access to seconds, maximizing data usage and value without compromising privacy.

Authoritative Virtual Data Sources:

With Immuta, AstrumU exposed complex joins and data transformations for downstream users, enabling its data team to easily track data to its original source. Transparent data use and lineage make it significantly easier for AstrumU to prove compliance, and clear data identification has helped its data analytics team improve their product.

Audit Consistency:

Auditing is a crucial part of the compliance process, particularly when sensitive data is involved. Immuta’s advanced data monitoring and detection enable AstrumU to gather real-time insights into all data use and activity, and to create detailed reports of what data was accessed, by whom, when, and for what purpose. The ability to monitor data on-demand and from a centralized policy tier modernized AstrumU’s previous approach, simplified its compliance process, and increased consistency across data use silos.

The Result

Since implementing Immuta and Databricks, AstrumU has effectively complied with SOC-2 guidelines and reduced the risk of data breaches, without overextending its team or compromising the value of its data.

Using Immuta’s dynamic data masking features and attribute-based access control, AstrumU can demonstrate its product to prospective customers in real time without fear of exposing sensitive information, which has significantly strengthened its sales process and strategy. In fact, this capability has directly empowered AstrumU’s sales team to generate more business opportunities and build a faster, more successful sales pipeline. AstrumU has also gained legitimacy and enhanced its reputation within the HRTech and EduTech industries.

If you are struggling to govern your own data, looking for a tool to help with compliance, or are frustrated by a decline in data teams’ productivity, request a demo of Immuta. Or, explore our customer page to see how other companies are using Immuta’s innovative data governance and data access controls.