With nearly half of data teams focusing on creating, publishing, and sharing data products in the next year, and 68% planning to implement a data marketplace, there’s no question that the shift toward leveraging internal marketplaces to maximize the value of data products is well underway. As with any data modernization effort, this puts extreme strain on governance and compliance teams to establish the systems and processes that will help them meet their data marketplace objectives while maintaining governance and compliance goals.
We launched the Immuta Data Marketplace solution to help streamline and secure how data products are published, accessed, and shared within organizations. And today, we’re excited to announce four new innovations that further accelerate data product publishing, data marketplace access requests, and automatic provisioning flows, while remaining compliant with complex and evolving regulatory standards.
Before diving into those capabilities, consider the circle of life of a data product. The diagram below illustrates it at its most simplistic level, with black indicating where data product planning and engineering occur, and white representing the data marketplace:
A glaring omission of this diagram is the layers of governance and compliance required. In most of the organizations we work with, the humans in charge of governance and compliance are not the same humans in charge of creating value with data products.
There is a key separation of duties between these teams, but they also must work seamlessly together – too much governance will slow data product delivery and innovation; too little governance will place the organization at risk and create complexity for your data consumers who have zero understanding of what data they can use and why.
Our latest features help these two teams bridge those gaps in harmony, accelerating data product delivery and provisioning, while also meeting stringent compliance needs. In this blog, we’ll explore the top concerns facing data teams in managing efficient, secure data product delivery via data marketplaces, and explain how Immuta’s new features help overcome obstacles without introducing risk.
You want to empower data engineering teams to create and publish data products that drive business value. By enabling them to manage that process, you reduce friction and increase efficiency. But, you also do not want your data engineers to publish any data to the marketplace without oversight, since that may result in exposing sensitive information and introducing risk.
Our first new innovation is dynamic data domain assignment. While Immuta has always allowed permissioning for who can publish data products to the marketplace from specific domains, dynamic data domain assignment takes that a step further. Now, governance teams can control dynamic assignment of data to a domain based on where it originates and/or how it is tagged.
This allows you to configure sandboxes (typically CREATE privilege on a database or schema) for data engineering teams to create data products. When they consider those data products ready to publish, the data engineers can simply tag them, which will dynamically assign them to the appropriate domain(s) for publishing. This allows each domain, which is typically a line of business or area of concern within lines of businesses, to have its own areas for data product creation, as well as the ability to quickly, but compliantly, promote data for publishing on its own.
Take the following example from one of Immuta’s customers: You have critical data that you only want non-human data processing pipelines to have access to, but you can’t know ahead of time when that data may be required for a new pipeline. So, you require the creator of the new pipeline to request access through the marketplace, on behalf of the non-human system account that will do the processing. How do you ensure that only system accounts can ever get access?
This is what our next innovation, prevention policies, solves. Prevention policies allow you to set restrictions on who can gain access to data, no matter what. Continuing our example, if a human requested and was granted access to this data instead of a system account, our example prevention policy would still block them from gaining access. Prevention policies create a permanent “global guardrail” around who can consume certain data, no matter what actions are taken in the marketplace. And you can configure these prevention policies to consider different metadata about your users as the policy logic.
When it comes to providing access to sensitive data, scrutiny is key. Many of our customers therefore require very specific questions to be answered in order to grant access to data. In many cases, these questions need to be tracked and audited for compliance, and cannot be free form.
Our third innovation for the Immuta Data Marketplace is customizable request forms. These allow data product publishers and/or stewards assigned with managing access requests to build a set of highly customizable questions that must be answered as part of an access request. If desired, the questions can also be reused across data products to increase efficiency and consistency.
The form questions support four answer types: free text, dropdown, check boxes, and access duration. The answers provide the necessary context to make an accurate determination of whether to grant or deny access.
The customizable request forms can also be integrated with existing data catalogs to initiate an access request on demand. They will set the foundation for our next innovation, dynamic review flows, which will consider the questions’ answers, requestor’s identity, and data’s contents, in order to dynamically approve, deny, or designate a review chain.
Many of our customers express a concern that when data is sensitive, there is no reason to have access to it longer than the minimum amount of time needed. And those same customers wish to avoid drawn out and time consuming entitlement reviews and access recertification processes, where managers must manually review access of all their reports at some periodicity. Those recertifications become moot if access is always withdrawn within some period of time.
To support this, we created our fourth innovation, timebound approvals. With timebound approvals, the requestor can specify how long they need access for (see access duration answer type in the previous section), and the reviewer can either accept that proposed amount of time, reject it, or edit it to be longer or shorter. And as Immuta already automatically provisions data access upon approval, so does it automatically de-provision access when that time limit has been reached.
These four innovations align the goals of governance and compliance with the delivery of data products, allowing the charters of both parties to be met rapidly and effectively. As you can see from the final diagram, these innovations cover both the inputs and workflows for managing requests and provisioning data products, providing complete compliance coverage without slowing down consumers’ speed to accessing data.
If you are interested in learning more about these new capabilities, please contact us. Customers, please contact your Customer Success Manager.
Take a self-guided tour or a get custom tour from our team.
