With the world of data analytics and governance constantly changing, it is critical for organizations to stay on the cutting edge. The sharing of ideas between innovators and industry leaders help provide a glimpse into the dynamic core of this field, highlighting the trends central to staying competitive both today and beyond.
A panel of tech pioneers recently spoke about these data trends at the Citi Ventures Fall Enterprise Tech Series. Immuta Founder and CEO Matt Carroll joined fellow leaders from Citi, Snowflake, Databricks, Starburst, Matillion, and Redis, to underscore some of the most pressing themes in data use.
Discussing the acceleration of cloud adoption, the tension between data use and increased regulations, and how to cut through the complexity of data analysis, these data leaders emphasized the importance of addressing today’s data use trends in order to maintain a successful business. Here are our top four takeaways from the event.
Today’s leaders agree that as data becomes increasingly important to an organization’s business strategy, data teams are rapidly transitioning from physical, on-premises storage models to the cloud.
Cloud data platforms are helping companies move from slower, fixed models of data storage and analysis to scalable, dynamic, and more efficient models. “We’re seeing constant migration as fast as possible,” Carroll said, calling attention to the integral role this shift is playing in contemporary business strategy. Whether moving storage from Teradata to Snowflake or transitioning Hadoop clusters to Databricks, there is a general immediacy surrounding data relocation to the cloud.
Following the shift to the cloud, companies need the ability to scale workloads and users across platforms. This process, while necessary for data enablement, can involve tedious manual work for data engineers and architects. Another necessity of cloud maturation is being able to exchange data with third parties, which in turn involves more manual policy creation and auditing.
This is where an automated data access control framework is key, as it eliminates the need for manual policy creation and streamlines cloud adoption. Ultimately, it makes the transition to the cloud both more efficient and more effective.
A prevailing view of data’s role in business amongst these innovators is that the more data we can responsibly use, the better. “We want more personal data, because we want unique insights and vectors around our customers and what they’re doing and why they’re doing it,” said Carroll. Companies have the desire to use their customer data to analyze insights, create strategies, and drive overall success.
However, the increase in and evolution of regulations governing data is all but constant. Many of these regulations are produced and enforced by governmental bodies, such as HIPAA, GDPR, and CCPA. Other rules are put in place internally by organizations that are subject to specific requirements. According to a survey Immuta conducted with Gradient Flow, 30% of respondents claimed they are subject to internal, company-specific rules.
While these regulatory measures are necessary, they can often become a burden on how a business handles data. When you want real-time insights and analysis, the need to guarantee compliance with potentially hundreds or thousands of rules can be a major bottleneck. This disconnect between usability and compliance tends to drive the creation of overly restrictive manual policy creation, which again can create unnecessary strain.
Automated data access control works to alleviate the tension between privacy and utility by helping to bridge the gap and create a compliant environment that enables secure, real-time data access.
When you take the inherent intricacy of migrating data storage from legacy systems to a cloud-based infrastructure, and add the layer of governance and regulations on top, cloud data use can easily become very complex. There is no denying that the move to cloud is a business necessity, but the process tends to become convoluted and burdensome to those tasked with manual policy creation and risk assessment.
“Any query could have hundreds of thousands of policies on it; you just can’t write code to do that,” Carroll said. When relying on manual policy creation and enforcement, the time-to-value for an organization’s data is significantly longer.
So, how do these layers of complexity stack up? First, platform complexity among different cloud systems leads to disparate access and privacy control mechanisms. On top of that, the aforementioned rule complexity creates a massive buildup of both legacy and new rules that exacerbate time to available data. Lastly, the amount of new users on these platforms expands exponentially, creating the need for massive amounts of roles in order to guarantee secure access for everyone involved.
When these tiers of complexity pile up, businesses are faced with an important question: How can we make effective use of our data resources without placing exorbitant strain on our organization? How can we guarantee access without all of the excess?
The key to creating a successful, scalable, and useful cloud data storage system is to separate policy creation from the platforms. “With all these systems…there’s no single way to build policy across it consistently,” said Carroll. The approach then tends to come down to creating manual row- or column-level security, manually auditing data access, or other similarly work-intensive measures.
Instead, the need to scale can be met by adopting an automated data access control model with universal cloud compatibility, which allows for policy to be abstracted from a specific platform and consistently enforced across any systems in an organization’s data stack. With such a model, data teams can create policies that stretch across cloud compute environments without affecting data users or their analysis. Cloud adoption can then be carried out without worrying about manual policy creation or the layers of platform and user complexity.
Immuta’s data access control solution creates a scalable cloud environment for any organization, regardless of its data stack. Providing automated data access control, cross-platform policy orchestration and management, and multi-dimensional observability creates an environment where governance and business insights can exist hand-in-hand.
“The argument I always make,” said Carroll, “is that the ability to scale and do more requires that you do less.” Immuta helps users avoid role-explosion and excess work, hooking natively into cloud platforms and providing groups with the ability to operate adeptly and securely.
To see how Immuta’s SaaS deployment can facilitate quick, scalable, and secure migration to the cloud, request a free trial!
