Introduction & Executive Summary

Compliance, privacy, and ethics have always been make-or-break mandates for enterprise organizations. But the stakes have never been higher than they are right now: Generative AI is allowing companies to test new limits and innovations, faster than ever.

For all of the innovation and potential that generative AI brings, it also presents a world of uncertainty and security risk: 80% of data experts agree that AI is making data security more challenging.

In their eagerness to embrace large language models (LLMs) and keep up with the rapid pace of adoption, employees at all levels are sending vast amounts of data into unknown and unproven AI models. The potentially devastating security costs of doing so aren’t yet clear.

In our 2024 State of Data Security Report, 88% of data leaders said employees at their organization were using AI, whether officially adopted by the company or not. Yet they expressed widespread concern about sensitive data exposure, training data poisoning, and unauthorized use of AI models. As adoption speeds up, these risks will only become more pressing.

“The emergence of generative AI and GPT technologies has brought about new concerns regarding data privacy and security.”

Diego Souza
Chief Information Security Officer at Cummins

As fast as AI is evolving, standards, regulations, and controls aren’t adapting fast enough to keep up. To manage the risks that come with this sea change, organizations need to secure their generative AI data pipelines and outputs with an airtight security and governance strategy.

We asked nearly 700 engineering leaders, data security professionals, and governance experts for their outlook on AI security and governance. In this report, we cover their optimism around the technology, the security challenges they’re facing, and what they’re doing to adopt it.

Key Insights

01 AI Is Making Data Security More Challenging

Organizations can’t adopt AI fast enough. The hype and excitement around it are too tempting to resist, which explains why more than half of data experts (54%) say that their organization already leverages at least four AI systems or applications. More than three-quarters (79%) also report that their budget for AI systems, applications, and development has increased in the last 12 months.

But this fast-paced adoption also carries massive uncertainty. AI is still a black box when it comes to security and governance.

Leaders cite a wide range of data security threats with AI and LLMs:

• 55% say inadvertent exposure of sensitive information by LLMs is one of the biggest threats.
• 52% are concerned about inadvertent exposure of sensitive information to LLMs via user prompts.
• 52% worry about adversarial attacks by malicious actors via AI models.
• 57% say that they’ve seen a significant increase in AI-powered attacks in the past year.

02 Confidence in AI Adoption & Implementation Is High

Despite all of the security and governance risks, data leaders are highly optimistic about their security strategy — raising potential concerns that they are overconfident:

• 85% say they feel confident that their data security strategy will keep pace with the evolution
of AI.
• 66% rate their ability to balance data utility with privacy concerns as effective or highly effective.

To keep pace with the evolution of AI, data leaders must look at how they can scale and automate data security and governance, and give a strong voice to those teams responsible for managing risk in the organization.

03 Policies and Processes Are Changing

With new technology comes new responsibilities — and new security, privacy, and compliance risks. In response to the growth of AI adoption, IT and engineering leaders are adapting organizational governance standards, with 83% of respondents reporting their organization has updated its internal privacy and governance guidelines:

• 78% of data leaders say that their organization has conducted risk assessments specific to AI security.
• 72% are driving transparency by monitoring AI predictions for anomalies.
• 61% have purpose-based access controls in place to prevent unauthorized usage of AI models.
• 37% say they have a comprehensive strategy in place to remain compliant with recent and forthcoming AI regulations and data security needs.

04 Data Experts Are Forward-Facing

While it’s easy to worry about the challenges that AI adoption presents, many data leaders are also very excited about how the technology will help improve security and governance. From the ability to adopt new tools, to unlocking new ways of automating data management processes and implementing better safeguards, they are looking to the future with optimism.

Data leaders believe that one of the most promising AI security advancements is AI-driven threat detection systems (40%). This makes sense, given how AI and machine learning are able to automate processes and quickly analyze vast data sets. Another auspicious advancement is the use of AI as an advanced encryption method (28%) that can help generate robust cryptographic keys and optimize encryption algorithms.

Data leaders are also interested in the potential for AI to serve as a tool in service of data security. Respondents say that some of the main advantages of AI for data security operations will include:

• Anomaly detection (14%)
• Security app development (14%)
• Phishing attack identification (13%)
• Security awareness training (13%)

Methodology

Immuta commissioned an independent market research agency, UserEvidence, to conduct the 2024 AI Security and Governance Survey.

AI Security and Governance Survey.

The study surveyed 697 data leaders and professionals from the US, UK, Canada, and Australia. Respondents represent global cloud-based enterprise companies across public and private sectors, with the majority (57%) in the technology sector. Other sectors surveyed include:

• Manufacturing
• Financial services
• Business and professional services
• Public sectors, such as government
• Private healthcare, including pharma
• Several other sectors

Over a third of respondents (34%) are senior leaders or C-level executives, and more than a quarter of respondents (28%) are mid-level managers.

All respondents use data analytics, governance, or transformation tools in their roles, with 76% regularly using those tools at their organization. The most common job titles included: