The Zero-Day Vulnerabilities Of Generative AI

Posted by Neil Sheppard on July 25, 2024
The Zero-Day Vulnerabilities Of Generative AI

Generative AI remains a novel technology, and we're still discovering the risks and threats it may pose. Learn how SAP LeanIX supports you in adopting AI safely and confidently.

Generative artificial intelligence (AI) is a revolutionary technology with the potential to change the way we do business. Unfortunately, it's also a technology that we don't yet fully understand.

Generative AI has been around for a while, but the release of ChatGPT in 2022 announced that it was ready for prime time. Since then, organizations have labored to discover what it is best suited for and how to implement it within the tech stack. Given that most are still in the exploration phase, the potential risks from the use of this technology are yet to be fully understood.

Despite the novelty and uncertainty, organizations cannot employ a wait-and-see approach when it comes to AI adoption. Those who can successfully harness its power will enjoy a distinct competitive advantage. Those who can't will be left behind.

This doesn't mean that organizations should stumble blindly into using generative AI. To do so would mean traversing a minefield of risks. From cybersecurity threats to hefty regulatory penalties, we're already seeing a variety of negative consequences arising in association with a willy-nilly approach to generative AI.

To lay the groundwork for future success, organizations need to implement generative AI safely. Adopting this technology calls for the simultaneous adoption of a proper AI governance framework to avoid both anticipated and unanticipated risks. To support that framework, you will also want a tool providing a comprehensive overview of your AI usage.

To find out how SAP LeanIX empowers safe AI adoption, book a demo:

Request demo

 

The Risks Of Generative AI

Generative artificial intelligence (AI), like any other new technology, remains an unknown until it's been extensively used in real-world conditions. We simply can't know what threats the technology poses until it's been fully adopted by the market.

We can guarantee that we will find a number of vulnerabilities in generative AI tools that their developers never expected to arise. Such 'zero-day' vulnerabilities can't be avoided and must simply be discovered and addressed as they reveal themselves.

For example, Johann Rehberger, Chief Hacking Officer at WUNDERWUZZI, added a malicious prompt for generative AI to the transcripts of his YouTube videos. When an AI attempted to read the transcript, it was reprogrammed to display "AI injection succeeded" and tell a joke.

This was, of course, an innocent action intended to raise awareness of generative AI's potential dangers. Unfortunately, the same process could be used to do something more malicious:

“If people build applications to have the LLM read your emails and take some action based on the contents of those emails—make purchases, summarize content—an attacker may send emails that contain prompt-injection attacks.”

William Zhang, Machine Learning Engineer, Robust Intelligence

Think about it. Although an AI-enabled email reading assistant sounds like a great time saver, a hacker could send you an email with a malicious prompt that told the AI to delete all your emails or forward them to the hacker. Since these kinds of problems are emergent – we won't be able to deal with them or even know about them until they make their first appearance – assumptions about the safety and security of AI are going to be wrong for a long time.

However, as we mentioned above, avoiding generative AI altogether could put your organization at a real disadvantage. Driven by this fear (the fear of being left behind), 80% of organizations are already leveraging AI to some degree, our recent AI survey showed.

Ironically, if we all refuse to leverage generative AI, zero-day vulnerabilities will never be discovered and the technology will never become any safer to use. In other words, the key to addressing these safety issues lies in finding a way to work with AI as safely as possible.

 

Leveraging Generative AI Without The Risk

Ensuring you have complete visibility into generative AI's use across your IT landscape is the first step to minimizing risk, both known and unknown. Such visibility will not only help you understand how teams are using generative AI. It will also show you where you may be vulnerable and at risk.

Our AI survey found that 90% of IT experts across the world agree that such visibility is critical when it comes to managing risk, even if we're only talking about the risk associated with running afoul of evolving AI regulation. At the same time, only 14% of respondents said they possess anything approaching complete oversight of AI in their IT estate.

When you know who in your company is utilizing AI and what they're using it for, you can start to assess your risk. Where team members are using the official free version of ChatGPT, for example, to write internal, non-confidential communications, there's little to no risk.

On the other hand, Samsung was recently caught out when employees input proprietary code into the public ChatGPT interface for bug fixing. This could pass on the confidential code to ChatGPT's vendor and, potentially, to other users.

Data and IP leakage through AI use is a major threat, while other uses are inconsequential. This is why it's crucial to get the whole picture and manage risk accordingly.

Controlling your AI portfolio in this way calls for a comprehensive view across your IT landscape. To create and maintain that view, you'll want a tool that can track and monitor your AI usage, while also supporting best-practice governance.

Far from restricting usage, this allows you to adopt AI confidently and safely. Proper strategy and governance, in other words, enable AI adoption and fulfill the current enterprise demand for the technology.

 

SAP LeanIX For Generative AI Adoption And Governance

Generative artificial intelligence (AI) adoption and governance are two sides of the same coin. To leverage AI confidently, you also need to account for any and all related risks.

SAP LeanIX gives you complete oversight of your IT landscape, including all of the AI tools you're using. On top of that, it lets you create your own custom dashboards to track your AI portfolio and share them with your stakeholders.

Finally, we've extended our meta-model to make it possible to track AI at every level of your enterprise architecture and understand exactly how it supports everything from business capabilities to corporate strategy.

To find out exactly how SAP LeanIX empowers safe AI adoption, book a demo:

Request demo

Subscribe to the LeanIX Blog and never miss a post again!

Related Posts