Getty Images
Generative artificial intelligence models often encounter security threats like prompt injections and data exfiltration. Cybersecurity companies are combatting these risks by utilizing AI to safeguard large language models (LLMs), but this approach comes with associated expenses.
This article is a part of “How AI Is Changing Everything,” a series focusing on AI adoption across various industries. Generative artificial intelligence is a relatively recent technology, leading to new security challenges that organizations may not be fully prepared for.
Chatbots powered by extensive language models face vulnerability to innovative attacks such as prompt injections, where specially crafted prompts alter the model’s behavior, and data exfiltration, which involves repetitive prompting to extract sensitive information.
These attacks exploit the unpredictable nature of LLMs and have caused significant financial losses. Chuck Herrin, the Field Chief Information Security Officer at F5, a multicloud application and security firm, highlighted a notable security breach involving OpenAI and DeepSeek, a Chinese company’s reasoning model.
DeepSeek’s release of DeepSeek-R1, closely trailing OpenAI’s top models, raised suspicions due to similarities in response construction and model identification. Allegations of intellectual property theft and concerns over distillation techniques utilized by DeepSeek led to substantial market impact.
Exploiting vulnerabilities in AI is feasible as LLMs respond to a wide range of prompts without memorizing specific training data. However, repeated prompting and analysis can emulate a model’s behavior, potentially accessing training data and intellectual property.
To enhance AI security, safeguarding the application programming interface alongside the broader ecosystem is crucial. LLMs’ complex neural networks lack transparency, making it challenging to pinpoint specific elements responsible for responses to prompts, further emphasizing the need for robust security measures.
Sanjay Kalra, the head of product management at Zscaler, emphasized the importance of safeguarding data within an LLM similar to how organizations protect databases. In a traditional approach, data is typically stored in a database, allowing for easy deletion. However, with LLM chatbots, rolling back information is not as straightforward. Kalra highlighted the significance of security fundamentals like access control and user logging, emphasizing the need for a layered cybersecurity approach to address AI vulnerabilities.
Cybersecurity companies are addressing these challenges through a combination of traditional security measures and AI solutions. Authentication and authorization play a crucial role in verifying users for LLMs, while AI watchdogs are being trained to analyze user prompts and model responses for potential threats. These AI models act as an additional layer of defense against malicious activities that could compromise the LLM.
Deploying security-tuned AI models, such as OpenAI’s GPT-4.1, can enhance protection but comes with a significant cost. To make security measures more practical, smaller language models with fewer parameters, like Meta’s Llama 3-8B and Mistral’s Ministral 3B, are being utilized. Zscaler also develops its own internal models through its AI and machine learning team to enhance security measures.
As organizations navigate evolving AI technologies, a multilayered security approach combining cybersecurity fundamentals with AI models tailored for security purposes can help strengthen an LLM’s defense mechanisms.
