Lakera: A New Platform to Protect Large Language Models from Malicious Prompts

Large language models (LLMs) are powerful AI systems that can generate natural language texts from simple prompts. They have many potential applications, such as content creation, summarization, translation, and more. However, they also pose significant security risks, as they can be manipulated by malicious users who can inject specially crafted prompts to exploit their vulnerabilities.

For example, a malicious user could try to trick an LLM-powered chatbot into revealing sensitive information, such as passwords or personal data, by using linguistic techniques such as role-playing, obfuscation, or sidestepping. This is known as prompt injection, and it is one of the main threats that LLMs face.

To address this challenge, a Swiss startup called Lakera has launched a platform that aims to protect LLMs from prompt injections and other security weaknesses. Lakera offers a cloud-based service that monitors and filters the inputs and outputs of LLMs, such as GPT-4, and alerts the users of any potential risks. Lakera also helps prevent data leakage, content moderation, misinformation, and factual inaccuracies that LLMs may generate.

Lakera has developed a comprehensive database of insights by leveraging various sources, including open source datasets, in-house research, and data collected from an interactive game called Gandalf. Gandalf is a game-like tool that allows users to test the security of LLMs by trying to hack them using linguistic tricks. The insights gained from Gandalf are then integrated into Lakera Guard, the company's flagship product.

Lakera's launch is timely, as the company aligns with the EU AI Act, which is set to introduce regulations to safeguard generative AI models. The Act emphasizes the need for LLM providers to identify risks and implement appropriate measures. Lakera's founders have been involved in advisory roles for the Act, contributing their technical expertise to shape the regulatory landscape.

Lakera recognizes that enterprises may hesitate to adopt generative AI due to security concerns. The company works closely with startups and leading enterprises to ensure the secure integration of generative AI applications. By addressing security obstacles, Lakera aims to facilitate the smooth deployment of these applications while mitigating risks.

(1) Lakera launches to protect large language models from malicious prompts .... https://techcrunch.com/2023/10/12/lakera-launches-to-protect-large-language-models-from-malicious-prompts/.
(2) Lakera launches to protect large language models from malicious prompts. https://finance.yahoo.com/news/lakera-launches-protect-large-language-122550893.html.
(3) New Language Model Protection Startup Lakera Aims to Safeguard Against .... https://robots.net/news/new-language-model-protection-startup-lakera-aims-to-safeguard-against-malicious-prompts/.