Title: Vocabulary Attack to Hijack Large Language Model Applications Show Chinese title

Title: 词汇攻击以劫持大型语言模型应用

Abstract: The fast advancements in Large Language Models (LLMs) are driving an increasing number of applications. Together with the growing number of users, we also see an increasing number of attackers who try to outsmart these systems. They want the model to reveal confidential information, specific false information, or offensive behavior. To this end, they manipulate their instructions for the LLM by inserting separators or rephrasing them systematically until they reach their goal. Our approach is different. It inserts words from the model vocabulary. We find these words using an optimization procedure and embeddings from another LLM (attacker LLM). We prove our approach by goal hijacking two popular open-source LLMs from the Llama2 and the Flan-T5 families, respectively. We present two main findings. First, our approach creates inconspicuous instructions and therefore it is hard to detect. For many attack cases, we find that even a single word insertion is sufficient. Second, we demonstrate that we can conduct our attack using a different model than the target model to conduct our attack with. Show Chinese abstract

Abstract: 大型语言模型（LLMs）的快速发展正在推动越来越多的应用。 随着用户数量的增加，我们还看到越来越多的攻击者试图欺骗这些系统。 他们希望模型泄露机密信息、特定的虚假信息或表现出冒犯性行为。 为此，他们通过插入分隔符或系统地重新表述指令来操纵给LLM的指令，直到达到他们的目标。 我们的方法不同。 它从模型词汇表中插入单词。 我们使用优化过程和另一个LLM（攻击者LLM）的嵌入来找到这些单词。 我们通过目标劫持两个流行的开源LLMs分别来自Llama2和Flan-T5系列来证明我们的方法。 我们提出了两个主要发现。 首先，我们的方法创建了不显眼的指令，因此很难被检测到。 对于许多攻击案例，我们发现即使插入一个单词也是足够的。 其次，我们证明我们可以使用与目标模型不同的模型来进行我们的攻击。