标题： ICPC：基于上下文提示压缩的快速推理 显示英文标题

标题： ICPC: In-context Prompt Compression with Faster Inference

摘要： 尽管大型语言模型（LLMs）最近取得了成功，但由于LLM输入的固定大小，向LLM提供长提示仍然具有挑战性。 作为解决方法，提示压缩通过删除提示中的冗余标记成为一种有前景的解决方案。 然而，在现有工作中使用LLM需要额外的计算资源并导致内存开销。 为了解决这个问题，我们提出了ICPC（上下文提示压缩），这是一种新颖且可扩展的提示压缩方法，能够自适应地减少提示长度。 ICPC的关键思想是使用编码器计算提示中每个词出现的概率，并通过信息函数计算每个词携带的信息，这有效地减少了提示压缩过程中的信息损失并提高了压缩速度。 实证表明，ICPC可以有效压缩不同类别的长文本，从而在不同类型的自然语言处理任务上实现更好的性能和速度。 显示英文摘要

摘要： Despite the recent success of Large Language Models (LLMs), it remains challenging to feed LLMs with long prompts due to the fixed size of LLM inputs. As a remedy, prompt compression becomes a promising solution by removing redundant tokens in the prompt. However, using LLM in the existing works requires additional computation resources and leads to memory overheads. To address it, we propose ICPC (In-context Prompt Compression), a novel and scalable prompt compression method that adaptively reduces the prompt length. The key idea of ICPC is to calculate the probability of each word appearing in the prompt using encoders and calculate information carried by each word through the information function, which effectively reduces the information loss during prompt compression and increases the speed of compression. Empirically, we demonstrate that ICPC can effectively compress long texts of different categories and thus achieve better performance and speed on different types of NLP tasks.