标题： 概念的柏拉图表示下大型语言模型之间的跨模型可迁移性 显示英文标题

标题： Cross-model Transferability among Large Language Models on the Platonic Representations of Concepts

摘要： 理解大型语言模型（LLMs）的内部机制是一个关键的研究领域。先前的研究表明，单个LLM的概念表示可以被捕捉为引导向量（SVs），从而能够控制LLM的行为（例如，生成有害内容）。我们的工作通过探索不同LLMs之间概念表示的复杂关系，采取了一种新颖的方法，并提出了一个引人注目的类比，即柏拉图的洞穴寓言。具体来说，我们引入了一种线性变换方法来连接这些表示，并提出了三个主要发现：1）可以通过简单的线性变换有效地对齐不同LLMs的概念表示，从而通过SVs实现高效的跨模型传输和行为控制。2）这种线性变换在概念上具有泛化能力，促进了不同概念的SVs在LLMs之间的对齐和控制。3）LLM概念表示之间存在弱到强的可迁移性，即从较小的LLMs中提取的SVs可以有效控制较大LLMs的行为。 显示英文摘要

摘要： Understanding the inner workings of Large Language Models (LLMs) is a critical research frontier. Prior research has shown that a single LLM's concept representations can be captured as steering vectors (SVs), enabling the control of LLM behavior (e.g., towards generating harmful content). Our work takes a novel approach by exploring the intricate relationships between concept representations across different LLMs, drawing an intriguing parallel to Plato's Allegory of the Cave. In particular, we introduce a linear transformation method to bridge these representations and present three key findings: 1) Concept representations across different LLMs can be effectively aligned using simple linear transformations, enabling efficient cross-model transfer and behavioral control via SVs. 2) This linear transformation generalizes across concepts, facilitating alignment and control of SVs representing different concepts across LLMs. 3) A weak-to-strong transferability exists between LLM concept representations, whereby SVs extracted from smaller LLMs can effectively control the behavior of larger LLMs.