标题： LLMs的值是否与人类结构对齐？ 因果视角 显示英文标题

标题： Are the Values of LLMs Structurally Aligned with Humans? A Causal Perspective

摘要： 随着大型语言模型（LLMs）越来越多地融入关键应用，使其行为与人类价值观保持一致带来了重大挑战。 当前的方法，如基于人类反馈的强化学习（RLHF），通常只关注一组有限的粗粒度价值观，并且资源消耗大。 此外，这些价值观之间的相关性仍然是隐式的，导致价值观引导结果的解释不清晰。 我们的工作认为，LLMs 的价值观维度下存在一个潜在的因果价值图，并且尽管进行了对齐训练，这种结构与人类价值观体系仍有显著差异。 我们利用这些因果价值图来指导两种轻量级的价值引导方法：基于角色的提示和稀疏自编码器（SAE）引导，有效缓解了意外的副作用。 此外，SAE 提供了一种更细粒度的价值引导方法。 在 Gemma-2B-IT 和 Llama3-8B-IT 上的实验展示了我们方法的有效性和可控性。 显示英文摘要

摘要： As large language models (LLMs) become increasingly integrated into critical applications, aligning their behavior with human values presents significant challenges. Current methods, such as Reinforcement Learning from Human Feedback (RLHF), typically focus on a limited set of coarse-grained values and are resource-intensive. Moreover, the correlations between these values remain implicit, leading to unclear explanations for value-steering outcomes. Our work argues that a latent causal value graph underlies the value dimensions of LLMs and that, despite alignment training, this structure remains significantly different from human value systems. We leverage these causal value graphs to guide two lightweight value-steering methods: role-based prompting and sparse autoencoder (SAE) steering, effectively mitigating unexpected side effects. Furthermore, SAE provides a more fine-grained approach to value steering. Experiments on Gemma-2B-IT and Llama3-8B-IT demonstrate the effectiveness and controllability of our methods.