Title: On the Fundamental Impossibility of Hallucination Control in Large Language Models Show Chinese title

Title: 关于大型语言模型中幻觉控制的根本不可能性

Abstract: This paper establishes a fundamental Impossibility Theorem: no LLM performing non-trivial knowledge aggregation can simultaneously achieve truthful knowledge representation, semantic information conservation, complete revelation of relevant knowledge, and knowledge-constrained optimality. This impossibility stems from the mathematical structure of information aggregation, not from engineering limitations. We prove this by modeling inference as an auction of ideas, where distributed components compete to influence responses using their encoded knowledge. The proof employs three independent approaches: mechanism design (Green-Laffont theorem), proper scoring rules (Savage), and transformer architecture analysis (log-sum-exp convexity). We introduce the semantic information measure and the emergence operator to analyze computationally bounded and unbounded reasoning. Bounded reasoning makes latent information accessible, enabling gradual insights and creativity, while unbounded reasoning makes all derivable knowledge immediately accessible while preserving the semantic content. We prove the conservation-reasoning dichotomy: meaningful reasoning necessarily violates information conservation. Our framework suggests that hallucination and imagination are mathematically identical, and both violate at least one of the four essential properties. The Jensen gap in transformer attention quantifies this violation as excess confidence beyond constituent evidence. This unified view explains why capable models must balance truthfulness against creativity. These results provide principled foundations for managing hallucination trade-offs in AI systems. Rather than eliminating hallucination, we should optimize these inevitable trade-offs for specific applications. We conclude with philosophical implications connecting the impossibility to fundamental limits of reason. Show Chinese abstract

Abstract: 本文建立了一个基本的不可能定理：任何执行非平凡知识聚合的大型语言模型，都无法同时实现真实的知识表示、语义信息守恒、相关知识的完全揭示以及知识约束最优性。 这种不可能性源于信息聚合的数学结构，而不是工程限制。 我们通过将推理建模为思想的拍卖来证明这一点，在这个过程中，分布式组件使用其编码的知识竞争以影响响应。 证明采用了三种独立的方法：机制设计（Green-Laffont定理）、适当评分规则（Savage）和变压器架构分析（log-sum-exp凸性）。 我们引入了语义信息度量和涌现算子来分析计算有限和无限的推理。 有限推理使潜在信息可访问，从而实现渐进的洞察力和创造力，而无限推理则使所有可推导的知识立即可访问，同时保持语义内容。 我们证明了保守推理二分法：有意义的推理必然违反信息守恒。 我们的框架表明，幻觉和想象在数学上是相同的，两者都至少违反四个基本属性中的一个。 变压器注意力中的Jensen差距量化了这种违反，表现为超出组成部分证据的过度自信。 这一统一观点解释了为什么强大模型必须在真实性与创造力之间取得平衡。 这些结果为管理人工智能系统中的幻觉权衡提供了原则性基础。 与其消除幻觉，我们应针对特定应用优化这些不可避免的权衡。 我们以哲学意义结束，将不可能性与理性基本限制联系起来。