标题： 大型语言模型中涌现能力的非遍历性框架 显示英文标题

标题： A non-ergodic framework for understanding emergent capabilities in Large Language Models

摘要： 大规模语言模型具有在规模扩大时意外出现的能力，但我们需要一个理论框架来解释它们为何以及如何出现。 我们证明语言模型实际上是非遍历系统，并提供了一个基于斯图尔特·卡夫曼的相邻可能理论（TAP）的数学框架来解释能力的出现。 我们的资源受限的TAP方程展示了架构、训练和上下文约束如何通过语义空间中的相变来塑造模型能力。 我们通过三种不同的语言模型实验证明，能力是通过约束相互作用和路径依赖性探索引导的离散转换出现的。 这个框架为理解语言模型中的涌现提供了理论基础，并指导了能够引导能力涌现的架构的发展。 显示英文摘要

摘要： Large language models have emergent capabilities that come unexpectedly at scale, but we need a theoretical framework to explain why and how they emerge. We prove that language models are actually non-ergodic systems while providing a mathematical framework based on Stuart Kauffman's theory of the adjacent possible (TAP) to explain capability emergence. Our resource-constrained TAP equation demonstrates how architectural, training, and contextual constraints interact to shape model capabilities through phase transitions in semantic space. We prove through experiments with three different language models that capacities emerge through discrete transitions guided by constraint interactions and path-dependent exploration. This framework provides a theoretical basis for understanding emergence in language models and guides the development of architectures that can guide capability emergence.