标题： 香农不变量：一种可扩展的信息分解方法 显示英文标题

标题： Shannon invariants: A scalable approach to information decomposition

摘要： 分布式系统，例如生物神经网络和人工神经网络，通过多个子系统之间的复杂交互来处理信息，从而在不同尺度上产生具有独特属性的高阶模式。 由于难以定义适当的多变量度量并确保其可扩展性到大规模系统，研究这些系统如何处理信息仍然具有挑战性。 为了解决这些挑战，我们引入了一种基于所谓的“香农不变量”的新框架——这些量以仅依赖于熵的定义方式捕捉高阶信息处理的基本属性，并且可以高效地计算大规模系统中的值。 我们的理论结果展示了如何利用香农不变量解决关于广泛使用的多变量信息论测度解释方面的长期模糊性。 此外，我们的实证结果揭示了不同深度学习架构在各层中的独特信息处理特征，这为我们提供了有关这些系统如何处理信息以及这一过程在训练过程中如何演变的新见解。 总体而言，我们的框架解决了分析高阶现象的根本局限性，并为理论发展和经验分析提供了广泛的机会。 显示英文摘要

摘要： Distributed systems, such as biological and artificial neural networks, process information via complex interactions engaging multiple subsystems, resulting in high-order patterns with distinct properties across scales. Investigating how these systems process information remains challenging due to difficulties in defining appropriate multivariate metrics and ensuring their scalability to large systems. To address these challenges, we introduce a novel framework based on what we call "Shannon invariants" -- quantities that capture essential properties of high-order information processing in a way that depends only on the definition of entropy and can be efficiently calculated for large systems. Our theoretical results demonstrate how Shannon invariants can be used to resolve long-standing ambiguities regarding the interpretation of widely used multivariate information-theoretic measures. Moreover, our practical results reveal distinctive information-processing signatures of various deep learning architectures across layers, which lead to new insights into how these systems process information and how this evolves during training. Overall, our framework resolves fundamental limitations in analyzing high-order phenomena and offers broad opportunities for theoretical developments and empirical analyses.