Title: Decoupling Knowledge and Reasoning in Transformers: A Modular Architecture with Generalized Cross-Attention Show Chinese title

Title: 解耦Transformer中的知识与推理：一种模块化架构与广义交叉注意力

Abstract: Transformers have achieved remarkable success across diverse domains, but their monolithic architecture presents challenges in interpretability, adaptability, and scalability. This paper introduces a novel modular Transformer architecture that explicitly decouples knowledge and reasoning through a generalized cross-attention mechanism to a globally shared knowledge base with layer-specific transformations, specifically designed for effective knowledge retrieval. Critically, we provide a rigorous mathematical derivation demonstrating that the Feed-Forward Network (FFN) in a standard Transformer is a specialized case (a closure) of this generalized cross-attention, revealing its role in implicit knowledge retrieval and validating our design. This theoretical framework provides a new lens for understanding FFNs and lays the foundation for future research exploring enhanced interpretability, adaptability, and scalability, enabling richer interplay with external knowledge bases and other systems. Show Chinese abstract

Abstract: 变压器在多个领域取得了显著的成功，但其单体架构在可解释性、适应性和可扩展性方面带来了挑战。 本文介绍了一种新颖的模块化变压器架构，通过一种广义的交叉注意力机制，将知识和推理显式解耦，并与全局共享的知识库进行分层特定转换，专门设计用于有效的知识检索。 关键的是，我们提供了严格的数学推导，证明标准变压器中的前馈网络（FFN）是这种广义交叉注意力的一个特例（闭包），揭示了其在隐式知识检索中的作用，并验证了我们的设计。 这一理论框架为理解FFN提供了一个新的视角，并为未来研究增强可解释性、适应性和可扩展性奠定了基础，使与外部知识库和其他系统的更丰富的互动成为可能。