标题： 基于线性复杂度的自回归图像生成：一种空间感知衰减视角 显示英文标题

标题： Autoregressive Image Generation with Linear Complexity: A Spatial-Aware Decay Perspective

摘要： 自回归（AR）模型因其能够有效捕捉视觉数据中的局部和全局结构而在图像生成中引起了广泛关注。 然而，流行的AR模型主要依赖于变压器架构，这些架构由于需要维护键值缓存而存在关于输入序列长度的二次计算复杂性和显著的内存开销。 尽管线性注意力机制在语言模型中成功减轻了这一负担，但我们的初步实验表明，由于无法捕捉视觉数据中的关键长距离依赖关系，它们会显著降低图像生成质量。 我们提出了线性注意力与空间感知衰减（LASAD），这是一种新的注意力机制，通过基于真实二维空间位置而非一维序列位置计算位置相关的衰减因子，从而显式地保留展平图像序列中的真实二维空间关系。 基于此机制，我们提出了LASADGen，一种自回归图像生成器，它以线性复杂度实现了对相关空间上下文的选择性注意力。 在ImageNet上的实验表明，LASADGen实现了最先进的图像生成性能和计算效率，弥合了线性注意力的效率与高质量生成所需的空间理解之间的差距。 显示英文摘要

摘要： Autoregressive (AR) models have garnered significant attention in image generation for their ability to effectively capture both local and global structures within visual data. However, prevalent AR models predominantly rely on the transformer architectures, which are beset by quadratic computational complexity concerning input sequence length and substantial memory overhead due to the necessity of maintaining key-value caches. Although linear attention mechanisms have successfully reduced this burden in language models, our initial experiments reveal that they significantly degrade image generation quality because of their inability to capture critical long-range dependencies in visual data. We propose Linear Attention with Spatial-Aware Decay (LASAD), a novel attention mechanism that explicitly preserves genuine 2D spatial relationships within the flattened image sequences by computing position-dependent decay factors based on true 2D spatial location rather than 1D sequence positions. Based on this mechanism, we present LASADGen, an autoregressive image generator that enables selective attention to relevant spatial contexts with linear complexity. Experiments on ImageNet show LASADGen achieves state-of-the-art image generation performance and computational efficiency, bridging the gap between linear attention's efficiency and spatial understanding needed for high-quality generation.