标题： DASViT：视觉变换器的可微架构搜索 显示英文标题

标题： DASViT: Differentiable Architecture Search for Vision Transformer

摘要： 设计有效的神经网络是深度学习的核心，神经网络架构搜索（NAS）已成为自动化这一过程的强大工具。在现有的NAS方法中，可微架构搜索（DARTS）因其效率和易用性而脱颖而出，激发了许多进步。自视觉变压器（ViT）兴起以来，研究人员已将NAS应用于探索ViT架构，通常专注于宏观层面的搜索空间，并依赖于进化算法等离散方法。虽然这些方法确保了可靠性，但在发现创新架构设计方面面临挑战，需要大量的计算资源，并且耗时。为解决这些限制，我们引入了视觉变压器的可微架构搜索（DASViT），它填补了ViT可微搜索的空白，并发现了新的设计。实验表明，DASViT提供的架构打破了传统的Transformer编码器设计，在多个数据集上优于ViT-B/16，并以更少的参数和FLOPs实现了更高的效率。 显示英文摘要

摘要： Designing effective neural networks is a cornerstone of deep learning, and Neural Architecture Search (NAS) has emerged as a powerful tool for automating this process. Among the existing NAS approaches, Differentiable Architecture Search (DARTS) has gained prominence for its efficiency and ease of use, inspiring numerous advancements. Since the rise of Vision Transformers (ViT), researchers have applied NAS to explore ViT architectures, often focusing on macro-level search spaces and relying on discrete methods like evolutionary algorithms. While these methods ensure reliability, they face challenges in discovering innovative architectural designs, demand extensive computational resources, and are time-intensive. To address these limitations, we introduce Differentiable Architecture Search for Vision Transformer (DASViT), which bridges the gap in differentiable search for ViTs and uncovers novel designs. Experiments show that DASViT delivers architectures that break traditional Transformer encoder designs, outperform ViT-B/16 on multiple datasets, and achieve superior efficiency with fewer parameters and FLOPs.