标题： 通过看到更少来学习更多：用于高效、可迁移和与人类对齐视觉的线条图预训练 显示英文标题

标题： Learning More by Seeing Less: Line Drawing Pretraining for Efficient, Transferable, and Human-Aligned Vision

摘要： 尽管计算机视觉领域取得了显著进展，现代识别系统仍然受限于其对丰富且冗余的视觉输入的依赖。 相比之下，人类可以轻松理解稀疏、最小的表示形式，如线条图——这表明结构而非外观是高效视觉理解的基础。 在本工作中，我们提出使用线条图作为以结构为先的预训练模态，以诱导更紧凑和可泛化的视觉表示。 我们证明，基于线条图预训练的模型在分类、检测和分割任务中表现出更强的形状偏差、更集中的注意力以及更高的数据效率。 值得注意的是，这些模型还表现出更低的内在维度，需要显著更少的主成分来捕捉表示方差——这与大脑中低维高效表示的类似观察结果相呼应。 除了性能提升外，线条图预训练生成的表示更具可压缩性，从而能够更好地蒸馏到轻量级学生模型中。 从线条预训练教师模型中蒸馏出的学生模型始终优于从颜色监督教师模型中训练得到的模型，突显了结构紧凑知识的优势。 最后，我们展示了通过我们提出的“学习绘画”方法，线条图预训练也可以扩展到无监督设置。 总之，我们的结果支持这样的观点：以结构为先的视觉学习促进了效率、泛化能力和与人类一致的归纳偏差——为构建更鲁棒和适应性强的视觉系统提供了一种简单而强大的策略。 显示英文摘要

摘要： Despite remarkable progress in computer vision, modern recognition systems remain limited by their dependence on rich, redundant visual inputs. In contrast, humans can effortlessly understand sparse, minimal representations like line drawings - suggesting that structure, rather than appearance, underlies efficient visual understanding. In this work, we propose using line drawings as a structure-first pretraining modality to induce more compact and generalizable visual representations. We show that models pretrained on line drawings develop stronger shape bias, more focused attention, and greater data efficiency across classification, detection, and segmentation tasks. Notably, these models also exhibit lower intrinsic dimensionality, requiring significantly fewer principal components to capture representational variance - echoing the similar observation in low dimensional efficient representation in the brain. Beyond performance improvements, line drawing pretraining produces more compressible representations, enabling better distillation into lightweight student models. Students distilled from line-pretrained teachers consistently outperform those trained from color-supervised teachers, highlighting the benefits of structurally compact knowledge. Finally, we demonstrate that the pretraining with line-drawing can also be extended to unsupervised setting via our proposed method "learning to draw". Together, our results support the view that structure-first visual learning fosters efficiency, generalization, and human-aligned inductive biases - offering a simple yet powerful strategy for building more robust and adaptable vision systems.