标题： 相同模型，更好性能：洗牌对DNA语言模型基准测试的影响 显示英文标题

标题： Same model, better performance: the impact of shuffling on DNA Language Models benchmarking

摘要： 大型语言模型在基因组学中越来越受欢迎，因为它们有望解码复杂的生物序列。 因此，研究人员需要一个标准化的基准来评估DNA语言模型（DNA LMs）的能力。 然而，评估DNA LMs是一个复杂的任务，它结合了基因组学领域的特定挑战和机器学习方法，其中看似微小的实现细节可能会显著损害基准的有效性。 我们通过BEND（基准测试DNA语言模型）展示了这一点，其中与硬件相关的超参数——数据加载工作数量和缓冲区大小——会导致相同模型的性能变化高达4%。 问题源于数据洗牌不足与领域特定数据特征的相互作用。 对三种DNA语言模型（HyenaDNA、DNABERT-2、ResNet-LM）的实验表明，这些伪影会影响绝对性能和相对模型排名。 我们提出一个简单的解决方案：在存储前对数据进行预洗牌，可以消除硬件依赖性，同时保持效率。 这项工作突显了标准机器学习实践如何与领域特定的数据特征意外交互，这对专业领域的基准设计有更广泛的影响。 显示英文摘要

摘要： Large Language Models are increasingly popular in genomics due to their potential to decode complex biological sequences. Hence, researchers require a standardized benchmark to evaluate DNA Language Models (DNA LMs) capabilities. However, evaluating DNA LMs is a complex task that intersects genomic's domain-specific challenges and machine learning methodologies, where seemingly minor implementation details can significantly compromise benchmark validity. We demonstrate this through BEND (Benchmarking DNA Language Models), where hardware-dependent hyperparameters -- number of data loading workers and buffer sizes -- create spurious performance variations of up to 4% for identical models. The problem stems from inadequate data shuffling interacting with domain specific data characteristics. Experiments with three DNA language models (HyenaDNA, DNABERT-2, ResNet-LM) show these artifacts affect both absolute performance and relative model rankings. We propose a simple solution: pre-shuffling data before storage eliminates hardware dependencies while maintaining efficiency. This work highlights how standard ML practices can interact unexpectedly with domain-specific data characteristics, with broader implications for benchmark design in specialized domains.