标题： 扩大ESM2架构以分析长蛋白质序列：长序列和量化方法 显示英文标题

标题： Scaling Up ESM2 Architectures for Long Protein Sequences Analysis: Long and Quantized Approaches

摘要： 利用Transformer架构的各种方法在自然语言处理（NLP）中取得了最先进的结果。 基于这一成功，已经提出了许多用于其他类型数据的架构，例如生物学领域，特别是蛋白质序列。 其中值得注意的是ESM2架构，它们在数十亿个蛋白质上进行了预训练，构成了该领域各种最先进的方法的基础。 然而，ESM2架构在输入大小方面存在限制，将其限制为1,022个氨基酸，这需要使用预处理技术来处理超过此限制的序列。 在本文中，我们提出了ESM2架构的长版本和量化版本，将输入大小限制加倍至 2,048个氨基酸。 显示英文摘要

摘要： Various approaches utilizing Transformer architectures have achieved state-of-the-art results in Natural Language Processing (NLP). Based on this success, numerous architectures have been proposed for other types of data, such as in biology, particularly for protein sequences. Notably among these are the ESM2 architectures, pre-trained on billions of proteins, which form the basis of various state-of-the-art approaches in the field. However, the ESM2 architectures have a limitation regarding input size, restricting it to 1,022 amino acids, which necessitates the use of preprocessing techniques to handle sequences longer than this limit. In this paper, we present the long and quantized versions of the ESM2 architectures, doubling the input size limit to 2,048 amino acids.