标题： 文本到带隙：预训练语言模型作为半导体带隙预测的编码器 显示英文标题

标题： Text to Band Gap: Pre-trained Language Models as Encoders for Semiconductor Band Gap Prediction

摘要： 我们研究了基于Transformer的语言模型，如RoBERTa、T5和LLaMA，在直接从编码关键材料特征的文本表示中预测半导体材料带隙的应用，这些特征包括化学组成、晶体系统、空间群、每个晶胞中的原子数、价电子计数以及其他相关的电子和结构特性。量子化学模拟，如DFT，可以提供准确的预测，但计算成本高，限制了其在大规模材料筛选中的可行性。浅层机器学习模型提供了更快的替代方案，但通常需要大量的数据预处理，以将非数值材料特征转换为结构化的数值输入，这往往会导致关键描述信息的丢失。相比之下，我们的方法利用预训练的语言模型直接处理文本数据，消除了手动特征工程的需要。我们构建了两种格式的材料描述：结构化字符串，它在一致的模板中结合关键特征，以及使用ChatGPT API生成的自然语言叙述。对于每个模型，我们附加了一个自定义的回归头，并在一个精选的无机化合物数据集上进行特定任务的微调。我们的结果表明，经过微调的语言模型，特别是仅解码器的LLaMA-3架构，在预测精度和灵活性方面可以优于传统方法，与最佳浅层机器学习基线相比，其MAE为0.25 eV，R2为0.89，而该基线的MAE为0.32 eV，R2为0.84。值得注意的是，LLaMA-3在最小微调的情况下就能达到具有竞争力的准确性，这表明其架构能够为科学任务提供更具迁移性的表示。这项工作展示了微调语言模型在科学属性预测中的有效性，并为材料信息学提供了一个可扩展的、语言原生的框架。 显示英文摘要

摘要： We investigate the use of transformer-based language models, RoBERTa, T5, and LLaMA, for predicting the band gaps of semiconductor materials directly from textual representations that encode key material features such as chemical composition, crystal system, space group, number of atoms per unit cell, valence electron count, and other relevant electronic and structural properties. Quantum chemistry simulations such as DFT provide accurate predictions but are computationally intensive, limiting their feasibility for large-scale materials screening. Shallow ML models offer faster alternatives but typically require extensive data preprocessing to convert non-numerical material features into structured numerical inputs, often at the cost of losing critical descriptive information. In contrast, our approach leverages pretrained language models to process textual data directly, eliminating the need for manual feature engineering. We construct material descriptions in two formats: structured strings that combine key features in a consistent template, and natural language narratives generated using the ChatGPT API. For each model, we append a custom regression head and perform task-specific finetuning on a curated dataset of inorganic compounds. Our results show that finetuned language models, particularly the decoder-only LLaMA-3 architecture, can outperform conventional approaches in prediction accuracy and flexibility, achieving an MAE of 0.25 eV and R2 of 0.89, compared to the best shallow ML baseline, which achieved an MAE of 0.32 eV and R2 of 0.84. Notably, LLaMA-3 achieves competitive accuracy with minimal finetuning, suggesting its architecture enables more transferable representations for scientific tasks. This work demonstrates the effectiveness of finetuned language models for scientific property prediction and provides a scalable, language-native framework for materials informatics.