标题： 微调大型语言模型作为有效的符号回归器 显示英文标题

标题： Finetuning Large Language Model as an Effective Symbolic Regressor

摘要： 从观测数据推导控制方程，称为符号回归（SR），是科学发现的核心。大型语言模型（LLMs）通过利用其广泛的跨学科科学知识，在这项任务中表现出潜力。然而，现有的基于LLM的方法主要依赖于直接推理或提示工程，通常需要过多的推理迭代才能收敛到正确的公式，或者无法处理复杂的方程目标。这些在效果和泛化能力上的限制源于预训练LLMs在近似推理方面的熟练程度与SR任务对高精度需求之间的固有矛盾。为了弥合这一差距，我们提出对LLM进行微调以增强其SR能力。然而，缺乏针对SR定向微调的专用数据集仍然是一个关键障碍。因此，我们引入了SymbArena，专门设计用于优化LLM的SR能力。该基准包含148,102个多样化的方程，作为18.3亿个标记的语料库供LLM使用，实现了有效的训练和推理。此外，SymbArena提出了一种启发式度量标准，以精确量化形式一致性，超越了现有的以数值为导向的SR评估策略。通过这个基准，我们探索了主流的LLM微调技术用于SR任务，并建立了SymbolicChat，这是一个简单但有效的基于LLM的SR强基线。实验结果验证了SymbolicChat作为第一个在数值精度和符号形式准确性方面超过传统数值方法的LLM，其R2分数提高了两倍，形式一致性分数提高了8.37%。 显示英文摘要

摘要： Deriving governing equations from observational data, known as Symbolic Regression (SR), is a cornerstone of scientific discovery. Large Language Models (LLMs) have shown promise in this task by leveraging their vast cross-disciplinary scientific knowledge. However, existing LLM-based methods primarily rely on direct inference or prompt engineering, often requiring excessive inference iterations to converge on correct formulas or failing to treating complex equation targets. These limitations in effectiveness and generalization stem from an inherent tension between pre-trained LLMs' proficiency in approximate reasoning and the high-precision demands of SR tasks. To bridge this gap, we propose to fine-tune LLMs for enhanced SR capability. Yet, the absence of dedicated datasets for SR-oriented fine-tuning remains a critical barrier. We thus introduce SymbArena, specifically engineered to optimize LLMs for SR. This benchmark comprises 148,102 diverse equations formulated as corpora of 1.83 billion tokens for LLM utilization, enabling effective training and inference. Further, SymbArena proposes a heuristics metric to precisely quantify form-level consistency, going beyond existing SR numerical-oriented evaluation strategies. With this benchmark, we explore mainstream LLM fine-tuning techniques for SR tasks and establish SymbolicChat, a simple yet effective LLM-based SR strong baseline. Experimental results validate SymbolicChat as the first LLM to exceed traditional numerical methods in both numerical precision and symbolic form accuracy, outperforming the second-best LLM baseline with improvements of 2-fold gains in R2 score and 8.37% in form-level consistency score.