标题： 修复的艺术：用指令微调模型优化迭代程序修复 显示英文标题

标题： The Art of Repair: Optimizing Iterative Program Repair with Instruction-Tuned Models

摘要： 自动程序修复（APR）旨在减少识别和修复源代码中错误所需的手动工作量。在基于大型语言模型（LLM）的代理兴起之前，一种常见策略是增加生成补丁的数量，有时达到数千个，以在基准测试中获得更好的修复效果。最近，LLMs通过反馈引导实现了自我迭代的能力，在多轮迭代中改进补丁。然而，文献往往关注多次迭代而忽视了不同数量的输出。我们研究了一种平衡这两种方法的APR流水线，即生成多个输出和多轮迭代，同时对每个错误限制总补丁数不超过10个。我们将三种最先进的指令微调LLM——DeepSeekCoder-Instruct、Codellama-Instruct和Llama3.1-Instruct应用于APR任务。我们进一步使用三种大小（1K、30K、65K）和两种技术（全微调和LoRA）对每个模型进行微调，使我们能够在两个APR基准：HumanEval-Java和Defects4J上评估它们的修复能力。我们的结果显示，仅使用微调数据集的一小部分（<1%），我们可以将可能补丁的数量提高多达78%，这挑战了先前研究表明使用全微调时收益有限的研究。然而，我们发现超过某些阈值会导致收益递减，这可能是由于过拟合造成的。此外，我们表明基础模型从迭代式地创建补丁中受益匪浅，而不是一次性生成所有补丁。此外，在复杂的基准测试中，迭代策略的优势更加明显。即使是经过微调的模型，虽然从迭代中获益较少，但在复杂基准测试中仍能获得优势。这项研究强调了需要结合多输出生成和迭代精炼的平衡APR策略的重要性。 显示英文摘要

摘要： Automatic program repair (APR) aims to reduce the manual efforts required to identify and fix errors in source code. Before the rise of LLM-based agents, a common strategy was to increase the number of generated patches, sometimes to the thousands, to achieve better repair results on benchmarks. More recently, self-iterative capabilities enabled LLMs to refine patches over multiple rounds guided by feedback. However, literature often focuses on many iterations and disregards different numbers of outputs. We investigate an APR pipeline that balances these two approaches, the generation of multiple outputs and multiple rounds of iteration, while imposing a limit of 10 total patches per bug. We apply three SOTA instruction-tuned LLMs - DeepSeekCoder-Instruct, Codellama-Instruct, Llama3.1-Instruct - to the APR task. We further fine-tune each model on an APR dataset with three sizes (1K, 30K, 65K) and two techniques (Full Fine-Tuning and LoRA), allowing us to assess their repair capabilities on two APR benchmarks: HumanEval-Java and Defects4J. Our results show that by using only a fraction (<1%) of the fine-tuning dataset, we can achieve improvements of up to 78% in the number of plausible patches generated, challenging prior studies that reported limited gains using Full Fine-Tuning. However, we find that exceeding certain thresholds leads to diminishing outcomes, likely due to overfitting. Moreover, we show that base models greatly benefit from creating patches in an iterative fashion rather than generating them all at once. In addition, the benefit of iterative strategies becomes more pronounced in complex benchmarks. Even fine-tuned models, while benefiting less from iterations, still gain advantages, particularly on complex benchmarks. The research underscores the need for balanced APR strategies that combine multi-output generation and iterative refinement.