标题： 正确答案的解释性论点提取在住院医师医学考试中 显示英文标题

标题： Explanatory Argument Extraction of Correct Answers in Resident Medical Exams

摘要： 开发用于协助医学专家日常活动的所需技术是人工智能研究领域当前的热门话题。 因此，最近提出了一些大型语言模型（LLMs）和自动化基准，旨在通过自然语言作为人机交互的媒介，促进循证医学中的信息提取。 最典型的基准仅限于选择题或长答案，并且仅以英文提供。 为了弥补这些不足，本文我们提出了一个新的数据集，与之前的工作不同：(i) 不仅包括正确答案的解释性论点，还包括解释为什么错误答案不正确的论点；(ii) 解释是由医学医生撰写的，以回答西班牙住院医师考试中的问题。 此外，这个新的基准使我们能够设置一个新颖的抽取任务，即识别由医学医生撰写的正确答案的解释。 我们设置的另一个好处是我们可以利用抽取式问答范式，自动评估大型语言模型的性能，而无需依赖医学专家进行昂贵的人工评估。 对西班牙语语言模型的全面实验表明，有时多语言模型的表现优于单语模型，甚至超过了已适应医学领域的模型。 此外，单语模型的结果是混合的，据称较小且较差的模型表现出了竞争力。 无论如何，获得的结果表明，我们的新数据集和方法可以作为一种有效的方法，帮助医学从业者识别医学问题的相关循证解释。 显示英文摘要

摘要： Developing the required technology to assist medical experts in their everyday activities is currently a hot topic in the Artificial Intelligence research field. Thus, a number of large language models (LLMs) and automated benchmarks have recently been proposed with the aim of facilitating information extraction in Evidence-Based Medicine (EBM) using natural language as a tool for mediating in human-AI interaction. The most representative benchmarks are limited to either multiple-choice or long-form answers and are available only in English. In order to address these shortcomings, in this paper we present a new dataset which, unlike previous work: (i) includes not only explanatory arguments for the correct answer, but also arguments to reason why the incorrect answers are not correct; (ii) the explanations are written originally by medical doctors to answer questions from the Spanish Residency Medical Exams. Furthermore, this new benchmark allows us to setup a novel extractive task which consists of identifying the explanation of the correct answer written by medical doctors. An additional benefit of our setting is that we can leverage the extractive QA paradigm to automatically evaluate performance of LLMs without resorting to costly manual evaluation by medical experts. Comprehensive experimentation with language models for Spanish shows that sometimes multilingual models fare better than monolingual ones, even outperforming models which have been adapted to the medical domain. Furthermore, results across the monolingual models are mixed, with supposedly smaller and inferior models performing competitively. In any case, the obtained results show that our novel dataset and approach can be an effective technique to help medical practitioners in identifying relevant evidence-based explanations for medical questions.