标题： 评估大型语言模型在空间计量经济学研究中的能力 显示英文标题

标题： Evaluating Large Language Model Capabilities in Assessing Spatial Econometrics Research

摘要： 本文研究了大型语言模型（LLMs）评估空间计量经济学实证发现的经济合理性与理论一致性能力。 我们从28篇发表的论文（2005-2024年）中创建了原创且刻意改动的“反事实”摘要，这些摘要由一组多样化的LLMs进行了评估。 LLMs在变量选择、系数可信度以及发表适宜性方面提供了定性评估和结构化的二元分类。 结果显示，尽管LLMs能够出色地评估变量选择的一致性（顶级模型如GPT-4o的整体F1得分为0.87），但在评估更深层次的方面如系数可信度和整体发表适宜性时，其表现存在显著差异。 进一步的结果表明，LLM的选择、论文的具体特征以及这两者之间的相互作用显著影响评估的准确性，特别是在细微判断方面。 这些发现突显了LLMs在辅助初步、较浅层检查方面的当前优势及其在进行全面深入经济推理方面的局限性，表明其在同行评审中的潜在辅助角色仍需强大的人类监督。 显示英文摘要

摘要： This paper investigates Large Language Models (LLMs) ability to assess the economic soundness and theoretical consistency of empirical findings in spatial econometrics. We created original and deliberately altered "counterfactual" summaries from 28 published papers (2005-2024), which were evaluated by a diverse set of LLMs. The LLMs provided qualitative assessments and structured binary classifications on variable choice, coefficient plausibility, and publication suitability. The results indicate that while LLMs can expertly assess the coherence of variable choices (with top models like GPT-4o achieving an overall F1 score of 0.87), their performance varies significantly when evaluating deeper aspects such as coefficient plausibility and overall publication suitability. The results further revealed that the choice of LLM, the specific characteristics of the paper and the interaction between these two factors significantly influence the accuracy of the assessment, particularly for nuanced judgments. These findings highlight LLMs' current strengths in assisting with initial, more surface-level checks and their limitations in performing comprehensive, deep economic reasoning, suggesting a potential assistive role in peer review that still necessitates robust human oversight.