标题： 信号还是噪声？ 在情境变化和人类专家基准下评估大型语言模型在简历筛选中的表现 显示英文标题

标题： Signal or Noise? Evaluating Large Language Models in Resume Screening Across Contextual Variations and Human Expert Benchmarks

摘要： 本研究调查了大型语言模型（LLMs）在根据职位描述筛选简历时是否表现出一致的行为（信号）或随机变化（噪声），以及它们的性能与人类专家相比如何。 使用受控数据集，我们在相同和随机化的简历下，针对三种LLMs（Claude、GPT和Gemini）在不同情境（无公司、Firm1[跨国公司]、Firm2[初创公司]、简化情境）中进行了测试，并与三位人类招聘专家进行了基准比较。 方差分析显示，在八种LLM单独条件下有四种存在显著均值差异，并且LLM与人类评估之间存在持续显著差异（p < 0.01）。 配对t检验显示，GPT对公司情境适应性强（p < 0.001），Gemini部分适应（Firm1的p = 0.038），而Claude适应性最小（p > 0.1），而在所有情境下，所有LLMs与人类专家存在显著差异。 元认知分析突显了与人类评估方法明显不同的自适应加权模式。 研究结果表明，LLMs在详细提示下提供可解释的模式，但与人类判断存在显著差异，这为其在自动化招聘系统中的部署提供了依据。 显示英文摘要

摘要： This study investigates whether large language models (LLMs) exhibit consistent behavior (signal) or random variation (noise) when screening resumes against job descriptions, and how their performance compares to human experts. Using controlled datasets, we tested three LLMs (Claude, GPT, and Gemini) across contexts (No Company, Firm1 [MNC], Firm2 [Startup], Reduced Context) with identical and randomized resumes, benchmarked against three human recruitment experts. Analysis of variance revealed significant mean differences in four of eight LLM-only conditions and consistently significant differences between LLM and human evaluations (p < 0.01). Paired t-tests showed GPT adapts strongly to company context (p < 0.001), Gemini partially (p = 0.038 for Firm1), and Claude minimally (p > 0.1), while all LLMs differed significantly from human experts across contexts. Meta-cognition analysis highlighted adaptive weighting patterns that differ markedly from human evaluation approaches. Findings suggest LLMs offer interpretable patterns with detailed prompts but diverge substantially from human judgment, informing their deployment in automated hiring systems.