标题： MCEval：一种用于LLMs公平多语言文化评估的动态框架 显示英文标题

标题： MCEval: A Dynamic Framework for Fair Multilingual Cultural Evaluation of LLMs

摘要： 大型语言模型表现出文化偏见和有限的跨文化理解能力，特别是在服务多元化的全球用户群体时。 我们提出了MCEval，一种新颖的多语言评估框架，该框架采用动态文化问题构建，并通过反事实改写和混杂因素改写实现因果分析。 我们的全面评估涵盖了13种文化和13种语言，系统地评估了不同语言场景下的文化意识和文化偏见。 该框架提供了39,897个文化意识实例和17,940个文化偏见实例。 实验结果揭示了不同语言场景下的性能差异，表明最佳文化表现不仅与训练数据分布有关，还与语言与文化的匹配度有关。 评估结果还暴露了公平性问题，其中在英语场景中表现成功的方案在其他场景中会造成显著劣势。 MCEval是第一个全面的多语言文化评估框架，为深入理解LLMs的文化理解提供了更深层次的见解。 显示英文摘要

摘要： Large language models exhibit cultural biases and limited cross-cultural understanding capabilities, particularly when serving diverse global user populations. We propose MCEval, a novel multilingual evaluation framework that employs dynamic cultural question construction and enables causal analysis through Counterfactual Rephrasing and Confounder Rephrasing. Our comprehensive evaluation spans 13 cultures and 13 languages, systematically assessing both cultural awareness and cultural bias across different linguistic scenarios. The framework provides 39,897 cultural awareness instances and 17,940 cultural bias instances. Experimental results reveal performance disparities across different linguistic scenarios, demonstrating that optimal cultural performance is not only linked to training data distribution, but also is related to language-culture alignment. The evaluation results also expose the fairness issue, where approaches appearing successful in the English scenario create substantial disadvantages. MCEval represents the first comprehensive multilingual cultural evaluation framework that provides deeper insights into LLMs' cultural understanding.