标题： 病理学中评估人工智能解决方案的测试数据集建议 显示英文标题

标题： Recommendations on test datasets for evaluating AI solutions in pathology

摘要： 自动从数字组织学图像中提取信息的人工智能（AI）解决方案在改善病理诊断方面显示出巨大潜力。在常规使用之前，评估其预测性能并获得监管批准非常重要。这种评估需要适当的测试数据集。然而，编制这样的数据集具有挑战性，并且缺乏具体的建议。一个由各种利益相关者组成的委员会，包括商业AI开发者、病理学家和研究人员，讨论了测试数据集的关键方面，并对病理学中的测试数据集进行了广泛的文献回顾。在这里，我们总结了结果并得出了关于测试数据集收集的一般建议。我们探讨了几个问题：需要哪些图像以及需要多少张？如何处理低患病率的子集？如何检测潜在的偏倚？数据集应该如何报告？不同国家的监管要求是什么？这些建议旨在帮助AI开发者展示其产品的实用性，并帮助监管机构和最终用户验证所报告的性能指标。为了制定足够有代表性的测试数据集的标准，从而在未来减少用户干预并更好地支持诊断工作流程，还需要进一步的研究。 显示英文摘要

摘要： Artificial intelligence (AI) solutions that automatically extract information from digital histology images have shown great promise for improving pathological diagnosis. Prior to routine use, it is important to evaluate their predictive performance and obtain regulatory approval. This assessment requires appropriate test datasets. However, compiling such datasets is challenging and specific recommendations are missing. A committee of various stakeholders, including commercial AI developers, pathologists, and researchers, discussed key aspects and conducted extensive literature reviews on test datasets in pathology. Here, we summarize the results and derive general recommendations for the collection of test datasets. We address several questions: Which and how many images are needed? How to deal with low-prevalence subsets? How can potential bias be detected? How should datasets be reported? What are the regulatory requirements in different countries? The recommendations are intended to help AI developers demonstrate the utility of their products and to help regulatory agencies and end users verify reported performance measures. Further research is needed to formulate criteria for sufficiently representative test datasets so that AI solutions can operate with less user intervention and better support diagnostic workflows in the future.