标题： 我们有多好？ 通过人机交互增强评估细胞AI基础模型在肾脏病理学中的表现 显示英文标题

标题： How Good Are We? Evaluating Cell AI Foundation Models in Kidney Pathology with Human-in-the-Loop Enrichment

摘要： 训练人工智能基础模型已成为一种有前景的大规模学习方法，用于解决现实世界的医疗保健挑战，包括数字病理学。 尽管这些模型中的许多都是通过使用广泛和多样的训练数据集来开发用于疾病诊断和组织定量等任务，但它们在某些可能最简单的任务（例如单个器官内的细胞核分割（如肾脏））上的部署准备情况仍不确定。 本文旨在通过全面评估最近的细胞基础模型在一个精心挑选的多中心、多疾病和多物种外部测试数据集上的性能，来回答这个关键问题，“我们做得怎么样？” 此外，我们还通过开发和评估人机协同的数据增强策略，解决了一个更具挑战性的问题，“我们如何改进？”这些策略旨在提高模型性能的同时，尽量减少对像素级人工标注的依赖。 为了解决第一个问题，我们整理了一个包含2,542张肾脏全切片图像（WSI）的多中心、多疾病和多物种数据集。 选择了三种最先进的（SOTA）细胞基础模型——Cellpose、StarDist和CellViT进行评估。 为了解决第二个问题，我们通过一个带有人工参与的框架，从不同的基础模型中提炼预测结果，探索数据增强算法，旨在以最小的人工努力进一步提升基础模型的性能。 我们的实验结果表明，所有三个基础模型在使用增强数据进行模型微调后都优于其基线模型。 有趣的是，基线模型中F1分数最高的模型在微调后并没有产生最佳的分割结果。 这项研究为针对真实世界数据应用的细胞视觉基础模型的开发和部署建立了一个基准。 显示英文摘要

摘要： Training AI foundation models has emerged as a promising large-scale learning approach for addressing real-world healthcare challenges, including digital pathology. While many of these models have been developed for tasks like disease diagnosis and tissue quantification using extensive and diverse training datasets, their readiness for deployment on some arguably simplest tasks, such as nuclei segmentation within a single organ (e.g., the kidney), remains uncertain. This paper seeks to answer this key question, "How good are we?", by thoroughly evaluating the performance of recent cell foundation models on a curated multi-center, multi-disease, and multi-species external testing dataset. Additionally, we tackle a more challenging question, "How can we improve?", by developing and assessing human-in-the-loop data enrichment strategies aimed at enhancing model performance while minimizing the reliance on pixel-level human annotation. To address the first question, we curated a multicenter, multidisease, and multispecies dataset consisting of 2,542 kidney whole slide images (WSIs). Three state-of-the-art (SOTA) cell foundation models-Cellpose, StarDist, and CellViT-were selected for evaluation. To tackle the second question, we explored data enrichment algorithms by distilling predictions from the different foundation models with a human-in-the-loop framework, aiming to further enhance foundation model performance with minimal human efforts. Our experimental results showed that all three foundation models improved over their baselines with model fine-tuning with enriched data. Interestingly, the baseline model with the highest F1 score does not yield the best segmentation outcomes after fine-tuning. This study establishes a benchmark for the development and deployment of cell vision foundation models tailored for real-world data applications.