Title: Evaluation of Multi-indicator And Multi-organ Medical Image Segmentation Models Show Chinese title

Title: 多指标和多器官医学图像分割模型的评估

Abstract: In recent years, "U-shaped" neural networks featuring encoder and decoder structures have gained popularity in the field of medical image segmentation. Various variants of this model have been developed. Nevertheless, the evaluation of these models has received less attention compared to model development. In response, we propose a comprehensive method for evaluating medical image segmentation models for multi-indicator and multi-organ (named MIMO). MIMO allows models to generate independent thresholds which are then combined with multi-indicator evaluation and confidence estimation to screen and measure each organ. As a result, MIMO offers detailed information on the segmentation of each organ in each sample, thereby aiding developers in analyzing and improving the model. Additionally, MIMO can produce concise usability and comprehensiveness scores for different models. Models with higher scores are deemed to be excellent models, which is convenient for clinical evaluation. Our research tests eight different medical image segmentation models on two abdominal multi-organ datasets and evaluates them from four perspectives: correctness, confidence estimation, Usable Region and MIMO. Furthermore, robustness experiments are tested. Experimental results demonstrate that MIMO offers novel insights into multi-indicator and multi-organ medical image evaluation and provides a specific and concise measure for the usability and comprehensiveness of the model. Code: https://github.com/SCUT-ML-GUO/MIMO Show Chinese abstract

Abstract: 近年来，“U型”神经网络因其编码器-解码器结构在医学图像分割领域广受欢迎。各种该模型的变体已经被开发出来。然而，与模型开发相比，这些模型的评估却较少受到关注。为此，我们提出了一种综合方法来评估多指标和多器官（命名为MIMO）的医学图像分割模型。MIMO允许模型生成独立的阈值，然后结合多指标评估和置信度估计来筛选和衡量每个器官。因此，MIMO为每个样本中每个器官的分割提供了详细信息，从而帮助开发者分析和改进模型。此外，MIMO还可以为不同的模型生成简洁的可用性和全面性评分，评分较高的模型被认为是优秀的模型，这便于临床评估。我们的研究在两个腹部多器官数据集上测试了八种不同的医学图像分割模型，并从正确性、置信度估计、可用区域和MIMO四个角度进行评估。此外，还进行了鲁棒性实验。实验结果表明，MIMO为多指标和多器官医学图像评估提供了新的见解，并为模型的可用性和全面性提供了一个具体且简洁的度量标准。代码：https://github.com/SCUT-ML-GUO/MIMO