标题： 分类模型的样本量规划 显示英文标题

标题： Sample Size Planning for Classification Models

摘要： 在生物光谱学中，用于分类器训练和测试的适当标注且统计独立的样本（如患者、批次等）稀缺且昂贵。学习曲线展示了模型性能作为训练样本大小的函数，并可帮助确定训练良好分类器所需的样本量。然而，实际上仅仅构建一个良好的模型是不够的：其性能还必须得到验证。我们讨论了典型的少量样本情况下的学习曲线，每类样本数量为 5 至 25 个独立样本。尽管分类模型达到了可接受的性能，但由于测试样本量同样有限，随机测试不确定性可能会完全掩盖学习曲线。因此，我们确定了验证过程中需要的合理精度所对应的测试样本量，发现通常需要 75 至 100 个样本来测试一个良好但非完美的分类器。这样的数据集随后可以基于已实现的性能进行细化的样本量规划。我们还演示了如何计算必要的样本量以证明一种分类器优于另一种：这通常需要数百个统计独立的测试样本，甚至在理论上不可能实现。我们通过大约 2550 个单细胞拉曼光谱的数据集（五类：红细胞、白细胞以及三种肿瘤细胞系 BT-20、MCF-7 和 OCI-AML3）以及广泛的模拟实验进行了验证，这些模拟实验允许精确测定相关模型的实际性能。 显示英文摘要

摘要： In biospectroscopy, suitably annotated and statistically independent samples (e. g. patients, batches, etc.) for classifier training and testing are scarce and costly. Learning curves show the model performance as function of the training sample size and can help to determine the sample size needed to train good classifiers. However, building a good model is actually not enough: the performance must also be proven. We discuss learning curves for typical small sample size situations with 5 - 25 independent samples per class. Although the classification models achieve acceptable performance, the learning curve can be completely masked by the random testing uncertainty due to the equally limited test sample size. In consequence, we determine test sample sizes necessary to achieve reasonable precision in the validation and find that 75 - 100 samples will usually be needed to test a good but not perfect classifier. Such a data set will then allow refined sample size planning on the basis of the achieved performance. We also demonstrate how to calculate necessary sample sizes in order to show the superiority of one classifier over another: this often requires hundreds of statistically independent test samples or is even theoretically impossible. We demonstrate our findings with a data set of ca. 2550 Raman spectra of single cells (five classes: erythrocytes, leukocytes and three tumour cell lines BT-20, MCF-7 and OCI-AML3) as well as by an extensive simulation that allows precise determination of the actual performance of the models in question.