标题： 粪便研究经不起嗅觉测试：一项关于人体肠道微生物组研究的系统综述表明机器学习被广泛误用 显示英文标题

标题： Stool Studies Don't Pass the Sniff Test: A Systematic Review of Human Gut Microbiome Research Suggests Widespread Misuse of Machine Learning

摘要： 在机器学习文化中，需要独立的测试集来进行模型的正确验证。模型验证中的失败，包括测试集遗漏和测试集泄露，使得无法确定训练好的模型是否适合其用途。在本文中，我们对人类肠道微生物组分类研究进行了系统综述和定量分析，目的是衡量测试集遗漏和测试集泄露对受试者工作特征曲线下面积（AUC）报告的频率和影响。在纳入分析的102篇文章中，我们发现只有12%的研究报告了真实的测试集AUC，这意味着88%的研究公布的AUC值无法仅凭表面价值进行信任。我们的研究结果对那些声称肠道微生物组在人类疾病中有高诊断或预后潜力的研究的普遍有效性提出了严重质疑。 显示英文摘要

摘要： In the machine learning culture, an independent test set is required for proper model verification. Failures in model verification, including test set omission and test set leakage, make it impossible to know whether or not a trained model is fit for purpose. In this article, we present a systematic review and quantitative analysis of human gut microbiome classification studies, conducted to measure the frequency and impact of test set omission and test set leakage on area under the receiver operating curve (AUC) reporting. Among 102 articles included for analysis, we find that only 12% of studies report a bona fide test set AUC, meaning that the published AUCs for 88% of studies cannot be trusted at face value. Our findings cast serious doubt on the general validity of research claiming that the gut microbiome has high diagnostic or prognostic potential in human disease.