标题： 在Med-BERT之上使用下一访问令牌预测头推进胰腺癌预测 显示英文标题

标题： Advancing Pancreatic Cancer Prediction with a Next Visit Token Prediction Head on top of Med-BERT

摘要： 背景：最近，许多在大量数据上预训练的基础模型在使用电子健康记录（EHRs）进行疾病预测方面表现出有效性。然而，如何最好地利用这些模型，特别是在非常小的微调队列情况下，仍有一些未解答的问题。 方法：我们使用了Med-BERT，一个针对EHR的基础模型，并将疾病二分类预测任务重新构建为标记预测任务和下一个就诊掩码标记预测任务，以与Med-BERT的预训练任务格式对齐，从而在少量样本和完全监督设置中提高胰腺癌（PaCa）预测的准确性。 结果：将任务重新构建为标记预测任务，称为Med-BERT-Sum，在少量样本场景和较大数据样本中均表现出略微优越的性能。此外，将预测任务重新构建为下一个就诊掩码标记预测任务（Med-BERT-Mask）在少量样本场景中，数据量范围从10到500个样本时，显著优于传统的二分类（BC）预测任务（Med-BERT-BC），提升幅度为3%至7%。这些发现表明，将下游任务与Med-BERT的预训练目标对齐可以显著增强模型的预测能力，从而提高其预测罕见和常见疾病的效果。 结论：重新格式化疾病预测任务以与基础模型的预训练对齐可以提高预测准确性，从而实现早期检测和及时干预。这种方法可以提高胰腺癌及其他潜在癌症的治疗效果、生存率和总体患者预后。 显示英文摘要

摘要： Background: Recently, numerous foundation models pretrained on extensive data have demonstrated efficacy in disease prediction using Electronic Health Records (EHRs). However, there remains some unanswered questions on how to best utilize such models especially with very small fine-tuning cohorts. Methods: We utilized Med-BERT, an EHR-specific foundation model, and reformulated the disease binary prediction task into a token prediction task and a next visit mask token prediction task to align with Med-BERT's pretraining task format in order to improve the accuracy of pancreatic cancer (PaCa) prediction in both few-shot and fully supervised settings. Results: The reformulation of the task into a token prediction task, referred to as Med-BERT-Sum, demonstrates slightly superior performance in both few-shot scenarios and larger data samples. Furthermore, reformulating the prediction task as a Next Visit Mask Token Prediction task (Med-BERT-Mask) significantly outperforms the conventional Binary Classification (BC) prediction task (Med-BERT-BC) by 3% to 7% in few-shot scenarios with data sizes ranging from 10 to 500 samples. These findings highlight that aligning the downstream task with Med-BERT's pretraining objectives substantially enhances the model's predictive capabilities, thereby improving its effectiveness in predicting both rare and common diseases. Conclusion: Reformatting disease prediction tasks to align with the pretraining of foundation models enhances prediction accuracy, leading to earlier detection and timely intervention. This approach improves treatment effectiveness, survival rates, and overall patient outcomes for PaCa and potentially other cancers.