Title: Actor Critic with Experience Replay-based automatic treatment planning for prostate cancer intensity modulated radiotherapy Show Chinese title

Title: 基于经验回放的自动治疗计划的演员评论家方法用于前列腺癌调强放疗

Abstract: Background: Real-time treatment planning in IMRT is challenging due to complex beam interactions. AI has improved automation, but existing models require large, high-quality datasets and lack universal applicability. Deep reinforcement learning (DRL) offers a promising alternative by mimicking human trial-and-error planning. Purpose: Develop a stochastic policy-based DRL agent for automatic treatment planning with efficient training, broad applicability, and robustness against adversarial attacks using Fast Gradient Sign Method (FGSM). Methods: Using the Actor-Critic with Experience Replay (ACER) architecture, the agent tunes treatment planning parameters (TPPs) in inverse planning. Training is based on prostate cancer IMRT cases, using dose-volume histograms (DVHs) as input. The model is trained on a single patient case, validated on two independent cases, and tested on 300+ plans across three datasets. Plan quality is assessed using ProKnow scores, and robustness is tested against adversarial attacks. Results: Despite training on a single case, the model generalizes well. Before ACER-based planning, the mean plan score was 6.20$\pm$1.84; after, 93.09% of cases achieved a perfect score of 9, with a mean of 8.93$\pm$0.27. The agent effectively prioritizes optimal TPP tuning and remains robust against adversarial attacks. Conclusions: The ACER-based DRL agent enables efficient, high-quality treatment planning in prostate cancer IMRT, demonstrating strong generalizability and robustness. Show Chinese abstract

Abstract: 背景：由于复杂的射束相互作用，调强放射治疗（IMRT）的实时治疗计划制定具有挑战性。人工智能提高了自动化水平，但现有模型需要大量高质量的数据集，并且缺乏普遍适用性。深度强化学习（DRL）通过模仿人类的试错规划提供了一种有前景的替代方案。目的：开发一种基于随机策略的DRL代理，使用快速梯度符号方法（FGSM）实现高效的自动治疗计划制定，具有广泛的适用性和对对抗攻击的鲁棒性。方法：使用带有经验回放的演员-评论家（ACER）架构，代理在逆向规划中调整治疗计划参数（TPPs）。训练基于前列腺癌IMRT病例，使用剂量体积直方图（DVHs）作为输入。该模型在一个患者病例上进行训练，在两个独立病例上进行验证，并在三个数据集中的300多个计划上进行测试。计划质量使用ProKnow分数进行评估，并测试其对对抗攻击的鲁棒性。结果：尽管仅在一个病例上进行训练，该模型表现出良好的泛化能力。在基于ACER的规划之前，平均计划得分为6.20$\pm$1.84；之后，93.09%的病例获得了完美的9分，平均分为8.93$\pm$0.27。代理有效地优先考虑最佳TPP调整，并对对抗攻击保持鲁棒性。结论：基于ACER的DRL代理实现了前列腺癌IMRT中高效、高质量的治疗计划制定，表现出强大的泛化能力和鲁棒性。