标题： 针对帕金森病深部脑刺激的样本高效强化学习控制器 显示英文标题

标题： Sample-Efficient Reinforcement Learning Controller for Deep Brain Stimulation in Parkinson's Disease

摘要： 深部脑刺激（DBS）是帕金森病（PD）的一种已确立的干预方法，但传统的开环系统缺乏适应性，由于持续刺激而能效低下，并且对个体神经动态的个性化有限。自适应DBS（aDBS）提供了一种闭环替代方案，利用诸如β频段振荡等生物标志物来动态调节刺激。虽然强化学习（RL）在个性化aDBS控制方面具有前景，但现有方法存在高样本复杂度、二元动作空间中不稳定探索以及在资源受限硬件上的部署能力有限的问题。我们提出了SEA-DBS，这是一种样本高效的Actor-Critic框架，解决了基于RL的自适应神经刺激的核心挑战。SEA-DBS集成了一个预测奖励模型，以减少对实时反馈的依赖，并在二元动作空间中采用基于Gumbel Softmax的探索方法，实现稳定、可微分的策略更新。这些组件共同提高了样本效率、探索鲁棒性和与资源受限神经调节硬件的兼容性。我们在帕金森氏基底节活动的生物现实模拟上评估了SEA-DBS，证明了其收敛速度更快、对病理性β频段功率的抑制更强，并且对训练后的FP16量化具有韧性。我们的结果表明，SEA-DBS为实时、资源受限的神经调节提供了一个实用且有效的基于RL的aDBS框架。 显示英文摘要

摘要： Deep brain stimulation (DBS) is an established intervention for Parkinson's disease (PD), but conventional open-loop systems lack adaptability, are energy-inefficient due to continuous stimulation, and provide limited personalization to individual neural dynamics. Adaptive DBS (aDBS) offers a closed-loop alternative, using biomarkers such as beta-band oscillations to dynamically modulate stimulation. While reinforcement learning (RL) holds promise for personalized aDBS control, existing methods suffer from high sample complexity, unstable exploration in binary action spaces, and limited deployability on resource-constrained hardware. We propose SEA-DBS, a sample-efficient actor-critic framework that addresses the core challenges of RL-based adaptive neurostimulation. SEA-DBS integrates a predictive reward model to reduce reliance on real-time feedback and employs Gumbel Softmax-based exploration for stable, differentiable policy updates in binary action spaces. Together, these components improve sample efficiency, exploration robustness, and compatibility with resource-constrained neuromodulatory hardware. We evaluate SEA-DBS on a biologically realistic simulation of Parkinsonian basal ganglia activity, demonstrating faster convergence, stronger suppression of pathological beta-band power, and resilience to post-training FP16 quantization. Our results show that SEA-DBS offers a practical and effective RL-based aDBS framework for real-time, resource-constrained neuromodulation.