标题： 带约束的带电粒子跟踪的多智能体强化学习优化 显示英文标题

标题： Constrained Optimization of Charged Particle Tracking with Multi-Agent Reinforcement Learning

摘要： 强化学习在建模复杂的物理驱动系统方面表现出巨大的成功，通过与模拟或真实环境交互，提供端到端可训练的解决方案，最大化标量奖励信号。 在本工作中，我们提出了一种多智能体强化学习方法，具有分配约束，用于在像素化粒子探测器中重建粒子轨迹。 我们的方法通过联合最小化读出帧中重建轨迹上的总粒子散射量，协作优化一个参数化的策略，该策略作为多维分配问题的启发式方法。 为了满足约束条件，确保粒子击中点的唯一分配，我们提出了一种安全层，为每个联合动作求解一个线性分配问题。 此外，为了强制成本边缘，增加局部策略预测与优化器映射决策边界之间的距离，我们建议在黑盒梯度估计中使用额外组件，迫使策略找到总分配成本较低的解决方案。 我们在为质子成像开发的粒子探测器生成的模拟数据上，实证展示了我们方法的有效性，与多个单智能体和多智能体基线方法相比。 我们进一步证明了带有成本边缘的约束在优化和泛化中的有效性，这些约束通过更广泛的高重建性能区域以及减少的预测不稳定性来实现。 我们的结果为基于强化学习的跟踪技术的进一步发展奠定了基础，既提供了受约束策略的性能提升，又通过个体和团队奖励的选择增强了优化跟踪算法的灵活性。 显示英文摘要

摘要： Reinforcement learning demonstrated immense success in modelling complex physics-driven systems, providing end-to-end trainable solutions by interacting with a simulated or real environment, maximizing a scalar reward signal. In this work, we propose, building upon previous work, a multi-agent reinforcement learning approach with assignment constraints for reconstructing particle tracks in pixelated particle detectors. Our approach optimizes collaboratively a parametrized policy, functioning as a heuristic to a multidimensional assignment problem, by jointly minimizing the total amount of particle scattering over the reconstructed tracks in a readout frame. To satisfy constraints, guaranteeing a unique assignment of particle hits, we propose a safety layer solving a linear assignment problem for every joint action. Further, to enforce cost margins, increasing the distance of the local policies predictions to the decision boundaries of the optimizer mappings, we recommend the use of an additional component in the blackbox gradient estimation, forcing the policy to solutions with lower total assignment costs. We empirically show on simulated data, generated for a particle detector developed for proton imaging, the effectiveness of our approach, compared to multiple single- and multi-agent baselines. We further demonstrate the effectiveness of constraints with cost margins for both optimization and generalization, introduced by wider regions with high reconstruction performance as well as reduced predictive instabilities. Our results form the basis for further developments in RL-based tracking, offering both enhanced performance with constrained policies and greater flexibility in optimizing tracking algorithms through the option for individual and team rewards.