Title: Opportunities of Hybrid Model-based Reinforcement Learning for Cell Therapy Manufacturing Process Control Show Chinese title

Title: 混合模型基础强化学习在细胞治疗制造过程控制中的机会

Abstract: Driven by the key challenges of cell therapy manufacturing, including high complexity, high uncertainty, and very limited process observations, we propose a hybrid model-based reinforcement learning (RL) to efficiently guide process control. We first create a probabilistic knowledge graph (KG) hybrid model characterizing the risk- and science-based understanding of biomanufacturing process mechanisms and quantifying inherent stochasticity, e.g., batch-to-batch variation. It can capture the key features, including nonlinear reactions, nonstationary dynamics, and partially observed state. This hybrid model can leverage existing mechanistic models and facilitate learning from heterogeneous process data. A computational sampling approach is used to generate posterior samples quantifying model uncertainty. Then, we introduce hybrid model-based Bayesian RL, accounting for both inherent stochasticity and model uncertainty, to guide optimal, robust, and interpretable dynamic decision making. Cell therapy manufacturing examples are used to empirically demonstrate that the proposed framework can outperform the classical deterministic mechanistic model assisted process optimization. Show Chinese abstract

Abstract: 由细胞治疗制造中的关键挑战驱动，包括高复杂性、高不确定性以及非常有限的过程观察，我们提出了一种基于混合模型的强化学习（RL），以高效地指导过程控制。 我们首先创建了一个概率知识图（KG）混合模型，该模型描述了生物制造过程机制的风险和科学基础的理解，并量化了固有的随机性，例如批次间的变异。 它可以捕捉关键特征，包括非线性反应、非平稳动态和部分可观测的状态。 这种混合模型可以利用现有的机理模型，并促进从异构过程数据中进行学习。 采用计算采样方法生成量化模型不确定性的后验样本。 然后，我们引入基于混合模型的贝叶斯强化学习，同时考虑固有的随机性和模型不确定性，以指导最优、稳健和可解释的动态决策。 使用细胞治疗制造示例来实证证明，所提出的框架可以优于经典的确定性机理模型辅助过程优化。