标题： 动态场景图预测 显示英文标题

标题： FDSG: Forecasting Dynamic Scene Graphs

摘要： 动态场景图生成通过建模实体关系及其时间演化，将场景图生成从图像扩展到视频。然而，现有方法要么从观测帧中生成场景图而没有显式建模时间动态，要么仅预测关系同时假设实体标签和位置是静态的。这些限制阻碍了实体和关系动态的有效外推，限制了视频场景理解。我们提出了Forecasting Dynamic Scene Graphs（FDSG），这是一种新颖的框架，可以预测未观测帧的未来实体标签、边界框和关系，同时为观测帧生成场景图。我们的场景图预测模块利用查询分解和神经随机微分方程来建模实体和关系动态。一个时间聚合模块进一步通过交叉注意力集成预测和观测信息来优化预测。为了评估FDSG，我们引入了Scene Graph Forecasting，这是一个新的全未来场景图预测任务。Action Genome上的实验表明，FDSG在动态场景图生成、场景图预期和场景图预测方面优于最先进的方法。代码将在发表后发布。 显示英文摘要

摘要： Dynamic scene graph generation extends scene graph generation from images to videos by modeling entity relationships and their temporal evolution. However, existing methods either generate scene graphs from observed frames without explicitly modeling temporal dynamics, or predict only relationships while assuming static entity labels and locations. These limitations hinder effective extrapolation of both entity and relationship dynamics, restricting video scene understanding. We propose Forecasting Dynamic Scene Graphs (FDSG), a novel framework that predicts future entity labels, bounding boxes, and relationships, for unobserved frames, while also generating scene graphs for observed frames. Our scene graph forecast module leverages query decomposition and neural stochastic differential equations to model entity and relationship dynamics. A temporal aggregation module further refines predictions by integrating forecasted and observed information via cross-attention. To benchmark FDSG, we introduce Scene Graph Forecasting, a new task for full future scene graph prediction. Experiments on Action Genome show that FDSG outperforms state-of-the-art methods on dynamic scene graph generation, scene graph anticipation, and scene graph forecasting. Codes will be released upon publication.