Skip to main content
CenXiv.org
This website is in trial operation, support us!
We gratefully acknowledge support from all contributors.
Contribute
Donate
cenxiv logo > cs > arXiv:2504.04956v1

Help | Advanced Search

Computer Science > Graphics

arXiv:2504.04956v1 (cs)
[Submitted on 7 Apr 2025 (this version) , latest version 8 Apr 2025 (v2) ]

Title: REWIND: Real-Time Egocentric Whole-Body Motion Diffusion with Exemplar-Based Identity Conditioning

Title: REWIND:基于示例的身份条件实时第一人称全身运动扩散

Authors:Jihyun Lee, Weipeng Xu, Alexander Richard, Shih-En Wei, Shunsuke Saito, Shaojie Bai, Te-Li Wang, Minhyuk Sung, Tae-Kyun (T-K)Kim, Jason Saragih
Abstract: We present REWIND (Real-Time Egocentric Whole-Body Motion Diffusion), a one-step diffusion model for real-time, high-fidelity human motion estimation from egocentric image inputs. While an existing method for egocentric whole-body (i.e., body and hands) motion estimation is non-real-time and acausal due to diffusion-based iterative motion refinement to capture correlations between body and hand poses, REWIND operates in a fully causal and real-time manner. To enable real-time inference, we introduce (1) cascaded body-hand denoising diffusion, which effectively models the correlation between egocentric body and hand motions in a fast, feed-forward manner, and (2) diffusion distillation, which enables high-quality motion estimation with a single denoising step. Our denoising diffusion model is based on a modified Transformer architecture, designed to causally model output motions while enhancing generalizability to unseen motion lengths. Additionally, REWIND optionally supports identity-conditioned motion estimation when identity prior is available. To this end, we propose a novel identity conditioning method based on a small set of pose exemplars of the target identity, which further enhances motion estimation quality. Through extensive experiments, we demonstrate that REWIND significantly outperforms the existing baselines both with and without exemplar-based identity conditioning.
Abstract: 我们提出REWIND(实时第一视角全身运动扩散模型),这是一种用于从第一视角图像输入中实时、高保真估计人类运动的一步扩散模型。 尽管现有的第一视角全身(即身体和手部)运动估计方法由于基于扩散的迭代运动优化以捕捉身体和手部姿态之间的相关性,因此是非实时且非因果的,而REWIND以完全因果和实时的方式运行。 为了实现实时推理,我们引入了(1)级联的身体-手部去噪扩散,以快速、前馈的方式有效建模第一视角身体和手部运动之间的相关性,以及(2)扩散蒸馏,这使得在单次去噪步骤中就能实现高质量的运动估计。 我们的去噪扩散模型基于一种修改后的Transformer架构,该架构旨在因果地建模输出运动,同时增强对未见过的运动长度的泛化能力。 此外,当存在身份先验时,REWIND可选地支持身份条件运动估计。 为此,我们提出了一种基于目标身份的一小组姿态示例的新身份条件方法,这进一步提高了运动估计质量。 通过大量实验,我们证明REWIND在有和没有基于示例的身份条件的情况下均显著优于现有基线方法。
Comments: Accepted to CVPR 2025, project page: https://jyunlee.github.io/projects/rewind/
Subjects: Graphics (cs.GR) ; Computer Vision and Pattern Recognition (cs.CV)
Cite as: arXiv:2504.04956 [cs.GR]
  (or arXiv:2504.04956v1 [cs.GR] for this version)
  https://doi.org/10.48550/arXiv.2504.04956
arXiv-issued DOI via DataCite

Submission history

From: Jihyun Lee [view email]
[v1] Mon, 7 Apr 2025 11:44:11 UTC (1,243 KB)
[v2] Tue, 8 Apr 2025 03:01:03 UTC (1,243 KB)
Full-text links:

Access Paper:

    View a PDF of the paper titled
  • View Chinese PDF
  • View PDF
  • HTML (experimental)
  • TeX Source
license icon view license
Current browse context:
cs.GR
< prev   |   next >
new | recent | 2025-04
Change to browse by:
cs
cs.CV

References & Citations

  • NASA ADS
  • Google Scholar
  • Semantic Scholar
a export BibTeX citation Loading...

BibTeX formatted citation

×
Data provided by:

Bookmark

BibSonomy logo Reddit logo

Bibliographic and Citation Tools

Bibliographic Explorer (What is the Explorer?)
Connected Papers (What is Connected Papers?)
Litmaps (What is Litmaps?)
scite Smart Citations (What are Smart Citations?)

Code, Data and Media Associated with this Article

alphaXiv (What is alphaXiv?)
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub (What is DagsHub?)
Gotit.pub (What is GotitPub?)
Hugging Face (What is Huggingface?)
Papers with Code (What is Papers with Code?)
ScienceCast (What is ScienceCast?)

Demos

Replicate (What is Replicate?)
Hugging Face Spaces (What is Spaces?)
TXYZ.AI (What is TXYZ.AI?)

Recommenders and Search Tools

Influence Flower (What are Influence Flowers?)
CORE Recommender (What is CORE?)
IArxiv Recommender (What is IArxiv?)
  • Author
  • Venue
  • Institution
  • Topic

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.

Which authors of this paper are endorsers? | Disable MathJax (What is MathJax?)
  • About
  • Help
  • contact arXivClick here to contact arXiv Contact
  • subscribe to arXiv mailingsClick here to subscribe Subscribe
  • Copyright
  • Privacy Policy
  • Web Accessibility Assistance
  • arXiv Operational Status
    Get status notifications via email or slack

京ICP备2025123034号