标题： SignBot：学习人类与人形机器人手语交互 显示英文标题

标题： SignBot: Learning Human-to-Humanoid Sign Language Interaction

摘要： 手语是一种自然的视觉语言形式，通过动作和表情来传递意义，是听力障碍（DHH）人士重要的沟通方式。然而，能够熟练掌握手语的人数仍然有限，这凸显了技术进步以弥补沟通差距并促进与少数群体互动的需求。 基于近期在具身人形机器人领域的进展，我们提出了SignBot，这是一种针对人类与机器人手语交互的新框架。 SignBot整合了一个受小脑启发的动作控制组件以及一个以大脑为导向的理解与交互模块。 具体而言， SignBot包括：1) 动作重定向，将人类手语数据集转换为机器人兼容的动力学；2) 动作控制，利用基于学习的方法开发出一种稳健的人形机器人控制策略来跟踪手语手势；以及3) 生成性交互，包含手语的翻译者、响应者和生成器，从而实现机器人与人类之间自然有效的沟通。 模拟和真实世界实验结果表明，SignBot可以有效促进人类与机器人之间的交互，并且能够在多种机器人和数据集上执行手语动作。 SignBot标志着在具身人形机器人平台上自动手语交互方面的重要进步，为改善听力障碍人群的沟通无障碍提供了有前景的解决方案。 显示英文摘要

摘要： Sign language is a natural and visual form of language that uses movements and expressions to convey meaning, serving as a crucial means of communication for individuals who are deaf or hard-of-hearing (DHH). However, the number of people proficient in sign language remains limited, highlighting the need for technological advancements to bridge communication gaps and foster interactions with minorities. Based on recent advancements in embodied humanoid robots, we propose SignBot, a novel framework for human-robot sign language interaction. SignBot integrates a cerebellum-inspired motion control component and a cerebral-oriented module for comprehension and interaction. Specifically, SignBot consists of: 1) Motion Retargeting, which converts human sign language datasets into robot-compatible kinematics; 2) Motion Control, which leverages a learning-based paradigm to develop a robust humanoid control policy for tracking sign language gestures; and 3) Generative Interaction, which incorporates translator, responser, and generator of sign language, thereby enabling natural and effective communication between robots and humans. Simulation and real-world experimental results demonstrate that SignBot can effectively facilitate human-robot interaction and perform sign language motions with diverse robots and datasets. SignBot represents a significant advancement in automatic sign language interaction on embodied humanoid robot platforms, providing a promising solution to improve communication accessibility for the DHH community.