Title: Person Identification from Egocentric Human-Object Interactions using 3D Hand Pose Show Chinese title

Title: 从第一视角人-物体交互中的人体识别使用3D手部姿态

Abstract: Human-Object Interaction Recognition (HOIR) and user identification play a crucial role in advancing augmented reality (AR)-based personalized assistive technologies. These systems are increasingly being deployed in high-stakes, human-centric environments such as aircraft cockpits, aerospace maintenance, and surgical procedures. This research introduces I2S (Interact2Sign), a multi stage framework designed for unobtrusive user identification through human object interaction recognition, leveraging 3D hand pose analysis in egocentric videos. I2S utilizes handcrafted features extracted from 3D hand poses and per forms sequential feature augmentation: first identifying the object class, followed by HOI recognition, and ultimately, user identification. A comprehensive feature extraction and description process was carried out for 3D hand poses, organizing the extracted features into semantically meaningful categories: Spatial, Frequency, Kinematic, Orientation, and a novel descriptor introduced in this work, the Inter-Hand Spatial Envelope (IHSE). Extensive ablation studies were conducted to determine the most effective combination of features. The optimal configuration achieved an impressive average F1-score of 97.52% for user identification, evaluated on a bimanual object manipulation dataset derived from the ARCTIC and H2O datasets. I2S demonstrates state-of-the-art performance while maintaining a lightweight model size of under 4 MB and a fast inference time of 0.1 seconds. These characteristics make the proposed framework highly suitable for real-time, on-device authentication in security-critical, AR-based systems. Show Chinese abstract

Abstract: 人体-物体交互识别（HOIR）和用户识别在推动基于增强现实（AR）的个性化辅助技术方面起着关键作用。 这些系统正越来越多地部署在高风险、以人类为中心的环境中，如飞机驾驶舱、航天维护和外科手术中。 本研究介绍了I2S（Interact2Sign），这是一个多阶段框架，旨在通过人体-物体交互识别实现无侵入式的用户识别，利用第一人称视频中的3D手部姿态分析。 I2S利用从3D手部姿态中提取的手工特征，并执行顺序特征增强：首先识别物体类别，然后进行HOI识别，最后进行用户识别。 对3D手部姿态进行了全面的特征提取和描述过程，将提取的特征组织成语义上有意义的类别：空间、频率、运动学、方向，以及本研究中引入的一种新描述符，即跨手空间包络（IHSE）。 进行了广泛的消融实验，以确定最有效的特征组合。 最优配置在从ARCTIC和H2O数据集中派生的双侧物体操作数据集上实现了令人印象深刻的平均F1分数为97.52%。 I2S在保持模型大小低于4 MB且推理时间仅为0.1秒的情况下表现出最先进的性能。 这些特性使所提出的框架非常适合在安全关键的AR系统中进行实时、设备端的身份验证。