Title: Geo-Sign: Hyperbolic Contrastive Regularisation for Geometrically Aware Sign Language Translation Show Chinese title

Title: Geo-Sign：用于感知几何的手语翻译的双曲对比正则化

Abstract: Recent progress in Sign Language Translation (SLT) has focussed primarily on improving the representational capacity of large language models to incorporate Sign Language features. This work explores an alternative direction: enhancing the geometric properties of skeletal representations themselves. We propose Geo-Sign, a method that leverages the properties of hyperbolic geometry to model the hierarchical structure inherent in sign language kinematics. By projecting skeletal features derived from Spatio-Temporal Graph Convolutional Networks (ST-GCNs) into the Poincar\'e ball model, we aim to create more discriminative embeddings, particularly for fine-grained motions like finger articulations. We introduce a hyperbolic projection layer, a weighted Fr\'echet mean aggregation scheme, and a geometric contrastive loss operating directly in hyperbolic space. These components are integrated into an end-to-end translation framework as a regularisation function, to enhance the representations within the language model. This work demonstrates the potential of hyperbolic geometry to improve skeletal representations for Sign Language Translation, improving on SOTA RGB methods while preserving privacy and improving computational efficiency. Code available here: https://github.com/ed-fish/geo-sign. Show Chinese abstract

Abstract: 手语翻译（SLT）的最新进展主要集中在提高大型语言模型的表征能力以纳入手语特征。本文探索了一个替代方向：增强骨骼表示的几何属性本身。我们提出了 Geo-Sign，一种利用双曲几何特性对手语运动学中固有的层次结构进行建模的方法。通过将从时空图卷积网络（ST-GCNs）得出的骨骼特征投影到庞加莱球模型中，我们的目标是创建更具区分性的嵌入，特别是针对细微的动作，如手指动作。我们引入了双曲投影层、加权弗雷歇均值聚合方案以及直接在双曲空间中运行的几何对比损失。这些组件作为正则化函数集成到端到端翻译框架中，以增强语言模型中的表示。这项工作展示了双曲几何在改善手语翻译的骨骼表示方面的潜力，同时超越了当前最先进的基于RGB的方法，同时保持隐私并提高了计算效率。代码可在以下地址获取：https://github.com/ed-fish/geo-sign。