Title: FreeTacMan: Robot-free Visuo-Tactile Data Collection System for Contact-rich Manipulation Show Chinese title

Title: FreeTacMan：用于富接触操作的无机器人视觉-触觉数据收集系统

Abstract: Enabling robots with contact-rich manipulation remains a pivotal challenge in robot learning, which is substantially hindered by the data collection gap, including its inefficiency and limited sensor setup. While prior work has explored handheld paradigms, their rod-based mechanical structures remain rigid and unintuitive, providing limited tactile feedback and posing challenges for human operators. Motivated by the dexterity and force feedback of human motion, we propose FreeTacMan, a human-centric and robot-free data collection system for accurate and efficient robot manipulation. Concretely, we design a wearable data collection device with dual visuo-tactile grippers, which can be worn by human fingers for intuitive and natural control. A high-precision optical tracking system is introduced to capture end-effector poses, while synchronizing visual and tactile feedback simultaneously. FreeTacMan achieves multiple improvements in data collection performance compared to prior works, and enables effective policy learning for contact-rich manipulation tasks with the help of the visuo-tactile information. We will release the work to facilitate reproducibility and accelerate research in visuo-tactile manipulation. Show Chinese abstract

Abstract: 在机器人学习中，赋予机器人丰富的接触操作能力仍然是一个关键挑战，而数据收集差距（包括其低效性和有限的传感器设置）严重阻碍了这一进程。尽管先前的研究探索了手持范式，但基于杆的机械结构仍然僵化且不直观，提供的触觉反馈有限，并给人类操作员带来了挑战。 受人类运动的灵巧性和力反馈的启发，我们提出了FreeTacMan，这是一种以人为中心且无需机器人的数据收集系统，旨在实现准确高效的机器人操作。具体而言，我们设计了一种带有双视觉-触觉夹爪的可穿戴数据收集设备，可以佩戴在人类手指上，以实现直观和自然的控制。 引入了一个高精度光学跟踪系统来捕捉末端执行器的姿态，同时同步视觉和触觉反馈。与之前的工作相比，FreeTacMan在数据收集性能方面实现了多项改进，并通过视觉-触觉信息有效促进了丰富接触操作任务的策略学习。 我们将发布这项工作，以促进视觉-触觉操作的可重复性和研究的加速发展。