Title: Object Recognition and Force Estimation with the GelSight Baby Fin Ray Show Chinese title

Title: 基于GelSight Baby Fin Ray的目标识别与力估计

Abstract: Recent advances in soft robotic hands and tactile sensing have enabled both to perform an increasing number of complex tasks with the aid of machine learning. In particular, we presented the GelSight Baby Fin Ray in our previous work, which integrates a camera with a soft, compliant Fin Ray structure. Camera-based tactile sensing gives the GelSight Baby Fin Ray the ability to capture rich contact information like forces, object geometries, and textures. Moreover, our previous work showed that the GelSight Baby Fin Ray can dig through clutter, and classify in-shell nuts. To further examine the potential of the GelSight Baby Fin Ray, we leverage learning to distinguish nut-in-shell textures and to perform force and position estimation. We implement ablation studies with popular neural network structures, including ResNet50, GoogLeNet, and 3- and 5-layer convolutional neural network (CNN) structures. We conclude that machine learning is a promising technique to extract useful information from high-resolution tactile images and empower soft robotics to better understand and interact with the environments. Show Chinese abstract

Abstract: 近年来，软体机械手和触觉感知的进展使得它们在机器学习的帮助下能够执行越来越多的复杂任务。 特别是，在我们之前的工作中，我们介绍了GelSight Baby Fin Ray，它将相机与柔软、顺应的Fin Ray结构结合在一起。 基于相机的触觉感知使GelSight Baby Fin Ray能够捕捉丰富的接触信息，如力、物体几何形状和纹理。 此外，我们之前的工作表明，GelSight Baby Fin Ray可以穿过杂乱的环境，并对壳内坚果进行分类。 为了进一步检验GelSight Baby Fin Ray的潜力，我们利用学习来区分壳内坚果的纹理以及进行力和位置估计。 我们使用流行的神经网络结构进行了消融研究，包括ResNet50、GoogLeNet以及3层和5层卷积神经网络（CNN）结构。 我们得出结论，机器学习是一种有前景的技术，可以从高分辨率触觉图像中提取有用信息，并增强软体机器人更好地理解和与环境互动的能力。