Title: On Robustness of Vision-Language-Action Model against Multi-Modal Perturbations Show Chinese title

Title: 关于视觉-语言-动作模型对多模态扰动的鲁棒性

Abstract: In Vision-Language-Action (VLA) models, robustness to real-world perturbations is critical for deployment. Existing methods target simple visual disturbances, overlooking the broader multi-modal perturbations that arise in actions, instructions, environments, and observations. Here, we first evaluate the robustness of mainstream VLAs under 17 perturbations across four modalities. We find (1) actions as the most fragile modality, (2) Existing visual-robust VLA do not gain robustness in other modality, and (3) pi0 demonstrates superior robustness with a diffusion-based action head. To build multi-modal robust VLAs, we propose RobustVLA against perturbations in VLA inputs and outputs. For output robustness, we perform offline robust optimization against worst-case action noise that maximizes mismatch in flow matching objective. This can be seen as adversarial training, label smoothing, and outlier penalization. For input robustness, we enforce consistent actions across input variations that preserve task semantics. To account for multiple perturbations, we formulate robustness as a multi-armed bandit problem and apply an upper confidence bound algorithm to automatically identify the most harmful noise. Experiments on LIBERO demonstrate our RobustVLA delivers absolute gains over baselines of 12.6% on the pi0 backbone and 10.4% on the OpenVLA backbone across all 17 perturbations, achieving 50.6x faster inference than existing visual-robust VLAs, and a 10.4% gain under mixed perturbations. Our RobustVLA is particularly effective on real-world FR5 robot with limited demonstrations, showing absolute gains by 65.6% under perturbations of four modalities. Show Chinese abstract

Abstract: 在视觉-语言-动作（VLA）模型中，对现实世界扰动的鲁棒性对于部署至关重要。 现有方法针对简单的视觉干扰，忽视了在动作、指令、环境和观察中出现的更广泛的多模态扰动。 在这里，我们首先评估主流VLA在四个模态下的17种扰动下的鲁棒性。 我们发现（1）动作是最脆弱的模态，（2）现有的视觉鲁棒VLA在其他模态中并未获得鲁棒性，以及（3）pi0在基于扩散的动作头方面表现出优越的鲁棒性。 为了构建多模态鲁棒的VLA，我们提出了RobustVLA，以应对VLA输入和输出中的扰动。 对于输出鲁棒性，我们进行离线鲁棒优化，以对抗最大化流匹配目标不匹配的最坏情况动作噪声。 这可以看作是对抗训练、标签平滑和异常值惩罚。 对于输入鲁棒性，我们在保留任务语义的输入变化中强制一致的动作。 为了考虑多种扰动，我们将鲁棒性形式化为一个多臂老虎机问题，并应用上置信界算法自动识别最具破坏性的噪声。 在LIBERO上的实验表明，我们的RobustVLA在所有17种扰动下，相对于基线在pi0主干上实现了12.6%的绝对增益，在OpenVLA主干上实现了10.4%的绝对增益，推理速度比现有的视觉鲁棒VLA快50.6倍，并在混合扰动下实现了10.4%的增益。 我们的RobustVLA在具有有限演示的真实世界FR5机器人上特别有效，在四种模态的扰动下实现了65.6%的绝对增益。