Title: Information Sifting Funnel: Privacy-preserving Collaborative Inference Against Model Inversion Attacks Show Chinese title

Title: 信息筛选漏斗：针对模型逆向攻击的隐私保护协作推理

Abstract: The complexity of neural networks and inference tasks, coupled with demands for computational efficiency and real-time feedback, poses significant challenges for resource-constrained edge devices. Collaborative inference mitigates this by assigning shallow feature extraction to edge devices and offloading features to the cloud for further inference, reducing computational load. However, transmitted features remain susceptible to model inversion attacks (MIAs), which can reconstruct original input data. Current defenses, such as perturbation and information bottleneck techniques, offer explainable protection but face limitations, including the lack of standardized criteria for assessing MIA difficulty, challenges in mutual information estimation, and trade-offs among usability, privacy, and deployability. To address these challenges, we introduce the first criterion to evaluate MIA difficulty in collaborative inference, supported by theoretical analysis of existing attacks and defenses, validated using experiments with the Mutual Information Neural Estimator (MINE). Based on these findings, we propose SiftFunnel, a privacy-preserving framework for collaborative inference. The edge model is trained with linear and non-linear correlation constraints to reduce redundant information in transmitted features, enhancing privacy protection. Label smoothing and a cloud-based upsampling module are added to balance usability and privacy. To improve deployability, the edge model incorporates a funnel-shaped structure and attention mechanisms, preserving both privacy and usability. Extensive experiments demonstrate that SiftFunnel outperforms state-of-the-art defenses against MIAs, achieving superior privacy protection with less than 3% accuracy loss and striking an optimal balance among usability, privacy, and practicality. Show Chinese abstract

Abstract: 神经网络和推理任务的复杂性，以及对计算效率和实时反馈的需求，给资源受限的边缘设备带来了重大挑战。协作推理通过将浅层特征提取分配给边缘设备，并将特征卸载到云端进行进一步推理，从而缓解了这一问题，减少了计算负载。然而，传输的特征仍容易受到模型反转攻击（MIAs）的影响，这可能导致原始输入数据的重建。当前的防御措施，如扰动和信息瓶颈技术，提供了可解释的保护，但存在局限性，包括缺乏评估MIA难度的标准准则、互信息估计的挑战以及可用性、隐私性和可部署性之间的权衡。为了解决这些挑战，我们引入了第一个用于评估协作推理中MIA难度的标准，该标准基于对现有攻击和防御的理论分析，并通过使用互信息神经估计器（MINE）的实验进行了验证。基于这些发现，我们提出了SiftFunnel，这是一种用于协作推理的隐私保护框架。边缘模型在进行线性和非线性相关性约束训练，以减少传输特征中的冗余信息，从而增强隐私保护。添加了标签平滑和基于云的上采样模块，以平衡可用性和隐私性。为了提高可部署性，边缘模型结合了漏斗形结构和注意力机制，同时保持隐私性和可用性。大量实验表明，SiftFunnel在对抗MIAs方面优于最先进的防御措施，在小于3%的准确率损失下实现了优越的隐私保护，并在可用性、隐私性和实用性之间达到了最佳平衡。