标题： FedRSClip：使用视觉语言模型的遥感场景分类联邦学习 显示英文标题

标题： FedRSClip: Federated Learning for Remote Sensing Scene Classification Using Vision-Language Models

摘要： 遥感数据通常分布在多个机构中，由于隐私问题和数据共享限制，在集中式训练框架中利用大规模数据集具有挑战性。联邦学习通过在不需数据集中的情况下，实现跨分布式数据源的协作模型训练，提供了一个有前景的解决方案。然而，当前的视觉-语言模型（VLMs），通常包含数十亿个参数，对于基于模型参数更新的传统联邦学习方法来说，会带来显著的通信挑战，因为这将产生大量的通信成本。在本文中，我们提出了FedRSCLIP，这是第一个基于VLM，特别是CLIP的遥感图像分类联邦学习框架。FedRSCLIP通过引入提示学习，仅优化一小部分可调参数，解决了联邦环境中的数据异构性和大规模模型传输问题。该框架引入了双提示机制，包括用于全局知识共享的共享提示和用于客户端特定适应的私有提示。为了保持共享提示和私有提示之间的语义连贯性，我们提出了双提示对齐约束，以在不同客户端分布之间平衡全局一致性和局部适应性。此外，为了增强跨模态表示学习，我们引入了跨模态特征对齐约束，以对齐文本和图像提示之间的多模态特征。为了验证我们提出的模型的有效性，我们构建了一个Fed-RSIC数据集，基于三个现有的遥感图像分类数据集，专门设计用于模拟各种联邦学习配置。实验结果表明FedRSCLIP在遥感图像分类中的有效性和优越性。 显示英文摘要

摘要： Remote sensing data is often distributed across multiple institutions, and due to privacy concerns and data-sharing restrictions, leveraging large-scale datasets in a centralized training framework is challenging. Federated learning offers a promising solution by enabling collaborative model training across distributed data sources without requiring data centralization. However, current Vision-Language Models (VLMs), which typically contain billions of parameters, pose significant communication challenges for traditional federated learning approaches based on model parameter updates, as they would incur substantial communication costs. In this paper, we propose FedRSCLIP, the first federated learning framework designed for remote sensing image classification based on a VLM, specifically CLIP. FedRSCLIP addresses the challenges of data heterogeneity and large-scale model transmission in federated environments by introducing Prompt Learning, which optimizes only a small set of tunable parameters. The framework introduces a dual-prompt mechanism, comprising Shared Prompts for global knowledge sharing and Private Prompts for client-specific adaptation. To maintain semantic coherence between shared and private prompts, we propose the Dual Prompt Alignment Constraint to balance global consistency and local adaptability across diverse client distributions. Additionally, to enhance cross-modal representation learning, we introduce the Cross-Modal Feature Alignment Constraint to align multimodal features between text and image prompts. To validate the effectiveness of our proposed model, we construct a Fed-RSIC dataset based on three existing remote sensing image classification datasets, specifically designed to simulate various federated learning configurations. Experimental results demonstrate the effectiveness and superiority of FedRSCLIP in remote sensing image classification.