标题： 基于视觉语言模型的跨模态语义通信系统 显示英文标题

标题： Visual Language Model based Cross-modal Semantic Communication Systems

摘要： 语义通信（SC）近年来已成为一种新的通信范式，通过创新的语义传输概念成功超越了香农物理容量极限。 然而，现有的图像语义通信（ISC）系统在动态环境中面临一些挑战，包括语义密度低、灾难性遗忘和不确定的信噪比（SNR）。 为了解决这些挑战，我们提出了一种基于视觉-语言模型的跨模态语义通信（VLM-CSC）系统。 VLM-CSC包含三个新组件：（1）跨模态知识库（CKB）用于从发射端语义稀疏的图像中提取高密度文本语义，并在接收端基于文本语义重建原始图像。高密度语义的传输有助于缓解带宽压力。 （2）基于记忆的编码器和解码器（MED）采用混合长/短期记忆机制，使语义编码器和解码器在语义特征分布发生漂移时能够克服灾难性遗忘。 （3）噪声注意力模块（NAM）利用注意力机制根据SNR自适应调整语义编码和信道编码，确保CSC系统的鲁棒性。 实验模拟验证了CSC系统的有效性、适应性和鲁棒性。 显示英文摘要

摘要： Semantic Communication (SC) has emerged as a novel communication paradigm in recent years, successfully transcending the Shannon physical capacity limits through innovative semantic transmission concepts. Nevertheless, extant Image Semantic Communication (ISC) systems face several challenges in dynamic environments, including low semantic density, catastrophic forgetting, and uncertain Signal-to-Noise Ratio (SNR). To address these challenges, we propose a novel Vision-Language Model-based Cross-modal Semantic Communication (VLM-CSC) system. The VLM-CSC comprises three novel components: (1) Cross-modal Knowledge Base (CKB) is used to extract high-density textual semantics from the semantically sparse image at the transmitter and reconstruct the original image based on textual semantics at the receiver. The transmission of high-density semantics contributes to alleviating bandwidth pressure. (2) Memory-assisted Encoder and Decoder (MED) employ a hybrid long/short-term memory mechanism, enabling the semantic encoder and decoder to overcome catastrophic forgetting in dynamic environments when there is a drift in the distribution of semantic features. (3) Noise Attention Module (NAM) employs attention mechanisms to adaptively adjust the semantic coding and the channel coding based on SNR, ensuring the robustness of the CSC system. The experimental simulations validate the effectiveness, adaptability, and robustness of the CSC system.