标题： SpaLLM-Guard：使用开源和商业大语言模型进行短信垃圾邮件检测 显示英文标题

标题： SpaLLM-Guard: Pairing SMS Spam Detection Using Open-source and Commercial LLMs

摘要： 日益增长的短信垃圾信息威胁，由不断演变的对抗技术和概念漂移驱动，需要更加稳健和自适应的检测方法。 在本文中，我们评估了大型语言模型（LLMs）在短信垃圾信息检测中的潜力，比较了它们在零样本、少量样本、微调和思维链提示方法上的性能。 使用一个全面的短信消息数据集，我们评估了GPT-4、DeepSeek、LLAMA-2和Mixtral等知名LLMs的垃圾信息检测能力。 我们的研究结果表明，尽管零样本学习提供了便利性，但它对于有效的垃圾信息检测不可靠。 少量样本学习，特别是通过精心选择的示例，可以提高检测效果，但不同模型之间表现出一定的差异性。 微调成为最有效的策略，Mixtral达到了98.6%的准确率，并且误报率和漏报率均低于2%，符合稳健垃圾信息检测的标准。 此外，我们探讨了这些模型对对抗攻击的鲁棒性，发现微调显著提高了对可见和不可见篡改的鲁棒性。 最后，我们研究了概念漂移的影响，并证明微调的LLMs，尤其是结合少量样本学习时，可以减轻其影响，在不断演变的垃圾信息数据集上仍能保持高性能。 本研究强调了微调和定制化学习策略的重要性，以有效部署LLMs进行现实世界的短信垃圾信息检测。 显示英文摘要

摘要： The increasing threat of SMS spam, driven by evolving adversarial techniques and concept drift, calls for more robust and adaptive detection methods. In this paper, we evaluate the potential of large language models (LLMs), both open-source and commercial, for SMS spam detection, comparing their performance across zero-shot, few-shot, fine-tuning, and chain-of-thought prompting approaches. Using a comprehensive dataset of SMS messages, we assess the spam detection capabilities of prominent LLMs such as GPT-4, DeepSeek, LLAMA-2, and Mixtral. Our findings reveal that while zero-shot learning provides convenience, it is unreliable for effective spam detection. Few-shot learning, particularly with carefully selected examples, improves detection but exhibits variability across models. Fine-tuning emerges as the most effective strategy, with Mixtral achieving 98.6% accuracy and a balanced false positive and false negative rate below 2%, meeting the criteria for robust spam detection. Furthermore, we explore the resilience of these models to adversarial attacks, finding that fine-tuning significantly enhances robustness against both perceptible and imperceptible manipulations. Lastly, we investigate the impact of concept drift and demonstrate that fine-tuned LLMs, especially when combined with few-shot learning, can mitigate its effects, maintaining high performance even on evolving spam datasets. This study highlights the importance of fine-tuning and tailored learning strategies to deploy LLMs effectively for real-world SMS spam detection