标题： 关于针对Android恶意软件分析的代码LLM基准测试 显示英文标题

标题： On Benchmarking Code LLMs for Android Malware Analysis

摘要： 大型语言模型（LLMs）在各种代码智能任务中展示了强大的能力。然而，它们在Android恶意软件分析中的有效性仍未被充分探索。由于恶意逻辑隐藏在大量函数中，并且经常缺乏有意义的函数名称，反编译的Android恶意软件代码为分析带来了独特的挑战。本文提出了CAMA，这是一个基准框架，旨在系统地评估代码LLMs在Android恶意软件分析中的有效性。CAMA指定了结构化的模型输出，以支持关键的恶意软件分析任务，包括恶意函数识别和恶意软件目的总结。在此基础上，它集成了三个领域特定的评估指标（一致性、保真度和语义相关性），从而实现严格的稳定性和有效性评估以及跨模型比较。我们构建了一个包含13个家族、近年来收集的118个Android恶意软件样本的基准数据集，该数据集涵盖了超过750万个不同的函数，并使用CAMA评估了四种流行的开源代码LLMs。我们的实验提供了有关代码LLMs如何解释反编译代码的见解，并量化了对函数重命名的敏感性，突显了它们在恶意软件分析中的潜力和当前局限性。 显示英文摘要

摘要： Large Language Models (LLMs) have demonstrated strong capabilities in various code intelligence tasks. However, their effectiveness for Android malware analysis remains underexplored. Decompiled Android malware code presents unique challenges for analysis, due to the malicious logic being buried within a large number of functions and the frequent lack of meaningful function names. This paper presents CAMA, a benchmarking framework designed to systematically evaluate the effectiveness of Code LLMs in Android malware analysis. CAMA specifies structured model outputs to support key malware analysis tasks, including malicious function identification and malware purpose summarization. Built on these, it integrates three domain-specific evaluation metrics (consistency, fidelity, and semantic relevance), enabling rigorous stability and effectiveness assessment and cross-model comparison. We construct a benchmark dataset of 118 Android malware samples from 13 families collected in recent years, encompassing over 7.5 million distinct functions, and use CAMA to evaluate four popular open-source Code LLMs. Our experiments provide insights into how Code LLMs interpret decompiled code and quantify the sensitivity to function renaming, highlighting both their potential and current limitations in malware analysis.