标题： SPENCER：用于高效代码检索的自适应模型蒸馏 显示英文标题

标题： SPENCER: Self-Adaptive Model Distillation for Efficient Code Retrieval

摘要： 代码检索旨在根据用户的自然语言查询为用户提供所需的代码片段。 随着深度学习技术的发展，采用预训练模型进行此任务已成为主流。 考虑到检索效率，大多数先前的方法为此任务采用双编码器，分别将描述和代码片段编码为表示向量。 然而，双编码器的模型结构往往会限制模型的性能，因为在训练过程中，模型底层缺乏代码片段和描述之间的交互。 为了在保持效率的同时提高模型的有效性，我们提出了一种框架，该框架采用自适应模型蒸馏用于高效代码检索，命名为SPENCER。 SPENCER首先采用双编码器缩小搜索空间，然后采用交叉编码器提高准确性。 为了提高SPENCER的效率，我们提出了一种新颖的模型蒸馏技术，可以在保持整体性能的同时大幅减少双编码器的推理时间。 我们还为模型蒸馏提出了一个教学助手选择策略，该策略可以在模型蒸馏过程中自适应地为不同的预训练模型选择合适的教学助手模型，以确保模型性能。 大量实验表明，与仅基于双编码器的模型相比，双编码器和交叉编码器的结合提高了整体性能。 此外，我们的模型蒸馏技术在将双编码器的推理时间减少70%的同时，保留了超过98%的整体性能。 显示英文摘要

摘要： Code retrieval aims to provide users with desired code snippets based on users' natural language queries. With the development of deep learning technologies, adopting pre-trained models for this task has become mainstream. Considering the retrieval efficiency, most of the previous approaches adopt a dual-encoder for this task, which encodes the description and code snippet into representation vectors, respectively. However, the model structure of the dual-encoder tends to limit the model's performance, since it lacks the interaction between the code snippet and description at the bottom layer of the model during training. To improve the model's effectiveness while preserving its efficiency, we propose a framework, which adopts Self-AdaPtive Model Distillation for Efficient CodE Retrieval, named SPENCER. SPENCER first adopts the dual-encoder to narrow the search space and then adopts the cross-encoder to improve accuracy. To improve the efficiency of SPENCER, we propose a novel model distillation technique, which can greatly reduce the inference time of the dual-encoder while maintaining the overall performance. We also propose a teaching assistant selection strategy for our model distillation, which can adaptively select the suitable teaching assistant models for different pre-trained models during the model distillation to ensure the model performance. Extensive experiments demonstrate that the combination of dual-encoder and cross-encoder improves overall performance compared to solely dual-encoder-based models for code retrieval. Besides, our model distillation technique retains over 98% of the overall performance while reducing the inference time of the dual-encoder by 70%.