Title: Large Language Models are Few-shot Multivariate Time Series Classifiers Show Chinese title

Title: 大型语言模型是Few-shot多变量时间序列分类器

Abstract: Large Language Models (LLMs) have been extensively applied in time series analysis. Yet, their utility in the few-shot classification (i.e., a crucial training scenario due to the limited training data available in industrial applications) concerning multivariate time series data remains underexplored. We aim to leverage the extensive pre-trained knowledge in LLMs to overcome the data scarcity problem within multivariate time series. Specifically, we propose LLMFew, an LLM-enhanced framework to investigate the feasibility and capacity of LLMs for few-shot multivariate time series classification. This model introduces a Patch-wise Temporal Convolution Encoder (PTCEnc) to align time series data with the textual embedding input of LLMs. We further fine-tune the pre-trained LLM decoder with Low-rank Adaptations (LoRA) to enhance its feature representation learning ability in time series data. Experimental results show that our model outperformed state-of-the-art baselines by a large margin, achieving 125.2% and 50.2% improvement in classification accuracy on Handwriting and EthanolConcentration datasets, respectively. Moreover, our experimental results demonstrate that LLM-based methods perform well across a variety of datasets in few-shot MTSC, delivering reliable results compared to traditional models. This success paves the way for their deployment in industrial environments where data are limited. Show Chinese abstract

Abstract: 大型语言模型（LLMs）已在时间序列分析中得到了广泛应用。然而，它们在处理多变量时间序列数据的少量样本分类问题中的效用仍未被充分探索，而少量样本分类（即由于工业应用中可用训练数据有限，这是一种关键的训练场景）对于时间序列分析至关重要。我们旨在利用LLMs中广泛预训练的知识来克服多变量时间序列中的数据稀缺性问题。具体而言，我们提出了LLMFew，这是一个增强LLM的框架，用于研究LLM在少量样本多变量时间序列分类任务中的可行性和能力。该模型引入了基于Patch的时间卷积编码器（PTCEnc），以使时间序列数据与LLMs的文本嵌入输入对齐。我们进一步使用低秩适应（LoRA）微调预训练的LLM解码器，以增强其在时间序列数据中的特征表示学习能力。实验结果表明，我们的模型大幅超越了最先进的基线模型，在手写和乙醇浓度数据集上的分类准确率分别提高了125.2%和50.2%。此外，我们的实验结果表明，基于LLM的方法在少量样本多变量时间序列分类（MTSC）的各种数据集中表现良好，与传统模型相比提供了可靠的结果。这一成功为它们在工业环境中有限数据的情况下部署铺平了道路。