Title: On Significance of Subword tokenization for Low Resource and Efficient Named Entity Recognition: A case study in Marathi Show Chinese title

Title: 低资源和高效命名实体识别中子词分词的重要性：马拉地语案例研究

Abstract: Named Entity Recognition (NER) systems play a vital role in NLP applications such as machine translation, summarization, and question-answering. These systems identify named entities, which encompass real-world concepts like locations, persons, and organizations. Despite extensive research on NER systems for the English language, they have not received adequate attention in the context of low resource languages. In this work, we focus on NER for low-resource language and present our case study in the context of the Indian language Marathi. The advancement of NLP research revolves around the utilization of pre-trained transformer models such as BERT for the development of NER models. However, we focus on improving the performance of shallow models based on CNN, and LSTM by combining the best of both worlds. In the era of transformers, these traditional deep learning models are still relevant because of their high computational efficiency. We propose a hybrid approach for efficient NER by integrating a BERT-based subword tokenizer into vanilla CNN/LSTM models. We show that this simple approach of replacing a traditional word-based tokenizer with a BERT-tokenizer brings the accuracy of vanilla single-layer models closer to that of deep pre-trained models like BERT. We show the importance of using sub-word tokenization for NER and present our study toward building efficient NLP systems. The evaluation is performed on L3Cube-MahaNER dataset using tokenizers from MahaBERT, MahaGPT, IndicBERT, and mBERT. Show Chinese abstract

Abstract: 命名实体识别（NER）系统在自然语言处理（NLP）应用中起着至关重要的作用，例如机器翻译、摘要和问答。 这些系统识别命名实体，包括现实世界概念，如地点、人物和组织。 尽管对英语语言的NER系统进行了大量研究，但在低资源语言的背景下，它们尚未得到足够的关注。 在这项工作中，我们专注于低资源语言的NER，并在印度语言马拉地语的背景下展示了我们的案例研究。 NLP研究的进步围绕着使用预训练的Transformer模型，如BERT，来开发NER模型。 然而，我们关注的是通过结合两者的优势来提高基于CNN和LSTM的浅层模型的性能。 在Transformer时代，这些传统深度学习模型仍然相关，因为它们具有高的计算效率。 我们提出了一种混合方法，通过将基于BERT的子词分词器集成到普通的CNN/LSTM模型中，以实现高效的NER。 我们表明，用基于BERT的分词器替换传统的基于单词的分词器这一简单方法，可以使普通单层模型的准确性接近像BERT这样的深度预训练模型。 我们展示了使用子词分词对于NER的重要性，并介绍了我们构建高效NLP系统的研究所做的工作。 评估是在L3Cube-MahaNER数据集上进行的，使用了MahaBERT、MahaGPT、IndicBERT和mBERT的分词器。