Title: On Exploring the Reasoning Capability of Large Language Models with Knowledge Graphs Show Chinese title

Title: 关于利用知识图谱探索大型语言模型的推理能力

Abstract: This paper examines the capacity of LLMs to reason with knowledge graphs using their internal knowledge graph, i.e., the knowledge graph they learned during pre-training. Two research questions are formulated to investigate the accuracy of LLMs in recalling information from pre-training knowledge graphs and their ability to infer knowledge graph relations from context. To address these questions, we employ LLMs to perform four distinct knowledge graph reasoning tasks. Furthermore, we identify two types of hallucinations that may occur during knowledge reasoning with LLMs: content and ontology hallucination. Our experimental results demonstrate that LLMs can successfully tackle both simple and complex knowledge graph reasoning tasks from their own memory, as well as infer from input context. Show Chinese abstract

Abstract: 本文研究了大型语言模型利用其内部知识图谱（即预训练期间学习的知识图谱）进行推理的能力。 为探讨大型语言模型在从预训练知识图谱中回忆信息的准确性以及从上下文中推断知识图谱关系的能力，我们提出了两个研究问题。 为解决这些问题，我们使用大型语言模型执行四项不同的知识图谱推理任务。 此外，我们识别了在使用大型语言模型进行知识推理时可能出现的两种幻觉类型：内容幻觉和本体幻觉。 我们的实验结果表明，大型语言模型能够成功处理来自自身记忆的简单和复杂知识图谱推理任务，并能够从输入上下文中进行推断。