标题： BIMA：双向最大似然学习方法在大型视觉-语言模型中的幻觉预测与缓解研究 显示英文标题

标题： BIMA: Bijective Maximum Likelihood Learning Approach to Hallucination Prediction and Mitigation in Large Vision-Language Models

摘要： 大型视觉-语言模型已在各个领域得到广泛应用。然而，开发一个可信的系统，并具有大规模模型的最小可解释特性，这构成了重大挑战。这些系统导致的谬误函数最常见的术语之一是幻觉，即语言模型生成的响应与视觉内容不对应。为了缓解这个问题，已经开发了几种方法，其中一个突出的方向是改进解码过程。本文提出了一种新的双射最大似然学习（BIMA）方法，利用归一化流理论来减轻幻觉问题。所提出的BIMA方法可以高效地减轻现有视觉-语言模型中的幻觉问题，带来显著的改进。值得注意的是，BIMA在POPE基准上的平均F1得分为85.06%，并且分别显著减少了CHAIRS和CHAIRI 7.6%和2.6%。据我们所知，这是最早考虑双射方法以减少由大型视觉-语言模型引起的幻觉的研究之一。 显示英文摘要

摘要： Large vision-language models have become widely adopted to advance in various domains. However, developing a trustworthy system with minimal interpretable characteristics of large-scale models presents a significant challenge. One of the most prevalent terms associated with the fallacy functions caused by these systems is hallucination, where the language model generates a response that does not correspond to the visual content. To mitigate this problem, several approaches have been developed, and one prominent direction is to ameliorate the decoding process. In this paper, we propose a new Bijective Maximum Likelihood Learning (BIMA) approach to hallucination mitigation using normalizing flow theories. The proposed BIMA method can efficiently mitigate the hallucination problem in prevailing vision-language models, resulting in significant improvements. Notably, BIMA achieves the average F1 score of 85.06% on POPE benchmark and remarkably reduce CHAIRS and CHAIRI by 7.6% and 2.6%, respectively. To the best of our knowledge, this is one of the first studies that contemplates the bijection means to reduce hallucination induced by large vision-language models.