# Ensembling Multiple Hallucination Detectors Trained on VLLM Internal Representations

Source: [Ensembling Multiple Hallucination Detectors Trained on VLLM Internal Representations](<https://www.sota2.com/research/paper/68fa7c281e209131cea8cb9d>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-12-30T05:00:04.958Z

Authors: Yuto Nakamizo, Ryuhei Miyazato, Hikaru Tanabe, Ryuta Yamakura, Kiori Hatanaka

Published: 2025-10-16T06:09:26.000Z

## Abstract

This paper presents the 5th place solution by our team, y3h2, for the Meta CRAG-MM Challenge at KDD Cup 2025. The CRAG-MM benchmark is a visual question answering (VQA) dataset focused on factual questions about images, including egocentric images. The competition was contested based on VQA accuracy, as judged by an LLM-based automatic evaluator. Since incorrect answers result in negative scores, our strategy focused on reducing hallucinations from the internal representations of the VLM. Specifically, we trained logistic regression-based hallucination detection models using both the hidden\_state and the outputs of specific attention heads. We then employed an ensemble of these models. As a result, while our method sacrificed some correct answers, it significantly reduced hallucinations and allowed us to place among the top entries on the final leaderboard.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2510.14330>)

[PDF](<https://arxiv.org/pdf/2510.14330.pdf>)
