# I've got the "Answer"! Interpretation of LLMs Hidden States in Question Answering

Source: [I've got the "Answer"! Interpretation of LLMs Hidden States in Question Answering](<https://www.sota2.com/research/paper/68fa737a1e209131cea5cc81>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-28T13:57:06.978Z

Authors: Valeriya Goloviznina, Evgeny Kotelnikov

Published: 2024-06-04T07:43:12.000Z

## Abstract

Interpretability and explainability of AI are becoming increasingly important in light of the rapid development of large language models (LLMs). This paper investigates the interpretation of LLMs in the context of the knowledge-based question answering. The main hypothesis of the study is that correct and incorrect model behavior can be distinguished at the level of hidden states. The quantized models LLaMA-2-7B-Chat, Mistral-7B, Vicuna-7B and the MuSeRC question-answering dataset are used to test this hypothesis. The results of the analysis support the proposed hypothesis. We also identify the layers which have a negative effect on the model's behavior. As a prospect of practical application of the hypothesis, we propose to train such "weak" layers additionally in order to improve the quality of the task solution.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2406.02060>)

[PDF](<https://arxiv.org/pdf/2406.02060.pdf>)