# Hallucination Detection on MetaQA 1-hop LLaMA2-7B responses (val)

Source: [Hallucination Detection on MetaQA 1-hop LLaMA2-7B responses (val)](<https://www.sota2.com/research/sota/hallucination-detection-on-metaqa-1-hop-llama2-7b-responses-val>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-05-27T08:10:30.266Z

Evaluation of model performance on hallucination detection using the validation split of responses generated by LLaMA2-7B on the 1-hop subset of the MetaQA dataset.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Hallucination Detection |
| Dataset (including any stated split) | MetaQA 1-hop LLaMA2-7B responses (val) |
| Metric | AUC |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 7 of 7 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [AUC](<https://www.sota2.com/research/sota/hallucination-detection-on-metaqa-1-hop-llama2-7b-responses-val.md?metric=AUC>): Higher is better; unit: PERCENT.

- [F1 (Class 1)](<https://www.sota2.com/research/sota/hallucination-detection-on-metaqa-1-hop-llama2-7b-responses-val.md?metric=F1%20(Class%201)>): Higher is better; unit: PERCENT.

- [F1 (Macro Avg)](<https://www.sota2.com/research/sota/hallucination-detection-on-metaqa-1-hop-llama2-7b-responses-val.md?metric=F1%20(Macro%20Avg)>): Higher is better; unit: PERCENT.

- [Precision (Class 1)](<https://www.sota2.com/research/sota/hallucination-detection-on-metaqa-1-hop-llama2-7b-responses-val.md?metric=Precision%20(Class%201)>): Higher is better; unit: PERCENT.

- [Recall (Class 1)](<https://www.sota2.com/research/sota/hallucination-detection-on-metaqa-1-hop-llama2-7b-responses-val.md?metric=Recall%20(Class%201)>): Higher is better; unit: PERCENT.

## Recorded results

| Method | AUC | Result date | Report paper |
| --- | --- | --- | --- |
| SSR + SAS | 83.41 | Not recorded | 6a167aa8dbd7150506f80076 |
| Embedding Divergence | 69.14 | Not recorded | 6a167aa8dbd7150506f80076 |
| BERTScore | 63.66 | Not recorded | 6a167aa8dbd7150506f80076 |
| NLI Contradiction | 57.41 | Not recorded | 6a167aa8dbd7150506f80076 |
| Token confidence | 50.2 | Not recorded | 6a167aa8dbd7150506f80076 |
| Max token probability | 50 | Not recorded | 6a167aa8dbd7150506f80076 |
| Perplexity | 45.78 | Not recorded | 6a167aa8dbd7150506f80076 |

### SSR + SAS

AUC: 83.41. Result ID: 6a16a58cfbc373592c0fa306.

[Reporting paper](<https://www.sota2.com/research/paper/6a167aa8dbd7150506f80076.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a167aa8dbd7150506f80076.md>)

Recorded evaluation configuration: {"Category":"Mechanistic Detector"}. This record does not assert independent reproduction.

### Embedding Divergence

AUC: 69.14. Result ID: 6a16a58cfbc373592c0fa304.

[Reporting paper](<https://www.sota2.com/research/paper/6a167aa8dbd7150506f80076.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa66441e209131cea0f5ab.md>)

[Implementation](<https://github.com/UKPLab/sentence-transformers>)

Recorded evaluation configuration: {"Category":"Semantic Consistency"}. This record does not assert independent reproduction.

### BERTScore

AUC: 63.66. Result ID: 6a16a58cfbc373592c0fa303.

[Reporting paper](<https://www.sota2.com/research/paper/6a167aa8dbd7150506f80076.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa67171e209131cea140d5.md>)

Recorded evaluation configuration: {"Category":"Semantic Consistency"}. This record does not assert independent reproduction.

### NLI Contradiction

AUC: 57.41. Result ID: 6a16a58cfbc373592c0fa305.

[Reporting paper](<https://www.sota2.com/research/paper/6a167aa8dbd7150506f80076.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa62151e209131ce9f76f2.md>)

[Implementation](<https://www.sota2.com/nlp.stanford.edu/projects/snli/>)

Recorded evaluation configuration: {"Category":"Semantic Consistency"}. This record does not assert independent reproduction.

### Token confidence

AUC: 50.2. Result ID: 6a16a58cfbc373592c0fa301.

[Reporting paper](<https://www.sota2.com/research/paper/6a167aa8dbd7150506f80076.md>)

Recorded evaluation configuration: {"Category":"Model-Internal Confidence"}. This record does not assert independent reproduction.

### Max token probability

AUC: 50. Result ID: 6a16a58cfbc373592c0fa302.

[Reporting paper](<https://www.sota2.com/research/paper/6a167aa8dbd7150506f80076.md>)

Recorded evaluation configuration: {"Category":"Model-Internal Confidence"}. This record does not assert independent reproduction.

### Perplexity

AUC: 45.78. Result ID: 6a16a58cfbc373592c0fa300.

[Reporting paper](<https://www.sota2.com/research/paper/6a167aa8dbd7150506f80076.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa70d71e209131cea4d943.md>)

Recorded evaluation configuration: {"Category":"Model-Internal Confidence"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.
