Hallucination Detection on SQuAD (AUROC)
0.89AUROCID
Evaluation results
| Method | Links | |
|---|---|---|
| IDBackbone=LLaMA-3.1-8B-Instruct, First-sentence truncation=true, Forward horizon (w)=72026.05 | 0.89 | |
| FEPoIDBackbone=LLaMA-3.1-8B-Instruct, First-sentence truncation=true, Forward horizon (w)=72026.05 | 0.89 | |
| IDBackbone=LlaMA-3.1-8B, Forward horizon (w)=7, Representation extraction method=FST, Model tuning status=non-instruction-tuned (base model)2026.05 | 0.8884 | |
| FEPoIDBackbone=LlaMA-3.1-8B, Forward horizon (w)=7, Representation extraction method=FST, Model tuning status=non-instruction-tuned (base model)2026.05 | 0.8884 | |
| Val LossBackbone=LlaMA-3.1-8B, Forward horizon (w)=7, Representation extraction method=FST, Model tuning status=non-instruction-tuned (base model)2026.05 | 0.887 | |
| Val LossBackbone=LLaMA-3.1-8B-Instruct, First-sentence truncation=true, Forward horizon (w)=72026.05 | 0.8865 | |
| Val LossBackbone=Mistral-7B-Instruct-v0.3, First-sentence truncation=true, Forward horizon (w)=72026.05 | 0.8784 | |
| IDBackbone=Mistral-7B-Instruct-v0.3, First-sentence truncation=true, Forward horizon (w)=72026.05 | 0.8769 | |
| FEPoIDBackbone=Mistral-7B-Instruct-v0.3, First-sentence truncation=true, Forward horizon (w)=72026.05 | 0.8769 | |
| Attention probe, soft targetRegime=Post-gen., Model=Qwen2.5-32B2026.06 | 0.8754 | |
| CurvatureBackbone=Mistral-7B-Instruct-v0.3, First-sentence truncation=true, Forward horizon (w)=72026.05 | 0.8704 | |
| Val LossLLM Backbone=Mistral-7B-Instruct-v0.3, Method Category=Hidden-State Probing2026.05 | 0.8679 | |
| IDLLM Backbone=Mistral-7B-Instruct-v0.3, Method Category=Hidden-State Probing2026.05 | 0.8679 | |
| FEPoIDLLM Backbone=Mistral-7B-Instruct-v0.3, Method Category=Hidden-State Probing2026.05 | 0.8679 | |
| CurvatureBackbone=LLaMA-3.1-8B-Instruct, First-sentence truncation=true, Forward horizon (w)=72026.05 | 0.8625 | |
| CurvatureBackbone=LlaMA-3.1-8B, Forward horizon (w)=7, Representation extraction method=FST, Model tuning status=non-instruction-tuned (base model)2026.05 | 0.8613 | |
| Attention probe, soft targetRegime=Pre-gen., Model=Qwen2.5-32B2026.06 | 0.8566 | |
| CurvatureLLM Backbone=Mistral-7B-Instruct-v0.3, Method Category=Hidden-State Probing2026.05 | 0.8553 | |
| RankMEBackbone=LlaMA-3.1-8B, Forward horizon (w)=7, Representation extraction method=FST, Model tuning status=non-instruction-tuned (base model)2026.05 | 0.854 | |
| Attention probeRegime=Pre-gen., Model=Qwen2.5-32B2026.06 | 0.8519 | |
| Attention probeRegime=Post-gen., Model=Qwen2.5-32B2026.06 | 0.8479 | |
| Lexical SimilarityBackbone=LLaMA-3.1-8B-Instruct, First-sentence truncation=true, Forward horizon (w)=72026.05 | 0.8433 | |
| Attention probeRegime=Post-gen., Model=Qwen2.5-7B2026.06 | 0.8339 | |
| Attention probe, soft targetRegime=Post-gen., Model=Qwen2.5-7B2026.06 | 0.8333 | |
| Attention probe, soft targetRegime=Post-gen., Model=Llama-3-8B2026.06 | 0.8213 | |
| EigenScoreBackbone=LLaMA-3.1-8B-Instruct, First-sentence truncation=true, Forward horizon (w)=72026.05 | 0.8152 | |
| Attention probeRegime=Post-gen., Model=Llama-3-8B2026.06 | 0.8149 | |
| RankMEBackbone=LLaMA-3.1-8B-Instruct, First-sentence truncation=true, Forward horizon (w)=72026.05 | 0.8106 | |
| RGNBackbone=LLaMA-3.1-8B-Instruct, First-sentence truncation=true, Forward horizon (w)=72026.05 | 0.8104 | |
| Attention probe, soft targetRegime=Pre-gen., Model=Qwen2.5-7B2026.06 | 0.8015 | |
| Attention probe, soft targetRegime=Post-gen., Model=Qwen2.5-3B2026.06 | 0.8006 | |
| Attention probeRegime=Post-gen., Model=Gemma-2-9B2026.06 | 0.7977 | |
| Linear probeRegime=Post-gen., Model=Llama-3-8B2026.06 | 0.7967 | |
| EntropyRegime=Post-gen., Model=Qwen2.5-32B2026.06 | 0.7954 | |
| RGNBackbone=Mistral-7B-Instruct-v0.3, First-sentence truncation=true, Forward horizon (w)=72026.05 | 0.7929 | |
| IDBackbone=LlaMA-3.1-8B base, Forward horizon (w)=7, FST=false2026.05 | 0.7892 | |
| FEPoIDBackbone=LlaMA-3.1-8B base, Forward horizon (w)=7, FST=false2026.05 | 0.7892 | |
| Attention probe, soft targetRegime=Post-gen., Model=Gemma-2-9B2026.06 | 0.7852 | |
| Pred. EntropyBackbone=LLaMA-3.1-8B-Instruct, First-sentence truncation=true, Forward horizon (w)=72026.05 | 0.785 | |
| Val LossBackbone=LlaMA-3.1-8B base, Forward horizon (w)=7, FST=false2026.05 | 0.7811 | |
| Semantic EntropyBackbone=LLaMA-3.1-8B-Instruct, First-sentence truncation=true, Forward horizon (w)=72026.05 | 0.7806 | |
| Attention probe, soft targetRegime=Pre-gen., Model=Gemma-2-9B2026.06 | 0.78 | |
| Attention probe, soft targetRegime=Pre-gen., Model=Llama-3-8B2026.06 | 0.7701 | |
| Attention probeRegime=Pre-gen., Model=Qwen2.5-7B2026.06 | 0.7695 | |
| Linear probeRegime=Post-gen., Model=Qwen2.5-32B2026.06 | 0.7681 | |
| Pred. EntropyBackbone=Mistral-7B-Instruct-v0.3, First-sentence truncation=true, Forward horizon (w)=72026.05 | 0.7633 | |
| RGNLLM Backbone=Mistral-7B-Instruct-v0.3, Method Category=Hidden-State Probing2026.05 | 0.7553 | |
| RankMEBackbone=Mistral-7B-Instruct-v0.3, First-sentence truncation=true, Forward horizon (w)=72026.05 | 0.7548 | |
| Attention probeRegime=Post-gen., Model=Qwen2.5-3B2026.06 | 0.7532 | |
| EigenScoreLLM Backbone=Mistral-7B-Instruct-v0.32026.05 | 0.7508 | |
| LN-EntropyModel=gpt-4o-mini-2024-07-18, Category=Single-run2026.05 | 0.748 | |
| CurvatureBackbone=LlaMA-3.1-8B base, Forward horizon (w)=7, FST=false2026.05 | 0.7432 | |
| LN-Pred. EntropyBackbone=LLaMA-3.1-8B-Instruct, First-sentence truncation=true, Forward horizon (w)=72026.05 | 0.7425 | |
| Attention probeRegime=Pre-gen., Model=Llama-3-8B2026.06 | 0.7393 | |
| Linear probeRegime=Post-gen., Model=Gemma-2-9B2026.06 | 0.7372 | |
| EigenScoreBackbone=Mistral-7B-Instruct-v0.3, First-sentence truncation=true, Forward horizon (w)=72026.05 | 0.735 | |
| RankMELLM Backbone=Mistral-7B-Instruct-v0.3, Method Category=Hidden-State Probing2026.05 | 0.7341 | |
| Attention probeRegime=Pre-gen., Model=Gemma-2-9B2026.06 | 0.7323 | |
| Pred. EntropyLLM Backbone=Mistral-7B-Instruct-v0.32026.05 | 0.7316 | |
| RGNBackbone=LlaMA-3.1-8B base, Forward horizon (w)=7, FST=false2026.05 | 0.7313 | |
| RGNBackbone=LlaMA-3.1-8B, Forward horizon (w)=7, Representation extraction method=FST, Model tuning status=non-instruction-tuned (base model)2026.05 | 0.7305 | |
| IDBackbone=LlaMA-3.2-3B, Forward horizon (w)=7, FST extraction status=without FST2026.05 | 0.7275 | |
| FEPoIDBackbone=LlaMA-3.2-3B, Forward horizon (w)=7, FST extraction status=without FST2026.05 | 0.7275 | |
| Lexical SimilarityBackbone=Mistral-7B-Instruct-v0.3, First-sentence truncation=true, Forward horizon (w)=72026.05 | 0.727 | |
| EntropyRegime=Post-gen., Model=Qwen2.5-7B2026.06 | 0.7254 | |
| SNRBackbone=LlaMA-3.1-8B, Forward horizon (w)=7, Representation extraction method=FST, Model tuning status=non-instruction-tuned (base model)2026.05 | 0.723 | |
| Lexical SimilarityLLM Backbone=Mistral-7B-Instruct-v0.32026.05 | 0.7169 | |
| Val LossBackbone=LlaMA-3.2-3B, Forward horizon (w)=7, FST extraction status=without FST2026.05 | 0.7153 | |
| RankMEBackbone=LlaMA-3.1-8B base, Forward horizon (w)=7, FST=false2026.05 | 0.7132 | |
| SNRBackbone=Mistral-7B-Instruct-v0.3, First-sentence truncation=true, Forward horizon (w)=72026.05 | 0.7112 | |
| Linear probeRegime=Pre-gen., Model=Qwen2.5-32B2026.06 | 0.7027 | |
| EntropyRegime=Post-gen., Model=Qwen2.5-3B2026.06 | 0.6963 | |
| Attention probe, soft targetRegime=Pre-gen., Model=Qwen2.5-3B2026.06 | 0.6924 | |
| Linear probeRegime=Post-gen., Model=Qwen2.5-7B2026.06 | 0.6899 | |
| SNRLLM Backbone=Mistral-7B-Instruct-v0.3, Method Category=Hidden-State Probing2026.05 | 0.687 | |
| EmbRegressModel=Meta-Llama-3-8B-Instruct, Category=Multi-trajectory / LLM-based2026.05 | 0.678 | |
| SNRBackbone=LLaMA-3.1-8B-Instruct, First-sentence truncation=true, Forward horizon (w)=72026.05 | 0.6693 | |
| CurvatureBackbone=LlaMA-3.2-3B, Forward horizon (w)=7, FST extraction status=without FST2026.05 | 0.6616 | |
| RGNBackbone=LlaMA-3.2-3B, Forward horizon (w)=7, FST extraction status=without FST2026.05 | 0.6567 | |
| LN-Pred. EntropyBackbone=Mistral-7B-Instruct-v0.3, First-sentence truncation=true, Forward horizon (w)=72026.05 | 0.6564 | |
| Attention probeRegime=Pre-gen., Model=Qwen2.5-3B2026.06 | 0.6539 | |
| Linear probeRegime=Pre-gen., Model=Gemma-2-9B2026.06 | 0.6537 | |
| FEPoIDBackbone=LlaMA-3.2-1B, Forward horizon (w)=3, FST extraction status=without FST2026.05 | 0.6514 | |
| RankMEBackbone=LlaMA-3.2-3B, Forward horizon (w)=7, FST extraction status=without FST2026.05 | 0.6476 | |
| CurvatureBackbone=LlaMA-3.2-1B, Forward horizon (w)=3, FST extraction status=without FST2026.05 | 0.6442 | |
| Val LossBackbone=LlaMA-3.2-1B, Forward horizon (w)=3, FST extraction status=without FST2026.05 | 0.6442 | |
| EntropyRegime=Post-gen., Model=Gemma-2-9B2026.06 | 0.6426 | |
| Semantic EntropyLLM Backbone=Mistral-7B-Instruct-v0.32026.05 | 0.6409 | |
| Linear probeRegime=Pre-gen., Model=Llama-3-8B2026.06 | 0.6399 | |
| LN-Pred. EntropyLLM Backbone=Mistral-7B-Instruct-v0.32026.05 | 0.639 | |
| SNRBackbone=LlaMA-3.1-8B base, Forward horizon (w)=7, FST=false2026.05 | 0.6387 | |
| FEPoIDLLM Backbone=LlaMA-3.1-8B-Instruct, Method Category=Hidden-State Probing2026.05 | 0.6377 | |
| Semantic EntropyBackbone=Mistral-7B-Instruct-v0.3, First-sentence truncation=true, Forward horizon (w)=72026.05 | 0.6314 | |
| SNRBackbone=LlaMA-3.2-1B, Forward horizon (w)=3, FST extraction status=without FST2026.05 | 0.6225 | |
| CurvatureLLM Backbone=LlaMA-3.1-8B-Instruct, Method Category=Hidden-State Probing2026.05 | 0.6183 | |
| Val LossLLM Backbone=LlaMA-3.1-8B-Instruct, Method Category=Hidden-State Probing2026.05 | 0.6164 | |
| RGNLLM Backbone=LlaMA-3.1-8B-Instruct, Method Category=Hidden-State Probing2026.05 | 0.613 | |
| IDLLM Backbone=LlaMA-3.1-8B-Instruct, Method Category=Hidden-State Probing2026.05 | 0.613 | |
| RankMELLM Backbone=LlaMA-3.1-8B-Instruct, Method Category=Hidden-State Probing2026.05 | 0.6071 | |
| Lexical SimilarityLLM Backbone=LlaMA-3.1-8B-Instruct2026.05 | 0.5988 |