Hallucination Detection on BioASQ
81.13AUROCSemantic Entropy
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| Semantic EntropyBase Model=Ministral 8B, Base Model Accuracy=50.3%2026.03 | 81.13 | 79.74 | — | |
| SE ProbeBase Model=Ministral 8B, Base Model Accuracy=50.3%2026.03 | 79.83 | 80.34 | — | |
| Combined: PC + SEBase Model=Ministral 8B, Base Model Accuracy=50.3%2026.03 | 79.13 | 79.44 | — | |
| Semantic EntropyBase Model=Gemma 3 4B, Base Model Accuracy=36.4%2026.03 | 77.73 | 82.44 | — | |
| SE ProbeBase Model=Gemma 3 4B, Base Model Accuracy=36.4%2026.03 | 77.64 | 82.54 | — | |
| Combined: PC + SE ProbeBase Model=Ministral 8B, Base Model Accuracy=50.3%2026.03 | 77.43 | 77.84 | — | |
| Combined: PC + NLLBase Model=Ministral 8B, Base Model Accuracy=50.3%2026.03 | 76.73 | 77.05 | — | |
| Combined: PC + SEBase Model=Gemma 3 4B, Base Model Accuracy=36.4%2026.03 | 76.14 | 82.33 | — | |
| PC ProbeBase Model=Ministral 8B, Base Model Accuracy=50.3%2026.03 | 75.93 | 76.74 | — | |
| Combined: PC + SE ProbeBase Model=Gemma 3 4B, Base Model Accuracy=36.4%2026.03 | 75.34 | 81.73 | — | |
| Combined: PC + NLLBase Model=Gemma 3 4B, Base Model Accuracy=36.4%2026.03 | 75.14 | 81.54 | — | |
| PC ProbeBase Model=Gemma 3 4B, Base Model Accuracy=36.4%2026.03 | 74.64 | 78.84 | — | |
| Semantic EntropyBase Model=Llama 3.2 3B, Base Model Accuracy=48.3%2026.03 | 73.74 | 69.95 | — | |
| Combined: PC + SEBase Model=Qwen3 4B, Base Model Accuracy=40.4%2026.03 | 72.24 | 79.04 | — | |
| Semantic EntropyBase Model=Qwen3 4B, Base Model Accuracy=40.4%2026.03 | 71.73 | 76.54 | — | |
| Combined: PC + NLLBase Model=Qwen3 4B, Base Model Accuracy=40.4%2026.03 | 71.14 | 77.14 | — | |
| PC ProbeBase Model=Qwen3 4B, Base Model Accuracy=40.4%2026.03 | 70.94 | 78.14 | — | |
| Combined: PC + SE ProbeBase Model=Qwen3 4B, Base Model Accuracy=40.4%2026.03 | 70.74 | 77.34 | — | |
| Combined: PC + SEBase Model=Llama 3.2 3B, Base Model Accuracy=48.3%2026.03 | 70.23 | 72.55 | — | |
| SE ProbeBase Model=Llama 3.2 3B, Base Model Accuracy=48.3%2026.03 | 69.74 | 72.15 | — | |
| Combined: PC + SE ProbeBase Model=Llama 3.2 3B, Base Model Accuracy=48.3%2026.03 | 69.33 | 71.34 | — | |
| Sequence NLLBase Model=Ministral 8B, Base Model Accuracy=50.3%2026.03 | 68.14 | 67.85 | — | |
| Combined: PC + NLLBase Model=Llama 3.2 3B, Base Model Accuracy=48.3%2026.03 | 67.43 | 68.55 | — | |
| PC ProbeBase Model=Llama 3.2 3B, Base Model Accuracy=48.3%2026.03 | 66.93 | 68.35 | — | |
| Sequence NLLBase Model=Qwen3 4B, Base Model Accuracy=40.4%2026.03 | 66.34 | 72.84 | — | |
| Sequence NLLBase Model=Llama 3.2 3B, Base Model Accuracy=48.3%2026.03 | 65.64 | 63.35 | — | |
| Sequence NLLBase Model=Gemma 3 4B, Base Model Accuracy=36.4%2026.03 | 64.74 | 73.74 | — | |
| SE ProbeBase Model=Qwen3 4B, Base Model Accuracy=40.4%2026.03 | 62.24 | 68.15 | — | |
| Max PoolingBackbone=LLaMA-3.3-Instruct-70B2026.05 | 0.917 | — | — | |
| Max PoolingBackbone=LLaMA-3.1-8B2026.05 | 0.904 | — | — | |
| Mean PoolingBackbone=LLaMA-3.3-Instruct-70B2026.05 | 0.904 | — | — | |
| HaMI (SP)Backbone=LLaMA-3.1-8B2026.05 | 0.903 | — | — | |
| HaMI (SP)Backbone=Mistral-Nemo-Instruct (12B)2026.05 | 0.902 | — | — | |
| Max PoolingBackbone=Mistral-Nemo-Instruct (12B)2026.05 | 0.897 | — | — | |
| HaMI (SP)Backbone=LLaMA-3.3-Instruct-70B2026.05 | 0.896 | — | — | |
| Mean PoolingBackbone=LLaMA-3.1-8B2026.05 | 0.892 | — | — | |
| Mean PoolingBackbone=Mistral-Nemo-Instruct (12B)2026.05 | 0.89 | — | — | |
| HaMI (Original)Backbone=LLaMA-3.3-Instruct-70B2026.05 | 0.888 | — | — | |
| HaMI (Original)Backbone=Mistral-Nemo-Instruct (12B)2026.05 | 0.873 | — | — | |
| HaMI (Original)Backbone=LLaMA-3.1-8B2026.05 | 0.862 | — | — | |
| QAODQ⊕V⊥Model=Qwen3-30B-A3B2026.05 | 0.8604 | — | 0.7338 | |
| QAODQ⊕V⊥Model=Qwen3-14B2026.05 | 0.8569 | — | 0.7351 | |
| QAODQ⊕V⊥Model=Llama-2-7B-chat2026.05 | 0.8345 | — | 0.7608 | |
| SAPLMAModel=Qwen3-30B-A3B2026.05 | 0.8324 | — | 0.6822 | |
| SAPLMAModel=Qwen3-14B2026.05 | 0.8166 | — | 0.6859 | |
| SINdexModel=Qwen3-30B-A3B2026.05 | 0.8137 | — | 0.6767 | |
| QAODQ⊕V⊥Model=gemma-2-2b2026.05 | 0.811 | — | 0.747 | |
| SINdexModel=Qwen3-14B2026.05 | 0.805 | — | 0.6749 | |
| QAODV⊥Model=Qwen3-30B-A3B2026.05 | 0.7995 | — | 0.6785 | |
| PfalseModel=Qwen3-30B-A3B2026.05 | 0.7933 | — | 0.6681 | |
| SAPLMAModel=Llama-2-7B-chat2026.05 | 0.7921 | — | 0.6935 | |
| SINdexModel=Llama-2-7B-chat2026.05 | 0.7919 | — | 0.6844 | |
| PfalseModel=Qwen3-14B2026.05 | 0.7917 | — | 0.6697 | |
| QAODV⊥Model=Qwen3-14B2026.05 | 0.7839 | — | 0.6714 | |
| MINDModel=Qwen3-30B-A3B2026.05 | 0.7787 | — | 0.6578 | |
| QAODV⊥Model=Llama-2-7B-chat2026.05 | 0.7746 | — | 0.7432 | |
| MINDModel=Llama-2-7B-chat2026.05 | 0.7681 | — | 0.7266 | |
| MINDModel=Qwen3-14B2026.05 | 0.7681 | — | 0.6541 | |
| SAPLMAModel=gemma-2-2b2026.05 | 0.7566 | — | 0.7106 | |
| QAODV⊥Model=gemma-2-2b2026.05 | 0.7464 | — | 0.7093 | |
| PfalseModel=Llama-2-7B-chat2026.05 | 0.7376 | — | 0.6871 | |
| MINDModel=gemma-2-2b2026.05 | 0.7342 | — | 0.7094 | |
| PikModel=Qwen3-14B2026.05 | 0.7322 | — | 0.6413 | |
| SINdexModel=gemma-2-2b2026.05 | 0.7291 | — | 0.7146 | |
| PikModel=gemma-2-2b2026.05 | 0.7251 | — | 0.7035 | |
| PikModel=Llama-2-7B-chat2026.05 | 0.7248 | — | 0.6762 | |
| LLM-CheckModel=Llama-2-7B-chat2026.05 | 0.714 | — | 0.7113 | |
| PikModel=Qwen3-30B-A3B2026.05 | 0.7032 | — | 0.592 | |
| CES (unsup)Model=gpt-4o-mini-2024-07-18, Category=Single-run2026.05 | 0.698 | — | — | |
| CES (sup)Model=gpt-4o-mini-2024-07-18, Category=Single-run2026.05 | 0.697 | — | — | |
| LN-EntropyModel=gpt-4o-mini-2024-07-18, Category=Single-run2026.05 | 0.69 | — | — | |
| PerplexityModel=gpt-4o-mini-2024-07-18, Category=Single-run2026.05 | 0.69 | — | — | |
| PfalseModel=gemma-2-2b2026.05 | 0.6545 | — | 0.7234 | |
| Logit EntropyModel=gemma-2-2b2026.05 | 0.648 | — | 0.6989 | |
| KLE-FullModel=gpt-4o-mini-2024-07-18, Category=Multi-trajectory / LLM-based2026.05 | 0.644 | — | — | |
| KLEModel=gpt-4o-mini-2024-07-18, Category=Multi-trajectory / LLM-based2026.05 | 0.643 | — | — | |
| LLM-CheckModel=gemma-2-2b2026.05 | 0.6399 | — | 0.6914 | |
| DSEModel=gpt-4o-mini-2024-07-18, Category=Multi-trajectory / LLM-based2026.05 | 0.636 | — | — | |
| LLM-CheckModel=Qwen3-30B-A3B2026.05 | 0.6225 | — | 0.5522 | |
| Logit EntropyModel=Qwen3-14B2026.05 | 0.6188 | — | 0.4652 | |
| PPLModel=Qwen3-14B2026.05 | 0.6133 | — | 0.4628 | |
| EmbRegressModel=Meta-Llama-3-8B-Instruct, Category=Multi-trajectory / LLM-based2026.05 | 0.601 | — | — | |
| PPLModel=Qwen3-30B-A3B2026.05 | 0.6007 | — | 0.5234 | |
| Logit EntropyModel=Qwen3-30B-A3B2026.05 | 0.5973 | — | 0.5234 | |
| SEModel=gpt-4o-mini-2024-07-18, Category=Multi-trajectory / LLM-based2026.05 | 0.597 | — | — | |
| LLM-CheckModel=Qwen3-14B2026.05 | 0.5967 | — | 0.5742 | |
| PPLModel=gemma-2-2b2026.05 | 0.5932 | — | 0.6982 | |
| KLEModel=Meta-Llama-3-8B-Instruct, Category=Multi-trajectory / LLM-based2026.05 | 0.592 | — | — | |
| KLE-FullModel=Meta-Llama-3-8B-Instruct, Category=Multi-trajectory / LLM-based2026.05 | 0.591 | — | — | |
| EigenscoreModel=Meta-Llama-3-8B-Instruct, Category=Multi-trajectory / LLM-based2026.05 | 0.591 | — | — | |
| SelfCheckGPTModel=gpt-4o-mini-2024-07-18, Category=Multi-trajectory / LLM-based2026.05 | 0.589 | — | — | |
| SelfCheckGPTModel=Meta-Llama-3-8B-Instruct, Category=Multi-trajectory / LLM-based2026.05 | 0.588 | — | — | |
| CES (unsup)Model=Meta-Llama-3-8B-Instruct, Category=Single-run2026.05 | 0.584 | — | — | |
| CES (sup)Model=Meta-Llama-3-8B-Instruct, Category=Single-run2026.05 | 0.584 | — | — | |
| LN-EntropyModel=Meta-Llama-3-8B-Instruct, Category=Single-run2026.05 | 0.582 | — | — | |
| PerplexityModel=Meta-Llama-3-8B-Instruct, Category=Single-run2026.05 | 0.582 | — | — | |
| DSEModel=Meta-Llama-3-8B-Instruct, Category=Multi-trajectory / LLM-based2026.05 | 0.581 | — | — | |
| EigenscoreModel=gpt-4o-mini-2024-07-18, Category=Multi-trajectory / LLM-based2026.05 | 0.581 | — | — | |
| Logit EntropyModel=Llama-2-7B-chat2026.05 | 0.571 | — | 0.6423 | |
| SEModel=Meta-Llama-3-8B-Instruct, Category=Multi-trajectory / LLM-based2026.05 | 0.567 | — | — |