List your company
ResearchFormerly WizWand

Hallucination Detection on BioASQ

81.13AUROCSemantic Entropy
-2.6555219.0964940.848562.60051Mar 22, 2026Apr 2, 2026Apr 13, 2026Apr 24, 2026May 5, 2026May 16, 2026May 27, 2026
Updated 5mo ago

Evaluation results

MethodLinks
Semantic EntropyBase Model=Ministral 8B, Base Model Accuracy=50.3%2026.0381.1379.74—
SE ProbeBase Model=Ministral 8B, Base Model Accuracy=50.3%2026.0379.8380.34—
Combined: PC + SEBase Model=Ministral 8B, Base Model Accuracy=50.3%2026.0379.1379.44—
Semantic EntropyBase Model=Gemma 3 4B, Base Model Accuracy=36.4%2026.0377.7382.44—
SE ProbeBase Model=Gemma 3 4B, Base Model Accuracy=36.4%2026.0377.6482.54—
Combined: PC + SE ProbeBase Model=Ministral 8B, Base Model Accuracy=50.3%2026.0377.4377.84—
Combined: PC + NLLBase Model=Ministral 8B, Base Model Accuracy=50.3%2026.0376.7377.05—
Combined: PC + SEBase Model=Gemma 3 4B, Base Model Accuracy=36.4%2026.0376.1482.33—
PC ProbeBase Model=Ministral 8B, Base Model Accuracy=50.3%2026.0375.9376.74—
Combined: PC + SE ProbeBase Model=Gemma 3 4B, Base Model Accuracy=36.4%2026.0375.3481.73—
Combined: PC + NLLBase Model=Gemma 3 4B, Base Model Accuracy=36.4%2026.0375.1481.54—
PC ProbeBase Model=Gemma 3 4B, Base Model Accuracy=36.4%2026.0374.6478.84—
Semantic EntropyBase Model=Llama 3.2 3B, Base Model Accuracy=48.3%2026.0373.7469.95—
Combined: PC + SEBase Model=Qwen3 4B, Base Model Accuracy=40.4%2026.0372.2479.04—
Semantic EntropyBase Model=Qwen3 4B, Base Model Accuracy=40.4%2026.0371.7376.54—
Combined: PC + NLLBase Model=Qwen3 4B, Base Model Accuracy=40.4%2026.0371.1477.14—
PC ProbeBase Model=Qwen3 4B, Base Model Accuracy=40.4%2026.0370.9478.14—
Combined: PC + SE ProbeBase Model=Qwen3 4B, Base Model Accuracy=40.4%2026.0370.7477.34—
Combined: PC + SEBase Model=Llama 3.2 3B, Base Model Accuracy=48.3%2026.0370.2372.55—
SE ProbeBase Model=Llama 3.2 3B, Base Model Accuracy=48.3%2026.0369.7472.15—
Combined: PC + SE ProbeBase Model=Llama 3.2 3B, Base Model Accuracy=48.3%2026.0369.3371.34—
Sequence NLLBase Model=Ministral 8B, Base Model Accuracy=50.3%2026.0368.1467.85—
Combined: PC + NLLBase Model=Llama 3.2 3B, Base Model Accuracy=48.3%2026.0367.4368.55—
PC ProbeBase Model=Llama 3.2 3B, Base Model Accuracy=48.3%2026.0366.9368.35—
Sequence NLLBase Model=Qwen3 4B, Base Model Accuracy=40.4%2026.0366.3472.84—
Sequence NLLBase Model=Llama 3.2 3B, Base Model Accuracy=48.3%2026.0365.6463.35—
Sequence NLLBase Model=Gemma 3 4B, Base Model Accuracy=36.4%2026.0364.7473.74—
SE ProbeBase Model=Qwen3 4B, Base Model Accuracy=40.4%2026.0362.2468.15—
Max PoolingBackbone=LLaMA-3.3-Instruct-70B2026.050.917——
Max PoolingBackbone=LLaMA-3.1-8B2026.050.904——
Mean PoolingBackbone=LLaMA-3.3-Instruct-70B2026.050.904——
HaMI (SP)Backbone=LLaMA-3.1-8B2026.050.903——
HaMI (SP)Backbone=Mistral-Nemo-Instruct (12B)2026.050.902——
Max PoolingBackbone=Mistral-Nemo-Instruct (12B)2026.050.897——
HaMI (SP)Backbone=LLaMA-3.3-Instruct-70B2026.050.896——
Mean PoolingBackbone=LLaMA-3.1-8B2026.050.892——
Mean PoolingBackbone=Mistral-Nemo-Instruct (12B)2026.050.89——
HaMI (Original)Backbone=LLaMA-3.3-Instruct-70B2026.050.888——
HaMI (Original)Backbone=Mistral-Nemo-Instruct (12B)2026.050.873——
HaMI (Original)Backbone=LLaMA-3.1-8B2026.050.862——
QAODQ⊕V⊥Model=Qwen3-30B-A3B2026.050.8604—0.7338
QAODQ⊕V⊥Model=Qwen3-14B2026.050.8569—0.7351
QAODQ⊕V⊥Model=Llama-2-7B-chat2026.050.8345—0.7608
SAPLMAModel=Qwen3-30B-A3B2026.050.8324—0.6822
SAPLMAModel=Qwen3-14B2026.050.8166—0.6859
SINdexModel=Qwen3-30B-A3B2026.050.8137—0.6767
QAODQ⊕V⊥Model=gemma-2-2b2026.050.811—0.747
SINdexModel=Qwen3-14B2026.050.805—0.6749
QAODV⊥Model=Qwen3-30B-A3B2026.050.7995—0.6785
PfalseModel=Qwen3-30B-A3B2026.050.7933—0.6681
SAPLMAModel=Llama-2-7B-chat2026.050.7921—0.6935
SINdexModel=Llama-2-7B-chat2026.050.7919—0.6844
PfalseModel=Qwen3-14B2026.050.7917—0.6697
QAODV⊥Model=Qwen3-14B2026.050.7839—0.6714
MINDModel=Qwen3-30B-A3B2026.050.7787—0.6578
QAODV⊥Model=Llama-2-7B-chat2026.050.7746—0.7432
MINDModel=Llama-2-7B-chat2026.050.7681—0.7266
MINDModel=Qwen3-14B2026.050.7681—0.6541
SAPLMAModel=gemma-2-2b2026.050.7566—0.7106
QAODV⊥Model=gemma-2-2b2026.050.7464—0.7093
PfalseModel=Llama-2-7B-chat2026.050.7376—0.6871
MINDModel=gemma-2-2b2026.050.7342—0.7094
PikModel=Qwen3-14B2026.050.7322—0.6413
SINdexModel=gemma-2-2b2026.050.7291—0.7146
PikModel=gemma-2-2b2026.050.7251—0.7035
PikModel=Llama-2-7B-chat2026.050.7248—0.6762
LLM-CheckModel=Llama-2-7B-chat2026.050.714—0.7113
PikModel=Qwen3-30B-A3B2026.050.7032—0.592
CES (unsup)Model=gpt-4o-mini-2024-07-18, Category=Single-run2026.050.698——
CES (sup)Model=gpt-4o-mini-2024-07-18, Category=Single-run2026.050.697——
LN-EntropyModel=gpt-4o-mini-2024-07-18, Category=Single-run2026.050.69——
PerplexityModel=gpt-4o-mini-2024-07-18, Category=Single-run2026.050.69——
PfalseModel=gemma-2-2b2026.050.6545—0.7234
Logit EntropyModel=gemma-2-2b2026.050.648—0.6989
KLE-FullModel=gpt-4o-mini-2024-07-18, Category=Multi-trajectory / LLM-based2026.050.644——
KLEModel=gpt-4o-mini-2024-07-18, Category=Multi-trajectory / LLM-based2026.050.643——
LLM-CheckModel=gemma-2-2b2026.050.6399—0.6914
DSEModel=gpt-4o-mini-2024-07-18, Category=Multi-trajectory / LLM-based2026.050.636——
LLM-CheckModel=Qwen3-30B-A3B2026.050.6225—0.5522
Logit EntropyModel=Qwen3-14B2026.050.6188—0.4652
PPLModel=Qwen3-14B2026.050.6133—0.4628
EmbRegressModel=Meta-Llama-3-8B-Instruct, Category=Multi-trajectory / LLM-based2026.050.601——
PPLModel=Qwen3-30B-A3B2026.050.6007—0.5234
Logit EntropyModel=Qwen3-30B-A3B2026.050.5973—0.5234
SEModel=gpt-4o-mini-2024-07-18, Category=Multi-trajectory / LLM-based2026.050.597——
LLM-CheckModel=Qwen3-14B2026.050.5967—0.5742
PPLModel=gemma-2-2b2026.050.5932—0.6982
KLEModel=Meta-Llama-3-8B-Instruct, Category=Multi-trajectory / LLM-based2026.050.592——
KLE-FullModel=Meta-Llama-3-8B-Instruct, Category=Multi-trajectory / LLM-based2026.050.591——
EigenscoreModel=Meta-Llama-3-8B-Instruct, Category=Multi-trajectory / LLM-based2026.050.591——
SelfCheckGPTModel=gpt-4o-mini-2024-07-18, Category=Multi-trajectory / LLM-based2026.050.589——
SelfCheckGPTModel=Meta-Llama-3-8B-Instruct, Category=Multi-trajectory / LLM-based2026.050.588——
CES (unsup)Model=Meta-Llama-3-8B-Instruct, Category=Single-run2026.050.584——
CES (sup)Model=Meta-Llama-3-8B-Instruct, Category=Single-run2026.050.584——
LN-EntropyModel=Meta-Llama-3-8B-Instruct, Category=Single-run2026.050.582——
PerplexityModel=Meta-Llama-3-8B-Instruct, Category=Single-run2026.050.582——
DSEModel=Meta-Llama-3-8B-Instruct, Category=Multi-trajectory / LLM-based2026.050.581——
EigenscoreModel=gpt-4o-mini-2024-07-18, Category=Multi-trajectory / LLM-based2026.050.581——
Logit EntropyModel=Llama-2-7B-chat2026.050.571—0.6423
SEModel=Meta-Llama-3-8B-Instruct, Category=Multi-trajectory / LLM-based2026.050.567——