Hallucination Detection on MedicalQA
78.95AUROCCombined: PC + SE
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| Combined: PC + SEBase Model=Gemma 3 4B, Base Model Accuracy=17.3%2026.03 | 78.95 | 94.42 | |
| Semantic EntropyBase Model=Ministral 8B, Base Model Accuracy=23.3%2026.03 | 77.24 | 90.12 | |
| Combined: PC + NLLBase Model=Gemma 3 4B, Base Model Accuracy=17.3%2026.03 | 77.05 | 93.42 | |
| PC ProbeBase Model=Gemma 3 4B, Base Model Accuracy=17.3%2026.03 | 75.55 | 92.13 | |
| Combined: PC + SE ProbeBase Model=Gemma 3 4B, Base Model Accuracy=17.3%2026.03 | 74.96 | 91.53 | |
| Semantic EntropyBase Model=Gemma 3 4B, Base Model Accuracy=17.3%2026.03 | 70.15 | 90.53 | |
| Sequence NLLBase Model=Llama 3.2 3B, Base Model Accuracy=22.0%2026.03 | 69.05 | 85.34 | |
| Semantic EntropyBase Model=Llama 3.2 3B, Base Model Accuracy=22.0%2026.03 | 68.85 | 86.63 | |
| Combined: PC + SEBase Model=Qwen3 4B, Base Model Accuracy=23.3%2026.03 | 68.14 | 87.63 | |
| Combined: PC + SE ProbeBase Model=Qwen3 4B, Base Model Accuracy=23.3%2026.03 | 68.04 | 88.23 | |
| PC ProbeBase Model=Qwen3 4B, Base Model Accuracy=23.3%2026.03 | 67.14 | 88.43 | |
| Combined: PC + NLLBase Model=Qwen3 4B, Base Model Accuracy=23.3%2026.03 | 66.75 | 86.14 | |
| Combined: PC + SEBase Model=Llama 3.2 3B, Base Model Accuracy=22.0%2026.03 | 66.65 | 87.53 | |
| Combined: PC + SE ProbeBase Model=Llama 3.2 3B, Base Model Accuracy=22.0%2026.03 | 65.45 | 87.03 | |
| Combined: PC + NLLBase Model=Llama 3.2 3B, Base Model Accuracy=22.0%2026.03 | 64.95 | 87.73 | |
| Semantic EntropyBase Model=Qwen3 4B, Base Model Accuracy=23.3%2026.03 | 64.85 | 85.03 | |
| SE ProbeBase Model=Qwen3 4B, Base Model Accuracy=23.3%2026.03 | 64.35 | 85.24 | |
| PC ProbeBase Model=Llama 3.2 3B, Base Model Accuracy=22.0%2026.03 | 64.25 | 86.83 | |
| Sequence NLLBase Model=Gemma 3 4B, Base Model Accuracy=17.3%2026.03 | 63.53 | 88.02 | |
| SE ProbeBase Model=Gemma 3 4B, Base Model Accuracy=17.3%2026.03 | 63.46 | 87.34 | |
| Combined: PC + SEBase Model=Ministral 8B, Base Model Accuracy=23.3%2026.03 | 62.84 | 85.93 | |
| SE ProbeBase Model=Llama 3.2 3B, Base Model Accuracy=22.0%2026.03 | 62.65 | 86.43 | |
| Sequence NLLBase Model=Qwen3 4B, Base Model Accuracy=23.3%2026.03 | 61.34 | 81.54 | |
| PC ProbeBase Model=Ministral 8B, Base Model Accuracy=23.3%2026.03 | 56.95 | 82.13 | |
| Combined: PC + SE ProbeBase Model=Ministral 8B, Base Model Accuracy=23.3%2026.03 | 56.55 | 81.83 | |
| Combined: PC + NLLBase Model=Ministral 8B, Base Model Accuracy=23.3%2026.03 | 56.05 | 81.43 | |
| Sequence NLLBase Model=Ministral 8B, Base Model Accuracy=23.3%2026.03 | 55.15 | 79.34 | |
| SE ProbeBase Model=Ministral 8B, Base Model Accuracy=23.3%2026.03 | 53.95 | 80.44 |