# Hallucination Detection on MedicalQA Source: [Hallucination Detection on MedicalQA]() Collection index: [llms.txt]() Data updated: 2026-03-24T14:53:30.904Z Evaluation of model performance on hallucination detection using the MedicalQA dataset, typically measured by AUROC and AUPRC. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Hallucination Detection | | Dataset (including any stated split) | MedicalQA | | Metric | AUROC | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 28 of 28 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [AUROC](): Higher is better; unit: PERCENT. - [AUPRC](): Higher is better; unit: PERCENT. ## Recorded results | Method | AUROC | Result date | Report paper | | --- | --- | --- | --- | | Combined: PC + SE | 78.95 | Not recorded | 69c21abc0636e5bf4afc4b70 | | Semantic Entropy | 77.24 | Not recorded | 69c21abc0636e5bf4afc4b70 | | Combined: PC + NLL | 77.05 | Not recorded | 69c21abc0636e5bf4afc4b70 | | PC Probe | 75.55 | Not recorded | 69c21abc0636e5bf4afc4b70 | | Combined: PC + SE Probe | 74.96 | Not recorded | 69c21abc0636e5bf4afc4b70 | | Semantic Entropy | 70.15 | Not recorded | 69c21abc0636e5bf4afc4b70 | | Sequence NLL | 69.05 | Not recorded | 69c21abc0636e5bf4afc4b70 | | Semantic Entropy | 68.85 | Not recorded | 69c21abc0636e5bf4afc4b70 | | Combined: PC + SE | 68.14 | Not recorded | 69c21abc0636e5bf4afc4b70 | | Combined: PC + SE Probe | 68.04 | Not recorded | 69c21abc0636e5bf4afc4b70 | | PC Probe | 67.14 | Not recorded | 69c21abc0636e5bf4afc4b70 | | Combined: PC + NLL | 66.75 | Not recorded | 69c21abc0636e5bf4afc4b70 | | Combined: PC + SE | 66.65 | Not recorded | 69c21abc0636e5bf4afc4b70 | | Combined: PC + SE Probe | 65.45 | Not recorded | 69c21abc0636e5bf4afc4b70 | | Combined: PC + NLL | 64.95 | Not recorded | 69c21abc0636e5bf4afc4b70 | | Semantic Entropy | 64.85 | Not recorded | 69c21abc0636e5bf4afc4b70 | | SE Probe | 64.35 | Not recorded | 69c21abc0636e5bf4afc4b70 | | PC Probe | 64.25 | Not recorded | 69c21abc0636e5bf4afc4b70 | | Sequence NLL | 63.53 | Not recorded | 69c21abc0636e5bf4afc4b70 | | SE Probe | 63.46 | Not recorded | 69c21abc0636e5bf4afc4b70 | | Combined: PC + SE | 62.84 | Not recorded | 69c21abc0636e5bf4afc4b70 | | SE Probe | 62.65 | Not recorded | 69c21abc0636e5bf4afc4b70 | | Sequence NLL | 61.34 | Not recorded | 69c21abc0636e5bf4afc4b70 | | PC Probe | 56.95 | Not recorded | 69c21abc0636e5bf4afc4b70 | | Combined: PC + SE Probe | 56.55 | Not recorded | 69c21abc0636e5bf4afc4b70 | | Combined: PC + NLL | 56.05 | Not recorded | 69c21abc0636e5bf4afc4b70 | | Sequence NLL | 55.15 | Not recorded | 69c21abc0636e5bf4afc4b70 | | SE Probe | 53.95 | Not recorded | 69c21abc0636e5bf4afc4b70 | ### Combined: PC + SE AUROC: 78.95. Result ID: 69c2a55a47f34ce924ab7824. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Gemma 3 4B","Base Model Accuracy":"17.3%"}. This record does not assert independent reproduction. ### Semantic Entropy AUROC: 77.24. Result ID: 69c2a55a47f34ce924ab7827. [Reporting paper]() Recorded evaluation configuration: {"Base Model":"Ministral 8B","Base Model Accuracy":"23.3%"}. This record does not assert independent reproduction. ### Combined: PC + NLL AUROC: 77.05. Result ID: 69c2a55a47f34ce924ab7823. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Gemma 3 4B","Base Model Accuracy":"17.3%"}. This record does not assert independent reproduction. ### PC Probe AUROC: 75.55. Result ID: 69c2a55a47f34ce924ab7822. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Gemma 3 4B","Base Model Accuracy":"17.3%"}. This record does not assert independent reproduction. ### Combined: PC + SE Probe AUROC: 74.96. Result ID: 69c2a55a47f34ce924ab7825. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Gemma 3 4B","Base Model Accuracy":"17.3%"}. This record does not assert independent reproduction. ### Semantic Entropy AUROC: 70.15. Result ID: 69c2a55a47f34ce924ab7820. [Reporting paper]() Recorded evaluation configuration: {"Base Model":"Gemma 3 4B","Base Model Accuracy":"17.3%"}. This record does not assert independent reproduction. ### Sequence NLL AUROC: 69.05. Result ID: 69c2a55a47f34ce924ab7811. [Reporting paper]() Recorded evaluation configuration: {"Base Model":"Llama 3.2 3B","Base Model Accuracy":"22.0%"}. This record does not assert independent reproduction. ### Semantic Entropy AUROC: 68.85. Result ID: 69c2a55a47f34ce924ab7812. [Reporting paper]() Recorded evaluation configuration: {"Base Model":"Llama 3.2 3B","Base Model Accuracy":"22.0%"}. This record does not assert independent reproduction. ### Combined: PC + SE AUROC: 68.14. Result ID: 69c2a55a47f34ce924ab781d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen3 4B","Base Model Accuracy":"23.3%"}. This record does not assert independent reproduction. ### Combined: PC + SE Probe AUROC: 68.04. Result ID: 69c2a55a47f34ce924ab781e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen3 4B","Base Model Accuracy":"23.3%"}. This record does not assert independent reproduction. ### PC Probe AUROC: 67.14. Result ID: 69c2a55a47f34ce924ab781b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen3 4B","Base Model Accuracy":"23.3%"}. This record does not assert independent reproduction. ### Combined: PC + NLL AUROC: 66.75. Result ID: 69c2a55a47f34ce924ab781c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen3 4B","Base Model Accuracy":"23.3%"}. This record does not assert independent reproduction. ### Combined: PC + SE AUROC: 66.65. Result ID: 69c2a55a47f34ce924ab7816. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Llama 3.2 3B","Base Model Accuracy":"22.0%"}. This record does not assert independent reproduction. ### Combined: PC + SE Probe AUROC: 65.45. Result ID: 69c2a55a47f34ce924ab7817. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Llama 3.2 3B","Base Model Accuracy":"22.0%"}. This record does not assert independent reproduction. ### Combined: PC + NLL AUROC: 64.95. Result ID: 69c2a55a47f34ce924ab7815. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Llama 3.2 3B","Base Model Accuracy":"22.0%"}. This record does not assert independent reproduction. ### Semantic Entropy AUROC: 64.85. Result ID: 69c2a55a47f34ce924ab7819. [Reporting paper]() Recorded evaluation configuration: {"Base Model":"Qwen3 4B","Base Model Accuracy":"23.3%"}. This record does not assert independent reproduction. ### SE Probe AUROC: 64.35. Result ID: 69c2a55a47f34ce924ab781a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen3 4B","Base Model Accuracy":"23.3%"}. This record does not assert independent reproduction. ### PC Probe AUROC: 64.25. Result ID: 69c2a55a47f34ce924ab7814. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Llama 3.2 3B","Base Model Accuracy":"22.0%"}. This record does not assert independent reproduction. ### Sequence NLL AUROC: 63.53. Result ID: 69c2a55a47f34ce924ab781f. [Reporting paper]() Recorded evaluation configuration: {"Base Model":"Gemma 3 4B","Base Model Accuracy":"17.3%"}. This record does not assert independent reproduction. ### SE Probe AUROC: 63.46. Result ID: 69c2a55a47f34ce924ab7821. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Gemma 3 4B","Base Model Accuracy":"17.3%"}. This record does not assert independent reproduction. ### Combined: PC + SE AUROC: 62.84. Result ID: 69c2a55a47f34ce924ab782b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Ministral 8B","Base Model Accuracy":"23.3%"}. This record does not assert independent reproduction. ### SE Probe AUROC: 62.65. Result ID: 69c2a55a47f34ce924ab7813. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Llama 3.2 3B","Base Model Accuracy":"22.0%"}. This record does not assert independent reproduction. ### Sequence NLL AUROC: 61.34. Result ID: 69c2a55a47f34ce924ab7818. [Reporting paper]() Recorded evaluation configuration: {"Base Model":"Qwen3 4B","Base Model Accuracy":"23.3%"}. This record does not assert independent reproduction. ### PC Probe AUROC: 56.95. Result ID: 69c2a55a47f34ce924ab7829. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Ministral 8B","Base Model Accuracy":"23.3%"}. This record does not assert independent reproduction. ### Combined: PC + SE Probe AUROC: 56.55. Result ID: 69c2a55a47f34ce924ab782c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Ministral 8B","Base Model Accuracy":"23.3%"}. This record does not assert independent reproduction. ### Combined: PC + NLL AUROC: 56.05. Result ID: 69c2a55a47f34ce924ab782a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Ministral 8B","Base Model Accuracy":"23.3%"}. This record does not assert independent reproduction. ### Sequence NLL AUROC: 55.15. Result ID: 69c2a55a47f34ce924ab7826. [Reporting paper]() Recorded evaluation configuration: {"Base Model":"Ministral 8B","Base Model Accuracy":"23.3%"}. This record does not assert independent reproduction. ### SE Probe AUROC: 53.95. Result ID: 69c2a55a47f34ce924ab7828. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Ministral 8B","Base Model Accuracy":"23.3%"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.