Medical Error Detection on MEDEC
77AccuracyMedGuards
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| MedGuardsbase LLM=Doubao-1.5 thinking pro2026.06 | 77 | — | — | — | — | — | |
| Medifact2026.06 | 73.7 | — | — | — | — | — | |
| MedGuardsbase LLM=Doubao-1.5 thinking pro2026.06 | 71.6 | — | — | — | — | — | |
| knowlab_AIMed2026.06 | 69.4 | — | — | — | — | — | |
| BLUEmedSetting=Few-Shot2026.04 | 69.13 | 68.28 | 73.33 | 63.87 | 74.45 | 72.44 | |
| EM-mixer2026.06 | 68 | — | — | — | — | — | |
| IKIM2026.06 | 67.8 | — | — | — | — | — | |
| BLUEmedSetting=Zero-Shot2026.04 | 67.67 | 66.67 | 72.01 | 62.05 | 70.24 | 68.88 | |
| IryoNLP2026.06 | 67.1 | — | — | — | — | — | |
| EM-mixer2026.06 | 64 | — | — | — | — | — | |
| knowlab_AIMed2026.06 | 62 | — | — | — | — | — | |
| IryoNLP2026.06 | 61 | — | — | — | — | — | |
| Medifact2026.06 | 60 | — | — | — | — | — | |
| IKIM2026.06 | 59 | — | — | — | — | — | |
| RAG-Single (WebMD)Setting=Few-Shot2026.04 | 58.96 | 43.16 | 77.5 | 29.9 | 50 | 50 | |
| RAG-Single (MayoClinic)Setting=Zero-Shot2026.04 | 58.63 | 59.44 | 60.74 | 58.2 | 50 | 50 | |
| RAG-Single (WebMD)Setting=Zero-Shot2026.04 | 57.79 | 60.5 | 59.02 | 62.06 | 62.93 | 50 | |
| RAG-Single (MayoClinic)Setting=Few-Shot2026.04 | 57.29 | 40.56 | 73.73 | 27.97 | 50 | 50 | |
| LLM-DebateSetting=Few-Shot2026.04 | 55.89 | 65.55 | 55.42 | 80.2 | 58.56 | 58.7 | |
| LLM-DebateSetting=Zero-Shot2026.04 | 55.28 | 66.91 | 54.44 | 86.82 | 59.51 | 59.47 |