Radiology Report Meta-Evaluation on ReEvalMed
100D ScoreHulu-Med-32B
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Hulu-Med-32BType=Open-source, Passes=1-Pass2026.06 | 100 | 23 | 61.5 | 77 | |
| Hulu-Med-32BType=Open-source, Inference Strategy=1-Pass2026.06 | 100 | 23 | 61.5 | 77 | |
| LingShu-32BType=Open-source, Passes=1-Pass2026.06 | 99.5 | 64 | 81.8 | 35.5 | |
| LingShu-32BType=Open-source, Inference Strategy=1-Pass2026.06 | 99.5 | 64 | 81.8 | 35.5 | |
| GPT-5.1Type=Proprietary, Passes=1-Pass2026.06 | 98.5 | 42.5 | 70.5 | 56 | |
| GPT-5.1Type=Proprietary, Inference Strategy=1-Pass2026.06 | 98.5 | 42.5 | 70.5 | 56 | |
| Claude Sonnet 4.5Type=Proprietary, Passes=2-Pass2026.06 | 98 | 51 | 74.5 | 47 | |
| Claude Sonnet 4.5Type=Proprietary, Inference Strategy=2-Pass2026.06 | 98 | 51 | 74.5 | 47 | |
| LingShu-32BType=Open-source, Passes=2-Pass2026.06 | 97.5 | 5.5 | 51.5 | 92 | |
| LingShu-32BType=Open-source, Inference Strategy=2-Pass2026.06 | 97.5 | 5.5 | 51.5 | 92 | |
| Qwen3-MaxType=Open-source, Passes=1-Pass2026.06 | 97 | 53.5 | 75.3 | 43.5 | |
| Qwen3-MaxType=Open-source, Inference Strategy=1-Pass2026.06 | 97 | 53.5 | 75.3 | 43.5 | |
| Qwen3-MaxType=Open-source, Passes=2-Pass2026.06 | 96.5 | 45 | 70.8 | 51.5 | |
| Qwen3-8BType=Open-source, Passes=1-Pass2026.06 | 96.5 | 8 | 52.3 | 88.5 | |
| Qwen3-8BType=Open-source, Inference Strategy=1-Pass2026.06 | 96.5 | 8 | 52.3 | 88.5 | |
| Qwen3-MaxType=Open-source, Inference Strategy=2-Pass2026.06 | 96.5 | 45 | 70.8 | 51.5 | |
| Hulu-Med-32BType=Open-source, Passes=2-Pass2026.06 | 96 | 52.5 | 74.3 | 43.5 | |
| Hulu-Med-32BType=Open-source, Inference Strategy=2-Pass2026.06 | 96 | 52.5 | 74.3 | 43.5 | |
| Qwen3-8BType=Open-source, Passes=2-Pass2026.06 | 95.5 | 9 | 52.3 | 86.5 | |
| Qwen3-8BType=Open-source, Inference Strategy=2-Pass2026.06 | 95.5 | 9 | 52.3 | 86.5 | |
| GPT-5.1Type=Proprietary, Passes=2-Pass2026.06 | 95 | 69.5 | 82.3 | 25.5 | |
| GPT-5.1Type=Proprietary, Inference Strategy=2-Pass2026.06 | 95 | 69.5 | 82.3 | 25.5 | |
| Gemini 3 ProType=Proprietary, Passes=1-Pass2026.06 | 94 | 57.5 | 75.8 | 36.5 | |
| Gemini 3 ProType=Proprietary, Inference Strategy=1-Pass2026.06 | 94 | 57.5 | 75.8 | 36.5 | |
| Claude Sonnet 4.5Type=Proprietary, Passes=1-Pass2026.06 | 91.5 | 84 | 87.8 | 7.5 | |
| Claude Sonnet 4.5Type=Proprietary, Inference Strategy=1-Pass2026.06 | 91.5 | 84 | 87.8 | 7.5 | |
| FineRadScoreType=LLM, Threshold=Maximin, Backbone=GPT-5.12026.06 | 86.5 | 68.5 | 77.5 | 18 | |
| RadFactType=LLM, Threshold=Maximin, Backbone=GPT-5.12026.06 | 80 | 56 | 68 | 24 | |
| CRIMSONType=LLM, Threshold=Maximin2026.06 | 75 | 80.5 | 77.8 | 5.5 | |
| MedGemma-4BType=Open-source, Passes=2-Pass2026.06 | 67 | 67.5 | 67.3 | 0.5 | |
| MedGemma-4BType=Open-source, Inference Strategy=2-Pass2026.06 | 67 | 67.5 | 67.3 | 0.5 | |
| Gemini 3 ProType=Proprietary, Passes=2-Pass2026.06 | 65 | 79.5 | 72.3 | 14.5 | |
| Gemini 3 ProType=Proprietary, Inference Strategy=2-Pass2026.06 | 65 | 79.5 | 72.3 | 14.5 | |
| GREENType=LLM, Threshold=Maximin2026.06 | 53.5 | 41.5 | 47.5 | 12 | |
| AlignScoreType=NLP, Threshold=Maximin2026.06 | 51 | 51 | 51 | 0 | |
| MedGemma-4BType=Open-source, Passes=1-Pass2026.06 | 49 | 9 | 29 | 40 | |
| MedGemma-4BType=Open-source, Inference Strategy=1-Pass2026.06 | 49 | 9 | 29 | 40 | |
| CheXbertType=Med., Threshold=Maximin2026.06 | 47 | 49 | 48 | 2 | |
| RaTEScoreType=Med., Threshold=Maximin2026.06 | 35.5 | 35.5 | 35.5 | 0 | |
| RadGraphType=Med., Threshold=Maximin2026.06 | 28.5 | 28.5 | 28.5 | 0 | |
| RadBERTScoreType=Med., Threshold=Maximin2026.06 | 26.5 | 26.5 | 26.5 | 0 | |
| BERTScoreType=NLP, Threshold=Maximin2026.06 | 24 | 24 | 24 | 0 | |
| BLEUType=NLP, Threshold=Maximin2026.06 | 19 | 19 | 19 | 0 |