Medical factuality evaluation on IU-Chest X-ray (test)
68.31AccuracyStandard RAG + BAIR
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| Standard RAG + BAIRBackbone=MedGemma-4B2026.05 | 68.31 | 24.25 | 8.4 | |
| MAD-RAG + BAIRBackbone=MedGemma-4B2026.05 | 68.25 | 24.19 | 8.45 | |
| MAD-RAGBackbone=MedGemma-4B2026.05 | 68.03 | 24.58 | 9.01 | |
| MS-PoE + BAIRBackbone=MedGemma-4B2026.05 | 67.6 | 21.01 | 7.43 | |
| MS-PoEBackbone=MedGemma-4B2026.05 | 67.17 | 20.68 | 7.85 | |
| MS-PoE + BAIRBackbone=CheXagent-8B2026.05 | 66.6 | 11.72 | 5.05 | |
| MS-PoEBackbone=CheXagent-8B2026.05 | 66.45 | 14.19 | 6.62 | |
| Standard RAGBackbone=MedGemma-4B2026.05 | 66.33 | 28 | 13.56 | |
| LongLLMLingua + BAIRBackbone=CheXagent-8B2026.05 | 66.24 | 16.64 | 8.36 | |
| Standard RAG + BAIRBackbone=CheXagent-8B2026.05 | 66.17 | 12.38 | 6.04 | |
| MAD-RAG + BAIRBackbone=CheXagent-8B2026.05 | 66.08 | 11.83 | 5.87 | |
| LongLLMLingua + BAIRBackbone=MedGemma-4B2026.05 | 65.91 | 25.21 | 12.46 | |
| Visual-focus Instr. + BAIRBackbone=MedGemma-4B2026.05 | 65.69 | 24.48 | 12.33 | |
| LongLLMLinguaBackbone=MedGemma-4B2026.05 | 65.66 | 25.14 | 12.79 | |
| Visual-focus Instr. + BAIRBackbone=CheXagent-8B2026.05 | 65.58 | 12.89 | 7.19 | |
| BaselineBackbone=CheXagent-8B2026.05 | 65.51 | — | — | |
| Visual-focus Instr.Backbone=MedGemma-4B2026.05 | 65.38 | 26.68 | 14.13 | |
| LongLLMLinguaBackbone=CheXagent-8B2026.05 | 65.33 | 23.63 | 13.63 | |
| Visual-focus Instr.Backbone=CheXagent-8B2026.05 | 64.94 | 15.89 | 9.81 | |
| MAD-RAGBackbone=CheXagent-8B2026.05 | 64.79 | 16.16 | 10.19 | |
| Standard RAGBackbone=CheXagent-8B2026.05 | 64.56 | 15.88 | 10.36 | |
| BaselineBackbone=MedGemma-4B2026.05 | 63.76 | — | — |