Medical Image Report Labeling on MIMIC-CXR (test)
45.8Macro F1 (14 Labels)CheXagent-2 + DPO
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| CheXagent-2 + DPOVariant=12024.10 | 45.8 | 59.7 | 56 | 64.1 | 56.4 | |
| CheXagent-22024.10 | 44.6 | 57.8 | 55.5 | 62.4 | 55.1 | |
| CheXagent + KTOJudge=BERTScore2024.10 | 44 | 58 | 54 | 62.3 | 54.6 | |
| CheXagent + IPOJudge=BERTScore2024.10 | 43.1 | 57 | 53.6 | 61.5 | 53.8 | |
| CheXagent-2 + DPOVariant=22024.10 | 43 | 59.7 | 53.5 | 63.8 | 55 | |
| CheXagent + DPOJudge=BERTScore2024.10 | 42.5 | 56.1 | 54.7 | 61.6 | 53.7 | |
| CheXagent + ORPOJudge=BERTScore2024.10 | 42.4 | 56.7 | 52.5 | 60.7 | 53.1 | |
| CheXagent + LC-DPOJudge=BERTScore2024.10 | 42.3 | 56.3 | 52.7 | 60.8 | 53 | |
| MAIRA-22024.10 | 41.6 | 58.1 | 50.4 | 59.1 | 52.3 | |
| CheXagent + DPOJudge=GREEN2024.10 | 41.5 | 54.1 | 51.8 | 58.3 | 51.4 | |
| CheXagent + ORPOJudge=GREEN2024.10 | 41.4 | 55 | 51.5 | 58.3 | 51.5 | |
| CheXagent + KTOJudge=GREEN2024.10 | 40.7 | 55 | 51.6 | 59.3 | 51.7 | |
| Med-PaLM MParameters=84B2024.10 | 39.8 | 53.6 | 51.6 | 57.9 | 50.7 | |
| LLaVA-Rad2024.10 | 39.5 | 57.3 | 47.7 | 57.4 | 50.5 | |
| CheXagent + IPOJudge=GREEN2024.10 | 39 | 52.9 | 48.9 | 56.5 | 49.3 | |
| CheXagentAlignment=None (Baseline)2024.10 | 38.9 | 50.9 | 47.6 | 54.1 | 47.9 | |
| MAIRA-12024.10 | 38.6 | 55.7 | 47.7 | 56 | 49.5 | |
| Med-PaLM MParameters=562B2024.10 | 37.8 | 51.6 | 49.9 | 56.3 | 48.9 | |
| CheXagent + LC-DPOJudge=GREEN2024.10 | 37.6 | 52.1 | 47.5 | 55.2 | 48.1 | |
| Med-PaLM MParameters=12B2024.10 | 37.3 | 51.4 | 50.6 | 56.5 | 49 | |
| GPT-4V2024.10 | 20.4 | 35.5 | 19.6 | 25.8 | 25.3 |