Chest X-ray Report Generation on MIMIC-CXR (test)
45.8F1 Macro (14)CheXagent-2 + DPO
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| CheXagent-2 + DPOJudge=GREEN2024.10 | 45.8 | 59.7 | 56 | 64.1 | 56.4 | — | — | — | |
| CheXagent-22024.10 | 44.6 | 57.8 | 55.5 | 62.4 | 55.1 | — | — | — | |
| CheXagent + KTOJudge=BERTScore2024.10 | 44 | 58 | 54 | 62.3 | 54.6 | — | — | — | |
| CheXagent + IPOJudge=BERTScore2024.10 | 43.1 | 57 | 53.6 | 61.5 | 53.8 | — | — | — | |
| CheXagent-2 + DPOJudge=BERTScore2024.10 | 43 | 59.7 | 53.5 | 63.8 | 55 | — | — | — | |
| CheXagent + DPOJudge=BERTScore2024.10 | 42.5 | 56.1 | 54.7 | 61.6 | 53.7 | — | — | — | |
| CheXagent + ORPOJudge=BERTScore2024.10 | 42.4 | 56.7 | 52.5 | 60.7 | 53.1 | — | — | — | |
| CheXagent + LC-DPOJudge=BERTScore2024.10 | 42.3 | 56.3 | 52.7 | 60.8 | 53 | — | — | — | |
| MAIRA-22024.10 | 41.6 | 58.1 | 50.4 | 59.1 | 52.3 | — | — | — | |
| CheXagent + DPOJudge=GREEN2024.10 | 41.5 | 54.1 | 51.8 | 58.3 | 51.4 | — | — | — | |
| CheXagent + ORPOJudge=GREEN2024.10 | 41.4 | 55 | 51.5 | 58.3 | 51.5 | — | — | — | |
| CheXagent + KTOJudge=GREEN2024.10 | 40.7 | 55 | 51.6 | 59.3 | 51.7 | — | — | — | |
| Med-PaLM M (84B)Model size=84B2024.10 | 39.8 | 53.6 | 51.6 | 57.9 | 50.7 | — | — | — | |
| LLaVA-Rad2024.10 | 39.5 | 57.3 | 47.7 | 57.4 | 50.5 | — | — | — | |
| CheXagent + IPOJudge=GREEN2024.10 | 39 | 52.9 | 48.9 | 56.5 | 49.3 | — | — | — | |
| CheXagent2024.10 | 38.9 | 50.9 | 47.6 | 54.1 | 47.9 | — | — | — | |
| MAIRA-12024.10 | 38.6 | 55.7 | 47.7 | 56 | 49.5 | — | — | — | |
| Med-PaLM M (562B)Model size=562B2024.10 | 37.8 | 51.6 | 49.9 | 56.3 | 48.9 | — | — | — | |
| CheXagent + LC-DPOJudge=GREEN2024.10 | 37.6 | 52.1 | 47.5 | 55.2 | 48.1 | — | — | — | |
| Med-PaLM M (12B)Model size=12B2024.10 | 37.3 | 51.4 | 50.6 | 56.5 | 49 | — | — | — | |
| GPT-4V2024.10 | 20.4 | 35.5 | 19.6 | 25.8 | 25.3 | — | — | — | |
| AdaAttYear=2017, Ratio of Pairs=100%2021.11 | — | — | — | — | — | 26.8 | 18.6 | 18.1 | |
| Att2inYear=2017, Ratio of Pairs=100%2021.11 | — | — | — | — | — | 32.2 | 23.9 | 24.9 | |
| KGAERatio of Pairs=0%2021.11 | — | — | — | — | — | 21.4 | 15.8 | 15.6 | |
| KGAE-SemiRatio of Pairs=60%2021.11 | — | — | — | — | — | 36 | 30.2 | 30.7 | |
| KGAE-SupervisedRatio of Pairs=100%2021.11 | — | — | — | — | — | 38.9 | 36.2 | 35.5 | |
| M² Trans.Year=2020, Ratio of Pairs=100%2021.11 | — | — | — | — | — | 19.7 | 14.5 | 13.3 | |
| NICYear=2015, Ratio of Pairs=100%2021.11 | — | — | — | — | — | 24.9 | 20.3 | 20.4 | |
| R2GenYear=2020, Ratio of Pairs=100%2021.11 | — | — | — | — | — | 33.3 | 27.3 | 27.6 | |
| TransformerYear=2020, Ratio of Pairs=100%2021.11 | — | — | — | — | — | 33.1 | 22.4 | 22.8 | |
| Up-DownYear=2018, Ratio of Pairs=100%2021.11 | — | — | — | — | — | 32 | 23.1 | 23.8 |