Open-Ended Medical Visual Chat on LLaVA-Med unseen 50 image-caption pairs 1.0 (evaluation set)
69.73Conversation ScoreBioMed-VITAL
Evaluation Results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| BioMed-VITALData Size=N2024.06 | 69.73 | 65.51 | 59.22 | 57.39 | 67.15 | 99.26 | 63.63 | 68.63 | |
| BioMed-VITALData Size=Top 80% * N2024.06 | 68.5 | 67.65 | 55.24 | 58.73 | 62.65 | 101.88 | 67.05 | 68.28 | |
| BioMed-VITAL-A1Data Size=N, Ablation=Only clinician-selected demonstrations2024.06 | 67.82 | 59.48 | 59.68 | 53.98 | 60.34 | 97.89 | 60.74 | 65.66 | |
| BioMed-VITAL-A2Data Size=N, Ablation=Only preference distillation2024.06 | 67.53 | 62.78 | 60.64 | 54.62 | 61.07 | 98.27 | 61.21 | 66.3 | |
| BioMed-VITALData Size=Top 50% * N2024.06 | 65.95 | 64.26 | 55.75 | 55.57 | 60.96 | 94.06 | 64.7 | 65.51 | |
| BioMed-VITAL-A0Data Size=N, Ablation=No clinician preference alignment2024.06 | 65.38 | 60.63 | 63.48 | 53.82 | 57.32 | 92.3 | 58.16 | 64.15 | |
| BioMed-VITALData Size=Top 10% * N2024.06 | 64.11 | 60.05 | 56.35 | 52.57 | 59.02 | 87.6 | 62.82 | 63.06 | |
| LLaVA-MedData Size=N2024.06 | 58.53 | 56.16 | 43.97 | 51.19 | 60.01 | 86.49 | 50.63 | 57.92 |