Safety Evaluation on RTVLM
7.89Racial Bias ScoreSilkieRT
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| SilkieRTBase Model=Qwen-VL-Chat, DPO Training Subset=VLFeedback Red Teaming Subset2024.10 | 7.89 | 7.24 | 8.31 | 5.31 | 7.19 | |
| Qwen-VL-Chat2024.10 | 6.38 | 6.89 | 7.44 | 2.14 | 5.71 | |
| HA-DPOBase Model=LLaVA-v1.52024.10 | 6.29 | 6.57 | 7.58 | 7.72 | 7.04 | |
| LLaVA-v1.52024.10 | 6.03 | 7.03 | 7.07 | 7.14 | 6.82 | |
| POVIDBase Model=LLaVA-v1.52024.10 | 5.56 | 6.25 | 8.21 | 7.95 | 6.99 | |
| LLaVA-SFT2024.10 | 5.51 | 6.67 | 7.98 | 4.86 | 6.26 | |
| LLaVA-RLHFBase Model=LLaVA-SFT2024.10 | 5.41 | 6.56 | 5.61 | 3.54 | 5.28 |