Safety Evaluation on VLSafe Orig.
0.23Unsafe RateVLGuard PH
Evaluation Results
| Method | Links | |
|---|---|---|
| VLGuard PHBackbone=LLaVA-v1.5-7B2024.10 | 0.23 | |
| VLGuard PHBackbone=LLaVA-v1.5-13B2024.10 | 0.56 | |
| LLaVA-v1.5-13BInput Protocol=Query2024.10 | 0.68 | |
| CMRM_sampleBackbone=LLaVA-v1.5-13B2024.10 | 0.79 | |
| VLGuard MixedBackbone=LLaVA-v1.5-13B2024.10 | 1.13 | |
| LLaVA-v1.5-13BInput Protocol=Caption2024.10 | 1.25 | |
| CMRM_sampleBackbone=ShareGPT4V2024.10 | 1.46 | |
| LLaVA-v1.5-7BInput Protocol=Query2024.10 | 1.91 | |
| CMRM_datasetBackbone=ShareGPT4V2024.10 | 1.91 | |
| VLGuard MixedBackbone=LLaVA-v1.5-7B2024.10 | 2.03 | |
| CMRM_sampleBackbone=LLaVA-v1.5-7B2024.10 | 3.15 | |
| ShareGPT4VInput Protocol=Query2024.10 | 4.13 | |
| CMRM_datasetBackbone=LLaVA-v1.5-13B2024.10 | 4.95 | |
| ShareGPT4VInput Protocol=Caption2024.10 | 5.32 | |
| CMRM_datasetBackbone=LLaVA-v1.5-7B2024.10 | 5.41 | |
| LLaVA-v1.5-7BInput Protocol=Caption2024.10 | 5.52 | |
| LLaVA-v1.5-13BInput Protocol=Visual (Orig.)2024.10 | 31.8 | |
| ShareGPT4VInput Protocol=Visual (Orig.)2024.10 | 57.09 | |
| LLaVA-v1.5-7BInput Protocol=Visual (Orig.)2024.10 | 61.53 |