Safety Evaluation on VSE-HH
86.7DSR (w/ vision)Gemini-2.5-Pro*
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Gemini-2.5-Pro*Frames=162025.05 | 86.7 | — | — | 99.5 | 12.8 | 1.6 | |
| Qwen2.5 VL-72B*Frames=162025.05 | 78.2 | — | — | 90 | 13.1 | 3.8 | |
| InternVL3-78B*Frames=82025.05 | 74.4 | — | — | 88.7 | 16.1 | 2.9 | |
| GPT-4o*Frames=162025.05 | 73 | — | — | 98.4 | 25.9 | 2.2 | |
| VideoLLaMA2-7BFrames=162025.05 | 69.8 | — | — | 75.3 | 7.3 | 2.3 | |
| Qwen2.5 VL-32B*Frames=162025.05 | 67.9 | — | — | 77.8 | 12.8 | 3.6 | |
| InternVL3-8B*Frames=162025.05 | 64.4 | — | — | 75.1 | 14.3 | 1.7 | |
| VideoChat2-Mistral-7BFrames=162025.05 | 62.1 | — | — | 71.7 | 13.5 | 2.8 | |
| Qwen2.5 VL-7BFrames=1fps2025.05 | 57.3 | — | — | 72.1 | 20.5 | 3.4 | |
| Qwen2.5 VL-3BFrames=1fps2025.05 | 54 | — | — | 77.1 | 30 | 2.4 | |
| VideoChat2-Phi3-3.8BFrames=162025.05 | 50.5 | — | — | 72.2 | 30 | 3.2 | |
| LLaVA-OV-7BFrames=162025.05 | 39.9 | — | — | 50.3 | 20.7 | 2.4 | |
| VideoLLaMA3-7BFrames=1fps2025.05 | 31.2 | — | — | 88.2 | 64.6 | 2.5 | |
| VideoGPT+-5BFrames=162025.05 | 30.7 | — | — | 71.6 | 57.1 | 2.1 | |
| PLLaVA-7BFrames=162025.05 | 26.8 | — | — | 58.3 | 54.1 | 3.2 | |
| LLaVA-Next-Video-7BFrames=162025.05 | 26.1 | — | — | 44.2 | 41 | 3.4 | |
| InternVL3-2B*Frames=162025.05 | 24 | — | — | 35.1 | 31.7 | 1.3 | |
| VideoChat-Flash-7BFrames=1fps2025.05 | 19 | — | — | 57.5 | 66.9 | 1.8 | |
| VideoLLaMA3-2BFrames=1fps2025.05 | 18.4 | — | — | 89.3 | 79.4 | 2.3 | |
| InternVideo2.5-8BFrames=1fps2025.05 | 16.5 | — | — | 53.5 | 69.2 | 1 | |
| VideoChat-Flash-2BFrames=1fps2025.05 | 8.5 | — | — | 13.6 | 38 | 1.8 | |
| MM-RLHFArch.=Qwen2.5 VL 7B2025.05 | — | 72.5 | 3.8 | — | — | — | |
| Qwen2.5 VLArch.=Qwen2.5 VL-3B2025.05 | — | 54 | 2.4 | — | — | — | |
| Qwen2.5 VLArch.=Qwen2.5 VL-7B2025.05 | — | 57.3 | 3.6 | — | — | — | |
| Self-ReminderArch.=Qwen2.5 VL-7B2025.05 | — | 65.2 | 4.4 | — | — | — | |
| SPA-VLArch.=Qwen2.5 VL 7B2025.05 | — | 86.9 | 2.3 | — | — | — | |
| VideoLLaMA3Arch.=VideoLLaMA3-2B2025.05 | — | 18.4 | 2.5 | — | — | — | |
| VideoLLaMA3Arch.=VideoLLaMA3-7B2025.05 | — | 31.2 | 2.5 | — | — | — | |
| VideoSafety-R1Arch.=Qwen2.5VL-3B2025.05 | — | 91.7 | 3.6 | — | — | — | |
| VideoSafety-R1Arch.=Qwen2.5VL-7B2025.05 | — | 95.4 | 3.4 | — | — | — | |
| VideoSafety-R1Arch.=VideoLLaMA-2B2025.05 | — | 89.5 | 4.6 | — | — | — | |
| VLGuardArch.=Qwen2.5 VL 7B2025.05 | — | 100 | 2 | — | — | — |