Safety Evaluation on PKU-SafeRLHF-V
77.8AccuracyDebate about images
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| Debate about imagesModel=GPT-4o2025.11 | 77.8 | 0.56 | 8 | |
| Debate with imagesModel=Claude-Sonnet-42025.11 | 76.8 | 0.53 | 12 | |
| Debate about imagesModel=Qwen2.5-VL-72B2025.11 | 76.3 | 0.53 | 13 | |
| Debate with imagesModel=GPT-4o2025.11 | 76.2 | 0.55 | 14 | |
| Debate with imagesModel=Gemini-2.5-Pro2025.11 | 74.9 | 0.5 | 25 | |
| Debate with imagesModel=Qwen2.5-VL-72B2025.11 | 74.8 | 0.5 | 22 | |
| CoT promptModel=Qwen2.5-VL-72B2025.11 | 74.7 | 0.49 | 15 | |
| Debate about imagesModel=Claude-Sonnet-42025.11 | 74.5 | 0.58 | 13 | |
| Direct promptModel=GPT-4o2025.11 | 74.4 | 0.5 | 11 | |
| Majority voteModel=Qwen2.5-VL-72B2025.11 | 73.9 | 0.48 | 13 | |
| Majority voteModel=Claude-Sonnet-42025.11 | 73.3 | 0.47 | 12 | |
| CoT promptModel=Claude-Sonnet-42025.11 | 72.9 | 0.46 | 12 | |
| Majority voteModel=GPT-4o2025.11 | 72.3 | 0.49 | 10 | |
| Debate about imagesModel=Gemini-2.5-Pro2025.11 | 72.2 | 0.45 | 26 | |
| Direct promptModel=Claude-Sonnet-42025.11 | 72.1 | 0.44 | 13 | |
| Direct promptModel=Qwen2.5-VL-72B2025.11 | 72 | 0.44 | 15 | |
| CoT promptModel=GPT-4o2025.11 | 70.1 | 0.45 | 13 | |
| Majority voteModel=Gemini-2.5-Pro2025.11 | 69.9 | 0.44 | 13 | |
| CoT promptModel=Gemini-2.5-Pro2025.11 | 68.4 | 0.41 | 16 | |
| Direct promptModel=Gemini-2.5-Pro2025.11 | 68 | 0.37 | 25 |