Anomaly Detection on Traffic Scene Anomaly Detection (balanced split of 1000 examples)
85AccuracyGemini 2.5 Pro
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Gemini 2.5 ProInput Type=image, Layered analysis=✗, Optimization=✗, Number of Queries=1, Model Category=Proprietary, Resolution=360p2025.10 | 85 | 94 | 70 | 80 | 24.9 | 938 | |
| GPT-5Input Type=image, Layered analysis=✗, Optimization=✗, Number of Queries=1, Model Category=Proprietary, Resolution=360p2025.10 | 83 | 88 | 77 | 82 | 28.7 | 51,851 | |
| Gemini 2.5 FlashInput Type=image, Layered analysis=✗, Optimization=✗, Number of Queries=1, Model Category=Proprietary, Resolution=360p2025.10 | 81 | 91 | 69 | 78 | 10.6 | 947 | |
| Gemini 1.5 FlashInput Type=image, Layered analysis=✗, Optimization=✗, Number of Queries=1, Model Category=Proprietary, Resolution=360p2025.10 | 81 | 90 | 70 | 79 | 2.9 | 958 | |
| Mistral Medium 3Input Type=image, Layered analysis=✗, Optimization=✗, Number of Queries=1, Model Category=Proprietary, Resolution=360p2025.10 | 81 | 94 | 66 | 78 | 6.4 | 1,158 | |
| Mistral Medium 3.1Input Type=image, Layered analysis=✗, Optimization=✗, Number of Queries=1, Model Category=Proprietary, Resolution=360p2025.10 | 80 | 80 | 80 | 80 | 7.9 | 1,259 | |
| Qwen-VL MaxInput Type=image, Layered analysis=✗, Optimization=✗, Number of Queries=1, Model Category=Proprietary, Resolution=360p2025.10 | 79 | 87 | 68 | 76 | 7.7 | 2,193 | |
| GPT-4oInput Type=image, Layered analysis=✗, Optimization=✗, Number of Queries=1, Model Category=Proprietary, Resolution=360p2025.10 | 78 | 94 | 60 | 73 | 7.4 | 1,032 | |
| Gemini 2.5 Pro PrevInput Type=image, Layered analysis=✗, Optimization=✗, Number of Queries=1, Model Category=Proprietary, Resolution=360p2025.10 | 77 | 87 | 64 | 74 | 23.6 | 898 | |
| Gemini 2.0 ThinkingInput Type=image, Layered analysis=✗, Optimization=✗, Number of Queries=1, Model Category=Proprietary, Resolution=360p2025.10 | 75 | 90 | 56 | 69 | 8.1 | 937 | |
| Gemini 2.5 Flash PrevInput Type=image, Layered analysis=✗, Optimization=✗, Number of Queries=1, Model Category=Proprietary, Resolution=360p2025.10 | 75 | 90 | 56 | 69 | 11.2 | 956 | |
| Qwen2.5-VL 72BInput Type=image, Layered analysis=✗, Optimization=✗, Number of Queries=1, Model Category=Open, Resolution=360p2025.10 | 75 | 82 | 64 | 72 | 9.4 | 961 | |
| Claude 3.5 SonnetInput Type=image, Layered analysis=✗, Optimization=✗, Number of Queries=1, Model Category=Proprietary, Resolution=360p2025.10 | 74 | 79 | 66 | 72 | 7.3 | 1,109 | |
| Mistral Small 3.1Input Type=image, Layered analysis=✗, Optimization=✗, Number of Queries=1, Model Category=Open, Resolution=360p2025.10 | 74 | 71 | 82 | 76 | 11.9 | 1,057 | |
| Gemini 2.0 Flash ExpInput Type=image, Layered analysis=✗, Optimization=✗, Number of Queries=1, Model Category=Proprietary, Resolution=360p2025.10 | 73 | 80 | 62 | 70 | 2.6 | 2,441 | |
| GPT-4 TurboInput Type=image, Layered analysis=✗, Optimization=✗, Number of Queries=1, Model Category=Proprietary, Resolution=360p2025.10 | 73 | 96 | 48 | 64 | 7.6 | 1,055 | |
| Claude 3.5 HaikuInput Type=image, Layered analysis=✗, Optimization=✗, Number of Queries=1, Model Category=Proprietary, Resolution=360p2025.10 | 72 | 89 | 50 | 64 | 7 | 1,144 | |
| Claude Sonnet 4Input Type=image, Layered analysis=✗, Optimization=✗, Number of Queries=1, Model Category=Proprietary, Resolution=360p2025.10 | 70 | 83 | 50 | 63 | 9.8 | 1,145 | |
| Mistral Small 3.2Input Type=image, Layered analysis=✗, Optimization=✗, Number of Queries=1, Model Category=Open, Resolution=360p2025.10 | 70 | 78 | 56 | 65 | 5.5 | 996 | |
| Claude Opus 4.1Input Type=image, Layered analysis=✗, Optimization=✗, Number of Queries=1, Model Category=Proprietary, Resolution=360p2025.10 | 68 | 95 | 38 | 54 | 19.4 | 1,238 | |
| Pixtral Large 2411Input Type=image, Layered analysis=✗, Optimization=✗, Number of Queries=1, Model Category=Open, Resolution=360p2025.10 | 67 | 64 | 78 | 70 | 5.8 | 1,817 | |
| GPT-4.1 MiniInput Type=image, Layered analysis=✗, Optimization=✗, Number of Queries=1, Model Category=Proprietary, Resolution=360p2025.10 | 66 | 94 | 34 | 50 | 5.4 | 1,075 | |
| Claude 3.7 SonnetInput Type=image, Layered analysis=✗, Optimization=✗, Number of Queries=1, Model Category=Proprietary, Resolution=360p2025.10 | 65 | 80 | 40 | 53 | 10.8 | 1,276 | |
| GPT-4o MiniInput Type=image, Layered analysis=✗, Optimization=✗, Number of Queries=1, Model Category=Proprietary, Resolution=360p2025.10 | 65 | 90 | 34 | 49 | 4.4 | 14,787 | |
| Pixtral 12BInput Type=image, Layered analysis=✗, Optimization=✗, Number of Queries=1, Model Category=Open, Resolution=360p2025.10 | 64 | 100 | 28 | 44 | 5.4 | 1,637 | |
| Claude Opus 4Input Type=image, Layered analysis=✗, Optimization=✗, Number of Queries=1, Model Category=Proprietary, Resolution=360p2025.10 | 62 | 83 | 31 | 45 | 13.1 | 1,224 | |
| Qwen2.5-VL 32BInput Type=image, Layered analysis=✗, Optimization=✗, Number of Queries=1, Model Category=Open, Resolution=360p2025.10 | 62 | 65 | 53 | 59 | 19.9 | 2,380 | |
| Qwen-VL PlusInput Type=image, Layered analysis=✗, Optimization=✗, Number of Queries=1, Model Category=Proprietary, Resolution=360p2025.10 | 58 | 90 | 18 | 30 | 2.8 | 2,155 | |
| LLaVA 1.5 7BInput Type=image, Layered analysis=✗, Optimization=✗, Number of Queries=1, Model Category=Open, Resolution=360p2025.10 | 58 | 79 | 22 | 34 | 6.7 | 1,029 | |
| GPT-4.1 NanoInput Type=image, Layered analysis=✗, Optimization=✗, Number of Queries=1, Model Category=Proprietary, Resolution=360p2025.10 | 56 | 80 | 16 | 27 | 4.3 | 1,262 | |
| Qwen2.5-VL 7BInput Type=image, Layered analysis=✗, Optimization=✗, Number of Queries=1, Model Category=Open, Resolution=360p2025.10 | 55 | 55 | 48 | 52 | 4.2 | 686 | |
| Qwen2.5-VL 3BInput Type=image, Layered analysis=✗, Optimization=✗, Number of Queries=1, Model Category=Open, Resolution=360p2025.10 | 50 | 50 | 29 | 37 | 3.6 | 1,489 | |
| Gemma3 12BInput Type=image, Layered analysis=✗, Optimization=✗, Number of Queries=1, Model Category=Open, Resolution=360p2025.10 | 44 | 46 | 70 | 56 | 9.2 | 1,254 |