Clock Reading on COCO, OpenImages, and Clock Movies Averaged (test)
78.9Overall AccuracySpecialized single-task model
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Specialized single-task modelModel Type=specialized clock reading model2024.09 | 78.9 | 84.2 | 82.9 | |
| Molmo-7B-D2024.09 | 68.2 | 78.6 | 76 | |
| MolmoE-1B2024.09 | 65.8 | 77.9 | 74.1 | |
| Molmo-72B2024.09 | 65.6 | 77.1 | 73.7 | |
| Molmo-7B-O2024.09 | 64.2 | 76.3 | 73.8 | |
| Qwen2-VL-72BAccess=open-weight2024.09 | 9.1 | 24.9 | 18.4 | |
| Claude 3.5 SonnetAccess=API-only2024.09 | 6.6 | 22.3 | 17.5 | |
| PaliGemma-mix-3BAccess=open-weight2024.09 | 6.1 | 21 | 15.8 | |
| LLaVA OneVision-72BAccess=open-weight and open-data2024.09 | 5.7 | 17.9 | 15.4 | |
| Llama-3.2V-90B-InstructAccess=open-weight2024.09 | 3.4 | 17.9 | 10.1 | |
| InternVL2-Llama-3-76BAccess=open-weight2024.09 | 3.3 | 16.3 | 9.9 | |
| GPT-4o-0513Access=API-only2024.09 | 2.7 | 14.2 | 8.6 | |
| xGen-MM-interleave-4BAccess=open-weight and open-data2024.09 | 2 | 11.9 | 8 | |
| Phi3.5-Vision-4BAccess=open-weight2024.09 | 1.9 | 12 | 7.6 | |
| Cambrian-1-34BAccess=open-weight and open-data2024.09 | 1.8 | 11.1 | 7.2 | |
| Pixtral-12BAccess=open-weight2024.09 | 1.7 | 11.9 | 6.7 | |
| Gemini 1.5 ProAccess=API-only2024.09 | 0.9 | 11.6 | 5.1 | |
| LLaVA-1.5-13BAccess=open-weight and open-data2024.09 | 0.8 | 11.6 | 5.7 |