Multimodal Reasoning on MMT-Bench
57.88AccuracyJigsaw
Evaluation Results
| Method | Links | |
|---|---|---|
| JigsawPrompting=CoT prompting, Backbone=Qwen-VL-2.5 3B2025.12 | 57.88 | |
| MixPrompting=CoT prompting, Backbone=Qwen-VL-2.5 3B2025.12 | 57.72 | |
| Jigsaw-R1Prompting=CoT prompting, Backbone=Qwen-VL-2.5 3B2025.12 | 57.53 | |
| Qwen 2.5 VLPrompting=CoT prompting, Backbone=Qwen-VL-2.5 3B2025.12 | 53.31 | |
| LLaVA-NeXT+CALLLM=Vicuna-13B, Resolution setting=High resolution2024.05 | 52.4 | |
| VLM-R1Prompting=CoT prompting, Backbone=Qwen-VL-2.5 3B2025.12 | 52.06 | |
| MGM+CALLLM=Vicuna-13B, Resolution setting=Low resolution2024.05 | 51.9 | |
| MGM+CALLLM=Vicuna-7B, Resolution setting=Low resolution2024.05 | 51.4 | |
| MGM-HD+CALLLM=Vicuna-13B, Resolution setting=High resolution2024.05 | 51.4 | |
| LLaVA-1.5+CALLLM=Vicuna-13B, Resolution setting=Low resolution2024.05 | 51.2 | |
| LLaVA-1.5LLM=Vicuna-13B, Resolution setting=Low resolution2024.05 | 51.1 | |
| MGM-HDLLM=Vicuna-13B, Resolution setting=High resolution2024.05 | 50.9 | |
| LLaVA-NeXT+CALLLM=Vicuna-7B, Resolution setting=High resolution2024.05 | 50.7 | |
| MGM-HD+CALLLM=Vicuna-7B, Resolution setting=High resolution2024.05 | 50.5 | |
| LLaVA-NeXTLLM=Vicuna-13B, Resolution setting=High resolution2024.05 | 50.4 | |
| MGMLLM=Vicuna-7B, Resolution setting=Low resolution2024.05 | 50.3 | |
| LLaVA-NeXTLLM=Vicuna-7B, Resolution setting=High resolution2024.05 | 49.5 | |
| MGMLLM=Vicuna-13B, Resolution setting=Low resolution2024.05 | 49.1 | |
| LLaVA-1.5+CALLLM=Vicuna-7B, Resolution setting=Low resolution2024.05 | 48.8 | |
| LLaVA-1.5LLM=Vicuna-7B, Resolution setting=Low resolution2024.05 | 48.6 | |
| MGM-HDLLM=Vicuna-7B, Resolution setting=High resolution2024.05 | 47.9 | |
| MGM+CALLLM=Gemma-2B, Resolution setting=Low resolution2024.05 | 45.4 | |
| MGMLLM=Gemma-2B, Resolution setting=Low resolution2024.05 | 43.4 |