Visual Question Answering on Super-CLEVR (test)
87.4AccuracyAtt-CoT
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Att-CoTModel=QVL-7B2026.06 | 87.4 | 63.2 | 0.231 | 0.52 | -1.22 | |
| Att-CoTModel=QVL-3B2026.06 | 86.5 | 57.1 | 0.643 | 0.43 | -0.58 | |
| Att-CoTModel=Gemma-12B2026.06 | 86.3 | 62.9 | 0.347 | 0.32 | -1.64 | |
| SFTModel=QVL-7B2026.06 | 85.5 | 60.4 | 0.478 | 0.35 | 0.75 | |
| SFTModel=QVL-3B2026.06 | 84.2 | 54.8 | 0.897 | 0.41 | 1.45 | |
| LLaVA-RModel=QVL-7B2026.06 | 83.9 | 61.1 | 0.36 | 0.35 | — | |
| Att-CoTModel=IVL-8B2026.06 | 83.7 | 56.9 | 0.32 | 0.18 | -1.24 | |
| LLaVA-CoTModel=QVL-7B2026.06 | 83.2 | 58.9 | 0.412 | 0.31 | — | |
| Att-CoTModel=Gemma-4B2026.06 | 83.1 | 56.5 | 0.806 | 0.55 | 1.82 | |
| LLaVA-RModel=QVL-3B2026.06 | 82.8 | 54.5 | 0.812 | 0.4 | — | |
| SFTModel=IVL-8B2026.06 | 82.1 | 54.2 | 0.515 | -0.03 | 0.33 | |
| LLaVA-RModel=IVL-8B2026.06 | 82 | 54.6 | 0.464 | 0.08 | — | |
| SFTModel=Gemma-12B2026.06 | 81.8 | 60.6 | 0.502 | 0.11 | 3.58 | |
| LLaVA-RModel=Gemma-12B2026.06 | 81.4 | 61.2 | 0.438 | 0.22 | — | |
| LLaVA-CoTModel=IVL-8B2026.06 | 80.6 | 53.1 | 0.527 | -0.01 | — | |
| LLaVA-CoTModel=Gemma-12B2026.06 | 80.6 | 59.3 | 0.516 | 0.09 | — | |
| LLaVA-CoTModel=QVL-3B2026.06 | 80.2 | 52.9 | 0.876 | 0.38 | — | |
| SFTModel=Gemma-4B2026.06 | 79.8 | 55.1 | 0.882 | 0.45 | 5.59 | |
| LLaVA-RModel=Gemma-4B2026.06 | 78.7 | 55.2 | 0.842 | 0.4 | — | |
| LLaVA-CoTModel=Gemma-4B2026.06 | 76.4 | 54.1 | 0.819 | 0.36 | — | |
| Att-CoTModel=IVL-4B2026.06 | 74.5 | 54.7 | 0.754 | 0.71 | -0.2 | |
| SFTModel=IVL-4B2026.06 | 65.5 | 52.6 | 0.925 | 0.65 | 8.86 | |
| LLaVA-RModel=IVL-4B2026.06 | 64.9 | 52.8 | 0.796 | 0.62 | — | |
| LLaVA-CoTModel=IVL-4B2026.06 | 62.8 | 49.4 | 0.855 | 0.6 | — |