Visual Question Answering on TallyQA
94.09AccuracyDirect Fine-tuning (r=1)
Evaluation Results
| Method | Links | |
|---|---|---|
| Direct Fine-tuning (r=1)Optimization steps=+0 steps2026.02 | 94.09 | |
| Direct Fine-tuning (r=1)Optimization steps=+700 steps2026.02 | 94.06 | |
| In-Squeeze (128 ... -> 1)Optimization mode=Min steps2026.02 | 94.03 | |
| Cont-Squeeze (128 -> 1)Optimization steps=+700 steps2026.02 | 93.78 | |
| In-Squeeze (128 ... -> 1)Optimization mode=Standard2026.02 | 93.75 | |
| Cont-Squeeze (128 -> 1)Optimization steps=+200 steps2026.02 | 93.56 | |
| Direct Fine-tuning (r=1)Optimization steps=+200 steps2026.02 | 93.53 | |
| Single Task SoTA2023.12 | 90.7 | |
| UIO-2XLModel Size=3.2B2023.12 | 87.4 | |
| UIO-2XXLModel Size=6.8B2023.12 | 86.2 | |
| VisRefModel=InternVL3.5-8B2026.02 | 84.5 | |
| AutoMoT2026.03 | 81.4 | |
| OpenEMMA2026.03 | 80 | |
| TSRModel=InternVL3.5-8B2026.02 | 79.6 | |
| STModel=InternVL3.5-8B2026.02 | 79.4 | |
| Gemma 3 4B ITZero-shot=true2026.02 | 78.9 | |
| VisRefModel=Qwen3-VL-8B2026.02 | 78.9 | |
| UIO-2LModel Size=1.1B2023.12 | 78.6 | |
| PaliGemmaParameters=3B2025.12 | 76.8 | |
| TSRModel=Qwen3-VL-8B2026.02 | 75.1 | |
| STModel=Qwen3-VL-8B2026.02 | 74.3 | |
| VisRefModel=SAIL-VL2-8B2026.02 | 73.9 | |
| Cont-Squeeze (128 -> 1)Optimization steps=+0 steps2026.02 | 72.5 | |
| LLaVa-1.5Model Size=13B, Evaluation Mode=Open-source release2023.12 | 72.4 | |
| LLaVA-1.5LLM=Vicuna-13B2025.12 | 72.3 | |
| PaLIParameters=17B2025.12 | 71.9 | |
| TSRModel=SAIL-VL2-8B2026.02 | 71.7 | |
| VL-JEPASFTParameters=1.6B2025.12 | 69.9 | |
| ReCogDrive2026.03 | 69.6 | |
| STModel=SAIL-VL2-8B2026.02 | 69.3 | |
| InstructBLIPModel Size=8.2B, Evaluation Mode=Open-source release2023.12 | 68.2 | |
| InstructBLIPLLM=Vicuna-13B2025.12 | 68 | |
| PaLIParameters=3B2025.12 | 65.8 | |
| SmolVLMParameters=2B2025.12 | 64.7 | |
| Robotron-Drivereproduced=true2026.03 | 63.4 | |
| PaLIParameters=700M2025.12 | 62.3 | |
| VALORBase Model=Qwen3-8B2025.12 | 51 | |
| VALOR-RLBase Model=Qwen3-8B2025.12 | 49.5 | |
| ViGoRLBase Model=Qwen2.5-VL-7B2025.12 | 49.3 | |
| GRITBase Model=Qwen2.5-VL-3B2025.12 | 46.4 | |
| OracleData Source Type=Real Data Reweighting, Sampling Strategy Category=Gold, Compute Budget=280K datapoints2026.03 | 45.2 | |
| D.P.Data Source Type=Real Data Reweighting, Sampling Strategy Category=Ours, Compute Budget=280K datapoints2026.03 | 45.1 | |
| SmolVLMParameters=500M2025.12 | 44.8 | |
| ICONSData Source Type=Real Data Reweighting, Sampling Strategy Category=Baseline, Compute Budget=280K datapoints2026.03 | 42.8 | |
| UniformData Source Type=Real Data Reweighting, Sampling Strategy Category=Baseline, Compute Budget=280K datapoints2026.03 | 41.8 | |
| D.P.Data Source Type=Synthetic Data Selection, Sampling Strategy Category=Ours, Compute Budget=280K datapoints2026.03 | 36.2 | |
| ICONSData Source Type=Synthetic Data Selection, Sampling Strategy Category=Baseline, Compute Budget=280K datapoints2026.03 | 35.5 | |
| SmolVLMParameters=256M2025.12 | 32.3 | |
| UniformData Source Type=Synthetic Data Selection, Sampling Strategy Category=Baseline, Compute Budget=280K datapoints2026.03 | 30.4 |