List your company
ResearchFormerly WizWand

Visual Question Answering on TallyQA

94.09AccuracyDirect Fine-tuning (r=1)
27.852445.048762.24579.4413Dec 28, 2023May 11, 2024Sep 24, 2024Feb 6, 2025Jun 22, 2025Nov 4, 2025Mar 20, 2026
Updated 7mo ago

Evaluation results

MethodLinks
Direct Fine-tuning (r=1)Optimization steps=+0 steps2026.0294.09
Direct Fine-tuning (r=1)Optimization steps=+700 steps2026.0294.06
In-Squeeze (128 ... -> 1)Optimization mode=Min steps2026.0294.03
Cont-Squeeze (128 -> 1)Optimization steps=+700 steps2026.0293.78
In-Squeeze (128 ... -> 1)Optimization mode=Standard2026.0293.75
Cont-Squeeze (128 -> 1)Optimization steps=+200 steps2026.0293.56
Direct Fine-tuning (r=1)Optimization steps=+200 steps2026.0293.53
Single Task SoTA2023.1290.7
UIO-2XLModel Size=3.2B2023.1287.4
UIO-2XXLModel Size=6.8B2023.1286.2
VisRefModel=InternVL3.5-8B2026.0284.5
AutoMoT2026.0381.4
OpenEMMA2026.0380
TSRModel=InternVL3.5-8B2026.0279.6
STModel=InternVL3.5-8B2026.0279.4
Gemma 3 4B ITZero-shot=true2026.0278.9
VisRefModel=Qwen3-VL-8B2026.0278.9
UIO-2LModel Size=1.1B2023.1278.6
PaliGemmaParameters=3B2025.1276.8
TSRModel=Qwen3-VL-8B2026.0275.1
STModel=Qwen3-VL-8B2026.0274.3
VisRefModel=SAIL-VL2-8B2026.0273.9
Cont-Squeeze (128 -> 1)Optimization steps=+0 steps2026.0272.5
LLaVa-1.5Model Size=13B, Evaluation Mode=Open-source release2023.1272.4
LLaVA-1.5LLM=Vicuna-13B2025.1272.3
PaLIParameters=17B2025.1271.9
TSRModel=SAIL-VL2-8B2026.0271.7
VL-JEPASFTParameters=1.6B2025.1269.9
ReCogDrive2026.0369.6
STModel=SAIL-VL2-8B2026.0269.3
InstructBLIPModel Size=8.2B, Evaluation Mode=Open-source release2023.1268.2
InstructBLIPLLM=Vicuna-13B2025.1268
PaLIParameters=3B2025.1265.8
SmolVLMParameters=2B2025.1264.7
Robotron-Drivereproduced=true2026.0363.4
PaLIParameters=700M2025.1262.3
VALORBase Model=Qwen3-8B2025.1251
VALOR-RLBase Model=Qwen3-8B2025.1249.5
ViGoRLBase Model=Qwen2.5-VL-7B2025.1249.3
GRITBase Model=Qwen2.5-VL-3B2025.1246.4
OracleData Source Type=Real Data Reweighting, Sampling Strategy Category=Gold, Compute Budget=280K datapoints2026.0345.2
D.P.Data Source Type=Real Data Reweighting, Sampling Strategy Category=Ours, Compute Budget=280K datapoints2026.0345.1
SmolVLMParameters=500M2025.1244.8
ICONSData Source Type=Real Data Reweighting, Sampling Strategy Category=Baseline, Compute Budget=280K datapoints2026.0342.8
UniformData Source Type=Real Data Reweighting, Sampling Strategy Category=Baseline, Compute Budget=280K datapoints2026.0341.8
D.P.Data Source Type=Synthetic Data Selection, Sampling Strategy Category=Ours, Compute Budget=280K datapoints2026.0336.2
ICONSData Source Type=Synthetic Data Selection, Sampling Strategy Category=Baseline, Compute Budget=280K datapoints2026.0335.5
SmolVLMParameters=256M2025.1232.3
UniformData Source Type=Synthetic Data Selection, Sampling Strategy Category=Baseline, Compute Budget=280K datapoints2026.0330.4