List your company
ResearchFormerly WizWand

Visual Reasoning on NLVR2 (test)

85.15AccuracySimVLM_HUGE
56.695664.082871.4778.8572Nov 3, 2021Jul 29, 2022Apr 24, 2023Jan 18, 2024Oct 12, 2024Jul 8, 2025Apr 3, 2026
Updated 6mo ago

Evaluation results

MethodLinks
SimVLM_HUGEPre-training volume=1.8B, Pre-training scale=>10M2021.1185.15
VinVL_LARGEPre-training scale=<10M2021.1183.98
X-VLMclip2022.1083.48
MADTPReduce Ratio=0.3, GFLOPS=92.60, Backbone=BLIP2024.0383.2
ALBEFPre-training volume=14M, Pre-training scale=>10M2021.1183.14
UncompressedReduce Ratio=/, GFLOPS=132.54, Backbone=BLIP2024.0383.08
METERBackbone=CLIP-ViT-BASE, Pre-training scale=<10M2021.1183.05
MADTPReduce Ratio=0.5, GFLOPS=66.16, Backbone=BLIP2024.0382.85
METERBackbone=Swin-BASE, Pre-training scale=<10M2021.1182.47
MADTPReduce Ratio=0.6, GFLOPS=52.92, Backbone=BLIP2024.0382.42
X-VLMclipperformance_target=98%2022.1081.81
SimVLM_BASEPre-training volume=1.8B, Pre-training scale=>10M2021.1181.77
EfficientVLM2022.1081.72
VILLA_LARGEPre-training scale=<10M2021.1181.47
MADTPReduce Ratio=0.7, GFLOPS=39.69, Backbone=BLIP2024.0381.23
EVPGFramework=VisProg, Supervision Type=Supervised, Backbone=BLIP_lavis2025.1281.2
UPopReduce Ratio=0.3, GFLOPS=89.36, Backbone=BLIP2024.0381.13
ALBEFPre-training volume=4M, Pre-training scale=<10M2021.1180.5
EVPGFramework=VisProg, Supervision Type=Supervised, Backbone=InternVL1.52025.1280.5
STPReduce Ratio=0.3, GFLOPS=94.08, Backbone=BLIP2024.0380.01
UNITER_LARGEPre-training scale=<10M2021.1179.98
CoMPGFLOPs=26.33±0.382026.0479.67
X-VLMsmall2022.1079.26
MADTPReduce Ratio=0.8, GFLOPS=26.46, Backbone=BLIP2024.0379.22
OSCARB2022.1078.36
InternVL1.5Framework=VisProg, Supervision Type=Zero-shot (baseline)2025.1278.2
Visual ParsingPre-training scale=<10M2021.1178.05
MADTPGFLOPs=26.77±0.232026.0477.64
STPReduce Ratio=0.5, GFLOPS=68.31, Backbone=BLIP2024.0377.61
UPopReduce Ratio=0.5, GFLOPS=65.29, Backbone=BLIP2024.0377.61
OSCARBperformance_target=98%2022.1076.79
VILT-NLVRFinetuned=true, Runs=12022.1176.3
ViLTPre-training scale=<10M2021.1176.13
ViLT2022.1076.1
SDVPFramework=VisProg, Supervision Type=Supervised, Backbone=BLIP_hg2025.1275.2
DistilDualEnc2022.1074.3
MiniVLM2022.1073.93
BLIP_lavisFramework=VisProg, Supervision Type=Zero-shot (baseline)2025.1273.6
UPopReduce Ratio=0.6, GFLOPS=50.35, Backbone=BLIP2024.0373.55
PixelBERTPre-training scale=<10M2021.1172.2
UPopReduce Ratio=0.7, GFLOPS=39.93, Backbone=BLIP2024.0368.76
BLIP_hgFramework=VisProg, Supervision Type=Zero-shot (baseline)2025.1268.3
VISPROGPrompting strategy=voting, Finetuned=false, Runs=5, Context examples per run=162022.1162.4
VISPROGPrompting strategy=curated, Finetuned=false, Runs=1, Context examples per run=122022.1161.8
VISPROGPrompting strategy=random, Finetuned=false, Runs=1, Context examples per run=162022.1161.3
UPopReduce Ratio=0.8, GFLOPS=19.08, Backbone=BLIP2024.0357.79