List your company
ResearchFormerly WizWand

Natural Language Visual Reasoning on NLVR2 (dev)

91.51AccuracyBEIT-3
81.858884.364486.8789.3756Jan 2, 2021Sep 24, 2021Jun 17, 2022Mar 10, 2023Dec 1, 2023Aug 23, 2024May 16, 2025
Updated 7mo ago

Evaluation results

MethodLinks
BEIT-32022.0891.51
BEIT-32023.0591.5
BEiT-3# Params=1.9B2023.0191.5
VLMO-Large++# Pretrain Images=1.0B2021.1188.62
ONE-PEACE2023.0587.8
X-FMbase# Params=327M, Training Data=More Data2023.0187.6
X-FMbase# Params=327M2023.0186.3
X2-VLMbase# Params=255M, Training Data=More Data2023.0186.2
CoCaInput Modality=image only, Evaluation Protocol=lightweight finetuning2022.0586.1
CoCa2022.0886.1
CoCa2023.0586.1
CoCa#Images=3B, Para.=2.1B2022.1286.1
CoCa# Params=2.1B2023.0186.1
X2-VLMbaseBackbone=ResNet-50, # Params=255M2023.0185.9
VLMO-Large# Pretrain Images=4M, Model Size=Large2021.1185.64
VLMo-LPretrain Images=4M2022.0685.64
VLMo2022.0585.64
VLMoInput Modality=image only, Evaluation Protocol=lightweight finetuning2022.0585.6
X-VLMData=16M+, EPIC=true2022.1185.2
METERData=16M+, EPIC=true2022.1185
X-VLMData=4M+, EPIC=true2022.1184.6
FIBER-BPretrain Images=4M2022.0684.59
mPLUG2022.0584.58
PTP-BLIP#Images=14M, Para.=220M2022.1284.55
SimVLM_HUGEPre-training volume=1.8B, Pre-training scale=>10M2021.1184.53
SimVLMModel Size=Huge2021.0884.53
SimVLM-Huge# Pretrain Images=1.8B2021.1184.53
SimVLM2022.0884.53
SimVLM-HPretrain Images=1.8B2022.0684.53
SimVLMInput Modality=image only, Evaluation Protocol=lightweight finetuning2022.0584.5
SimVLM2023.0584.5
X-VLMPre-training Images Scale=16M2021.1184.41
X-VLMPretrain Images=4M2022.0684.41
X-VLM2023.0584.41
X-VLMData=16M+, EPIC=false2022.1184.3
SimVLM-BASE# Pre-train Images=1.8B, Model scale=Base2023.0584.2
X-VLM# Params=216M2023.0184.2
X-VLMPre-training Images Scale=4M2021.1184.16
SimVLMModel Size=Large2021.0884.13
SimVLM-Large# Pretrain Images=1.8B2021.1184.13
SimVLM-LARGE# Pre-train Images=1.8B, Model scale=Large2023.0584.13
SimVLMSize=Large2022.0584.13
ALBEFData=16M+, EPIC=true2022.1184.1
SCLPre-training image scale=<10M images2022.1183.63
METERData=4M+, EPIC=true2022.1183.5
MAPPre-training dataset=<10M images, Model size=Base size2022.1083.3
X-VLMData=4M+, EPIC=false2022.1183.3
ManagerTower-BASE# Pre-train Images=4M, Model scale=Base2023.0582.81
VLMbase# Params=175M2023.0182.8
VLMO-Base# Pretrain Images=4M, Model Size=Base2021.1182.77
VLMo-BPretrain Images=4M2022.0682.77
VLMoPre-training image scale=<10M images2022.1182.77
VLMo-BASE# Pre-train Images=4M, Model scale=Base2023.0582.77
VLMo-BasePre-training dataset=<10M images, Model size=Base size2022.1082.77
VinVLInput Modality=image only, Evaluation Protocol=lightweight finetuning2022.0582.7
METERData=16M+, EPIC=false2022.1182.7
OSCAR+ w/ VinVLModel Scale=large2021.0182.67
BLIPPre-train #Images=14M2022.0182.67
VinVL_LARGEPre-training scale=<10M2021.1182.67
VinVLModel Size=Large2021.0882.67
VinVL-Large# Pretrain Images=5.7M2021.1182.67
VinVL_largePre-training Images Scale=5.6M2021.1182.67
VinVL2022.0882.67
VinVL_L#Images=5.6M, Para.=347M2022.1282.67
BLIP#Images=14M, Para.=220M2022.1282.67
BLIP2022.0582.67
BLIPPre-train # Images=14M2023.1282.67
CCLM_basePre-training Data=4M2022.0682.66
MAFAPre-train # Images=4M, Pre-train Dataset=4M-Clean2023.1282.66
FALCON#Images=4M2025.0582.61
ALBEFInput Modality=image only, Evaluation Protocol=lightweight finetuning2022.0582.6
ALBEFData=16M+, EPIC=false2022.1182.6
ALBEFPre-training Images=14M2021.0782.55
ALBEFPre-train #Images=14M2022.0182.55
ALBEFPre-training volume=14M, Pre-training scale=>10M2021.1182.55
ALBEFPre-training Images Scale=14M2021.1182.55
ALBEF2022.0882.55
ALBEF(14M)Pre-training image scale=>10M images2022.1182.55
ALBEF-BASE (14M)# Pre-train Images=14M, Model scale=Base2023.0582.55
ALBEF2023.0582.55
ALBEF#Images=14M, Para.=210M2022.1282.55
ALBEF2022.0582.55
SimVLM-BasePre-training dataset=>10M images, Model size=Base size2022.1082.55
ALBEFPre-train # Images=14M2023.1282.55
MAFAPre-train # Images=4M, Pre-train Dataset=4M-Noisy2023.1282.52
MAFA#Images=4M2025.0582.52
BLIPbase# Params=240M2023.0182.5
BLIPPre-train #Images=129M2022.0182.48
BLIP-BASE# Pre-train Images=129M, Model scale=Base2023.0582.48
METERBackbone=CLIP-ViT-BASE, Pre-training scale=<10M2021.1182.33
METER-CLIP2021.1182.33
METERPre-training image scale=<10M images2022.1182.33
METER-CLIP-BASE# Pre-train Images=4M, Model scale=Base2023.0582.33
METER2022.0582.33
METERPre-training dataset=<10M images, Model size=Base size2022.1082.33
METER# Params=341M2023.0182.3
METERBackbone=Swin-BASE, Pre-training scale=<10M2021.1182.23
METER-Swin2021.1182.23
METER-Swin-BPretrain Images=4M2022.0682.23
METER-Swin-BASE# Pre-train Images=4M, Model scale=Base2023.0582.23