Natural Language Visual Reasoning on NLVR2
88.8AccuracyGPT-4V
Evaluation results
| Method | Links | ||
|---|---|---|---|
| GPT-4VParameter=-2026.04 | 88.8 | — | |
| Ours (masked) (LLaVA-OV-7B)Model Size=7B, Backbone=LLaVA-OV, Attention Masking=True2026.01 | 87.3 | — | |
| Idefics2Parameter=8B2026.04 | 86.9 | — | |
| LLaVA-OV-7BModel Size=7B2026.01 | 84.2 | — | |
| Qwen2.5-VL + S2H-DPOParameter=7B2026.04 | 74.67 | — | |
| Qwen2.5-VLParameter=7B2026.04 | 74.28 | — | |
| Qwen2.5-VL + MIA-DPOParameter=7B2026.04 | 74.18 | — | |
| Ours (LLaVA-OV-1.5B)Model Size=1.5B, Backbone=LLaVA-OV, Attention Masking=False2026.01 | 73.7 | — | |
| EWCCheckpoint=Task 2: NLVR2, Backbone=ViLT2026.03 | 72.85 | — | |
| EWC-DRCheckpoint=Task 2: NLVR2, Backbone=ViLT2026.03 | 72.77 | — | |
| LLaVA-OV-1.5BModel Size=1.5B2026.01 | 70.9 | — | |
| EWC-DRCheckpoint=Task 3: SNLI-VE, Backbone=ViLT2026.03 | 70.65 | 9.31 | |
| Ours (masked) (LLaVA-OV-1.5B)Model Size=1.5B, Backbone=LLaVA-OV, Attention Masking=True2026.01 | 69 | — | |
| Ours (LLaVA-OV-0.5B)Model Size=0.5B, Backbone=LLaVA-OV, Attention Masking=False2026.01 | 68 | — | |
| MMICL2023.09 | 66.6 | — | |
| EWC-DRCheckpoint=Task 4: VCR, Backbone=ViLT2026.03 | 66.51 | 27.48 | |
| Ours (masked) (LLaVA-OV-0.5B)Model Size=0.5B, Backbone=LLaVA-OV, Attention Masking=True2026.01 | 65.1 | — | |
| EWCCheckpoint=Task 3: SNLI-VE, Backbone=ViLT2026.03 | 64.95 | 34.59 | |
| LLaVA-OV-0.5BModel Size=0.5B2026.01 | 61.2 | — | |
| InstructBLIPParameter=13B2026.04 | 60.3 | — | |
| LLaVA-v1.6Parameter=7B2026.04 | 58.9 | — | |
| Qwen-VL-ChatParameter=7B2026.04 | 58.7 | — | |
| CogVLMParameter=17B2026.04 | 58.6 | — | |
| Emu2-ChatParameter=37B2026.04 | 58.2 | — | |
| VideoLLaVAParameter=7B2026.04 | 56.5 | — | |
| LLaVA-v1.5 + S2H-DPOParameter=7B2026.04 | 55.59 | — | |
| LLaVA-v1.5 + MIA-DPOParameter=7B2026.04 | 54.2 | — | |
| InstructionBlip2023.09 | 53.95 | — | |
| EWCCheckpoint=Task 4: VCR, Backbone=ViLT2026.03 | 52.13 | 90.66 | |
| LLaVA-v1.5Parameter=7B2026.04 | 52.1 | — | |
| LLaVA-v1.5 + LLaVA-RLHFParameter=7B2026.04 | 51.8 | — | |
| LLaVA-v1.5 + HA-DPOParameter=7B2026.04 | 51.6 | — | |
| FuyuParameter=8B2026.04 | 51.1 | — | |
| Qwen3-VL + S2H-DPOParameter=2B2026.04 | 50.61 | — | |
| Qwen3-VLParameter=2B2026.04 | 49.71 | — | |
| OTTER2023.09 | 47.2 | — | |
| Qwen2VL-2BModel Size=2B2026.01 | 41.6 | — | |
| Qwen2VL-7BModel Size=7B2026.01 | 41.5 | — | |
| LLaVA-v1.5 + POVIDParameter=7B2026.04 | 21.4 | — | |
| InternVL2-2BModel Size=2B2026.01 | 18.9 | — | |
| InternVL2-8BModel Size=8B2026.01 | 8.7 | — |