Visual Question Answering on VQAv2 (test-std)
82.3AccuracyCoCa
Evaluation Results
| Method | Links | |
|---|---|---|
| CoCaVocabulary=Closed2022.05 | 82.3 | |
| FlamingoVocabulary=Open, Parameters=80B2022.05 | 82.1 | |
| GIT2Vocabulary=Open, Parameters=5.1B2022.05 | 81.92 | |
| mPlugVocabulary=Closed2022.05 | 81.26 | |
| FlorenceVocabulary=Closed2022.05 | 80.36 | |
| SimVLMVocabulary=Closed2022.05 | 80.34 | |
| OFAVocabulary=Closed2022.05 | 80.02 | |
| GITVocabulary=Open, Parameters=0.7B2022.05 | 78.81 | |
| BLIPVocabulary=Closed2022.05 | 78.32 | |
| METERVocabulary=Closed2022.05 | 77.64 | |
| UncompressedModel=BLIP, Pruning Mode=/, Pruning Ratio=/, GFLOPs=186.12026.04 | 77.5 | |
| UncompressedModel=BLIP, Pruning Mode=/, Pruning Ratio=/, GFLOPs=186.12026.04 | 77.5 | |
| BLIPEvaluation protocol=Full fine-tuning, # Tunable=337M2025.12 | 77.48 | |
| Null-LoRAEvaluation protocol=Frozen backbone, # Tunable=9.5M2025.12 | 77.48 | |
| Ours‡Params. (M)=13.6, Mem. (G)=14.92026.04 | 76.91 | |
| Ours†Params. (M)=10.9, Mem. (G)=12.62026.04 | 76.87 | |
| Fully-FTParams. (M)=236.8, Mem. (G)=82.02026.04 | 76.86 | |
| UFOVocabulary=Closed2022.05 | 76.76 | |
| CLIP-ViLVocabulary=Closed2022.05 | 76.7 | |
| VinVLVocabulary=Closed2022.05 | 76.6 | |
| CoMPModel=BLIP, Pruning Mode=C, Pruning Ratio=0.7, GFLOPs=59.72026.04 | 76.5 | |
| CoMPModel=BLIP, Pruning Mode=C, Pruning Ratio=0.7, GFLOPs=59.72026.04 | 76.5 | |
| MADTPModel=BLIP, Pruning Mode=T, Pruning Ratio=0.7, GFLOPs=61.62026.04 | 76.2 | |
| MADTPModel=BLIP, Pruning Mode=T, Pruning Ratio=0.7, GFLOPs=61.62026.04 | 76.2 | |
| DoRAEvaluation protocol=Frozen backbone, # Tunable=10.8M, rank (r)=322025.12 | 76.17 | |
| ALBEFEvaluation protocol=Full fine-tuning, # Tunable=266M2025.12 | 76.04 | |
| ALBEFVocabulary=Closed2022.05 | 76.04 | |
| SHERLParams. (M)=13.0, Mem. (G)=14.02026.04 | 75.82 | |
| UniAdapterEvaluation protocol=Frozen backbone, # Tunable=19.5M, rank (r)=5122025.12 | 75.56 | |
| UniPTParams. (M)=10.3, Mem. (G)=11.62026.04 | 75.53 | |
| LSTParams. (M)=13.4, Mem. (G)=25.62026.04 | 75.44 | |
| UNIMOVocabulary=Closed2022.05 | 75.27 | |
| LoRAEvaluation protocol=Frozen backbone, # Tunable=10.6M, rank (r)=322025.12 | 75.2 | |
| VILLAVocabulary=Closed2022.05 | 74.87 | |
| UPopModel=BLIP, Pruning Mode=P, Pruning Ratio=0.7, GFLOPs=62.32026.04 | 74.6 | |
| UPopModel=BLIP, Pruning Mode=P, Pruning Ratio=0.7, GFLOPs=62.32026.04 | 74.6 | |
| PixelBERTVocabulary=Closed2022.05 | 74.55 | |
| Visual ParsingVocabulary=Closed2022.05 | 74.17 | |
| UNITEREvaluation protocol=Full fine-tuning, # Tunable=330M2025.12 | 74.03 | |
| UNITERVocabulary=Closed2022.05 | 74.02 | |
| OSCAREvaluation protocol=Full fine-tuning, # Tunable=330M2025.12 | 73.82 | |
| OSCARVocabulary=Closed2022.05 | 73.82 | |
| SOHOEvaluation protocol=Full fine-tuning, # Tunable=155M2025.12 | 73.47 | |
| VL-T5/BARTEvaluation protocol=Full fine-tuning, # Tunable=165M2025.12 | 71.3 |