Visual Commonsense Reasoning (Q->A) on VCR sampled 5000 (val)
0.84AccuracyMerlotReserve
Evaluation Results
| Method | Links | |
|---|---|---|
| MerlotReserveEvaluation Protocol=Supervised2023.05 | 0.84 | |
| VisualBERTEvaluation Protocol=Supervised2023.05 | 0.708 | |
| R2CEvaluation Protocol=Supervised2023.05 | 0.638 | |
| IdealGPTEvaluation Protocol=Zero-Shot, Validation Sample Size=5000, Answerer=BLIP2-FlanT5-XL2023.05 | 0.507 | |
| MiniGPT4Evaluation Protocol=Zero-Shot, Validation Sample Size=50002023.05 | 0.406 | |
| LLaVAEvaluation Protocol=Zero-Shot, Validation Sample Size=50002023.05 | 0.283 | |
| Random Guess2023.05 | 0.25 |