Counterfactual Reasoning on CVQA
71.41AccuracyBlind-VLM
Evaluation results
| Method | Links | |
|---|---|---|
| Blind-VLMLLM=Phi2, Evaluation Mode=Boolean2026.02 | 71.41 | |
| StableLSDLLM=Phi2, Evaluation Mode=Boolean2026.02 | 70.44 | |
| SPOTLLM=Phi2, Evaluation Mode=Boolean2026.02 | 69.47 | |
| DINOSAURLLM=Phi2, Evaluation Mode=Boolean2026.02 | 69.29 | |
| FT-DINOSAURLLM=Phi2, Evaluation Mode=Boolean2026.02 | 68.85 | |
| Slot DiffusionLLM=Phi2, Evaluation Mode=Boolean2026.02 | 68.23 | |
| mFRESALLM=Phi2, Evaluation Mode=Boolean2026.02 | 66.64 | |
| VQDINOMLPLLM=Phi2, Evaluation Mode=Boolean2026.02 | 66.37 | |
| DINOSAURv2LLM=Phi2, Evaluation Mode=Boolean2026.02 | 65.75 | |
| Blind-VLMLLM=Qwen2-7B, Evaluation Mode=Boolean2026.02 | 65.57 | |
| DINOSAURv2LLM=Qwen2-7B, Evaluation Mode=Boolean2026.02 | 64.07 | |
| DINOv2LLM=Phi2, Evaluation Mode=Boolean2026.02 | 63.72 | |
| DINOSAURLLM=Qwen2-7B, Evaluation Mode=Boolean2026.02 | 63.72 | |
| StableLSDLLM=Qwen2-7B, Evaluation Mode=Boolean2026.02 | 62.39 | |
| mFRESALLM=Qwen2-7B, Evaluation Mode=Boolean2026.02 | 60.44 | |
| Slot DiffusionLLM=Qwen2-7B, Evaluation Mode=Boolean2026.02 | 59.56 | |
| SPOTLLM=Qwen2-7B, Evaluation Mode=Boolean2026.02 | 57.61 | |
| FT-DINOSAURLLM=Qwen2-7B, Evaluation Mode=Boolean2026.02 | 57.17 | |
| VQDINOMLPLLM=Qwen2-7B, Evaluation Mode=Boolean2026.02 | 55.66 | |
| DINOv2LLM=Qwen2-7B, Evaluation Mode=Boolean2026.02 | 53.54 | |
| DINOv2LLM=Qwen2-7B, Evaluation Mode=Direct2026.02 | 45.74 | |
| VQDINOMLPLLM=Qwen2-7B, Evaluation Mode=Direct2026.02 | 42.34 | |
| DINOSAURv2LLM=Qwen2-7B, Evaluation Mode=Direct2026.02 | 42.09 | |
| FT-DINOSAURLLM=Qwen2-7B, Evaluation Mode=Direct2026.02 | 42 | |
| SPOTLLM=Qwen2-7B, Evaluation Mode=Direct2026.02 | 41.83 | |
| mFRESALLM=Qwen2-7B, Evaluation Mode=Direct2026.02 | 41.39 | |
| DINOSAURLLM=Qwen2-7B, Evaluation Mode=Direct2026.02 | 41.13 | |
| StableLSDLLM=Qwen2-7B, Evaluation Mode=Direct2026.02 | 41.04 | |
| Slot DiffusionLLM=Qwen2-7B, Evaluation Mode=Direct2026.02 | 39.39 | |
| FT-DINOSAURLLM=Phi2, Evaluation Mode=Direct2026.02 | 39.13 | |
| StableLSDLLM=Phi2, Evaluation Mode=Direct2026.02 | 38.26 | |
| mFRESALLM=Phi2, Evaluation Mode=Direct2026.02 | 38.09 | |
| DINOv2LLM=Phi2, Evaluation Mode=Direct2026.02 | 36.96 | |
| SPOTLLM=Phi2, Evaluation Mode=Direct2026.02 | 36.35 | |
| DINOSAURLLM=Phi2, Evaluation Mode=Direct2026.02 | 35.74 | |
| VQDINOMLPLLM=Phi2, Evaluation Mode=Direct2026.02 | 35.13 | |
| DINOSAURv2LLM=Phi2, Evaluation Mode=Direct2026.02 | 34.52 | |
| Slot DiffusionLLM=Phi2, Evaluation Mode=Direct2026.02 | 33.83 | |
| Blind-VLMLLM=Qwen2-7B, Evaluation Mode=Direct2026.02 | 32.69 | |
| Blind-VLMLLM=Phi2, Evaluation Mode=Direct2026.02 | 28 |