Cultural Visual Question Answering on CVQA (eval)
5.52Accuracy DeltaConAct
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| ConActVLM=Qwen2.5-VL-7B, Eval. Setting=Self-Deactivation2025.10 | 5.52 | 12.61 | |
| ConActVLM=Qwen2.5-VL-7B, Eval. Setting=Self–Cross Gap2025.10 | 4.88 | 8.36 | |
| MADVLM=Qwen2.5-VL-7B, Eval. Setting=Self-Deactivation2025.10 | 4.64 | 12.03 | |
| LAPEVLM=LLaVA-v1.6-Mistral-7B, Eval. Setting=Cross-Deactivation Avg.2025.10 | 4.44 | 11.28 | |
| LAPEVLM=LLaVA-v1.6-Mistral-7B, Eval. Setting=Self-Deactivation2025.10 | 4.43 | 11.44 | |
| ConActVLM=Pangea-7B, Eval. Setting=Self-Deactivation2025.10 | 4.33 | 12.99 | |
| MADVLM=Pangea-7B, Eval. Setting=Self-Deactivation2025.10 | 4.2 | 13.55 | |
| ConActVLM=Pangea-7B, Eval. Setting=Self–Cross Gap2025.10 | 3.61 | 5.65 | |
| MADVLM=Qwen2.5-VL-7B, Eval. Setting=Self–Cross Gap2025.10 | 3.33 | 6.07 | |
| MADVLM=Pangea-7B, Eval. Setting=Self–Cross Gap2025.10 | 2.86 | 4.75 | |
| LAPVLM=LLaVA-v1.6-Mistral-7B, Eval. Setting=Cross-Deactivation Avg.2025.10 | 2.74 | 17.74 | |
| LAPVLM=LLaVA-v1.6-Mistral-7B, Eval. Setting=Self-Deactivation2025.10 | 2.5 | 17.82 | |
| MADVLM=LLaVA-v1.6-Mistral-7B, Eval. Setting=Self-Deactivation2025.10 | 1.46 | 7.74 | |
| ConActVLM=LLaVA-v1.6-Mistral-7B, Eval. Setting=Self-Deactivation2025.10 | 1.39 | 9.58 | |
| MADVLM=Pangea-7B, Eval. Setting=Cross-Deactivation Avg.2025.10 | 1.34 | 8.8 | |
| MADVLM=Qwen2.5-VL-7B, Eval. Setting=Cross-Deactivation Avg.2025.10 | 1.31 | 5.96 | |
| LAPVLM=Qwen2.5-VL-7B, Eval. Setting=Cross-Deactivation Avg.2025.10 | 1.07 | 17.21 | |
| RNDVLM=Pangea-7B, Eval. Setting=Self-Deactivation2025.10 | 1.02 | 6.45 | |
| LAPVLM=Pangea-7B, Eval. Setting=Self-Deactivation2025.10 | 1 | 24.1 | |
| LAPVLM=Qwen2.5-VL-7B, Eval. Setting=Self-Deactivation2025.10 | 0.96 | 17.05 | |
| MADVLM=LLaVA-v1.6-Mistral-7B, Eval. Setting=Self–Cross Gap2025.10 | 0.93 | 1.18 | |
| LAPVLM=Pangea-7B, Eval. Setting=Cross-Deactivation Avg.2025.10 | 0.89 | 23.82 | |
| ConActVLM=LLaVA-v1.6-Mistral-7B, Eval. Setting=Self–Cross Gap2025.10 | 0.76 | 1.95 | |
| LAPEVLM=Pangea-7B, Eval. Setting=Self-Deactivation2025.10 | 0.74 | 6.52 | |
| ConActVLM=Pangea-7B, Eval. Setting=Cross-Deactivation Avg.2025.10 | 0.72 | 7.34 | |
| ConActVLM=Qwen2.5-VL-7B, Eval. Setting=Cross-Deactivation Avg.2025.10 | 0.64 | 4.25 | |
| ConActVLM=LLaVA-v1.6-Mistral-7B, Eval. Setting=Cross-Deactivation Avg.2025.10 | 0.63 | 7.63 | |
| LAPEVLM=Qwen2.5-VL-7B, Eval. Setting=Cross-Deactivation Avg.2025.10 | 0.61 | 4.12 | |
| LAPEVLM=Qwen2.5-VL-7B, Eval. Setting=Self-Deactivation2025.10 | 0.56 | 4.64 | |
| MADVLM=LLaVA-v1.6-Mistral-7B, Eval. Setting=Cross-Deactivation Avg.2025.10 | 0.53 | 6.56 | |
| RNDVLM=LLaVA-v1.6-Mistral-7B, Eval. Setting=Self-Deactivation2025.10 | 0.5 | 7.01 | |
| LAPEVLM=Pangea-7B, Eval. Setting=Self–Cross Gap2025.10 | 0.38 | 0.34 | |
| LAPEVLM=Pangea-7B, Eval. Setting=Cross-Deactivation Avg.2025.10 | 0.37 | 6.18 | |
| LAPVLM=LLaVA-v1.6-Mistral-7B, Eval. Setting=Self–Cross Gap2025.10 | 0.24 | 0.08 | |
| RNDVLM=Qwen2.5-VL-7B, Eval. Setting=Self-Deactivation2025.10 | 0.19 | 4.66 | |
| LAPVLM=Pangea-7B, Eval. Setting=Self–Cross Gap2025.10 | 0.11 | 0.28 | |
| LAPVLM=Qwen2.5-VL-7B, Eval. Setting=Self–Cross Gap2025.10 | 0.08 | 0.16 | |
| LAPEVLM=Qwen2.5-VL-7B, Eval. Setting=Self–Cross Gap2025.10 | 0.05 | 0.52 | |
| LAPEVLM=LLaVA-v1.6-Mistral-7B, Eval. Setting=Self–Cross Gap2025.10 | 0.01 | 0.15 |