Icon Question Answering on IconQA (test)
95.69Accuracy (Img)Human
Evaluation results
| Method | Links | |||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Human2021.10 | 95.69 | 93.91 | 93.56 | 94.63 | 97.63 | 94.41 | 93.31 | 92.73 | 95.66 | 97.94 | 97.45 | 87.51 | 96.29 | 86.55 | 97.06 | 85.67 | — | |
| Patch-TRM2021.10 | 82.66 | 75.19 | 83.62 | 81.87 | 77.81 | 87 | 55.62 | 62.39 | 68.75 | 77.98 | 82.13 | 98.24 | 56.73 | 97.98 | 92.49 | 95.73 | — | |
| ViLT2021.10 | 79.67 | 72.69 | 79.27 | 82.61 | 71.13 | 84.95 | 53.38 | 66.72 | 59.22 | 69.99 | 75.81 | 99.02 | 50.55 | 98.91 | 86.1 | 87.65 | — | |
| ViT2021.10 | 79.15 | 72.34 | 78.92 | 82.6 | 70.84 | 82.12 | 54.64 | 68.8 | 58.46 | 68.66 | 77.41 | 98.95 | 51.1 | 98.76 | 84.72 | 86.07 | — | |
| UNITER2021.10 | 78.71 | 72.39 | 78.53 | 81.31 | 71.01 | 83.67 | 48.34 | 61.25 | 60.81 | 69.77 | 78.37 | 99.41 | 49.18 | 99.38 | 86.1 | 87.84 | — | |
| DFAF2021.10 | 77.72 | 72.17 | 78.28 | 81.8 | 70.68 | 81.69 | 51.42 | 67.01 | 56.6 | 67.72 | 77.6 | 99.02 | 50.27 | 98.83 | 84.11 | 85.7 | — | |
| MCAN2021.10 | 77.36 | 71.25 | 74.52 | 79.86 | 68.94 | 82.73 | 49.7 | 62.49 | 54.79 | 68 | 76.2 | 99.08 | 47.32 | 98.99 | 83.25 | 84.87 | — | |
| ViLBERT2021.10 | 76.66 | 70.47 | 77.08 | 80.05 | 71.05 | 75.6 | 49.46 | 58.52 | 62.78 | 66.72 | 74.09 | 99.22 | 50.62 | 99.07 | 81.78 | 70.94 | — | |
| BAN2021.10 | 76.33 | 70.82 | 75.54 | 79.99 | 67.56 | 82.12 | 53.2 | 66.92 | 55.67 | 66.5 | 73.77 | 97.06 | 47.46 | 96.5 | 82.12 | 82.45 | — | |
| Top-Down2021.10 | 75.92 | 68.51 | 73.03 | 80.07 | 65.01 | 80.65 | 45.78 | 58.22 | 55.01 | 68.28 | 72.43 | 99.54 | 50 | 99.46 | 84.54 | 83.75 | — | |
| Random2021.10 | 41.7 | 36.87 | 0.29 | 30.3 | 18.38 | 41.2 | 36.49 | 34.25 | 34.81 | 35.82 | 34.84 | 3.62 | 11.12 | 0.36 | 45.16 | 38.81 | — | |
| Q-Only2021.10 | 41.64 | 36.86 | 28.45 | 38.03 | 33.63 | 48.19 | 37.14 | 35.37 | 33.66 | 48.09 | 33.06 | 40.46 | 28.02 | 38.07 | 45.25 | 40.76 | — | |
| I-Only2021.10 | 41.56 | 36.02 | 46.65 | 38.71 | 37.64 | 45.26 | 37.52 | 35.47 | 36.29 | 47.37 | 32.48 | 62.29 | 31.73 | 64.02 | 45.25 | 37.51 | — | |
| LoRAModel=Qwen2.5-VL-3B, Optimization=Base2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 78.9 | |
| LoRAModel=Qwen2.5-VL-3B, Optimization=SAM2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 79.7 | |
| LoRAModel=Qwen2.5-VL-3B, Optimization=GAM2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 80.8 | |
| LoRAModel=Qwen2.5-VL-3B, Optimization=FlatPO2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 81.6 | |
| Prefix TuningModel=Qwen2.5-VL-3B, Optimization=Base2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 75.5 | |
| Prefix TuningModel=Qwen2.5-VL-3B, Optimization=SAM2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 76.5 | |
| Prefix TuningModel=Qwen2.5-VL-3B, Optimization=GAM2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 77.8 | |
| Prefix TuningModel=Qwen2.5-VL-3B, Optimization=FlatPO2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 78 |