Short-answer Question Answering on HotPotQA (EM, F1)
59.6F1AGAR
Evaluation results
| Method | Links | ||
|---|---|---|---|
| AGAR2026.06 | 59.6 | — | |
| CGR2026.06 | 57.1 | — | |
| Random2026.06 | 56.9 | — | |
| Plain VQA2026.06 | 56.4 | — | |
| Uniform2026.06 | 53.4 | — | |
| VAR2026.06 | 52.9 | — | |
| VEA-Br2026.06 | 51 | — | |
| VEA-Con2026.06 | 50.5 | — | |
| AGLA2026.06 | 49.2 | — | |
| AGARBackbone=GLM-4.1V-9B-Thinking2026.06 | 48 | — | |
| AGLABackbone=GLM-4.1V-9B-Thinking2026.06 | 43.2 | — | |
| UniformBackbone=GLM-4.1V-9B-Thinking2026.06 | 41.4 | — | |
| RandomBackbone=GLM-4.1V-9B-Thinking2026.06 | 41 | — | |
| Plain VQABackbone=GLM-4.1V-9B-Thinking2026.06 | 40.9 | — | |
| VEA-ConBackbone=GLM-4.1V-9B-Thinking2026.06 | 40.7 | — | |
| APCDBackbone=Llama-2-13b-chat2026.05 | 39.2 | 40.8 | |
| VEA-BrBackbone=GLM-4.1V-9B-Thinking2026.06 | 39.2 | — | |
| ADBackbone=Llama-2-13b-chat2026.05 | 37 | 39.2 | |
| CGRBackbone=GLM-4.1V-9B-Thinking2026.06 | 36.9 | — | |
| VARBackbone=GLM-4.1V-9B-Thinking2026.06 | 36.9 | — | |
| DoLaBackbone=Llama-2-13b-chat2026.05 | 36.8 | 39.6 | |
| DoLa+ADBackbone=Llama-2-13b-chat2026.05 | 36.8 | 38.8 | |
| APCDBackbone=Llama-3-8b-it2026.05 | 33.3 | 35.2 | |
| Llama-2-13b-chatDecoding Strategy=Greedy2026.05 | 32.6 | 37.8 | |
| APCDBackbone=Llama-2-7b-chat2026.05 | 29.4 | 28.4 | |
| DoLaBackbone=Llama-3-8b-it2026.05 | 28.8 | 33.6 | |
| ADBackbone=Llama-3-8b-it2026.05 | 28.8 | 33.4 | |
| DoLa+ADBackbone=Llama-3-8b-it2026.05 | 28.6 | 33.3 | |
| Llama-3-8b-itDecoding Strategy=Greedy2026.05 | 28.3 | 33.1 | |
| CADBackbone=Llama-3-8b-it2026.05 | 26.5 | 30.1 | |
| DoLaBackbone=Llama-2-7b-chat2026.05 | 25.4 | 27.4 | |
| ITIBackbone=Llama-2-13b-chat2026.05 | 24.4 | 24.8 | |
| DoLa+ADBackbone=Llama-2-7b-chat2026.05 | 24.3 | 26.3 | |
| ADBackbone=Llama-2-7b-chat2026.05 | 24.2 | 26.1 | |
| CADBackbone=Llama-2-13b-chat2026.05 | 23.7 | 26.6 | |
| Llama-2-7b-chatDecoding Strategy=Greedy2026.05 | 23.4 | 25.5 | |
| ITIBackbone=Llama-2-7b-chat2026.05 | 22.6 | 25.8 | |
| ITIBackbone=Llama-3-8b-it2026.05 | 20.6 | 21.7 | |
| CADBackbone=Llama-2-7b-chat2026.05 | 18.8 | 24.3 |