List your company
ResearchFormerly WizWand

Natural Language Understanding on AGIEval

71.6AccuracyLlama 3 405B
11.69627.24842.858.352Nov 7, 2023Apr 16, 2024Sep 24, 2024Mar 5, 2025Aug 13, 2025Jan 21, 2026Jul 2, 2026
Updated 4mo ago

Evaluation results

MethodLinks
Llama 3 405BModel Type=Pre-trained2024.0771.6——
Llama 3 70BModel Type=Pre-trained2024.0764.6——
Mixtral 8x22BModel Type=Pre-trained2024.0761.5——
RLVRBackbone Model=Qwen2.5VL-7B-Instruct2026.0758.153.437.7
C3RL w/o RefBackbone Model=Qwen2.5VL-7B-Instruct2026.0758.168.219.7
C3RLBackbone Model=Qwen2.5VL-7B-Instruct2026.0757.868.815.4
RLCRBackbone Model=Qwen2.5VL-7B-Instruct2026.0757.566.420.5
SCBackbone Model=Qwen2.5VL-7B-Instruct2026.0753.771.120.8
BaseBackbone Model=Qwen2.5VL-7B-Instruct2026.075154.337.5
SaySelfBackbone Model=Qwen2.5VL-7B-Instruct2026.0750.874.514.7
SFT+RefBackbone Model=Qwen2.5VL-7B-Instruct2026.0749.175.310.5
Qwen2.52025.1248.13——
Llama 3 8BModel Type=Pre-trained2024.0747.8——
Gemma 7BModel Type=Pre-trained2024.0746——
Mistral 7BModel Type=Pre-trained2024.0742.7——
Qwen32025.1242.14——
RLVRBackbone Model=Llama-3.2-3B-Instruct2026.0737.352.246.8
C3RLBackbone Model=Llama-3.2-3B-Instruct2026.0737.36624.6
C3RL w/o RefBackbone Model=Llama-3.2-3B-Instruct2026.0737.265.221.8
NITPModel Scale=3B2026.0535.11——
BaseBackbone Model=Llama-3.2-3B-Instruct2026.0734.855.742.3
RLCRBackbone Model=Llama-3.2-3B-Instruct2026.0734.465.811.1
Gamayun2025.1233.34——
mPLUG-Owl2Shot setting=0-shot2023.1132.7——
SFT+RefBackbone Model=Llama-3.2-3B-Instruct2026.0732.159.424
NTPModel Scale=3B2026.0531.77——
NITPModel Scale=2B2026.0531.66——
NTPModel Scale=2B2026.0530.51——
Arcanazero-shot=true2024.1029.3——
SCBackbone Model=Llama-3.2-3B-Instruct2026.0728.664.435.2
LLaMA-2-ChatShot setting=0-shot2023.1128.5——
LLaMA-2-Chatzero-shot=true2024.1028.5——
Gemma32025.1228.05——
DeepSeek-VLVersion=7B Chat, Encoder=SigLIP+SAM, Evaluation Protocol=Generation-based (greedy decoding)2024.0327.8——
NTPModel Scale=0.5B2026.0527.01——
Llama3.22025.1226.32——
NITPModel Scale=0.5B2026.0526.15——
WizardLMShot setting=0-shot2023.1123.2——
WizardLMzero-shot=true2024.1023.2——
LLaMA-2Shot setting=0-shot2023.1121.8——
LLaMA-2zero-shot=true2024.1021.8——
Vicuna-v1.5Shot setting=0-shot2023.1121.2——
Vicuna-v1.5zero-shot=true2024.1021.2——
SaySelfBackbone Model=Llama-3.2-3B-Instruct2026.0719.461.17.2
DeepSeek-LLMVersion=7B Chat, Encoder=None, Evaluation Protocol=Generation-based (greedy decoding)2024.0319.3——
DeepSeek-VLVersion=1B Chat, Encoder=SigLIP, Evaluation Protocol=Generation-based (greedy decoding)2024.0314——