Reasoning on OBQA
97.67AccuracyCOSE
Evaluation Results
| Method | Links | |
|---|---|---|
| COSEModel=Llama-3.2-3B-Instruct2026.05 | 97.67 | |
| BaseModel=Qwen3-4B2026.05 | 97 | |
| MAEModel=Qwen3-4B2026.05 | 96.8 | |
| COSEModel=Qwen3-4B2026.05 | 96.65 | |
| AZRModel=Qwen3-4B2026.05 | 96.6 | |
| R-ZeroModel=Qwen3-4B2026.05 | 96.4 | |
| MAEModel=Llama-3.2-3B-Instruct2026.05 | 96 | |
| COSEModel=Qwen3-0.6B2026.05 | 95.96 | |
| R-ZeroModel=Llama-3.2-3B-Instruct2026.05 | 81.39 | |
| AZRModel=Llama-3.2-3B-Instruct2026.05 | 80.32 | |
| COSEModel=Qwen2.5-3B-Instruct2026.05 | 79.67 | |
| MAEModel=Qwen2.5-3B-Instruct2026.05 | 79.39 | |
| BaseModel=Llama-3.2-3B-Instruct2026.05 | 79 | |
| AZRModel=Qwen2.5-3B-Instruct2026.05 | 77.55 | |
| R-ZeroModel=Qwen2.5-3B-Instruct2026.05 | 76.8 | |
| MAEModel=Qwen3-0.6B2026.05 | 76.4 | |
| R-ZeroModel=Qwen3-0.6B2026.05 | 75.2 | |
| AZRModel=Qwen3-0.6B2026.05 | 73.4 | |
| BaseModel=Qwen3-0.6B2026.05 | 73 | |
| BaseModel=Qwen2.5-3B-Instruct2026.05 | 67.8 | |
| SmolLM-1.7B + OursNumber of Parameters=~ 1.7B, Zero-shot=true2026.04 | 31.6 | |
| SmolLM-360M + OursNumber of Parameters=~ 360M, Zero-shot=true2026.04 | 30.5 | |
| Self-Improving PretrainingTraining Dataset=SlimPajama, Pretraining Objective=Quality2026.01 | 30 | |
| LaMini-GPT-124MNumber of Parameters=~ 135M, Zero-shot=true2026.04 | 29.6 | |
| Self-Improving PretrainingTraining Dataset=SlimPajama, Pretraining Objective=Factuality2026.01 | 29 | |
| SmolLM-135M + OursNumber of Parameters=~ 135M, Zero-shot=true2026.04 | 28.6 | |
| Galactica-125MNumber of Parameters=~ 135M, Zero-shot=true2026.04 | 28.2 | |
| Self-Improving PretrainingTraining Dataset=RedPajama, Pretraining Objective=Safety2026.01 | 27.4 | |
| Llama BaseTraining Dataset=Original Llama Pretraining, Pretraining Objective=Standard next token prediction2026.01 | 27.2 | |
| Llama Pretrain BaselineTraining Dataset=SlimPajama, Pretraining Objective=Standard next token prediction2026.01 | 27 | |
| Llama Pretrain BaselineTraining Dataset=RedPajama, Pretraining Objective=Safety2026.01 | 26.6 | |
| OPT-2.7BNumber of Parameters=~ 1.7B, Zero-shot=true2026.04 | 25 | |
| OPT-1.3BNumber of Parameters=~ 1.7B, Zero-shot=true2026.04 | 23.4 | |
| Pythia-1.4BNumber of Parameters=~ 1.7B, Zero-shot=true2026.04 | 22.4 | |
| Pythia-2.8BNumber of Parameters=~ 1.7B, Zero-shot=true2026.04 | 22 | |
| Cerebras-GPT-2.7BNumber of Parameters=~ 1.7B, Zero-shot=true2026.04 | 20.6 | |
| Pythia-1BNumber of Parameters=~ 360M, Zero-shot=true2026.04 | 19.6 | |
| Pythia-410MNumber of Parameters=~ 360M, Zero-shot=true2026.04 | 17.8 | |
| OPT-350MNumber of Parameters=~ 360M, Zero-shot=true2026.04 | 17.6 | |
| OPT-125MNumber of Parameters=~ 135M, Zero-shot=true2026.04 | 16.6 | |
| Cerebras-GPT-1.3BNumber of Parameters=~ 1.7B, Zero-shot=true2026.04 | 16.6 | |
| Pythia-160MNumber of Parameters=~ 135M, Zero-shot=true2026.04 | 16 | |
| Cerebras-GPT-256MNumber of Parameters=~ 135M, Zero-shot=true2026.04 | 15.8 | |
| Cerebras-GPT-590MNumber of Parameters=~ 360M, Zero-shot=true2026.04 | 15.8 | |
| Pythia-70MNumber of Parameters=~ 135M, Zero-shot=true2026.04 | 13.2 | |
| Cerebras-GPT-111MNumber of Parameters=~ 135M, Zero-shot=true2026.04 | 11.8 |