Scientific Reasoning on ARC Challenge
92.5AccuracyQwen-3-32B
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Qwen-3-32BOpenness=Open-weights, Region=Non-European, Tuning=Instruction-tuned2026.02 | 92.5 | — | — | |
| DYPOBase Model=Qwen3-4B-Base2026.04 | 92.5 | — | — | |
| Qwen-3-30B-A3BOpenness=Open-weights, Region=Non-European, Tuning=Instruction-tuned2026.02 | 92.3 | — | — | |
| GRPO w/Clip-higher + LIEBackbone=Qwen3-4B-Base, Variant=Clip-higher, Strategy=Length-Incentivized Exploration2026.02 | 91.5 | — | — | |
| GSPO + LIEBackbone=Qwen3-4B-Base, Strategy=Length-Incentivized Exploration2026.02 | 91.4 | — | — | |
| GRPO + LIEBackbone=Qwen3-4B-Base, Strategy=Length-Incentivized Exploration2026.02 | 90.3 | — | — | |
| Llama-3.3-70BOpenness=Open-weights, Region=Non-European, Tuning=Instruction-tuned2026.02 | 90.1 | — | — | |
| Gemma-3-27BOpenness=Open-weights, Region=Non-European, Tuning=Instruction-tuned2026.02 | 90.1 | — | — | |
| Qwen-3-14BOpenness=Open-weights, Region=Non-European, Tuning=Instruction-tuned2026.02 | 90 | — | — | |
| GRPO w/Clip-higherBackbone=Qwen3-4B-Base, Variant=Clip-higher2026.02 | 89.6 | — | — | |
| Mistral-3.2-24BOpenness=Open-weights, Region=European, Tuning=Instruction-tuned2026.02 | 89.2 | — | — | |
| GSPOBackbone=Qwen3-4B-Base2026.02 | 88.4 | — | — | |
| Gemma-3-12BOpenness=Open-weights, Region=Non-European, Tuning=Instruction-tuned2026.02 | 87.2 | — | — | |
| GRPOBackbone=Qwen3-4B-Base2026.02 | 84.6 | — | — | |
| EuroLLM-22B (new)Openness=Fully-open, Region=European, Tuning=Instruction-tuned2026.02 | 82.7 | — | — | |
| EuroLLM-9B (new)Openness=Fully-open, Region=European, Tuning=Instruction-tuned2026.02 | 79.6 | — | — | |
| Best-of-KBackbone=LLaDA-8B-Instruct, Decoding Strategy=Best-of-K, K (Best-of-K)=82026.04 | 79.3 | — | — | |
| OLMo-3.1-32BOpenness=Fully-open, Region=Non-European, Tuning=Instruction-tuned2026.02 | 79 | — | — | |
| EuroLLM-22B (old)Openness=Fully-open, Region=European, Tuning=Instruction-tuned2026.02 | 78.7 | — | — | |
| Apertus-70BOpenness=Fully-open, Region=European, Tuning=Instruction-tuned2026.02 | 78.6 | — | — | |
| S³Backbone=LLaDA-8B-Instruct, Decoding Strategy=S³, N (S³)=4, b (S³)=2, K (Best-of-K)=642026.04 | 77.86 | — | — | |
| SFT -> RLBase Model=Qwen3-4B-Base2026.04 | 77.4 | — | — | |
| RLBase Model=Qwen3-4B-Base2026.04 | 76.7 | — | — | |
| Baseline DiffusionBackbone=LLaDA-8B-Instruct, Decoding Strategy=Standard Diffusion2026.04 | 76.11 | — | — | |
| SFTBase Model=Qwen3-4B-Base2026.04 | 73.8 | — | — | |
| EuroLLM-9B (old)Openness=Fully-open, Region=European, Tuning=Instruction-tuned2026.02 | 72.2 | — | — | |
| Apertus-8BOpenness=Fully-open, Region=European, Tuning=Instruction-tuned2026.02 | 69.9 | — | — | |
| DenseSparsity=0, Model Scale=72B, Model Family=Qwen 2.52025.10 | 68.52 | — | — | |
| Llama-3.1-8BOpenness=Open-weights, Region=Non-European, Tuning=Instruction-tuned2026.02 | 68.1 | — | — | |
| Qwen3-4B-Base2026.02 | 66.9 | — | — | |
| DenseModel=Qwen-3 14B, Sparsity=02025.10 | 66.64 | — | — | |
| DenseSparsity=0, Model Scale=32B, Model Family=Qwen 2.52025.10 | 66.3 | — | — | |
| DenseSparsity=0, Model Scale=14B, Model Family=Qwen 2.52025.10 | 64.51 | — | — | |
| DenseModel=Qwen-3 8B, Sparsity=02025.10 | 63.65 | — | — | |
| ARMORSparsity=2:4+2.4%, Model Scale=72B, Model Family=Qwen 2.52025.10 | 63.4 | — | — | |
| WandaSparsity=2:4, Model Scale=72B, Model Family=Qwen 2.52025.10 | 62.37 | — | — | |
| SparseGPTSparsity=2:4, Model Scale=72B, Model Family=Qwen 2.52025.10 | 62.29 | — | — | |
| NoWag-PSparsity=2:4, Model Scale=72B, Model Family=Qwen 2.52025.10 | 60.84 | — | — | |
| ARMORSparsity=2:4+3.44%, Model Scale=32B, Model Family=Qwen 2.52025.10 | 60.15 | — | — | |
| DenseSparsity=0, Model Scale=7B, Model Family=Qwen 2.52025.10 | 59.55 | — | — | |
| Qwen3-4B-Instruct-2507Model Size=4B, Training Strategy=Instruct-25072026.04 | 58.98 | — | — | |
| Qwen3-4B-BaseModel Size=4B, Training Strategy=Base2026.04 | 57.39 | — | — | |
| SparseGPTSparsity=2:4, Model Scale=32B, Model Family=Qwen 2.52025.10 | 57.08 | — | — | |
| MobileMoE-LActive Parameters=922M, Total Parameters=5.3B, Few-shot count=25-shot2026.05 | 57 | — | — | |
| ARMORModel=Qwen-3 14B, Sparsity=2:4+3.89%2025.10 | 56.31 | — | — | |
| WandaSparsity=2:4, Model Scale=32B, Model Family=Qwen 2.52025.10 | 55.72 | — | — | |
| NoWag-PSparsity=2:4, Model Scale=32B, Model Family=Qwen 2.52025.10 | 55.29 | — | — | |
| OLMo-3-7BOpenness=Fully-open, Region=Non-European, Tuning=Instruction-tuned2026.02 | 54.6 | — | — | |
| NITPModel scale=9bA1b, Evaluation=few-shot, Context length=81922026.05 | 53.95 | — | — | |
| ARMORSparsity=2:4+4.17%, Model Scale=14B, Model Family=Qwen 2.52025.10 | 53.75 | — | — | |
| SparseGPTModel=Qwen-3 14B, Sparsity=2:42025.10 | 53.58 | — | — | |
| SpB1.0-7BModel Size=7B2026.04 | 52.54 | — | — | |
| ConSA (head-wise, all-layers)Target Sparsity (ρ)=0.50, Granularity=head-wise, Constraint Scope=all-layers, Model=1.7B2026.06 | 52.05 | — | — | |
| ConSA (layer-wise)Target Sparsity (ρ)=0.50, Granularity=layer-wise, Model=1.7B2026.06 | 51.79 | — | — | |
| ConSA (head-wise, single-layer)Target Sparsity (ρ)=0.50, Granularity=head-wise, Constraint Scope=single-layer, Model=1.7B2026.06 | 51.71 | — | — | |
| SpB2.0-5BModel Size=5B2026.04 | 51.53 | — | — | |
| NTPModel scale=9bA1b, Evaluation=few-shot, Context length=81922026.05 | 51.2 | — | — | |
| Rule (head-wise)Target Sparsity (ρ)=0.50, Granularity=head-wise, Model=1.7B2026.06 | 51.19 | — | — | |
| Dense FATarget Sparsity (ρ)=0, Model=1.7B2026.06 | 51.02 | — | — | |
| MobileMoE-MActive Parameters=528M, Total Parameters=2.8B, Few-shot count=25-shot2026.05 | 51 | — | — | |
| Gemma3-4BModel Size=4B2026.04 | 50.85 | — | — | |
| WandaModel=Qwen-3 14B, Sparsity=2:42025.10 | 50.51 | — | — | |
| Qwen2.5-3BModel Size=3B2026.04 | 50.51 | — | — | |
| Rule (layer-wise)Target Sparsity (ρ)=0.50, Granularity=layer-wise, Model=1.7B2026.06 | 50.43 | — | — | |
| ARMORModel=Qwen-3 8B, Sparsity=2:4+5.03%2025.10 | 50.34 | — | — | |
| Qwen3-4B-BaseBase Model=Qwen3-4B-Base2026.04 | 49.4 | — | — | |
| NoWag-PModel=Qwen-3 14B, Sparsity=2:42025.10 | 48.89 | — | — | |
| ARMORSparsity=2:4+4.95%, Model Scale=7B, Model Family=Qwen 2.52025.10 | 48.63 | — | — | |
| SparseGPTSparsity=2:4, Model Scale=14B, Model Family=Qwen 2.52025.10 | 48.21 | — | — | |
| MobileMoE-SActive Parameters=272M, Total Parameters=1.3B, Few-shot count=25-shot2026.05 | 46.5 | — | — | |
| WandaSparsity=2:4, Model Scale=14B, Model Family=Qwen 2.52025.10 | 46.16 | — | — | |
| Llama3.2-3BModel Size=3B2026.04 | 45.76 | — | — | |
| LightReasoner2025.10 | 45.6 | — | — | |
| Sink AttentionModel Size=2B, Training Loss=base+aux2026.02 | 44.8 | — | — | |
| SparseGPTModel=Qwen-3 8B, Sparsity=2:42025.10 | 44.54 | — | — | |
| Gated AttentionModel Size=2B, Training Loss=base+aux2026.02 | 44.4 | — | — | |
| Gated AttentionModel Size=2B, Training Loss=base2026.02 | 44.36 | — | — | |
| Sink AttentionModel Size=2B, Training Loss=base2026.02 | 44.28 | — | — | |
| NoWag-PSparsity=2:4, Model Scale=14B, Model Family=Qwen 2.52025.10 | 43.77 | — | — | |
| SparseGPTSparsity=2:4, Model Scale=7B, Model Family=Qwen 2.52025.10 | 43.43 | — | — | |
| Vanilla AttentionModel Size=2B, Training Loss=base+aux2026.02 | 43.09 | — | — | |
| Baseline2025.10 | 42.8 | — | — | |
| Standard RLHFEvaluation Protocol=50-question stratified sample per subject2026.05 | 42 | — | — | |
| Practical (R̄_FPO)Evaluation Protocol=50-question stratified sample per subject2026.05 | 42 | — | — | |
| Relaxed (R̃_FPO)Evaluation Protocol=50-question stratified sample per subject2026.05 | 42 | — | — | |
| Gated AttentionModel Size=1B, Training Loss=base+aux2026.02 | 41.98 | — | — | |
| Vanilla AttentionModel Size=2B, Training Loss=base2026.02 | 41.97 | — | — | |
| WandaModel=Qwen-3 8B, Sparsity=2:42025.10 | 41.55 | — | — | |
| Sink AttentionModel Size=1B, Training Loss=base2026.02 | 40.35 | — | — | |
| Sink AttentionModel Size=1B, Training Loss=base+aux2026.02 | 40.2 | — | — | |
| NoWag-PModel=Qwen-3 8B, Sparsity=2:42025.10 | 40.02 | — | — | |
| NoWag-PSparsity=2:4, Model Scale=7B, Model Family=Qwen 2.52025.10 | 39.16 | — | — | |
| Vanilla AttentionModel Size=1B, Training Loss=base+aux2026.02 | 38.99 | — | — | |
| Vanilla AttentionModel Size=1B, Training Loss=base2026.02 | 38.82 | — | — | |
| Gated AttentionModel Size=1B, Training Loss=base2026.02 | 38.73 | — | — | |
| WandaSparsity=2:4, Model Scale=7B, Model Family=Qwen 2.52025.10 | 37.8 | — | — | |
| NITPModel scale=3bA0.5b, Evaluation=few-shot, Context length=81922026.05 | 37.11 | — | — | |
| Gated AttentionModel Size=0.6B, Training Loss=base+aux2026.02 | 35.24 | — | — | |
| Gated AttentionModel Size=0.6B, Training Loss=base2026.02 | 34.89 | — | — | |
| Vanilla AttentionModel Size=0.6B, Training Loss=base+aux2026.02 | 34.47 | — | — |