Accuracy on AIME 24 (Reasoning)
86.3Accuracy on AIME 24Qwen3-Base SAT
Evaluation Results
| Method | Links | |
|---|---|---|
| Qwen3-Base SATSize=3×4B2026.04 | 86.3 | |
| TF-TTCLModel=DeepSeek-V3.22026.04 | 83.33 | |
| Qwen3 (thinking)Size=32B2026.04 | 81.4 | |
| Qwen3-A3BSize=30B2026.04 | 80.4 | |
| AWQBackbone=Qwen3-14B, Type=linear, Zero-shot=true2025.11 | 80 | |
| TF-GRPOModel=DeepSeek-V3.22026.04 | 80 | |
| OpenAI o3-mini (medium)Size=/2026.04 | 79.6 | |
| QwQSize=32B2026.04 | 79.5 | |
| ParoQuantBackbone=Qwen3-14B, Type=linear, Zero-shot=true2025.11 | 77.8 | |
| QTIPBackbone=Qwen3-14B, Type=vector, Zero-shot=true2025.11 | 76.7 | |
| TF-TTCLModel=Qwen-Plus2026.04 | 76.67 | |
| FP16Backbone=Qwen3-4B, Zero-shot=true2025.11 | 75.6 | |
| FP16Backbone=Qwen3-8B, Zero-shot=true2025.11 | 75.6 | |
| ParoQuantBackbone=Qwen3-8B, Type=linear, Zero-shot=true2025.11 | 75.6 | |
| ParoQuantBackbone=Qwen3-4B, Type=linear, Zero-shot=true2025.11 | 73.3 | |
| FP16Backbone=Qwen3-14B, Zero-shot=true2025.11 | 73.3 | |
| QTIPBackbone=Qwen3-8B, Type=vector, Zero-shot=true2025.11 | 72.2 | |
| AWQBackbone=Qwen3-8B, Type=linear, Zero-shot=true2025.11 | 72.2 | |
| E-QATBackbone=Qwen3-14B, Type=linear, Zero-shot=true2025.11 | 71.1 | |
| Qwen3-Base Debate + JudgeSize=3×8B2026.04 | 71.1 | |
| Llama 3.1-Base SATSize=3×8B2026.04 | 70.3 | |
| Chain-of-ThoughtModel=DeepSeek-V3.22026.04 | 70 | |
| TF-GRPOModel=Qwen-Plus2026.04 | 70 | |
| DeepSeek-R1 Distill LlamaSize=70B2026.04 | 70 | |
| Qwen3-Base Role-playSize=3×8B2026.04 | 69.9 | |
| E-QATBackbone=Qwen3-8B, Type=linear, Zero-shot=true2025.11 | 68.9 | |
| Qwen3-Base DebateSize=3×8B2026.04 | 68.7 | |
| QTIPBackbone=Qwen3-4B, Type=vector, Zero-shot=true2025.11 | 67.8 | |
| Base LLMModel=DeepSeek-V3.22026.04 | 66.67 | |
| DS-Qwen 14BModel Size=14B, 0-shot=true2025.05 | 63.33 | |
| AWQBackbone=Qwen3-4B, Type=linear, Zero-shot=true2025.11 | 62.2 | |
| Layer-wise CapreseModel Size=14B, Sparse Algorithm=CATS, 0-shot=true2025.05 | 61.67 | |
| CATSModel Size=14B, 0-shot=true2025.05 | 58.34 | |
| E2E CapreseModel Size=14B, Sparse Algorithm=CATS, 0-shot=true2025.05 | 58.34 | |
| E-QATBackbone=Qwen3-4B, Type=linear, Zero-shot=true2025.11 | 53.3 | |
| DS-Qwen 7BModel Size=7B, 0-shot=true2025.05 | 47.5 | |
| FP16Backbone=R1-Distill-Llama-8B, Zero-shot=true2025.11 | 42.2 | |
| Layer-wise CapreseModel Size=14B, Sparse Algorithm=GRIFFIN, 0-shot=true2025.05 | 41.67 | |
| Qwen3-Base-DAPOSize=8B2026.04 | 41.3 | |
| Chain-of-ThoughtModel=Qwen-Plus2026.04 | 40 | |
| E2E CapreseModel Size=14B, Sparse Algorithm=GRIFFIN, 0-shot=true2025.05 | 39.17 | |
| Qwen3-Base-GRPOSize=8B2026.04 | 39.1 | |
| QTIPBackbone=R1-Distill-Llama-8B, Type=vector, Zero-shot=true2025.11 | 37.8 | |
| ParoQuantBackbone=R1-Distill-Llama-8B, Type=linear, Zero-shot=true2025.11 | 36.6 | |
| Layer-wise CapreseModel Size=7B, Sparse Algorithm=CATS, 0-shot=true2025.05 | 35 | |
| AWQBackbone=R1-Distill-Llama-8B, Type=linear, Zero-shot=true2025.11 | 34.4 | |
| CATSModel Size=7B, 0-shot=true2025.05 | 34.17 | |
| E2E CapreseModel Size=7B, Sparse Algorithm=CATS, 0-shot=true2025.05 | 33.33 | |
| GRIFFINModel Size=14B, 0-shot=true2025.05 | 32.5 | |
| E-QATBackbone=R1-Distill-Llama-8B, Type=linear, Zero-shot=true2025.11 | 32.2 | |
| Qwen3-BaseSize=14B2026.04 | 31.7 | |
| Qwen3-BaseSize=32B2026.04 | 31 | |
| DS-Qwen 1.5BModel Size=1.5B, 0-shot=true2025.05 | 30 | |
| E2E CapreseModel Size=7B, Sparse Algorithm=GRIFFIN, 0-shot=true2025.05 | 30 | |
| Base LLMModel=Qwen-Plus2026.04 | 30 | |
| Layer-wise CapreseModel Size=7B, Sparse Algorithm=GRIFFIN, 0-shot=true2025.05 | 29.17 | |
| GRIFFINModel Size=7B, 0-shot=true2025.05 | 21.67 | |
| E2E CapreseModel Size=1.5B, Sparse Algorithm=CATS, 0-shot=true2025.05 | 20.83 | |
| Qwen2.5-BaseSize=72B2026.04 | 18.9 | |
| Layer-wise CapreseModel Size=1.5B, Sparse Algorithm=CATS, 0-shot=true2025.05 | 15.83 | |
| CATSModel Size=1.5B, 0-shot=true2025.05 | 11.67 | |
| GPT-4o-mini-2024-07-18Size=/2026.04 | 8.1 | |
| E2E CapreseModel Size=1.5B, Sparse Algorithm=GRIFFIN, 0-shot=true2025.05 | 6.67 | |
| GRIFFINModel Size=1.5B, 0-shot=true, Sparsity=50%2025.05 | 2.5 | |
| Layer-wise CapreseModel Size=1.5B, Sparse Algorithm=GRIFFIN, 0-shot=true2025.05 | 2.5 |