Science Question Answering on ARC Easy
98AccuracyBase
Evaluation results
| Method | Links | |
|---|---|---|
| BaseZero-shot=true2026.03 | 98 | |
| SFTZero-shot=true2026.03 | 97 | |
| GRPOZero-shot=true2026.03 | 97 | |
| Hard-routing MoEBackbone=Qwen2.5-7B-Instruct, # Params (%)=14.2G (100%), # Trans. (%)=14.2G (100%), Fine-tuning setting=DFL, Rank (rl)=322026.02 | 96.95 | |
| FPFTBackbone=Qwen2.5-7B-Instruct, # Params (%)=14.2G (100%), # Trans. (%)=14.2G (100%), Fine-tuning setting=DFL, Rank (rl)=322026.02 | 93.31 | |
| Sparse-and-Orthogonal LoRABackbone=Qwen2.5-7B-Instruct, # Params (%)=87M (0.60%), # Trans. (%)=42M (0.30%), Fine-tuning setting=DFL, Rank (rl)=322026.02 | 92.6 | |
| Exact computationModel=LLaDA-8B-Instruct2025.10 | 91.62 | |
| single-threshold support basisModel=LLaDA-8B-Instruct, Degree of Chebyshev polynomial=62025.10 | 91.58 | |
| LoRIBackbone=Qwen2.5-7B-Instruct, # Params (%)=168M (1.16%), # Trans. (%)=168M (1.16%), Fine-tuning setting=DFL, Rank (rl)=322026.02 | 90.82 | |
| Sparse-and-Orthogonal LoRA (Single)Backbone=Qwen2.5-7B-Instruct, # Params (%)=87M (0.60%), # Trans. (%)=42M (0.30%), Fine-tuning setting=DFL, Rank (rl)=322026.02 | 90.69 | |
| single-threshold support basisModel=LLaDA-8B-Instruct, Degree of Chebyshev polynomial=42025.10 | 89.9 | |
| LoRABackbone=Qwen2.5-7B-Instruct, # Params (%)=362M (2.5%), # Trans. (%)=362M (2.5%), Fine-tuning setting=DFL, Rank (rl)=322026.02 | 89.38 | |
| Hard-routing MoEBackbone=Qwen2.5-1.5B-Instruct, # Params (%)=3.2G (100%), # Trans. (%)=3.2G (100%), Fine-tuning setting=DFL, Rank (rl)=322026.02 | 87.56 | |
| CTBase Model=Qwen2.5-7B, #P/B=7.62, #AP/B=7.622026.03 | 87.33 | |
| PTBase Model=Qwen2.5-7B, #P/B=7.62, #AP/B=7.622026.03 | 87.12 | |
| FineRMoEBase Model=Qwen2.5-7B, #P/B=26.65, #AP/B=7.942026.03 | 86.95 | |
| C32A2Base Model=Qwen2.5-7B, #P/B=184.42, #AP/B=13.332026.03 | 86.32 | |
| NVShardBase Model=Qwen2.5-7B, #P/B=47.55, #AP/B=7.632026.03 | 84.97 | |
| FPFTBackbone=Qwen2.5-1.5B-Instruct, # Params (%)=3.2G (100%), # Trans. (%)=3.2G (100%), Fine-tuning setting=DFL, Rank (rl)=322026.02 | 84.49 | |
| Sparse-and-Orthogonal LoRABackbone=Qwen2.5-1.5B-Instruct, # Params (%)=16M (0.54%), # Trans. (%)=8M (0.27%), Fine-tuning setting=DFL, Rank (rl)=322026.02 | 84.41 | |
| Sparse-and-Orthogonal LoRA (Single)Backbone=Qwen2.5-1.5B-Instruct, # Params (%)=16M (0.54%), # Trans. (%)=8M (0.27%), Fine-tuning setting=DFL, Rank (rl)=322026.02 | 82.95 | |
| LoRIBackbone=Qwen2.5-1.5B-Instruct, # Params (%)=32M (1.05%), # Trans. (%)=32M (1.05%), Fine-tuning setting=DFL, Rank (rl)=322026.02 | 82.78 | |
| PTBase Model=Qwen2.5-1.5B, #P/B=1.54, #AP/B=1.542026.03 | 81.02 | |
| FineRMoEBase Model=Qwen2.5-1.5B, #P/B=5.64, #AP/B=1.852026.03 | 80.81 | |
| CTBase Model=Qwen2.5-1.5B, #P/B=1.54, #AP/B=1.542026.03 | 80.72 | |
| C32A2Base Model=Qwen2.5-1.5B, #P/B=37.62, #AP/B=2.942026.03 | 80.01 | |
| Mistral-7BRatio=0%, Evaluation Protocol=zero-shot2026.03 | 79.55 | |
| ShapLoRATunable Params=22.8M, Backbone=LLaMA-3 8B, Evaluation Protocol=Supervised fine-tuning (SFT), Decoding Strategy=Beam search (size 3)2026.01 | 79.3 | |
| DoRATunable Params=22.6M, Backbone=LLaMA-3 8B, Evaluation Protocol=Supervised fine-tuning (SFT), Decoding Strategy=Beam search (size 3)2026.01 | 77.9 | |
| NVShardBase Model=Qwen2.5-1.5B, #P/B=9.88, #AP/B=1.782026.03 | 77.82 | |
| MOELoRATunable Params=29.9M, Backbone=LLaMA-3 8B, Evaluation Protocol=Supervised fine-tuning (SFT), Decoding Strategy=Beam search (size 3)2026.01 | 77.5 | |
| Qwen2.5-7BModel Type=Dense, Active Parameters=7B, Total Parameters=7B2026.02 | 77.5 | |
| ReasoningBackbone=Qwen2.5-7B family2026.01 | 77.25 | |
| AutoLoRATunable Params=23.1M, Backbone=LLaMA-3 8B, Evaluation Protocol=Supervised fine-tuning (SFT), Decoding Strategy=Beam search (size 3)2026.01 | 76.9 | |
| OLMoE-1B-7BModel Type=MoE, Active Parameters=1B, Total Parameters=7B2026.02 | 76.3 | |
| SSPTunable Params=80.6M, Backbone=LLaMA-3 8B, Evaluation Protocol=Supervised fine-tuning (SFT), Decoding Strategy=Beam search (size 3)2026.01 | 75.2 | |
| IAPTTunable Params=20.9M, Backbone=LLaMA-3 8B, Evaluation Protocol=Supervised fine-tuning (SFT), Decoding Strategy=Beam search (size 3)2026.01 | 75.1 | |
| AdaLoRATunable Params=23.2M, Backbone=LLaMA-3 8B, Evaluation Protocol=Supervised fine-tuning (SFT), Decoding Strategy=Beam search (size 3)2026.01 | 75.1 | |
| LoRATunable Params=22.5M, Backbone=LLaMA-3 8B, Evaluation Protocol=Supervised fine-tuning (SFT), Decoding Strategy=Beam search (size 3)2026.01 | 74.4 | |
| LLaDA (Base)Pruning Ratio=0.02026.02 | 74.2 | |
| Qwen2.5Size=1.54B, Pre-training tokens=18T2025.12 | 74.1 | |
| Qwen2.5-3BModel Type=Dense, Active Parameters=3B, Total Parameters=3B2026.02 | 73.9 | |
| GamayunSize=1.47B, Pre-training tokens=2.6T2025.12 | 73.7 | |
| DenseBase Model=Llama-3.2-3B, Model architecture type=Dense, Few-shot=true2025.06 | 73.6 | |
| SmolLM2-1.7BModel Type=Dense, Active Parameters=1.7B, Total Parameters=1.7B2026.02 | 73.4 | |
| Learned-AdapterTunable Params=21.1M, Backbone=LLaMA-3 8B, Evaluation Protocol=Supervised fine-tuning (SFT), Decoding Strategy=Beam search (size 3)2026.01 | 73.1 | |
| (IA)3Tunable Params=23.1M, Backbone=LLaMA-3 8B, Evaluation Protocol=Supervised fine-tuning (SFT), Decoding Strategy=Beam search (size 3)2026.01 | 73.1 | |
| MiCRoBase Model=Llama-3.2-3B, Model architecture type=MiCRo, Few-shot=true2025.06 | 73 | |
| PHSATraining Tokens=100B, Inference Top-K=4, Training Top-K=42026.01 | 72.69 | |
| PHSATraining Tokens=100B, Inference Top-K=N/A, Training Top-K=42026.01 | 72.64 | |
| Qwen3Size=1.7B, Pre-training tokens=36T2025.12 | 72.6 | |
| InfLLM v2Training Tokens=100B, Inference Top-K=N/A, Training Top-K=22026.01 | 72.52 | |
| Qwen2.5-1.5BModel Type=Dense, Active Parameters=1.5B, Total Parameters=1.5B2026.02 | 72.5 | |
| Parallel-AdapterTunable Params=20.9M, Backbone=LLaMA-3 8B, Evaluation Protocol=Supervised fine-tuning (SFT), Decoding Strategy=Beam search (size 3)2026.01 | 72.4 | |
| ExpertWeaver-E64-A14-S2(Qwen2.5-7B)Model Type=MoE, Active Parameters=3.5B, Total Parameters=7B, Training Budget=200B tokens2026.02 | 72.4 | |
| InfLLM v2Training Tokens=100B, Inference Top-K=2, Training Top-K=22026.01 | 72.31 | |
| BitFitTunable Params=25.2M, Backbone=LLaMA-3 8B, Evaluation Protocol=Supervised fine-tuning (SFT), Decoding Strategy=Beam search (size 3)2026.01 | 72.3 | |
| InfLLM v2Training Tokens=100B, Inference Top-K=4, Training Top-K=42026.01 | 72.18 | |
| InfLLM v2Training Tokens=100B, Inference Top-K=N/A, Training Top-K=42026.01 | 72.1 | |
| DenseTraining Tokens=100B, Inference Top-K=22026.01 | 71.97 | |
| DenseTraining Tokens=100B, Inference Top-K=N/A2026.01 | 71.84 | |
| Sink-Aware (Ours)Pruning Ratio=0.32026.02 | 71.75 | |
| Llama-3.2-3BModel Type=Dense, Active Parameters=3B, Total Parameters=3B2026.02 | 71.6 | |
| MoBBase Model=Llama-3.2-3B, Model architecture type=MoB, Few-shot=true2025.06 | 71.6 | |
| SVD-LLMRatio=20%, Evaluation Protocol=zero-shot2026.03 | 71.34 | |
| EuroLMSize=1.66B, Pre-training tokens=4T2025.12 | 71.3 | |
| PHSATraining Tokens=100B, Inference Top-K=2, Training Top-K=22026.01 | 71.21 | |
| PHSATraining Tokens=100B, Inference Top-K=N/A, Training Top-K=22026.01 | 71.13 | |
| OLMoE-1B-7B*Model Type=MoE, Active Parameters=1B, Total Parameters=7B, Training Budget=500B tokens2026.02 | 71 | |
| Olmo2Size=1.48B, Pre-training tokens=4T2025.12 | 70.9 | |
| LinearBackbone=Qwen2.5-7B family2026.01 | 70.72 | |
| MoBBase Model=SmollM2-360M, Model architecture type=MoB, Few-shot=true2025.06 | 70 | |
| SoLARatio=20%, Evaluation Protocol=zero-shot2026.03 | 69.99 | |
| MiCRoBase Model=SmollM2-360M, Model architecture type=MiCRo, Few-shot=true2025.06 | 69.9 | |
| DenseBase Model=SmollM2-360M, Model architecture type=Dense, Few-shot=true2025.06 | 69.7 | |
| ExpertWeaver-E64-A14-S2(OLMo-7B)Model Type=MoE, Active Parameters=1B, Total Parameters=7B, Training Budget=200B tokens2026.02 | 69.3 | |
| LLaDA-structurePruning Ratio=0.32026.02 | 69.07 | |
| Gemma3Size=1B, Pre-training tokens=2T2025.12 | 68.9 | |
| PTBase Model=Qwen2.5-0.5B, #P/B=0.49, #AP/B=0.492026.03 | 68.9 | |
| SmolLM2Size=1.7B, Pre-training tokens=11T2025.12 | 68.8 | |
| OLMo-7BModel Type=Dense, Active Parameters=7B, Total Parameters=7B2026.02 | 68.7 | |
| P-tuning v2Tunable Params=21.0M, Backbone=LLaMA-3 8B, Evaluation Protocol=Supervised fine-tuning (SFT), Decoding Strategy=Beam search (size 3)2026.01 | 68.5 | |
| Llama3.2Size=1.24B, Pre-training tokens=9T2025.12 | 68.5 | |
| CTBase Model=Qwen2.5-0.5B, #P/B=0.49, #AP/B=0.492026.03 | 68.06 | |
| FineRMoEBase Model=Qwen2.5-0.5B, #P/B=1.68, #AP/B=0.652026.03 | 67.55 | |
| C32A2Base Model=Qwen2.5-0.5B, #P/B=10.36, #AP/B=0.952026.03 | 67.42 | |
| ReasonAnyBackbone=Qwen2.5-7B family2026.01 | 66.75 | |
| LLaMA-MoE-v1-3.5BModel Type=MoE, Active Parameters=3.5B2026.02 | 65.6 | |
| Qwen3-0.6B2026.03 | 65.6 | |
| NVShardBase Model=Qwen2.5-0.5B, #P/B=2.83, #AP/B=0.632026.03 | 65.4 | |
| DenseBase Model=Llama-3.2-1B, Model architecture type=Dense, Few-shot=true2025.06 | 64.3 | |
| Open-LLaMA-3B-v2Model Type=Dense, Active Parameters=3B, Total Parameters=3B2026.02 | 63.3 | |
| Sheared-LLaMA-2.7BModel Type=Dense, Active Parameters=2.7B, Total Parameters=2.7B2026.02 | 63.3 | |
| MoBBase Model=SmollM2-135M, Model architecture type=MoB, Few-shot=true2025.06 | 63 | |
| DenseBase Model=SmollM2-135M, Model architecture type=Dense, Few-shot=true2025.06 | 62.8 | |
| LoRABackbone=Qwen2.5-1.5B-Instruct, # Params (%)=65.5M (2.01%), # Trans. (%)=65.5M (2.01%), Fine-tuning setting=DFL, Rank (rl)=322026.02 | 62.3 | |
| INCITE-Base-3BModel Type=Dense, Active Parameters=3B, Total Parameters=3B2026.02 | 61.7 | |
| MoBBase Model=Llama-3.2-1B, Model architecture type=MoB, Few-shot=true2025.06 | 61.7 | |
| FinanceBackbone=Qwen2.5-7B family2026.01 | 61.02 | |
| S16A4Base Model=Qwen2.5-7B, #P/B=7.62, #AP/B=3.342026.03 | 60.98 |