# Science Question Answering on ARC Easy

Source: [Science Question Answering on ARC Easy](<https://www.sota2.com/research/sota/science-question-answering-on-arc-easy>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-05-26T13:44:00.468Z

Evaluation of model performance on the Easy subset of the Abstract Reasoning Corpus (ARC) for Science Question Answering.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Science Question Answering |
| Dataset (including any stated split) | ARC Easy |
| Metric | Accuracy |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 162 of 162 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Accuracy](<https://www.sota2.com/research/sota/science-question-answering-on-arc-easy.md?metric=Accuracy>): Higher is better; unit: PERCENT.

## Recorded results

| Method | Accuracy | Result date | Report paper |
| --- | --- | --- | --- |
| Base | 98 | Not recorded | 69a90de12d603e6de3e405d4 |
| SFT | 97 | Not recorded | 69a90de12d603e6de3e405d4 |
| GRPO | 97 | Not recorded | 69a90de12d603e6de3e405d4 |
| Hard-routing MoE | 96.95 | Not recorded | 699e81e06017c4460ffaccb4 |
| FPFT | 93.31 | Not recorded | 699e81e06017c4460ffaccb4 |
| Sparse-and-Orthogonal LoRA | 92.6 | Not recorded | 699e81e06017c4460ffaccb4 |
| Exact computation | 91.62 | Not recorded | 68fa7bd41e209131cea8ac23 |
| single-threshold support basis | 91.58 | Not recorded | 68fa7bd41e209131cea8ac23 |
| LoRI | 90.82 | Not recorded | 699e81e06017c4460ffaccb4 |
| Sparse-and-Orthogonal LoRA (Single) | 90.69 | Not recorded | 699e81e06017c4460ffaccb4 |
| single-threshold support basis | 89.9 | Not recorded | 68fa7bd41e209131cea8ac23 |
| LoRA | 89.38 | Not recorded | 699e81e06017c4460ffaccb4 |
| Hard-routing MoE | 87.56 | Not recorded | 699e81e06017c4460ffaccb4 |
| CT | 87.33 | Not recorded | 69b8e0090636e5bf4afbe9e0 |
| PT | 87.12 | Not recorded | 69b8e0090636e5bf4afbe9e0 |
| FineRMoE | 86.95 | Not recorded | 69b8e0090636e5bf4afbe9e0 |
| C32A2 | 86.32 | Not recorded | 69b8e0090636e5bf4afbe9e0 |
| NVShard | 84.97 | Not recorded | 69b8e0090636e5bf4afbe9e0 |
| FPFT | 84.49 | Not recorded | 699e81e06017c4460ffaccb4 |
| Sparse-and-Orthogonal LoRA | 84.41 | Not recorded | 699e81e06017c4460ffaccb4 |
| Sparse-and-Orthogonal LoRA (Single) | 82.95 | Not recorded | 699e81e06017c4460ffaccb4 |
| LoRI | 82.78 | Not recorded | 699e81e06017c4460ffaccb4 |
| PT | 81.02 | Not recorded | 69b8e0090636e5bf4afbe9e0 |
| FineRMoE | 80.81 | Not recorded | 69b8e0090636e5bf4afbe9e0 |
| CT | 80.72 | Not recorded | 69b8e0090636e5bf4afbe9e0 |
| C32A2 | 80.01 | Not recorded | 69b8e0090636e5bf4afbe9e0 |
| Mistral-7B | 79.55 | Not recorded | 69d48f957d0172140ec74311 |
| ShapLoRA | 79.3 | Not recorded | 69784700a2b28e761d87c8c8 |
| DoRA | 77.9 | Not recorded | 69784700a2b28e761d87c8c8 |
| NVShard | 77.82 | Not recorded | 69b8e0090636e5bf4afbe9e0 |
| MOELoRA | 77.5 | Not recorded | 69784700a2b28e761d87c8c8 |
| Qwen2.5-7B | 77.5 | Not recorded | 6995478c09c33900fcd6d5e1 |
| Reasoning | 77.25 | Not recorded | 69647fe953fdd5c36174478b |
| AutoLoRA | 76.9 | Not recorded | 69784700a2b28e761d87c8c8 |
| OLMoE-1B-7B | 76.3 | Not recorded | 6995478c09c33900fcd6d5e1 |
| SSP | 75.2 | Not recorded | 69784700a2b28e761d87c8c8 |
| IAPT | 75.1 | Not recorded | 69784700a2b28e761d87c8c8 |
| AdaLoRA | 75.1 | Not recorded | 69784700a2b28e761d87c8c8 |
| LoRA | 74.4 | Not recorded | 69784700a2b28e761d87c8c8 |
| LLaDA (Base) | 74.2 | Not recorded | 6997eac609c33900fcd76806 |
| Qwen2.5 | 74.1 | Not recorded | 69520aef53fdd5c36173d565 |
| Qwen2.5-3B | 73.9 | Not recorded | 6995478c09c33900fcd6d5e1 |
| Gamayun | 73.7 | Not recorded | 69520aef53fdd5c36173d565 |
| Dense | 73.6 | Not recorded | 68fa7c1d1e209131cea8c767 |
| SmolLM2-1.7B | 73.4 | Not recorded | 6995478c09c33900fcd6d5e1 |
| Learned-Adapter | 73.1 | Not recorded | 69784700a2b28e761d87c8c8 |
| (IA)3 | 73.1 | Not recorded | 69784700a2b28e761d87c8c8 |
| MiCRo | 73 | Not recorded | 68fa7c1d1e209131cea8c767 |
| PHSA | 72.69 | Not recorded | 695de86653fdd5c361743a1a |
| PHSA | 72.64 | Not recorded | 695de86653fdd5c361743a1a |
| Qwen3 | 72.6 | Not recorded | 69520aef53fdd5c36173d565 |
| InfLLM v2 | 72.52 | Not recorded | 695de86653fdd5c361743a1a |
| Qwen2.5-1.5B | 72.5 | Not recorded | 6995478c09c33900fcd6d5e1 |
| Parallel-Adapter | 72.4 | Not recorded | 69784700a2b28e761d87c8c8 |
| ExpertWeaver-E64-A14-S2(Qwen2.5-7B) | 72.4 | Not recorded | 6995478c09c33900fcd6d5e1 |
| InfLLM v2 | 72.31 | Not recorded | 695de86653fdd5c361743a1a |
| BitFit | 72.3 | Not recorded | 69784700a2b28e761d87c8c8 |
| InfLLM v2 | 72.18 | Not recorded | 695de86653fdd5c361743a1a |
| InfLLM v2 | 72.1 | Not recorded | 695de86653fdd5c361743a1a |
| Dense | 71.97 | Not recorded | 695de86653fdd5c361743a1a |
| Dense | 71.84 | Not recorded | 695de86653fdd5c361743a1a |
| Sink-Aware (Ours) | 71.75 | Not recorded | 6997eac609c33900fcd76806 |
| Llama-3.2-3B | 71.6 | Not recorded | 6995478c09c33900fcd6d5e1 |
| MoB | 71.6 | Not recorded | 68fa7c1d1e209131cea8c767 |
| SVD-LLM | 71.34 | Not recorded | 69d48f957d0172140ec74311 |
| EuroLM | 71.3 | Not recorded | 69520aef53fdd5c36173d565 |
| PHSA | 71.21 | Not recorded | 695de86653fdd5c361743a1a |
| PHSA | 71.13 | Not recorded | 695de86653fdd5c361743a1a |
| OLMoE-1B-7B\* | 71 | Not recorded | 6995478c09c33900fcd6d5e1 |
| Olmo2 | 70.9 | Not recorded | 69520aef53fdd5c36173d565 |
| Linear | 70.72 | Not recorded | 69647fe953fdd5c36174478b |
| MoB | 70 | Not recorded | 68fa7c1d1e209131cea8c767 |
| SoLA | 69.99 | Not recorded | 69d48f957d0172140ec74311 |
| MiCRo | 69.9 | Not recorded | 68fa7c1d1e209131cea8c767 |
| Dense | 69.7 | Not recorded | 68fa7c1d1e209131cea8c767 |
| ExpertWeaver-E64-A14-S2(OLMo-7B) | 69.3 | Not recorded | 6995478c09c33900fcd6d5e1 |
| LLaDA-structure | 69.07 | Not recorded | 6997eac609c33900fcd76806 |
| Gemma3 | 68.9 | Not recorded | 69520aef53fdd5c36173d565 |
| PT | 68.9 | Not recorded | 69b8e0090636e5bf4afbe9e0 |
| SmolLM2 | 68.8 | Not recorded | 69520aef53fdd5c36173d565 |

### Base

Accuracy: 98. Result ID: 69a92c5e6dbb7075b952142a.

[Reporting paper](<https://www.sota2.com/research/paper/69a90de12d603e6de3e405d4.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa788e1e209131cea79116.md>)

Recorded evaluation configuration: {"Zero-shot":"true"}. This record does not assert independent reproduction.

### SFT

Accuracy: 97. Result ID: 69a92c5e6dbb7075b952142b.

[Reporting paper](<https://www.sota2.com/research/paper/69a90de12d603e6de3e405d4.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6bf71e209131cea314c0.md>)

[Implementation](<https://github.com/openai/following-instructions-human-feedback>)

Recorded evaluation configuration: {"Zero-shot":"true"}. This record does not assert independent reproduction.

### GRPO

Accuracy: 97. Result ID: 69a92c5e6dbb7075b952142c.

[Reporting paper](<https://www.sota2.com/research/paper/69a90de12d603e6de3e405d4.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69a90de12d603e6de3e405d4.md>)

Recorded evaluation configuration: {"Zero-shot":"true"}. This record does not assert independent reproduction.

### Hard-routing MoE

Accuracy: 96.95. Result ID: 699eae585698665ec47b64a5.

[Reporting paper](<https://www.sota2.com/research/paper/699e81e06017c4460ffaccb4.md>)

Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B-Instruct","# Params (%)":"14.2G (100%)","# Trans. (%)":"14.2G (100%)","Fine-tuning setting":"DFL","Rank (rl)":32}. This record does not assert independent reproduction.

### FPFT

Accuracy: 93.31. Result ID: 699eae585698665ec47b64a2.

[Reporting paper](<https://www.sota2.com/research/paper/699e81e06017c4460ffaccb4.md>)

Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B-Instruct","# Params (%)":"14.2G (100%)","# Trans. (%)":"14.2G (100%)","Fine-tuning setting":"DFL","Rank (rl)":32}. This record does not assert independent reproduction.

### Sparse-and-Orthogonal LoRA

Accuracy: 92.6. Result ID: 699eae585698665ec47b64a7.

[Reporting paper](<https://www.sota2.com/research/paper/699e81e06017c4460ffaccb4.md>)

[Method origin paper](<https://www.sota2.com/research/paper/699e81e06017c4460ffaccb4.md>)

Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B-Instruct","# Params (%)":"87M (0.60%)","# Trans. (%)":"42M (0.30%)","Fine-tuning setting":"DFL","Rank (rl)":32}. This record does not assert independent reproduction.

### Exact computation

Accuracy: 91.62. Result ID: 69bcfcd947f34ce924a6cd99.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7bd41e209131cea8ac23.md>)

Recorded evaluation configuration: {"Model":"LLaDA-8B-Instruct"}. This record does not assert independent reproduction.

### single-threshold support basis

Accuracy: 91.58. Result ID: 69bcfcd947f34ce924a6cd9b.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7bd41e209131cea8ac23.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7bd41e209131cea8ac23.md>)

Recorded evaluation configuration: {"Model":"LLaDA-8B-Instruct","Degree of Chebyshev polynomial":"6"}. This record does not assert independent reproduction.

### LoRI

Accuracy: 90.82. Result ID: 699eae585698665ec47b64a4.

[Reporting paper](<https://www.sota2.com/research/paper/699e81e06017c4460ffaccb4.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7a661e209131cea82eb5.md>)

Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B-Instruct","# Params (%)":"168M (1.16%)","# Trans. (%)":"168M (1.16%)","Fine-tuning setting":"DFL","Rank (rl)":32}. This record does not assert independent reproduction.

### Sparse-and-Orthogonal LoRA (Single)

Accuracy: 90.69. Result ID: 699eae585698665ec47b64a6.

[Reporting paper](<https://www.sota2.com/research/paper/699e81e06017c4460ffaccb4.md>)

[Method origin paper](<https://www.sota2.com/research/paper/699e81e06017c4460ffaccb4.md>)

Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B-Instruct","# Params (%)":"87M (0.60%)","# Trans. (%)":"42M (0.30%)","Fine-tuning setting":"DFL","Rank (rl)":32}. This record does not assert independent reproduction.

### single-threshold support basis

Accuracy: 89.9. Result ID: 69bcfcd947f34ce924a6cd9a.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7bd41e209131cea8ac23.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7bd41e209131cea8ac23.md>)

Recorded evaluation configuration: {"Model":"LLaDA-8B-Instruct","Degree of Chebyshev polynomial":"4"}. This record does not assert independent reproduction.

### LoRA

Accuracy: 89.38. Result ID: 699eae585698665ec47b64a3.

[Reporting paper](<https://www.sota2.com/research/paper/699e81e06017c4460ffaccb4.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6b121e209131cea2bf59.md>)

[Implementation](<https://github.com/microsoft/LoRA>)

Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B-Instruct","# Params (%)":"362M (2.5%)","# Trans. (%)":"362M (2.5%)","Fine-tuning setting":"DFL","Rank (rl)":32}. This record does not assert independent reproduction.

### Hard-routing MoE

Accuracy: 87.56. Result ID: 699eae585698665ec47b649f.

[Reporting paper](<https://www.sota2.com/research/paper/699e81e06017c4460ffaccb4.md>)

Recorded evaluation configuration: {"Backbone":"Qwen2.5-1.5B-Instruct","# Params (%)":"3.2G (100%)","# Trans. (%)":"3.2G (100%)","Fine-tuning setting":"DFL","Rank (rl)":32}. This record does not assert independent reproduction.

### CT

Accuracy: 87.33. Result ID: 69b9071c47f34ce924a14522.

[Reporting paper](<https://www.sota2.com/research/paper/69b8e0090636e5bf4afbe9e0.md>)

Recorded evaluation configuration: {"Base Model":"Qwen2.5-7B","#P/B":"7.62","#AP/B":"7.62"}. This record does not assert independent reproduction.

### PT

Accuracy: 87.12. Result ID: 69b9071c47f34ce924a14521.

[Reporting paper](<https://www.sota2.com/research/paper/69b8e0090636e5bf4afbe9e0.md>)

Recorded evaluation configuration: {"Base Model":"Qwen2.5-7B","#P/B":"7.62","#AP/B":"7.62"}. This record does not assert independent reproduction.

### FineRMoE

Accuracy: 86.95. Result ID: 69b9071c47f34ce924a14527.

[Reporting paper](<https://www.sota2.com/research/paper/69b8e0090636e5bf4afbe9e0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69b8e0090636e5bf4afbe9e0.md>)

[Implementation](<https://github.com/liaoning97/FineRMoE>)

Recorded evaluation configuration: {"Base Model":"Qwen2.5-7B","#P/B":"26.65","#AP/B":"7.94"}. This record does not assert independent reproduction.

### C32A2

Accuracy: 86.32. Result ID: 69b9071c47f34ce924a14523.

[Reporting paper](<https://www.sota2.com/research/paper/69b8e0090636e5bf4afbe9e0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6e8e1e209131cea4091b.md>)

Recorded evaluation configuration: {"Base Model":"Qwen2.5-7B","#P/B":"184.42","#AP/B":"13.33"}. This record does not assert independent reproduction.

### NVShard

Accuracy: 84.97. Result ID: 69b9071c47f34ce924a14526.

[Reporting paper](<https://www.sota2.com/research/paper/69b8e0090636e5bf4afbe9e0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa79711e209131cea7da19.md>)

Recorded evaluation configuration: {"Base Model":"Qwen2.5-7B","#P/B":"47.55","#AP/B":"7.63"}. This record does not assert independent reproduction.

### FPFT

Accuracy: 84.49. Result ID: 699eae585698665ec47b649c.

[Reporting paper](<https://www.sota2.com/research/paper/699e81e06017c4460ffaccb4.md>)

Recorded evaluation configuration: {"Backbone":"Qwen2.5-1.5B-Instruct","# Params (%)":"3.2G (100%)","# Trans. (%)":"3.2G (100%)","Fine-tuning setting":"DFL","Rank (rl)":32}. This record does not assert independent reproduction.

### Sparse-and-Orthogonal LoRA

Accuracy: 84.41. Result ID: 699eae585698665ec47b64a1.

[Reporting paper](<https://www.sota2.com/research/paper/699e81e06017c4460ffaccb4.md>)

[Method origin paper](<https://www.sota2.com/research/paper/699e81e06017c4460ffaccb4.md>)

Recorded evaluation configuration: {"Backbone":"Qwen2.5-1.5B-Instruct","# Params (%)":"16M (0.54%)","# Trans. (%)":"8M (0.27%)","Fine-tuning setting":"DFL","Rank (rl)":32}. This record does not assert independent reproduction.

### Sparse-and-Orthogonal LoRA (Single)

Accuracy: 82.95. Result ID: 699eae585698665ec47b64a0.

[Reporting paper](<https://www.sota2.com/research/paper/699e81e06017c4460ffaccb4.md>)

[Method origin paper](<https://www.sota2.com/research/paper/699e81e06017c4460ffaccb4.md>)

Recorded evaluation configuration: {"Backbone":"Qwen2.5-1.5B-Instruct","# Params (%)":"16M (0.54%)","# Trans. (%)":"8M (0.27%)","Fine-tuning setting":"DFL","Rank (rl)":32}. This record does not assert independent reproduction.

### LoRI

Accuracy: 82.78. Result ID: 699eae585698665ec47b649e.

[Reporting paper](<https://www.sota2.com/research/paper/699e81e06017c4460ffaccb4.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7a661e209131cea82eb5.md>)

Recorded evaluation configuration: {"Backbone":"Qwen2.5-1.5B-Instruct","# Params (%)":"32M (1.05%)","# Trans. (%)":"32M (1.05%)","Fine-tuning setting":"DFL","Rank (rl)":32}. This record does not assert independent reproduction.

### PT

Accuracy: 81.02. Result ID: 69b9071c47f34ce924a1451a.

[Reporting paper](<https://www.sota2.com/research/paper/69b8e0090636e5bf4afbe9e0.md>)

Recorded evaluation configuration: {"Base Model":"Qwen2.5-1.5B","#P/B":"1.54","#AP/B":"1.54"}. This record does not assert independent reproduction.

### FineRMoE

Accuracy: 80.81. Result ID: 69b9071c47f34ce924a14520.

[Reporting paper](<https://www.sota2.com/research/paper/69b8e0090636e5bf4afbe9e0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69b8e0090636e5bf4afbe9e0.md>)

[Implementation](<https://github.com/liaoning97/FineRMoE>)

Recorded evaluation configuration: {"Base Model":"Qwen2.5-1.5B","#P/B":"5.64","#AP/B":"1.85"}. This record does not assert independent reproduction.

### CT

Accuracy: 80.72. Result ID: 69b9071c47f34ce924a1451b.

[Reporting paper](<https://www.sota2.com/research/paper/69b8e0090636e5bf4afbe9e0.md>)

Recorded evaluation configuration: {"Base Model":"Qwen2.5-1.5B","#P/B":"1.54","#AP/B":"1.54"}. This record does not assert independent reproduction.

### C32A2

Accuracy: 80.01. Result ID: 69b9071c47f34ce924a1451c.

[Reporting paper](<https://www.sota2.com/research/paper/69b8e0090636e5bf4afbe9e0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6e8e1e209131cea4091b.md>)

Recorded evaluation configuration: {"Base Model":"Qwen2.5-1.5B","#P/B":"37.62","#AP/B":"2.94"}. This record does not assert independent reproduction.

### Mistral-7B

Accuracy: 79.55. Result ID: 69d4b3a447f34ce924bc2c3c.

[Reporting paper](<https://www.sota2.com/research/paper/69d48f957d0172140ec74311.md>)

Recorded evaluation configuration: {"Ratio":"0%","Evaluation Protocol":"zero-shot"}. This record does not assert independent reproduction.

### ShapLoRA

Accuracy: 79.3. Result ID: 69966e6f5698665ec4b557d5.

[Reporting paper](<https://www.sota2.com/research/paper/69784700a2b28e761d87c8c8.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69784700a2b28e761d87c8c8.md>)

Recorded evaluation configuration: {"Tunable Params":"22.8M","Backbone":"LLaMA-3 8B","Evaluation Protocol":"Supervised fine-tuning (SFT)","Decoding Strategy":"Beam search (size 3)"}. This record does not assert independent reproduction.

### DoRA

Accuracy: 77.9. Result ID: 69966e6f5698665ec4b557d4.

[Reporting paper](<https://www.sota2.com/research/paper/69784700a2b28e761d87c8c8.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa74031e209131cea5f393.md>)

[Implementation](<https://github.com/NVlabs/DoRA>)

Recorded evaluation configuration: {"Tunable Params":"22.6M","Backbone":"LLaMA-3 8B","Evaluation Protocol":"Supervised fine-tuning (SFT)","Decoding Strategy":"Beam search (size 3)"}. This record does not assert independent reproduction.

### NVShard

Accuracy: 77.82. Result ID: 69b9071c47f34ce924a1451f.

[Reporting paper](<https://www.sota2.com/research/paper/69b8e0090636e5bf4afbe9e0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa79711e209131cea7da19.md>)

Recorded evaluation configuration: {"Base Model":"Qwen2.5-1.5B","#P/B":"9.88","#AP/B":"1.78"}. This record does not assert independent reproduction.

### MOELoRA

Accuracy: 77.5. Result ID: 69966e6f5698665ec4b557d3.

[Reporting paper](<https://www.sota2.com/research/paper/69784700a2b28e761d87c8c8.md>)

Recorded evaluation configuration: {"Tunable Params":"29.9M","Backbone":"LLaMA-3 8B","Evaluation Protocol":"Supervised fine-tuning (SFT)","Decoding Strategy":"Beam search (size 3)"}. This record does not assert independent reproduction.

### Qwen2.5-7B

Accuracy: 77.5. Result ID: 699ad8d95698665ec4684d34.

[Reporting paper](<https://www.sota2.com/research/paper/6995478c09c33900fcd6d5e1.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa74bf1e209131cea6378d.md>)

[Implementation](<https://github.com/QwenLM/Qwen2>)

Recorded evaluation configuration: {"Model Type":"Dense","Active Parameters":"7B","Total Parameters":"7B"}. This record does not assert independent reproduction.

### Reasoning

Accuracy: 77.25. Result ID: 69a1774c35dfcabb00e73852.

[Reporting paper](<https://www.sota2.com/research/paper/69647fe953fdd5c36174478b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa76a91e209131cea6e3eb.md>)

Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B family"}. This record does not assert independent reproduction.

### AutoLoRA

Accuracy: 76.9. Result ID: 69966e6f5698665ec4b557d2.

[Reporting paper](<https://www.sota2.com/research/paper/69784700a2b28e761d87c8c8.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa72951e209131cea574ec.md>)

Recorded evaluation configuration: {"Tunable Params":"23.1M","Backbone":"LLaMA-3 8B","Evaluation Protocol":"Supervised fine-tuning (SFT)","Decoding Strategy":"Beam search (size 3)"}. This record does not assert independent reproduction.

### OLMoE-1B-7B

Accuracy: 76.3. Result ID: 699ad8d95698665ec4684d42.

[Reporting paper](<https://www.sota2.com/research/paper/6995478c09c33900fcd6d5e1.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa773e1e209131cea71f14.md>)

Recorded evaluation configuration: {"Model Type":"MoE","Active Parameters":"1B","Total Parameters":"7B"}. This record does not assert independent reproduction.

### SSP

Accuracy: 75.2. Result ID: 69966e6f5698665ec4b557cf.

[Reporting paper](<https://www.sota2.com/research/paper/69784700a2b28e761d87c8c8.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6cbe1e209131cea35b95.md>)

Recorded evaluation configuration: {"Tunable Params":"80.6M","Backbone":"LLaMA-3 8B","Evaluation Protocol":"Supervised fine-tuning (SFT)","Decoding Strategy":"Beam search (size 3)"}. This record does not assert independent reproduction.

### IAPT

Accuracy: 75.1. Result ID: 69966e6f5698665ec4b557cc.

[Reporting paper](<https://www.sota2.com/research/paper/69784700a2b28e761d87c8c8.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa738f1e209131cea5d3db.md>)

Recorded evaluation configuration: {"Tunable Params":"20.9M","Backbone":"LLaMA-3 8B","Evaluation Protocol":"Supervised fine-tuning (SFT)","Decoding Strategy":"Beam search (size 3)"}. This record does not assert independent reproduction.

### AdaLoRA

Accuracy: 75.1. Result ID: 69966e6f5698665ec4b557d1.

[Reporting paper](<https://www.sota2.com/research/paper/69784700a2b28e761d87c8c8.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa71861e209131cea51c24.md>)

Recorded evaluation configuration: {"Tunable Params":"23.2M","Backbone":"LLaMA-3 8B","Evaluation Protocol":"Supervised fine-tuning (SFT)","Decoding Strategy":"Beam search (size 3)"}. This record does not assert independent reproduction.

### LoRA

Accuracy: 74.4. Result ID: 69966e6f5698665ec4b557d0.

[Reporting paper](<https://www.sota2.com/research/paper/69784700a2b28e761d87c8c8.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6b121e209131cea2bf59.md>)

[Implementation](<https://github.com/microsoft/LoRA>)

Recorded evaluation configuration: {"Tunable Params":"22.5M","Backbone":"LLaMA-3 8B","Evaluation Protocol":"Supervised fine-tuning (SFT)","Decoding Strategy":"Beam search (size 3)"}. This record does not assert independent reproduction.

### LLaDA (Base)

Accuracy: 74.2. Result ID: 699b12535698665ec4a5d648.

[Reporting paper](<https://www.sota2.com/research/paper/6997eac609c33900fcd76806.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7c3e1e209131cea8d1d9.md>)

Recorded evaluation configuration: {"Pruning Ratio":"0.0"}. This record does not assert independent reproduction.

### Qwen2.5

Accuracy: 74.1. Result ID: 6996b0db5698665ec4fb0f53.

[Reporting paper](<https://www.sota2.com/research/paper/69520aef53fdd5c36173d565.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa767e1e209131cea6d6a7.md>)

Recorded evaluation configuration: {"Size":"1.54B","Pre-training tokens":"18T"}. This record does not assert independent reproduction.

### Qwen2.5-3B

Accuracy: 73.9. Result ID: 699ad8d95698665ec4684d35.

[Reporting paper](<https://www.sota2.com/research/paper/6995478c09c33900fcd6d5e1.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa74bf1e209131cea6378d.md>)

[Implementation](<https://github.com/QwenLM/Qwen2>)

Recorded evaluation configuration: {"Model Type":"Dense","Active Parameters":"3B","Total Parameters":"3B"}. This record does not assert independent reproduction.

### Gamayun

Accuracy: 73.7. Result ID: 6996b0db5698665ec4fb0f51.

[Reporting paper](<https://www.sota2.com/research/paper/69520aef53fdd5c36173d565.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69520aef53fdd5c36173d565.md>)

Recorded evaluation configuration: {"Size":"1.47B","Pre-training tokens":"2.6T"}. This record does not assert independent reproduction.

### Dense

Accuracy: 73.6. Result ID: 69ddf71c47f34ce924c54ba5.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7c1d1e209131cea8c767.md>)

Recorded evaluation configuration: {"Base Model":"Llama-3.2-3B","Model architecture type":"Dense","Few-shot":"true"}. This record does not assert independent reproduction.

### SmolLM2-1.7B

Accuracy: 73.4. Result ID: 699ad8d95698665ec4684d3f.

[Reporting paper](<https://www.sota2.com/research/paper/6995478c09c33900fcd6d5e1.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa76d11e209131cea6f3fe.md>)

Recorded evaluation configuration: {"Model Type":"Dense","Active Parameters":"1.7B","Total Parameters":"1.7B"}. This record does not assert independent reproduction.

### Learned-Adapter

Accuracy: 73.1. Result ID: 69966e6f5698665ec4b557ca.

[Reporting paper](<https://www.sota2.com/research/paper/69784700a2b28e761d87c8c8.md>)

Recorded evaluation configuration: {"Tunable Params":"21.1M","Backbone":"LLaMA-3 8B","Evaluation Protocol":"Supervised fine-tuning (SFT)","Decoding Strategy":"Beam search (size 3)"}. This record does not assert independent reproduction.

### (IA)3

Accuracy: 73.1. Result ID: 69966e6f5698665ec4b557ce.

[Reporting paper](<https://www.sota2.com/research/paper/69784700a2b28e761d87c8c8.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6d491e209131cea38c7f.md>)

[Implementation](<https://github.com/r-three/t-few>)

Recorded evaluation configuration: {"Tunable Params":"23.1M","Backbone":"LLaMA-3 8B","Evaluation Protocol":"Supervised fine-tuning (SFT)","Decoding Strategy":"Beam search (size 3)"}. This record does not assert independent reproduction.

### MiCRo

Accuracy: 73. Result ID: 69ddf71c47f34ce924c54ba7.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7c1d1e209131cea8c767.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7c1d1e209131cea8c767.md>)

Recorded evaluation configuration: {"Base Model":"Llama-3.2-3B","Model architecture type":"MiCRo","Few-shot":"true"}. This record does not assert independent reproduction.

### PHSA

Accuracy: 72.69. Result ID: 6996a1f15698665ec4eb7669.

[Reporting paper](<https://www.sota2.com/research/paper/695de86653fdd5c361743a1a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/695de86653fdd5c361743a1a.md>)

Recorded evaluation configuration: {"Training Tokens":"100B","Inference Top-K":4,"Training Top-K":4}. This record does not assert independent reproduction.

### PHSA

Accuracy: 72.64. Result ID: 6996a1f15698665ec4eb7668.

[Reporting paper](<https://www.sota2.com/research/paper/695de86653fdd5c361743a1a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/695de86653fdd5c361743a1a.md>)

Recorded evaluation configuration: {"Training Tokens":"100B","Inference Top-K":"N/A","Training Top-K":4}. This record does not assert independent reproduction.

### Qwen3

Accuracy: 72.6. Result ID: 6996b0db5698665ec4fb0f56.

[Reporting paper](<https://www.sota2.com/research/paper/69520aef53fdd5c36173d565.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa788e1e209131cea79116.md>)

Recorded evaluation configuration: {"Size":"1.7B","Pre-training tokens":"36T"}. This record does not assert independent reproduction.

### InfLLM v2

Accuracy: 72.52. Result ID: 6996a1f15698665ec4eb7666.

[Reporting paper](<https://www.sota2.com/research/paper/695de86653fdd5c361743a1a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7bb81e209131cea8a238.md>)

Recorded evaluation configuration: {"Training Tokens":"100B","Inference Top-K":"N/A","Training Top-K":2}. This record does not assert independent reproduction.

### Qwen2.5-1.5B

Accuracy: 72.5. Result ID: 699ad8d95698665ec4684d36.

[Reporting paper](<https://www.sota2.com/research/paper/6995478c09c33900fcd6d5e1.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa74bf1e209131cea6378d.md>)

[Implementation](<https://github.com/QwenLM/Qwen2>)

Recorded evaluation configuration: {"Model Type":"Dense","Active Parameters":"1.5B","Total Parameters":"1.5B"}. This record does not assert independent reproduction.

### Parallel-Adapter

Accuracy: 72.4. Result ID: 69966e6f5698665ec4b557c9.

[Reporting paper](<https://www.sota2.com/research/paper/69784700a2b28e761d87c8c8.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6bb41e209131cea2fb2a.md>)

Recorded evaluation configuration: {"Tunable Params":"20.9M","Backbone":"LLaMA-3 8B","Evaluation Protocol":"Supervised fine-tuning (SFT)","Decoding Strategy":"Beam search (size 3)"}. This record does not assert independent reproduction.

### ExpertWeaver-E64-A14-S2(Qwen2.5-7B)

Accuracy: 72.4. Result ID: 699ad8d95698665ec4684d46.

[Reporting paper](<https://www.sota2.com/research/paper/6995478c09c33900fcd6d5e1.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6995478c09c33900fcd6d5e1.md>)

Recorded evaluation configuration: {"Model Type":"MoE","Active Parameters":"3.5B","Total Parameters":"7B","Training Budget":"200B tokens"}. This record does not assert independent reproduction.

### InfLLM v2

Accuracy: 72.31. Result ID: 6996a1f15698665ec4eb7667.

[Reporting paper](<https://www.sota2.com/research/paper/695de86653fdd5c361743a1a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7bb81e209131cea8a238.md>)

Recorded evaluation configuration: {"Training Tokens":"100B","Inference Top-K":2,"Training Top-K":2}. This record does not assert independent reproduction.

### BitFit

Accuracy: 72.3. Result ID: 69966e6f5698665ec4b557cd.

[Reporting paper](<https://www.sota2.com/research/paper/69784700a2b28e761d87c8c8.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6d5a1e209131cea39632.md>)

[Implementation](<https://www.github.com/benzakenelad/BitFit>)

Recorded evaluation configuration: {"Tunable Params":"25.2M","Backbone":"LLaMA-3 8B","Evaluation Protocol":"Supervised fine-tuning (SFT)","Decoding Strategy":"Beam search (size 3)"}. This record does not assert independent reproduction.

### InfLLM v2

Accuracy: 72.18. Result ID: 6996a1f15698665ec4eb766b.

[Reporting paper](<https://www.sota2.com/research/paper/695de86653fdd5c361743a1a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7bb81e209131cea8a238.md>)

Recorded evaluation configuration: {"Training Tokens":"100B","Inference Top-K":4,"Training Top-K":4}. This record does not assert independent reproduction.

### InfLLM v2

Accuracy: 72.1. Result ID: 6996a1f15698665ec4eb766a.

[Reporting paper](<https://www.sota2.com/research/paper/695de86653fdd5c361743a1a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7bb81e209131cea8a238.md>)

Recorded evaluation configuration: {"Training Tokens":"100B","Inference Top-K":"N/A","Training Top-K":4}. This record does not assert independent reproduction.

### Dense

Accuracy: 71.97. Result ID: 6996a1f15698665ec4eb7663.

[Reporting paper](<https://www.sota2.com/research/paper/695de86653fdd5c361743a1a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa703d1e209131cea49db1.md>)

[Implementation](<https://github.com/tensorflow/tensor2tensor>)

Recorded evaluation configuration: {"Training Tokens":"100B","Inference Top-K":2}. This record does not assert independent reproduction.

### Dense

Accuracy: 71.84. Result ID: 6996a1f15698665ec4eb7662.

[Reporting paper](<https://www.sota2.com/research/paper/695de86653fdd5c361743a1a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa703d1e209131cea49db1.md>)

[Implementation](<https://github.com/tensorflow/tensor2tensor>)

Recorded evaluation configuration: {"Training Tokens":"100B","Inference Top-K":"N/A"}. This record does not assert independent reproduction.

### Sink-Aware (Ours)

Accuracy: 71.75. Result ID: 699b12535698665ec4a5d64a.

[Reporting paper](<https://www.sota2.com/research/paper/6997eac609c33900fcd76806.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6997eac609c33900fcd76806.md>)

[Implementation](<https://github.com/VILA-Lab/Sink-Aware-Pruning>)

Recorded evaluation configuration: {"Pruning Ratio":"0.3"}. This record does not assert independent reproduction.

### Llama-3.2-3B

Accuracy: 71.6. Result ID: 699ad8d95698665ec4684d37.

[Reporting paper](<https://www.sota2.com/research/paper/6995478c09c33900fcd6d5e1.md>)

Recorded evaluation configuration: {"Model Type":"Dense","Active Parameters":"3B","Total Parameters":"3B"}. This record does not assert independent reproduction.

### MoB

Accuracy: 71.6. Result ID: 69ddf71c47f34ce924c54ba6.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7c1d1e209131cea8c767.md>)

Recorded evaluation configuration: {"Base Model":"Llama-3.2-3B","Model architecture type":"MoB","Few-shot":"true"}. This record does not assert independent reproduction.

### SVD-LLM

Accuracy: 71.34. Result ID: 69d4b3a447f34ce924bc2c3f.

[Reporting paper](<https://www.sota2.com/research/paper/69d48f957d0172140ec74311.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa777a1e209131cea73742.md>)

Recorded evaluation configuration: {"Ratio":"20%","Evaluation Protocol":"zero-shot"}. This record does not assert independent reproduction.

### EuroLM

Accuracy: 71.3. Result ID: 6996b0db5698665ec4fb0f54.

[Reporting paper](<https://www.sota2.com/research/paper/69520aef53fdd5c36173d565.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa74f31e209131cea64c25.md>)

Recorded evaluation configuration: {"Size":"1.66B","Pre-training tokens":"4T"}. This record does not assert independent reproduction.

### PHSA

Accuracy: 71.21. Result ID: 6996a1f15698665ec4eb7665.

[Reporting paper](<https://www.sota2.com/research/paper/695de86653fdd5c361743a1a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/695de86653fdd5c361743a1a.md>)

Recorded evaluation configuration: {"Training Tokens":"100B","Inference Top-K":2,"Training Top-K":2}. This record does not assert independent reproduction.

### PHSA

Accuracy: 71.13. Result ID: 6996a1f15698665ec4eb7664.

[Reporting paper](<https://www.sota2.com/research/paper/695de86653fdd5c361743a1a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/695de86653fdd5c361743a1a.md>)

Recorded evaluation configuration: {"Training Tokens":"100B","Inference Top-K":"N/A","Training Top-K":2}. This record does not assert independent reproduction.

### OLMoE-1B-7B\*

Accuracy: 71. Result ID: 699ad8d95698665ec4684d43.

[Reporting paper](<https://www.sota2.com/research/paper/6995478c09c33900fcd6d5e1.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa773e1e209131cea71f14.md>)

Recorded evaluation configuration: {"Model Type":"MoE","Active Parameters":"1B","Total Parameters":"7B","Training Budget":"500B tokens"}. This record does not assert independent reproduction.

### Olmo2

Accuracy: 70.9. Result ID: 6996b0db5698665ec4fb0f52.

[Reporting paper](<https://www.sota2.com/research/paper/69520aef53fdd5c36173d565.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7bf21e209131cea8b84e.md>)

Recorded evaluation configuration: {"Size":"1.48B","Pre-training tokens":"4T"}. This record does not assert independent reproduction.

### Linear

Accuracy: 70.72. Result ID: 69a1774c35dfcabb00e73853.

[Reporting paper](<https://www.sota2.com/research/paper/69647fe953fdd5c36174478b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa65651e209131cea09f96.md>)

[Implementation](<https://github.com/timgaripov/swa>)

Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B family"}. This record does not assert independent reproduction.

### MoB

Accuracy: 70. Result ID: 69ddf71c47f34ce924c54ba0.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7c1d1e209131cea8c767.md>)

Recorded evaluation configuration: {"Base Model":"SmollM2-360M","Model architecture type":"MoB","Few-shot":"true"}. This record does not assert independent reproduction.

### SoLA

Accuracy: 69.99. Result ID: 69d4b3a447f34ce924bc2c40.

[Reporting paper](<https://www.sota2.com/research/paper/69d48f957d0172140ec74311.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69d48f957d0172140ec74311.md>)

Recorded evaluation configuration: {"Ratio":"20%","Evaluation Protocol":"zero-shot"}. This record does not assert independent reproduction.

### MiCRo

Accuracy: 69.9. Result ID: 69ddf71c47f34ce924c54ba1.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7c1d1e209131cea8c767.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7c1d1e209131cea8c767.md>)

Recorded evaluation configuration: {"Base Model":"SmollM2-360M","Model architecture type":"MiCRo","Few-shot":"true"}. This record does not assert independent reproduction.

### Dense

Accuracy: 69.7. Result ID: 69ddf71c47f34ce924c54b9f.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7c1d1e209131cea8c767.md>)

Recorded evaluation configuration: {"Base Model":"SmollM2-360M","Model architecture type":"Dense","Few-shot":"true"}. This record does not assert independent reproduction.

### ExpertWeaver-E64-A14-S2(OLMo-7B)

Accuracy: 69.3. Result ID: 699ad8d95698665ec4684d45.

[Reporting paper](<https://www.sota2.com/research/paper/6995478c09c33900fcd6d5e1.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6995478c09c33900fcd6d5e1.md>)

Recorded evaluation configuration: {"Model Type":"MoE","Active Parameters":"1B","Total Parameters":"7B","Training Budget":"200B tokens"}. This record does not assert independent reproduction.

### LLaDA-structure

Accuracy: 69.07. Result ID: 699b12535698665ec4a5d649.

[Reporting paper](<https://www.sota2.com/research/paper/6997eac609c33900fcd76806.md>)

Recorded evaluation configuration: {"Pruning Ratio":"0.3"}. This record does not assert independent reproduction.

### Gemma3

Accuracy: 68.9. Result ID: 6996b0db5698665ec4fb0f4f.

[Reporting paper](<https://www.sota2.com/research/paper/69520aef53fdd5c36173d565.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa779f1e209131cea746cd.md>)

Recorded evaluation configuration: {"Size":"1B","Pre-training tokens":"2T"}. This record does not assert independent reproduction.

### PT

Accuracy: 68.9. Result ID: 69b9071c47f34ce924a14513.

[Reporting paper](<https://www.sota2.com/research/paper/69b8e0090636e5bf4afbe9e0.md>)

Recorded evaluation configuration: {"Base Model":"Qwen2.5-0.5B","#P/B":"0.49","#AP/B":"0.49"}. This record does not assert independent reproduction.

### SmolLM2

Accuracy: 68.8. Result ID: 6996b0db5698665ec4fb0f55.

[Reporting paper](<https://www.sota2.com/research/paper/69520aef53fdd5c36173d565.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa76d11e209131cea6f3fe.md>)

Recorded evaluation configuration: {"Size":"1.7B","Pre-training tokens":"11T"}. This record does not assert independent reproduction.

Page 1 of 3. This is an index page, not the full collection.

- [Next index page](<https://www.sota2.com/research/sota/science-question-answering-on-arc-easy.md?metric=Accuracy&page=2>): Follow this link to discover more pages.
