Science QA on ARC easy
77.9AccuracyVanilla
Evaluation Results
| Method | Links | |
|---|---|---|
| VanillaBackbone=Mistral-7B-Instruct2026.05 | 77.9 | |
| DT-CD⋆Backbone=Mistral-7B-Instruct2026.05 | 77.8 | |
| COFTBackbone=Mistral-7B-Instruct2026.05 | 77.8 | |
| SDDBackbone=Mistral-7B-Instruct2026.05 | 77.4 | |
| DExpertsBackbone=Mistral-7B-Instruct2026.05 | 77.2 | |
| VanillaBackbone=LLaMA-2-13B2026.05 | 74.6 | |
| COFTBackbone=LLaMA-2-13B2026.05 | 74.5 | |
| DT-CD⋆Backbone=LLaMA-2-13B2026.05 | 74.4 | |
| SDDBackbone=LLaMA-2-13B2026.05 | 74 | |
| DExpertsBackbone=LLaMA-2-13B2026.05 | 73.7 | |
| PrismScale=Large, Evaluation Protocol=Zero-shot2026.06 | 45.8 | |
| BaselineScale=Large, Evaluation Protocol=Zero-shot2026.06 | 45.5 | |
| PrismScale=Medium, Evaluation Protocol=Zero-shot2026.06 | 43.1 | |
| BaselineScale=Medium, Evaluation Protocol=Zero-shot2026.06 | 42.1 | |
| PrismScale=Small, Evaluation Protocol=Zero-shot2026.06 | 37.7 | |
| BaselineScale=Small, Evaluation Protocol=Zero-shot2026.06 | 37.6 | |
| Mamba-2model size=152M, mb=42026.06 | 36.4 | |
| SISAmodel size=152M, ds=32, mb=22026.06 | 35.8 | |
| SISAmodel size=152M, ds=64, mb=42026.06 | 35.4 | |
| SISAmodel size=152M, ds=128, mb=42026.06 | 35 | |
| Mamba-3model size=152M, mb=42026.06 | 34.9 | |
| SISAmodel size=152M, ds=16, mb=42026.06 | 34.7 | |
| Transformermodel size=152M, mb=42026.06 | 33.3 |