Question Answering on SciQ (Accuracy)
97.2AccuracyMSSRfull
Evaluation results
| Method | Links | ||
|---|---|---|---|
| MSSRfullBackbone=Qwen2.5-7B2026.03 | 97.2 | — | |
| MSSRfullBackbone=Gemma2-9B2026.03 | 96.9 | — | |
| MSSRsplBackbone=Qwen2.5-7B2026.03 | 96.6 | — | |
| LossBackbone=Qwen2.5-7B2026.03 | 96.2 | — | |
| LossBackbone=Gemma2-9B2026.03 | 96.1 | — | |
| GPT-NeoXParameters=20B, Evaluation=Five-shot2022.04 | 96 | — | |
| AccuBackbone=Gemma2-9B2026.03 | 95.9 | — | |
| FLAN-T5Model Variant=xlarge, Model Parameters=3B2023.07 | 95.7 | — | |
| MSSRsplBackbone=Gemma2-9B2026.03 | 95.7 | — | |
| Qwen3-1.7B#Params=1.7B/1.7B, #Tokens=36T, Zero-shot=true2026.02 | 95.6 | — | |
| Pythia (Deduplicated)Model size=12B, Evaluation protocol=Five-shot2023.04 | 95.5 | — | |
| AccuBackbone=Qwen2.5-7B2026.03 | 95.5 | — | |
| PythiaNumber of Parameters=12B, Evaluation Protocol=Five-shot2023.04 | 95.3 | — | |
| SPES-9B#Params=3.1B/9B, #Tokens=400B, Zero-shot=true, initialized from=pretrained dense model2026.02 | 95.3 | — | |
| MSSRschBackbone=Qwen2.5-7B2026.03 | 95.3 | — | |
| Pythia (Deduplicated)Model size=6.9B, Evaluation protocol=Five-shot2023.04 | 95.2 | — | |
| NoneBackbone=Gemma2-9B2026.03 | 95.2 | — | |
| MSSRschBackbone=Gemma2-9B2026.03 | 95.2 | — | |
| PythiaNumber of Parameters=6.9B, Evaluation Protocol=Five-shot2023.04 | 95.1 | — | |
| FixedBackbone=Qwen2.5-7B2026.03 | 95.1 | — | |
| GPT-3Model Variant=DaVinci, Zero-shot=true2022.04 | 94.9 | — | |
| FLAN-T5Model Variant=large, Model Parameters=780M2023.07 | 94.9 | — | |
| OLMoE-1B-7B#Params=1.3B/7B, #Tokens=5T, Zero-shot=true2026.02 | 94.9 | — | |
| StableLM 1.6Bshot=zero-shot, normalization=none2024.02 | 94.7 | — | |
| LLaMA2-7BNumber of Parameters=7B, Backbone=LLaMA2, Shots=02024.07 | 94.7 | — | |
| OLMoE-1B-7B-0924Model Type=MoE2026.04 | 94.6 | — | |
| Llama 2 7Bshot=zero-shot, normalization=none2024.02 | 94.5 | — | |
| MSSRfullBackbone=Llama-3.1-8B2026.03 | 94.5 | — | |
| MEFT1Model=OPT6.7B, Rank=64, #Param. (%)=0.25, Peak Memory (GB)=33.67, Activation Memory (GB)=8.01, Training Time (s)=200.4, Precision=FP162023.06 | 94.4 | — | |
| FixedBackbone=Gemma2-9B2026.03 | 94.4 | — | |
| PythiaNumber of Parameters=2.8B, Evaluation Protocol=Five-shot2023.04 | 94.3 | — | |
| Pythia (Deduplicated)Model size=2.8B, Evaluation protocol=Five-shot2023.04 | 94.2 | — | |
| Qwen2.5-1.5B#Params=1.5B/1.5B, #Tokens=18T, Zero-shot=true2026.02 | 94.1 | — | |
| Falcon-7Bshot=zero-shot, normalization=none2024.02 | 93.9 | — | |
| LLaMA 7Bshot=zero-shot, normalization=none2024.02 | 93.9 | — | |
| Llama2 7BCR=None2025.09 | 93.9 | — | |
| Llama3 8BCompression Ratio=Baseline2025.09 | 93.9 | — | |
| Llama3 8B2026.02 | 93.9 | — | |
| OLMo-7Bshot=zero-shot, normalization=none2024.02 | 93.8 | — | |
| Ettin-Dec-1BModel Size Category=XL, Parameters=908M-1.2B, Zero-shot=true, Closed-book=true2025.07 | 93.8 | — | |
| ALIGNModel Variant=large, Model Parameters=355M2023.07 | 93.7 | — | |
| MPT-7Bshot=zero-shot, normalization=none2024.02 | 93.7 | — | |
| Mula-1BModel Type=Dense2026.04 | 93.7 | — | |
| Qwen3-0.6B#Params=0.6B/0.6B, #Tokens=36T, Zero-shot=true2026.02 | 93.5 | — | |
| AccuBackbone=Llama-3.1-8B2026.03 | 93.4 | — | |
| Mula-7B-A1BModel Type=MoE2026.04 | 93.4 | — | |
| LoRAModel=OPT1.3B, Rank=64, #Param. (%)=0.64, Peak Memory (GB)=11.42, Activation Memory (GB)=6.27, Training Time (s)=36.6, Precision=FP162023.06 | 93.3 | — | |
| MSSRsplBackbone=Llama-3.1-8B2026.03 | 93.3 | — | |
| SmolLM2-1.7B#Params=1.7B/1.7B, #Tokens=11T, Zero-shot=true2026.02 | 93.2 | — | |
| MEFT1Model=OPT1.3B, Rank=64, #Param. (%)=0.64, Peak Memory (GB)=9.2, Activation Memory (GB)=4.05, Training Time (s)=45.2, Precision=FP162023.06 | 93.1 | — | |
| Qwen2.5-0.5B#Params=0.5B/0.5B, #Tokens=18T, Zero-shot=true2026.02 | 93 | — | |
| RPJ-INCITE-7Bshot=zero-shot, normalization=none2024.02 | 92.9 | — | |
| GPT-NeoXModel Size=20B, Zero-shot=true2022.04 | 92.8 | — | |
| Pythia (Deduplicated)Model size=1.4B, Evaluation protocol=Five-shot2023.04 | 92.6 | — | |
| NoneBackbone=Qwen2.5-7B2026.03 | 92.6 | — | |
| Full FTModel=OPT1.3B, #Param. (%)=100, Peak Memory (GB)=28.31, Activation Memory (GB)=8.23, Training Time (s)=128, Precision=FP162023.06 | 92.5 | — | |
| ALIGNModel Variant=base, Model Parameters=125M2023.07 | 92.4 | — | |
| GPT3.5Source Task=RACE2024.05 | 92.2 | — | |
| LossBackbone=Llama-3.1-8B2026.03 | 92.2 | — | |
| MSSRschBackbone=Llama-3.1-8B2026.03 | 92.1 | — | |
| PythiaNumber of Parameters=1B, Evaluation Protocol=Five-shot2023.04 | 92 | — | |
| PythiaNumber of Parameters=1.4B, Evaluation Protocol=Five-shot2023.04 | 92 | — | |
| SpanNormParam=5B, Tokens=200B, Architecture=Dense2026.01 | 92 | — | |
| NoneBackbone=Llama-3.1-8B2026.03 | 92 | — | |
| VERSATILEFFNBase Model Params=1.21B, Protocol=Zero-shot2025.12 | 91.9 | — | |
| GPT-3Model Variant=Curie, Zero-shot=true2022.04 | 91.8 | — | |
| Open-LLaMA-3B#Params=3B/3B, #Tokens=1T, Zero-shot=true2026.02 | 91.8 | — | |
| Ettin-Dec-400mModel Size Category=Large, Parameters=360-410M, Zero-shot=true, Closed-book=true2025.07 | 91.8 | — | |
| OLMo-1B-0724Model Size Category=XL, Parameters=908M-1.2B, Zero-shot=true, Closed-book=true2025.07 | 91.8 | — | |
| Pythia (Deduplicated)Model size=1B, Evaluation protocol=Five-shot2023.04 | 91.6 | — | |
| TOPSize=7B, Zero-shot=true2025.08 | 91.6 | — | |
| 4-LOOPBase Model Params=1.21B, Protocol=Zero-shot2025.12 | 91.5 | — | |
| GPT3.5Source Task=ARC-Easy2024.05 | 91.4 | — | |
| 6-LOOPBase Model Params=1.21B, Protocol=Zero-shot2025.12 | 91.4 | — | |
| GPT-JParameters=6B, Evaluation=Five-shot2022.04 | 91.3 | — | |
| Llama3.2-1B#Params=1.1B/1.1B, #Tokens=9T, Zero-shot=true2026.02 | 91.3 | — | |
| GPT3.5Source Task=Zero-shot2024.05 | 91.2 | — | |
| FixedBackbone=Llama-3.1-8B2026.03 | 91.2 | — | |
| Pythia 6.9Bshot=zero-shot, normalization=none2024.02 | 91.1 | — | |
| MOEBase Model Params=1.21B, Protocol=Zero-shot2025.12 | 91.1 | — | |
| 2-LOOPBase Model Params=1.21B, Protocol=Zero-shot2025.12 | 91.1 | — | |
| GPT-JModel Size=6B, Zero-shot=true2022.04 | 91 | — | |
| GPT3.5Source Task=Commonsense-QA2024.05 | 91 | — | |
| TransActNumber of Parameters=2.6B, Backbone=LLaMA, Shots=02024.07 | 91 | — | |
| Basis SharingCR=0.22025.09 | 91 | — | |
| COMPOTCR=0.22026.02 | 91 | — | |
| SpanNormParam=A2.4B-16B, Tokens=200B, Architecture=MoE2026.01 | 90.8 | — | |
| PreNormParam=5B, Tokens=200B, Architecture=Dense2026.01 | 90.7 | — | |
| GPT3.5Source Task=BoolQ2024.05 | 90.6 | — | |
| OpenELM-1.1B#Params=1.1B/1.1B, #Tokens=1.5T, Zero-shot=true2026.02 | 90.6 | — | |
| TinyLlama 1.1Bshot=zero-shot, normalization=none2024.02 | 90.5 | — | |
| FRAMESelection Metric=PPL&PD, Model Size=1.3B2025.02 | 90.5 | — | |
| ZeroShotModel=OPT6.7B, Zero-shot=true2023.06 | 90.1 | — | |
| LLM-Pruner-2.6BNumber of Parameters=2.6B, Backbone=LLaMA, Shots=02024.07 | 90.1 | — | |
| COMPOTCR=0.32026.02 | 90.1 | — | |
| GPT3.5Source Task=AG-news2024.05 | 89.9 | — | |
| Sheared-LLaMA-2.7BNumber of Parameters=2.7B, Backbone=LLaMA, Shots=02024.07 | 89.9 | — | |
| LLaMA-MoE-3.0B#Params=3.0B/7B, #Tokens=2.2T, Zero-shot=true, initialized from=pretrained dense model2026.02 | 89.9 | — | |
| SPES-7B#Params=1.6B/7B, #Tokens=500B, Zero-shot=true2026.02 | 89.9 | — | |
| GPT3.5Source Task=QQP2024.05 | 89.8 | — |