Question Answering on ARC Challenge (test)
91.2AccuracyMLP Probe
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| MLP ProbeEvaluation Protocol=Probe2026.02 | 91.2 | — | — | — | |
| Teacher 5-shotEvaluation Protocol=5-shot2026.02 | 89.7 | — | — | — | |
| LoRAModel=LiquidAI/LFM2-700M2026.05 | 68.8 | — | — | — | |
| Queryable LoRAModel=LiquidAI/LFM2-700M2026.05 | 67.2 | — | — | — | |
| Instruction-Queryable LoRAModel=Qwen/Qwen3-0.6B2026.05 | 65.6 | — | — | — | |
| FP16Model=LLAMA3-70B, Evaluation Protocol=Zero-shot, Bit-width=Full-precision2024.03 | 64.25 | — | — | — | |
| Instruction-Queryable LoRAModel=LiquidAI/LFM2-700M2026.05 | 62.5 | — | — | — | |
| Queryable LoRAModel=LiquidAI/LFM2.5-350M2026.05 | 62.5 | — | — | — | |
| LoRAModel=Qwen/Qwen3-0.6B2026.05 | 60.9 | — | — | — | |
| MISTRAL-7BModel Architecture=MISTRAL-7B, Configuration Type=Dense, Evaluation Protocol=Zero-shot, Sparsity Level=0%2026.01 | 60.07 | — | — | — | |
| FP16Model=LLaMA-2-13B, Quantization=None, Calibration=FP162026.04 | 59.4 | — | — | — | |
| QWEN3-14BModel Architecture=QWEN3-14B, Configuration Type=Dense, Evaluation Protocol=Zero-shot, Sparsity Level=0%2026.01 | 59.22 | — | — | — | |
| COVERCALModel=LLaMA-2-13B, Quantization=AWQ INT4, Calibration Samples=128, Calibration=COVERCAL2026.04 | 58.9 | — | — | — | |
| Max-ActVarModel=LLaMA-2-13B, Quantization=AWQ INT4, Calibration Samples=128, Calibration=Max-ActVar2026.04 | 58.2 | — | — | — | |
| RandomModel=LLaMA-2-13B, Quantization=AWQ INT4, Calibration Samples=128, Calibration=Random2026.04 | 57.8 | — | — | — | |
| Instruction-Queryable LoRAModel=LiquidAI/LFM2.5-350M2026.05 | 57.8 | — | — | — | |
| Queryable LoRAModel=Qwen/Qwen3-0.6B2026.05 | 57.8 | — | — | — | |
| LLAMA-3.1-8BModel Architecture=LLAMA-3.1-8B, Configuration Type=Dense, Evaluation Protocol=Zero-shot, Sparsity Level=0%2026.01 | 54.95 | — | — | — | |
| DEEPSEEK-R1-QWEN-8BModel Architecture=DEEPSEEK-R1-QWEN-8B, Configuration Type=Dense, Evaluation Protocol=Zero-shot, Sparsity Level=0%2026.01 | 54.78 | — | — | — | |
| FP16Model=LLAMA3-8B, Evaluation Protocol=Zero-shot, Bit-width=Full-precision2024.03 | 53.33 | — | — | — | |
| Instruction-Queryable LoRAModel=Qwen/Qwen2.5-0.5B-Instruct2026.05 | 53.1 | — | — | — | |
| SupervisedBackbone=DeBERTa-v3-base, Parameters=86M2026.02 | 52.3 | — | — | — | |
| Label SmoothingBackbone=DeBERTa-v3-base, Parameters=86M2026.02 | 51.6 | — | — | — | |
| Queryable LoRAModel=Qwen/Qwen2.5-0.5B-Instruct2026.05 | 51.6 | — | — | — | |
| Patient-KDBackbone=DeBERTa-v3-base, Parameters=86M, Distillation Strategy=Patient-KD2026.02 | 51.5 | — | — | — | |
| PROBE-KD (Logistic)Backbone=DeBERTa-v3-base, Parameters=86M, Probe Type=Logistic2026.02 | 51.5 | — | — | — | |
| LLaMA-2 7B Chat (cross-task prompting)Source Task=Commonsense-QA2024.05 | 51.4 | — | — | — | |
| LLaMA-2 7B Chat (cross-task prompting)Source Task=RACE2024.05 | 51.4 | — | — | — | |
| QWEN3-4BModel Architecture=QWEN3-4B, Configuration Type=Dense, Evaluation Protocol=Zero-shot, Sparsity Level=0%2026.01 | 51.11 | — | — | — | |
| LLaMA-2 7B Chat (cross-task prompting)Source Task=ARC-Easy2024.05 | 51 | — | — | — | |
| Logit-KDBackbone=DeBERTa-v3-base, Parameters=86M, Distillation Strategy=Logit-KD2026.02 | 50.9 | — | — | — | |
| PROBE-KD (MLP)Backbone=DeBERTa-v3-base, Parameters=86M, Probe Type=MLP2026.02 | 50.1 | — | — | — | |
| LoRAModel=Qwen/Qwen2.5-0.5B-Instruct2026.05 | 50 | — | — | — | |
| PROBE-KD (CCS)Backbone=DeBERTa-v3-base, Parameters=86M, Probe Type=CCS2026.02 | 49.7 | — | — | — | |
| G-DAUGSelection strategy=Influence2020.04 | 48.5 | 45.2 | 12.4 | 11 | |
| LoRAModel=LiquidAI/LFM2.5-350M2026.05 | 48.4 | — | — | — | |
| LoRAModel=ibm-granite/granite-4.0-350m2026.05 | 48.4 | — | — | — | |
| G-DAUGSelection strategy=Combo2020.04 | 48.2 | 43.8 | 13.1 | 10.7 | |
| G-DAUGSelection strategy=Random2020.04 | 48.1 | 43.4 | 12.9 | 10.8 | |
| QuaRotModel=LLAMA3-70B, Evaluation Protocol=Zero-shot, Bit-width=4-bit2024.03 | 47.53 | — | — | — | |
| G-DAUGSelection strategy=Diversity2020.04 | 47.5 | 42.2 | 13.9 | 10.8 | |
| DARTModel Architecture=MISTRAL-7B, Configuration Type=Sparse, Evaluation Protocol=Zero-shot, Sparsity Level=70%2026.01 | 46.42 | — | — | — | |
| LLAMA-3.2-3BModel Architecture=LLAMA-3.2-3B, Configuration Type=Dense, Evaluation Protocol=Zero-shot, Sparsity Level=0%2026.01 | 46.33 | — | — | — | |
| LLaMA-2 7B Chat (cross-task prompting)Source Task=BoolQ2024.05 | 45 | — | — | — | |
| DARTModel Architecture=QWEN3-14B, Configuration Type=Sparse, Evaluation Protocol=Zero-shot, Sparsity Level=70%2026.01 | 44.03 | — | — | — | |
| Instruction-Queryable LoRAModel=ibm-granite/granite-4.0-350m2026.05 | 43.8 | — | — | — | |
| QuaRotModel=LLAMA3-8B, Evaluation Protocol=Zero-shot, Bit-width=4-bit2024.03 | 43.34 | — | — | — | |
| Backtranslation2020.04 | 43.1 | 42.4 | 6.6 | 9.3 | |
| DEEPSEEK-R1-LLAMA-8BModel Architecture=DEEPSEEK-R1-LLAMA-8B, Configuration Type=Dense, Evaluation Protocol=Zero-shot, Sparsity Level=0%2026.01 | 42.66 | — | — | — | |
| LoRAModel=Qwen/Qwen2.5-Coder-0.5B-Instruct2026.05 | 42.2 | — | — | — | |
| LLaMA-2 7B Chat (cross-task prompting)Source Task=Zero-shot2024.05 | 42 | — | — | — | |
| ROBERTaSelection strategy=None (Baseline)2020.04 | 39.4 | 35.2 | 6.6 | 9.3 | |
| LLaMA-2 7B Chat (cross-task prompting)Source Task=AG-news2024.05 | 39.2 | — | — | — | |
| Instruction-Queryable LoRAModel=Qwen/Qwen2.5-Coder-0.5B-Instruct2026.05 | 39.1 | — | — | — | |
| DARTModel Architecture=DEEPSEEK-R1-LLAMA-8B, Configuration Type=Sparse, Evaluation Protocol=Zero-shot, Sparsity Level=70%2026.01 | 39.08 | — | — | — | |
| LLaMA-2 7B Chat (cross-task prompting)Source Task=SST22024.05 | 39 | — | — | — | |
| DARTModel Architecture=LLAMA-3.1-8B, Configuration Type=Sparse, Evaluation Protocol=Zero-shot, Sparsity Level=70%2026.01 | 38.99 | — | — | — | |
| DARTModel Architecture=DEEPSEEK-R1-QWEN-8B, Configuration Type=Sparse, Evaluation Protocol=Zero-shot, Sparsity Level=70%2026.01 | 38.91 | — | — | — | |
| SAES-SVDRatio=0.2, Model=LLaMA-13B, Fine-tuning=false, Mixed-rank strategies=false, Zero-shot evaluation=true2026.02 | 38.8 | — | — | — | |
| Feature-KDBackbone=DeBERTa-v3-base, Parameters=86M, Distillation Strategy=Feature-KD2026.02 | 38.6 | — | — | — | |
| Queryable LoRAModel=ibm-granite/granite-4.0-350m2026.05 | 37.5 | — | — | — | |
| LLaMA-2 7B Chat (cross-task prompting)Source Task=QQP2024.05 | 37.2 | — | — | — | |
| LLaMA-2 7B Chat (cross-task prompting)Source Task=Conll2003-POS2024.05 | 37 | — | — | — | |
| LLaMA-2 7B Chat (cross-task prompting)Source Task=MNLI2024.05 | 37 | — | — | — | |
| Dip-SVDRatio=0.2, Model=LLaMA-13B, Fine-tuning=false, Mixed-rank strategies=true, Zero-shot evaluation=true2026.02 | 36.9 | — | — | — | |
| SVD-LLMRatio=0.2, Model=LLaMA-13B, Fine-tuning=true, Mixed-rank strategies=false, Zero-shot evaluation=true2026.02 | 35.6 | — | — | — | |
| DARTModel Architecture=QWEN3-4B, Configuration Type=Sparse, Evaluation Protocol=Zero-shot, Sparsity Level=70%2026.01 | 35.49 | — | — | — | |
| DARTModel Architecture=LLAMA-3.2-3B, Configuration Type=Sparse, Evaluation Protocol=Zero-shot, Sparsity Level=70%2026.01 | 33.96 | — | — | — | |
| Queryable LoRAModel=Qwen/Qwen2.5-Coder-0.5B-Instruct2026.05 | 32.8 | — | — | — | |
| Instruction-Queryable LoRAModel=amd/ReasonLite-0.6B2026.05 | 32.8 | — | — | — | |
| Pythia-12BParameters=12B, Zero-shot=true2023.04 | 31.8 | — | — | — | |
| Pythia-6.9BParameters=6.9B, Zero-shot=true2023.04 | 31.3 | — | — | — | |
| AMOOptimizer=AMO, Model=Llama3.1-1.4B, Few-shot=02026.05 | 31.2 | — | — | — | |
| LLaMA-2 7B Chat (cross-task prompting)Source Task=Conll2003-NER2024.05 | 30.2 | — | — | — | |
| Queryable LoRAModel=amd/ReasonLite-0.6B2026.05 | 29.7 | — | — | — | |
| Pythia-2.8BParameters=2.8B, Zero-shot=true2023.04 | 29.5 | — | — | — | |
| SAES-SVDRatio=0.4, Model=LLaMA-13B, Fine-tuning=false, Mixed-rank strategies=false, Zero-shot evaluation=true2026.02 | 28.4 | — | — | — | |
| Dip-SVDRatio=0.4, Model=LLaMA-13B, Fine-tuning=false, Mixed-rank strategies=true, Zero-shot evaluation=true2026.02 | 28.3 | — | — | — | |
| Mamba (SSM)Model Scale=1.4B2024.07 | 28 | — | — | — | |
| Mistral (Full-Attention)Model Scale=1.4B2024.07 | 27.5 | — | — | — | |
| AMOOptimizer=AMO, Model=Llama3.1-760M, Few-shot=02026.05 | 26.79 | — | — | — | |
| BMoJo (Fading)Model Scale=1.4B2024.07 | 26.6 | — | — | — | |
| BMoJo (Fading + Eidetic)Model Scale=1.4B2024.07 | 26.6 | — | — | — | |
| Pythia-1.4BParameters=1.4B, Zero-shot=true2023.04 | 26 | — | — | — | |
| Hybrid (Sliding Attention + SSM)Model Scale=1.4B2024.07 | 25.4 | — | — | — | |
| Queryable LoRAModel=amd/ReasonLite-0.6B-Turbo2026.05 | 25 | — | — | — | |
| SVD-LLMRatio=0.4, Model=LLaMA-13B, Fine-tuning=true, Mixed-rank strategies=false, Zero-shot evaluation=true2026.02 | 24.8 | — | — | — | |
| FoSS2026.02 | 24.63 | — | — | — | |
| Pythia-1BParameters=1B, Zero-shot=true2023.04 | 24.4 | — | — | — | |
| CoG2026.02 | 24.34 | — | — | — | |
| Transformer (w FT)fine-tuned=true, fine-tuning dataset=WikiText-1032026.02 | 24 | — | — | — | |
| Transformer (w/o FT)fine-tuned=false2026.02 | 23.82 | — | — | — | |
| Mistral (Full-Attention)Model Scale=370M2024.07 | 23.5 | — | — | — | |
| LoRAModel=amd/ReasonLite-0.6B2026.05 | 23.4 | — | — | — | |
| BMoJo (Fading)Model Scale=370M2024.07 | 23 | — | — | — | |
| Hybrid (Sliding Attention + SSM)Model Scale=370M2024.07 | 22.4 | — | — | — | |
| Mamba (SSM)Model Scale=370M2024.07 | 22.3 | — | — | — | |
| BMoJo (Fading + Eidetic)Model Scale=370M2024.07 | 22.1 | — | — | — | |
| LoRAModel=amd/ReasonLite-0.6B-Turbo2026.05 | 21.9 | — | — | — | |
| Pythia-410MParameters=410M, Zero-shot=true2023.04 | 21.3 | — | — | — |