Question Answering on ARC-E
95.23AccuracyDirect Fine-tuning
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Direct Fine-tuningrank=1, additional_steps=2002026.02 | 95.23 | — | — | — | |
| In-Squeezerank=128 to 1, schedule=Min steps2026.02 | 95.23 | — | — | — | |
| Direct Fine-tuningrank=1, additional_steps=02026.02 | 95.12 | — | — | — | |
| Direct Fine-tuningrank=1, additional_steps=7002026.02 | 95.12 | — | — | — | |
| Cont-Squeezerank=128 to 1, additional_steps=2002026.02 | 95.03 | — | — | — | |
| Cont-Squeezerank=128 to 1, additional_steps=7002026.02 | 95.01 | — | — | — | |
| In-Squeezerank=128 to 1, schedule=Standard2026.02 | 94.92 | — | — | — | |
| TrajectoryTrain Dataset=ARC-E, Model=Qwen2.5-14b2026.03 | 94.28 | — | — | — | |
| FPModel=LLaDA-Instruct, Quantization=Full Precision2026.06 | 94 | — | — | — | |
| FAIR-CalibModel=LLaDA-1.5, Quantization=W4A42026.06 | 94 | — | — | — | |
| QuaRotModel=LLaDA-Instruct, Quantization=W4A42026.06 | 93.3 | — | — | — | |
| FPModel=LLaDA-1.5, Quantization=Full Precision2026.06 | 93.12 | — | — | — | |
| FlatQuantModel=LLaDA-1.5, Quantization=W4A42026.06 | 92.95 | — | — | — | |
| FAIR-CalibModel=LLaDA-Instruct, Quantization=W4A42026.06 | 92.77 | — | — | — | |
| FlatQuantModel=LLaDA-Instruct, Quantization=W4A42026.06 | 92.55 | — | — | — | |
| TrajectoryTrain Dataset=ARC-C, Model=Qwen2.5-14b2026.03 | 92.05 | — | — | — | |
| QuaRotModel=LLaDA-1.5, Quantization=W4A42026.06 | 91.89 | — | — | — | |
| PaLM 2-Lprompting=1-shot2023.05 | 89.7 | — | — | — | |
| LoRA-MixerBackbone=LLaMA3-8B, Adapter=Q/K/V/O, Routing Strategy=RSL2025.06 | 89.47 | — | — | — | |
| NVFP4+PTSBackbone=LongCat 560B2026.02 | 89.26 | — | — | 2.62 | |
| LoRABase Model=LLaMA-3 8B, Trainable Parameters=2.6%2024.04 | 89 | — | — | — | |
| RTNModel=LLaDA-Instruct, Quantization=W4A42026.06 | 88.71 | — | — | — | |
| RTNModel=LLaDA-1.5, Quantization=W4A42026.06 | 88.71 | — | — | — | |
| Stable-LoRAModel=3B2026.03 | 88.68 | — | — | — | |
| NVFP4Backbone=LongCat 560B2026.02 | 88.5 | — | — | 1.86 | |
| BaseBackbone=LLaMA3-8B, Adapter=Q/K/V/O2025.06 | 88.45 | — | — | — | |
| AdamWModel=3B2026.03 | 88.38 | — | — | — | |
| Few-shot AccuracyModel=Qwen2.5-14b2026.03 | 88.3 | — | — | — | |
| LoRA+Model=3B2026.03 | 88.26 | — | — | — | |
| LoRA-RITEModel=3B2026.03 | 88.26 | — | — | — | |
| DoRABase Model=LLaMA-3 8B, Trainable Parameters=2.6%2024.04 | 88.1 | — | — | — | |
| RiemannModel=3B2026.03 | 88.01 | — | — | — | |
| PaLM 2-Mprompting=1-shot2023.05 | 88 | — | — | — | |
| HiF4Backbone=LongCat 560B2026.02 | 87.95 | — | — | 1.31 | |
| MixDoRABase Model=LLaMA-3 8B, Trainable Parameters=3.0%2024.04 | 87.7 | — | — | — | |
| PhatgooseBackbone=LLaMA3-8B, Adapter=Q/K/V/O, Routing Strategy=sigmoid gating based on cosine similarity2025.06 | 87.13 | — | — | — | |
| NVFP4+PTSBackbone=DeepSeek-V3.1 671B2026.02 | 87.06 | — | — | 2.62 | |
| Stable-LoRAModel=1.5B2026.03 | 86.99 | — | — | — | |
| HiF4Backbone=DeepSeek-V3.1 671B2026.02 | 86.89 | — | — | 2.45 | |
| NVFP4Backbone=DeepSeek-V3.1 671B2026.02 | 86.85 | — | — | 2.41 | |
| Cont-Squeezerank=128 to 1, additional_steps=02026.02 | 86.74 | — | — | — | |
| TrajectoryTrain Dataset=OpenQA, Model=Qwen2.5-14b2026.03 | 86.7 | — | — | — | |
| BF16Backbone=LongCat 560B2026.02 | 86.64 | — | — | — | |
| MixLoRABase Model=LLaMA-3 8B, Trainable Parameters=3.0%2024.04 | 86.5 | — | — | — | |
| LoRA-RITEModel=1.5B2026.03 | 86.36 | — | — | — | |
| AdamWModel=1.5B2026.03 | 86.2 | — | — | — | |
| LoRA+Model=1.5B2026.03 | 85.94 | — | — | — | |
| RiemannModel=1.5B2026.03 | 85.82 | — | — | — | |
| PaLM 2-Sprompting=1-shot2023.05 | 85.6 | — | — | — | |
| PaLMprompting=1-shot2023.05 | 85 | — | — | — | |
| FalconModel Size=180B2023.11 | 84.7 | — | — | — | |
| BF16Backbone=DeepSeek-V3.1 671B2026.02 | 84.44 | — | — | — | |
| Vector (row/col)Model=Qwen3-14B, Zero-shot protocol=true2025.12 | 84.34 | — | — | — | |
| Linear ProbeTrain Dataset=ARC-E, Model=Qwen2.5-14b2026.03 | 84.32 | — | — | — | |
| BitDelta (scalar)Model=Qwen3-14B, Zero-shot protocol=true2025.12 | 84.13 | — | — | — | |
| BaselineModel=Qwen3-14B, Zero-shot protocol=true2025.12 | 84.09 | — | — | — | |
| LLaMA38BBackbone=LLaMA38B, Initialization Method=Vanilla, #GPU Hour=−, Shots=52026.05 | 83.92 | — | — | — | |
| MixDoRABase Model=LLaMA-2 13B, Trainable Parameters=2.5%2024.04 | 83.7 | — | — | — | |
| BitDelta (scalar)Model=Phi-4-reasoning, Zero-shot protocol=true2025.12 | 83.54 | — | — | — | |
| MixLoRABase Model=LLaMA-2 13B, Trainable Parameters=2.5%2024.04 | 83.5 | — | — | — | |
| Zero-shot AccuracyModel=Qwen2.5-14b2026.03 | 83.5 | — | — | — | |
| BaselineModel=Phi-4-reasoning, Zero-shot protocol=true2025.12 | 83.29 | — | — | — | |
| LoRABase Model=LLaMA-2 13B, Trainable Parameters=2.4%2024.04 | 83.2 | — | — | — | |
| DoRABase Model=LLaMA-2 13B, Trainable Parameters=2.4%2024.04 | 83.1 | — | — | — | |
| LLaMA-3-8B-LizardTraining Tokens (B)=0.04, Attention configuration=Linearized (Keep 50% Full Attn.)2025.07 | 83.1 | — | — | — | |
| Vector (row/col)Model=Llama-3.1-8B-Instruct, Zero-shot protocol=true2025.12 | 82.99 | — | — | — | |
| Vector (row/col)Model=Phi-4-reasoning, Zero-shot protocol=true2025.12 | 82.95 | — | — | — | |
| DenseBackbone=LLaMA-3.1-8B, Pruning Method=None, Evaluation Protocol=Zero-shot, FFN Sparsity=0%2026.01 | 82.53 | — | — | — | |
| BitDelta (scalar)Model=Llama-3.1-8B-Instruct, Zero-shot protocol=true2025.12 | 82.32 | — | — | — | |
| BaselineModel=Llama-3.1-8B-Instruct, Zero-shot protocol=true2025.12 | 81.81 | — | — | — | |
| FalconModel Size=40B2023.11 | 81.2 | — | — | — | |
| FP16Model=LLaMA-30B, Precision=W4A16, Zero-shot=true2026.05 | 80.64 | — | — | — | |
| FP16Model=LLaMA-30B, Precision=W3A16, Zero-shot=true2026.05 | 80.64 | — | — | — | |
| SARQC-GS(Saliency)Model=LLaMA-30B, Precision=W4A16, Zero-shot=true2026.05 | 80.6 | — | — | — | |
| GPTAQModel=LLaMA-30B, Precision=W4A16, Zero-shot=true2026.05 | 80.39 | — | — | — | |
| AWQModel=LLaMA-30B, Precision=W4A16, Zero-shot=true2026.05 | 80.26 | — | — | — | |
| SARQC-GS(Identity)Model=LLaMA-30B, Precision=W4A16, Zero-shot=true2026.05 | 80.22 | — | — | — | |
| LLaMA38BBackbone=LLaMA38B, Initialization Method=Vanilla, #GPU Hour=−, Shots=02026.05 | 80.22 | — | — | — | |
| LLaMA-2Model Size=70B2023.11 | 80.2 | — | — | — | |
| ZEPHYR-7B-beta2024.05 | 80.2 | — | — | — | |
| No pruning (Original)Backbone=Gemma-2 2B, Sparsity=0, Evaluation Protocol=Zero-shot2026.03 | 80.2 | — | — | — | |
| LLaMA-3-8BTraining Tokens (B)=15000, Attention configuration=Softmax2025.07 | 80.1 | — | — | — | |
| SARQC-GBS(Saliency)Model=LLaMA-30B, Precision=W4A16, Zero-shot=true2026.05 | 80.09 | — | — | — | |
| GPTQModel=LLaMA-30B, Precision=W4A16, Zero-shot=true2026.05 | 80.05 | — | — | — | |
| mPLUG-Owl2Shot setting=0-shot2023.11 | 79.9 | — | — | — | |
| SARQC-GBS(Identity)Model=LLaMA-30B, Precision=W4A16, Zero-shot=true2026.05 | 79.8 | — | — | — | |
| Clean FTModel=LLaMA-3-8B2026.03 | 79.6 | — | — | — | |
| FP16Model=LLaMA3.1-8B-Instruct, Precision=MXFP42026.04 | 79.6 | — | — | — | |
| SARQC-GBS(Saliency)Model=LLaMA-30B, Precision=W3A16, Zero-shot=true2026.05 | 79.46 | — | — | — | |
| LLaMA-2Model Size=34B2023.11 | 79.4 | — | — | — | |
| MISTRAL-7B2024.05 | 79.1 | — | — | — | |
| SARQC-GBS(Identity)Model=LLaMA-30B, Precision=W3A16, Zero-shot=true2026.05 | 79 | — | — | — | |
| ClusComp#Bit=4.09, Base Model=Llama-2-13B, Zero-shot=true2025.03 | 78.9 | — | 76.9 | — | |
| Linear ProbeTrain Dataset=ARC-C, Model=Qwen2.5-14b2026.03 | 78.81 | — | — | — | |
| Mamba2-LLaMA-3Training Tokens (B)=20, Attention configuration=Linearized (Keep 50% Full Attn.)2025.07 | 78.8 | — | — | — | |
| FP16Model=LLaMA3-8B-Instruct, Precision=MXFP42026.04 | 78.8 | — | — | — | |
| GPTAQModel=LLaMA-30B, Precision=W3A16, Zero-shot=true2026.05 | 78.7 | — | — | — | |
| ClusComp#Bit=2.81, Base Model=Llama-2-13B, Zero-shot=true2025.03 | 78.5 | — | 76.7 | — | |
| GPTQModel=LLaMA-30B, Precision=W3A16, Zero-shot=true2026.05 | 78.49 | — | — | — | |
| FlatQuantModel=LLaMA3.1-8B-Instruct, Precision=MXFP42026.04 | 78.3 | — | — | — |