Question Answering on ARC Easy
96.4Normalized AccLFTF
Evaluation results
| Method | Links | ||
|---|---|---|---|
| LFTFBackbone=Qwen-3-4B2026.01 | 96.4 | — | |
| FairSteerBackbone=Qwen-3-4B2026.01 | 96.39 | — | |
| BaseBackbone=Qwen-3-4B2026.01 | 96.37 | — | |
| KALEBackbone=Qwen2.5 32B, Category=Augmented-based2026.01 | 94.9 | — | |
| KALEBackbone=Gemma2 9B, Category=Augmented-based2026.01 | 94.7 | — | |
| KnowBiasBackbone=Qwen-3-4B2026.01 | 94.1 | — | |
| CRISPRBackbone=Qwen-3-4B, Debias target=race2026.01 | 94.02 | — | |
| CRISPRBackbone=Qwen-3-4B, Debias target=gender2026.01 | 93.77 | — | |
| ReGiFTBackbone=Qwen-3-4B2026.01 | 92.68 | — | |
| UNIFIEDQAModel Architecture=T5, Retrieval Context (IR)=Yes, Fine-tuning Protocol=Fine-tuned2020.05 | 92 | — | |
| DMTBackbone=Gemma2 9B, Category=SFT-based2026.01 | 91.12 | — | |
| SDFTBackbone=Gemma2 9B, Category=SFT-based2026.01 | 90.91 | — | |
| PEFTBackbone=Qwen-3-4B, Debias target=religion2026.01 | 90.06 | — | |
| T5Model Architecture=T5, Retrieval Context (IR)=Yes, Fine-tuning Protocol=Fine-tuned2020.05 | 90 | — | |
| MeanLearnBackbone=Gemma2 9B, Category=SFT-based2026.01 | 89.9 | — | |
| SFTBackbone=Gemma2 9B, Category=SFT-based2026.01 | 89.06 | — | |
| VanillaBackbone=Gemma2 9B, Category=Prompt-based2026.01 | 88.89 | — | |
| KALEBackbone=Orca2 7B, Category=Augmented-based2026.01 | 88.51 | — | |
| KG-SFTBackbone=Gemma2 9B, Category=SFT-based2026.01 | 88.26 | — | |
| GPT3MixBackbone=Gemma2 9B, Category=Augmented-based2026.01 | 88.22 | — | |
| GPT3MixBackbone=Qwen2.5 32B, Category=Augmented-based2026.01 | 87.33 | — | |
| CoTBackbone=Gemma2 9B, Category=Prompt-based2026.01 | 86.91 | — | |
| StructGPTBackbone=Gemma2 9B, Category=Retrieval-based2026.01 | 86.87 | — | |
| Llama-3-70B#Bit=16.00, Base Model=Llama-3-70B, Evaluation Protocol=Zero-shot2025.03 | 86.7 | — | |
| CRISPRBackbone=Qwen-3-4B, Debias target=religion2026.01 | 86.66 | — | |
| MeanLearnBackbone=Orca2 7B, Category=SFT-based2026.01 | 86.57 | — | |
| KALEBackbone=Llama3 8B2026.01 | 86.45 | — | |
| KALEBackbone=LlaMA3 8B, Category=Augmented-based2026.01 | 86.45 | — | |
| UNIFIEDQAModel Architecture=T5, Retrieval Context (IR)=No, Fine-tuning Protocol=Fine-tuned2020.05 | 86.4 | — | |
| AugGPTBackbone=Qwen2.5 32B, Category=Augmented-based2026.01 | 86.27 | — | |
| DMTBackbone=Qwen2.5 32B, Category=SFT-based2026.01 | 86.07 | — | |
| BiasEditBackbone=Qwen-3-4B, Debias target=religion2026.01 | 85.85 | — | |
| DeepSeek-R1 (BF16)Model=DeepSeek-R1, Setting=W16A16, Sparsity=0, Evaluation Protocol=Non-generative2025.12 | 85.52 | — | |
| SQ-formatModel=DeepSeek-R1, Setting=W(SQ5)A8, Sparsity=0.875, Evaluation Protocol=Non-generative2025.12 | 85.52 | — | |
| Mistral-7B-RewardBackbone=Mistral-7B, Preference Acquisition=Reward model-based classification2025.02 | 85.2 | — | |
| STaRBackbone=Qwen2.5 32B, Category=Augmented-based2026.01 | 84.6 | — | |
| STaRBackbone=Gemma2 9B, Category=Augmented-based2026.01 | 84.43 | — | |
| SDFTBackbone=Qwen2.5 32B, Category=SFT-based2026.01 | 84.3 | — | |
| GraphRAGBackbone=Qwen2.5 32B, Category=Retrieval-based2026.01 | 84.18 | — | |
| MeanLearnBackbone=Qwen2.5 32B, Category=SFT-based2026.01 | 84.18 | — | |
| TOGBackbone=Qwen2.5 32B, Category=Retrieval-based2026.01 | 84.13 | — | |
| KG-SFTBackbone=Qwen2.5 32B, Category=SFT-based2026.01 | 84.13 | — | |
| GraphRAGBackbone=Gemma2 9B, Category=Retrieval-based2026.01 | 84.13 | — | |
| SDBackbone=Qwen-3-4B2026.01 | 84.07 | — | |
| T5Model Architecture=T5, Retrieval Context (IR)=No, Fine-tuning Protocol=Fine-tuned2020.05 | 83.8 | — | |
| GalacticaModel Size=120B, Shots=0, Domain=in-domain2022.11 | 83.8 | — | |
| SFTBackbone=Qwen2.5 32B, Category=SFT-based2026.01 | 83.33 | — | |
| StructGPTBackbone=Qwen2.5 32B, Category=Retrieval-based2026.01 | 83.29 | — | |
| COT-DistillBackbone=Llama3 8B2026.01 | 82.92 | — | |
| UnifiedQA-BARTModel Architecture=BART, Retrieval Context (IR)=Yes, Fine-tuning Protocol=Fine-tuned2020.05 | 82.7 | — | |
| IPOBackbone=Mistral-7B, Framework=DPO2025.02 | 82.2 | — | |
| CoTBackbone=Qwen2.5 32B, Category=Prompt-based2026.01 | 82.07 | — | |
| AugGPTBackbone=Gemma2 9B, Category=Augmented-based2026.01 | 81.86 | — | |
| TOGBackbone=Gemma2 9B, Category=Retrieval-based2026.01 | 81.65 | — | |
| STaRBackbone=Orca2 7B, Category=Augmented-based2026.01 | 81.57 | — | |
| GemmaSize=8B, Tokens=6T, Context=8K2024.04 | 81.5 | — | |
| SPES-9B#Params=3.1B/9B, #Tokens=400B, Zero-shot=true, initialized from=pretrained dense model2026.02 | 81.5 | — | |
| BaselineBackbone=Qwen3-8B, Compression Ratio=1.00x, Zero-shot=true2026.06 | 80.93 | — | |
| Mistral 7B InstructBackbone=Mistral-7B2025.02 | 80.4 | — | |
| PEFTBackbone=Qwen-3-4B, Debias target=race2026.01 | 80.24 | — | |
| LoRA-Drop (25%)Backbone=Qwen2.5-14B, Evaluation Protocol=Zero-shot, Drop Ratio=25%, Speedup (x)=1.352026.01 | 80.2 | — | |
| SigmaScaleBackbone=Qwen3-8B, Compression Ratio=0.90x, Zero-shot=true2026.06 | 80.18 | — | |
| UnquantizedRank=0, Avg Bits=16, Backbone=LLaMa-3 8B, Evaluation Mode=Zero-shot2024.05 | 80.1 | — | |
| Llama-3-8B#Bit=16.00, Base Model=Llama-3-8B, Evaluation Protocol=Zero-shot2025.03 | 80.1 | — | |
| VanillaBackbone=Qwen2.5 32B, Category=Prompt-based2026.01 | 80.1 | — | |
| Full (baseline)Backbone=Qwen2.5-14B, Evaluation Protocol=Zero-shot, Speedup (x)=1.002026.01 | 80.1 | — | |
| Llama3-8BAttack Status=Before2025.11 | 80.09 | — | |
| AugGPTBackbone=Orca2 7B, Category=Augmented-based2026.01 | 80.05 | — | |
| KF+SIRModel Architecture=RoBERTa, Retrieval Context (IR)=Yes, Fine-tuning Protocol=standard2020.05 | 80 | — | |
| MistralSize=7B, Context=16K2024.04 | 80 | — | |
| LoRA-Drop (50%)Backbone=Qwen2.5-14B, Evaluation Protocol=Zero-shot, Drop Ratio=50%, Speedup (x)=1.682026.01 | 80 | — | |
| MEGALODONSize=7B, Tokens=2T, Context=32K2024.04 | 79.8 | — | |
| Mistral-7B-v0.3Attack Status=Before2025.11 | 79.67 | — | |
| GPT3MixBackbone=Orca2 7B, Category=Augmented-based2026.01 | 79.67 | — | |
| BaselineBackbone=Llama 3.1 8B Instruct, Compression Ratio=1.00x, Zero-shot=true2026.06 | 79.63 | — | |
| SigmaScale (KD)Backbone=Qwen3-8B, Compression Ratio=0.90x, Zero-shot=true2026.06 | 79.63 | — | |
| BART-largeModel Architecture=BART-large, Retrieval Context (IR)=Yes, Fine-tuning Protocol=Fine-tuned2020.05 | 79.6 | — | |
| FlexiDepthBackbone=Qwen2.5-14B, Evaluation Protocol=Zero-shot, Speedup (x)=1.492026.01 | 79.6 | — | |
| KG-SFTBackbone=LlaMA3 8B, Category=SFT-based2026.01 | 79.55 | — | |
| Llama-2-13B#Bit=16.00, Base Model=Llama-2-13B, Evaluation Protocol=Zero-shot2025.03 | 79.4 | — | |
| Llama2-13BAttack Status=Before2025.11 | 79.4 | — | |
| Unified Layer SkippingBackbone=Qwen2.5-14B, Evaluation Protocol=Zero-shot, Speedup (x)=1.342026.01 | 79.3 | — | |
| GraphRAGBackbone=Orca2 7B, Category=Retrieval-based2026.01 | 78.87 | — | |
| KG-SFTBackbone=Orca2 7B, Category=SFT-based2026.01 | 78.87 | — | |
| SigmaScaleBackbone=Llama 3.1 8B Instruct, Compression Ratio=0.90x, Zero-shot=true2026.06 | 78.62 | — | |
| LoRA-Drop (25%)Backbone=LLaMA3-8B, Evaluation Protocol=Zero-shot, Drop Ratio=25%, Speedup (x)=1.342026.01 | 78.1 | — | |
| OLMoE-1B-7B#Params=1.3B/7B, #Tokens=5T, Zero-shot=true2026.02 | 78 | — | |
| Full (baseline)Backbone=LLaMA3-8B, Evaluation Protocol=Zero-shot, Speedup (x)=1.002026.01 | 78 | — | |
| KALEBackbone=Mistral 7B, Category=Augmented-based2026.01 | 77.95 | — | |
| StructGPTBackbone=Orca2 7B, Category=Retrieval-based2026.01 | 77.95 | — | |
| LoRA-Drop (50%)Backbone=LLaMA3-8B, Evaluation Protocol=Zero-shot, Drop Ratio=50%, Speedup (x)=1.702026.01 | 77.9 | — | |
| SigmaScale (KD)Backbone=Llama 3.1 8B Instruct, Compression Ratio=0.90x, Zero-shot=true2026.06 | 77.86 | — | |
| SmolLM2-1.7B#Params=1.7B/1.7B, #Tokens=11T, Zero-shot=true2026.02 | 77.8 | — | |
| LoRA-Drop (75%)Backbone=Qwen2.5-14B, Evaluation Protocol=Zero-shot, Drop Ratio=75%, Speedup (x)=2.602026.01 | 77.8 | — | |
| FlexiDepthBackbone=LLaMA3-8B, Evaluation Protocol=Zero-shot, Speedup (x)=1.502026.01 | 77.5 | — | |
| FairSteerBackbone=Llama-3.1-8B2026.01 | 77.48 | — | |
| CoTBackbone=Orca2 7B, Category=Prompt-based2026.01 | 77.4 | — | |
| LLAMA2Size=13B, Tokens=2T, Context=4K2024.04 | 77.3 | — | |
| LoRA-Drop (25%)Backbone=Qwen2.5-7B, Evaluation Protocol=Zero-shot, Drop Ratio=25%, Speedup (x)=1.392026.01 | 77.2 | — | |
| Unified Layer SkippingBackbone=LLaMA3-8B, Evaluation Protocol=Zero-shot, Speedup (x)=1.362026.01 | 77.2 | — |