Question Answering on BoolQ
90.9AccuracyPaLM 2-L
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| PaLM 2-Lprompting=1-shot2023.05 | 90.9 | — | — | |
| PaLM-2 Lprotocol=1-shot2023.11 | 90.9 | — | — | |
| AUSteerModel=Llama-3.3-70B-Instruct2026.02 | 90.67 | — | — | |
| FLAN-T5Model Variant=xlarge, Model Parameters=3B2023.07 | 89.6 | — | — | |
| VanillaModel=Llama-3.3-70B-Instruct2026.02 | 89.54 | — | — | |
| FP16Type=-, Backbone=Qwen3-14B2025.11 | 89.4 | — | — | |
| QTIPType=vector, Backbone=Qwen3-14B2025.11 | 89.2 | — | — | |
| MoNEPruning ratio=25%, Backbone=Qwen3-30B-A3B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 89.11 | — | — | |
| AWQType=linear, Backbone=Qwen3-14B2025.11 | 89.1 | — | — | |
| PAROQType=linear, Backbone=Qwen3-14B2025.11 | 89.1 | — | — | |
| Falcon-180Bprotocol=1-shot2023.11 | 89 | — | — | |
| E-QATType=linear, Backbone=Qwen3-14B2025.11 | 89 | — | — | |
| MC-SMoEPruning ratio=25%, Backbone=Qwen3-30B-A3B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 88.75 | — | — | |
| PaLMprompting=1-shot2023.05 | 88.7 | — | — | |
| PaLMprotocol=1-shot2023.11 | 88.7 | — | — | |
| AUSteerModel=Qwen3-30B-A3B2026.02 | 88.69 | — | — | |
| Qwen3-30B-A3BPruning ratio=0%, Backbone=Qwen3-30B-A3B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 88.69 | — | — | |
| PaLM 2-Mprompting=1-shot2023.05 | 88.6 | — | — | |
| PaLM-2 Mprotocol=1-shot2023.11 | 88.6 | — | — | |
| RSPruning ratio=25%, Backbone=Qwen3-30B-A3B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 88.53 | — | — | |
| PaLM 2-Sprompting=1-shot2023.05 | 88.1 | — | — | |
| PaLM-2 Sprotocol=1-shot2023.11 | 88.1 | — | — | |
| ALIGNModel Variant=large, Model Parameters=355M2023.07 | 87.7 | — | — | |
| Hard-routing MoEBackbone=Qwen2.5-7B-Instruct, # Params (%)=14.2G (100%), # Trans. (%)=14.2G (100%), Fine-tuning setting=DFL, Rank (rl)=322026.02 | 87.67 | — | — | |
| PAROQType=linear, Backbone=Qwen3-8B2025.11 | 87 | — | — | |
| FLAPPruning ratio=25%, Backbone=Qwen2-57B-A14B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 86.91 | — | — | |
| QTIPType=vector, Backbone=Qwen3-8B2025.11 | 86.9 | — | — | |
| MoNEPruning ratio=25%, Backbone=Qwen2-57B-A14B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 86.88 | — | — | |
| VanillaModel=Qwen3-30B-A3B2026.02 | 86.82 | — | — | |
| BF16Model Size=8B, Quantization Method=None (BF16), Adaptive Block Scaling (4/6)=false2025.12 | 86.6 | — | — | |
| FP16Type=-, Backbone=Qwen3-8B2025.11 | 86.6 | — | — | |
| GPTQ + 4/6Model Size=8B, Quantization Method=GPTQ, Adaptive Block Scaling (4/6)=true2025.12 | 86.5 | — | — | |
| AWQModel Size=8B, Quantization Method=AWQ, Adaptive Block Scaling (4/6)=false2025.12 | 86.5 | — | — | |
| AWQ + 4/6Model Size=8B, Quantization Method=AWQ, Adaptive Block Scaling (4/6)=true2025.12 | 86.5 | — | — | |
| Qwen2-57B-A14BPruning ratio=0%, Backbone=Qwen2-57B-A14B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 86.45 | — | — | |
| MC-SMoEPruning ratio=25%, Backbone=Qwen2-57B-A14B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 86.45 | — | — | |
| GPTQModel Size=8B, Quantization Method=GPTQ, Adaptive Block Scaling (4/6)=false2025.12 | 86.4 | — | — | |
| SmoothQuantModel Size=8B, Quantization Method=SmoothQuant, Adaptive Block Scaling (4/6)=false2025.12 | 86.4 | — | — | |
| AWQType=linear, Backbone=Qwen3-8B2025.11 | 86.2 | — | — | |
| E-QATType=linear, Backbone=Qwen3-8B2025.11 | 86.1 | — | — | |
| SmoothQuant + 4/6Model Size=8B, Quantization Method=SmoothQuant, Adaptive Block Scaling (4/6)=true2025.12 | 85.9 | — | — | |
| RTN + 4/6Model Size=8B, Quantization Method=RTN, Adaptive Block Scaling (4/6)=true2025.12 | 85.8 | — | — | |
| FLAPPruning ratio=25%, Backbone=Qwen3-30B-A3B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 85.72 | — | — | |
| RTNModel Size=8B, Quantization Method=RTN, Adaptive Block Scaling (4/6)=false2025.12 | 85.5 | — | — | |
| PAROQType=linear, Backbone=Qwen3-4B2025.11 | 85.3 | — | — | |
| FP16Type=-, Backbone=Qwen3-4B2025.11 | 85.1 | — | — | |
| QTIPType=vector, Backbone=Qwen3-4B2025.11 | 85 | — | — | |
| AWQType=linear, Backbone=Qwen3-4B2025.11 | 85 | — | — | |
| FLAN-T5Model Variant=large, Model Parameters=780M2023.07 | 84.9 | — | — | |
| Krause-Qwen1.5-7BWindow size=32, top-k=162026.02 | 84.78 | — | — | |
| RSPruning ratio=25%, Backbone=Qwen2-57B-A14B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 84.77 | — | — | |
| Sparse-and-Orthogonal LoRABackbone=Qwen2.5-7B-Instruct, # Params (%)=87M (0.60%), # Trans. (%)=42M (0.30%), Fine-tuning setting=DFL, Rank (rl)=322026.02 | 84.76 | — | — | |
| Krause-Qwen1.5-7BWindow size=48, top-k=242026.02 | 84.66 | — | — | |
| Krause-Qwen1.5-7BWindow size=18, top-k=122026.02 | 84.63 | — | — | |
| E-QATType=linear, Backbone=Qwen3-4B2025.11 | 84.5 | — | — | |
| QTIPType=vector, Backbone=LLaMA-3.1-8B-Instruct2025.11 | 84.3 | — | — | |
| FPFTBackbone=Qwen2.5-7B-Instruct, # Params (%)=14.2G (100%), # Trans. (%)=14.2G (100%), Fine-tuning setting=DFL, Rank (rl)=322026.02 | 84.25 | — | — | |
| FP16Type=-, Backbone=LLaMA-3.1-8B-Instruct2025.11 | 84.1 | — | — | |
| Qwen1.5-7BFine-tuning protocol=LoRA2026.02 | 84.08 | — | — | |
| PAROQType=linear, Backbone=LLaMA-3.1-8B-Instruct2025.11 | 83.9 | — | — | |
| Qwen1.5-7BFine-tuning protocol=None2026.02 | 83.8 | — | — | |
| AWQType=linear, Backbone=LLaMA-3.1-8B-Instruct2025.11 | 83.5 | — | — | |
| E-QATType=linear, Backbone=LLaMA-3.1-8B-Instruct2025.11 | 83.5 | — | — | |
| Sparse-and-Orthogonal LoRA (Single)Backbone=Qwen2.5-7B-Instruct, # Params (%)=87M (0.60%), # Trans. (%)=42M (0.30%), Fine-tuning setting=DFL, Rank (rl)=322026.02 | 83.33 | — | — | |
| Hard-routing MoEBackbone=Qwen2.5-1.5B-Instruct, # Params (%)=3.2G (100%), # Trans. (%)=3.2G (100%), Fine-tuning setting=DFL, Rank (rl)=322026.02 | 83.23 | — | — | |
| DenseBackbone=LLaMA-3.1-8B, Pruning Method=None, Evaluation Protocol=Zero-shot, FFN Sparsity=0%2026.01 | 83.09 | — | — | |
| LoRIBackbone=Qwen2.5-7B-Instruct, # Params (%)=168M (1.16%), # Trans. (%)=168M (1.16%), Fine-tuning setting=DFL, Rank (rl)=322026.02 | 82.72 | — | — | |
| MUPPET2022.12 | 82.17 | — | — | |
| Sparse-and-Orthogonal LoRABackbone=Qwen2.5-1.5B-Instruct, # Params (%)=16M (0.54%), # Trans. (%)=8M (0.27%), Fine-tuning setting=DFL, Rank (rl)=322026.02 | 81.45 | — | — | |
| ColD-Fusion2022.12 | 81.39 | — | — | |
| LoRABackbone=Qwen2.5-7B-Instruct, # Params (%)=362M (2.5%), # Trans. (%)=362M (2.5%), Fine-tuning setting=DFL, Rank (rl)=322026.02 | 81.32 | — | — | |
| ALIGNModel Variant=base, Model Parameters=125M2023.07 | 81.1 | — | — | |
| Krause-Llama3-8B2026.02 | 80.59 | — | — | |
| AngularPruning ratio=25%, Backbone=Qwen3-30B-A3B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 80.52 | — | — | |
| Llama3-8Bfine-tuning=LoRA2026.02 | 80.41 | — | — | |
| MoonlightPruning ratio=0%, Backbone=Moonlight, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 80.4 | — | — | |
| Multitask2022.12 | 80.27 | — | — | |
| Sparse-and-Orthogonal LoRA (Single)Backbone=Qwen2.5-1.5B-Instruct, # Params (%)=16M (0.54%), # Trans. (%)=8M (0.27%), Fine-tuning setting=DFL, Rank (rl)=322026.02 | 80.21 | — | — | |
| Pythia-6.9B w/ Qwen2-7BShots=5, Teacher=Qwen2-7B2025.06 | 80 | — | — | |
| Deepseek-V2-LitePruning ratio=0%, Backbone=Deepseek-V2-Lite, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 79.88 | — | — | |
| Pythia-6.9B w/ Qwen2-7BShots=0, Teacher=Qwen2-7B2025.06 | 79.85 | — | — | |
| FPFTBackbone=Qwen2.5-1.5B-Instruct, # Params (%)=3.2G (100%), # Trans. (%)=3.2G (100%), Fine-tuning setting=DFL, Rank (rl)=322026.02 | 79.81 | — | — | |
| MoNEPruning ratio=25%, Backbone=Moonlight, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 79.57 | — | — | |
| Pythia-6.9B + Direct tuningShots=02025.06 | 79.3 | — | — | |
| LoRIBackbone=Qwen2.5-1.5B-Instruct, # Params (%)=32M (1.05%), # Trans. (%)=32M (1.05%), Fine-tuning setting=DFL, Rank (rl)=322026.02 | 79.29 | — | — | |
| S-MeZOBackbone=Mistral-7B, Evaluation Protocol=Fine-Tuning2024.02 | 79.2 | — | — | |
| Pythia-6.9B + Direct tuningShots=52025.06 | 78.87 | — | — | |
| Pythia-6.9B w/ LLAMA3-8BShots=5, Teacher=LLAMA3-8B2025.06 | 78.78 | — | — | |
| MC-SMoEPruning ratio=25%, Backbone=Moonlight, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 78.72 | — | — | |
| RSPruning ratio=25%, Backbone=Moonlight, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 78.69 | — | — | |
| MoNEPruning ratio=25%, Backbone=Deepseek-V2-Lite, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 78.47 | — | — | |
| LLaMA-13B (Touvron et al. 2023)Pruning Ratio=0%, Zero-shot=true2024.03 | 78.1 | — | — | |
| Pythia-6.9B w/ LLAMA3-8BShots=0, Teacher=LLAMA3-8B2025.06 | 77.83 | — | — | |
| DenseBackbone=Llama-2-7B, Sparsity=0%, Evaluation Protocol=Zero-shot2025.10 | 77.68 | — | — | |
| BF16Model Size=1.7B, Quantization Method=None (BF16), Adaptive Block Scaling (4/6)=false2025.12 | 77.6 | — | — | |
| Finetune2022.12 | 77.14 | — | — | |
| LoRABackbone=Qwen2.5-1.5B-Instruct, # Params (%)=65.5M (2.01%), # Trans. (%)=65.5M (2.01%), Fine-tuning setting=DFL, Rank (rl)=322026.02 | 76.94 | — | — | |
| Pythia-6.9B + Sentence distillShots=52025.06 | 76.88 | — | — | |
| Pythia-6.9B + Sentence distillShots=02025.06 | 76.64 | — | — | |
| MeZOBackbone=Mistral-7B, Evaluation Protocol=Fine-Tuning2024.02 | 76.6 | — | — |