Reading Comprehension on BOOLQ
94.47AccuracyIn-Squeeze
Evaluation results
| Method | Links | ||
|---|---|---|---|
| In-Squeezerank=128 to 1, schedule=Min steps2026.02 | 94.47 | — | |
| Direct Fine-tuningrank=1, additional_steps=02026.02 | 94.39 | — | |
| Cont-Squeezerank=128 to 1, additional_steps=2002026.02 | 94.39 | — | |
| Direct Fine-tuningrank=1, additional_steps=7002026.02 | 94.34 | — | |
| In-Squeezerank=128 to 1, schedule=Standard2026.02 | 94.3 | — | |
| Direct Fine-tuningrank=1, additional_steps=2002026.02 | 94.28 | — | |
| Cont-Squeezerank=128 to 1, additional_steps=02026.02 | 94.11 | — | |
| Cont-Squeezerank=128 to 1, additional_steps=7002026.02 | 94.06 | — | |
| T5 + UDGSetting=Supervised2022.01 | 91.4 | — | |
| MSSRfullBackbone=Gemma2-9B2026.03 | 91.1 | — | |
| MSSRfullBackbone=Qwen2.5-7B2026.03 | 91.1 | — | |
| MSSRschBackbone=Gemma2-9B2026.03 | 90.6 | — | |
| MSSRsplBackbone=Gemma2-9B2026.03 | 90.3 | — | |
| AccuBackbone=Gemma2-9B2026.03 | 90.1 | — | |
| AccuBackbone=Qwen2.5-7B2026.03 | 89.8 | — | |
| Inflection-12023.11 | 89.7 | — | |
| MSSRschBackbone=Qwen2.5-7B2026.03 | 89.6 | — | |
| MSSRsplBackbone=Qwen2.5-7B2026.03 | 89.3 | — | |
| AdaLoRAModel=Qwen2.5 3B, #Param.=61.32 M, rank=32, Evaluation Protocol=Zero-shot2025.09 | 89.14 | — | |
| LossBackbone=Gemma2-9B2026.03 | 89 | — | |
| NoneBackbone=Gemma2-9B2026.03 | 88.9 | — | |
| MSSRfullBackbone=Llama-3.1-8B2026.03 | 88.9 | — | |
| TASOModel=Qwen2.5 3B, #Param.=2.06 M, Evaluation Protocol=Zero-shot2025.09 | 88.64 | — | |
| TASOModel=LLaMA3.2 3B, #Param.=1.67 M, Evaluation Protocol=Zero-shot2025.09 | 88.53 | — | |
| LossBackbone=Qwen2.5-7B2026.03 | 88.5 | — | |
| DoRAModel=Qwen2.5 3B, #Param.=65.98 M, rank=32, Evaluation Protocol=Zero-shot2025.09 | 88.37 | — | |
| IA3Model=Qwen2.5 3B, #Param.=1.35 M, Evaluation Protocol=Zero-shot2025.09 | 88.26 | — | |
| Llama 3-8B E8T2Shots=0-shot2024.12 | 88.23 | — | |
| AdapterModel=Qwen2.5 3B, #Param.=67.10 M, Evaluation Protocol=Zero-shot2025.09 | 88.22 | — | |
| FixedBackbone=Qwen2.5-7B2026.03 | 88.2 | — | |
| MSSRsplBackbone=Llama-3.1-8B2026.03 | 88.1 | — | |
| PaLM2023.11 | 88 | — | |
| PaLM-540BModel Category=LLM, Evaluation Mode=Zero-shot2023.09 | 88 | — | |
| Model-centricNumber of experts=62024.07 | 88 | — | |
| LoRAModel=Qwen2.5 3B, #Param.=59.87 M, rank=32, Evaluation Protocol=Zero-shot2025.09 | 87.95 | — | |
| Mixtral-8x22BNumber of experts=82024.07 | 87.9 | — | |
| FalconModel Size=180B2023.11 | 87.8 | — | |
| AdaLoRAModel=LLaMA3.2 3B, #Param.=49.51 M, rank=32, Evaluation Protocol=Zero-shot2025.09 | 87.8 | — | |
| NoneBackbone=Qwen2.5-7B2026.03 | 87.8 | — | |
| Fine-tuneModel=Qwen2.5 3B, #Param.=3151.91 M, Evaluation Protocol=Zero-shot2025.09 | 87.76 | — | |
| Data-centricNumber of experts=62024.07 | 87.6 | — | |
| FixedBackbone=Gemma2-9B2026.03 | 87.6 | — | |
| FixedBackbone=Llama-3.1-8B2026.03 | 87.6 | — | |
| LoRAModel=Qwen2.5 3B, #Param.=14.97 M, rank=8, Evaluation Protocol=Zero-shot2025.09 | 87.5 | — | |
| EnumerateNumber of experts=62024.07 | 87.4 | — | |
| Qwen3-4BParams=4B2025.12 | 86.09 | — | |
| VERAModel=LLaMA3.2 3B, #Param.=1.09 M, rank=1024, Evaluation Protocol=Zero-shot2025.09 | 85.85 | — | |
| VERAModel=Qwen2.5 3B, #Param.=1.42 M, rank=1024, Evaluation Protocol=Zero-shot2025.09 | 85.43 | — | |
| LossBackbone=Llama-3.1-8B2026.03 | 85.4 | — | |
| MSSRschBackbone=Llama-3.1-8B2026.03 | 85.3 | — | |
| AdapterModel=LLaMA3.2 3B, #Param.=50.32 M, Evaluation Protocol=Zero-shot2025.09 | 85.28 | — | |
| Model-centricNumber of experts=42024.07 | 85.2 | — | |
| AccuBackbone=Llama-3.1-8B2026.03 | 85.2 | — | |
| LLaMA-2Model Size=70B2023.11 | 85 | — | |
| MT-NLGSetting=Few-shot2022.01 | 84.83 | — | |
| DoRAModel=LLaMA3.2 3B, #Param.=53.73 M, rank=32, Evaluation Protocol=Zero-shot2025.09 | 84.63 | — | |
| IA3Model=LLaMA3.2 3B, #Param.=0.91 M, Evaluation Protocol=Zero-shot2025.09 | 84.51 | — | |
| Qwen2.5-3BParams=3B2025.12 | 83.88 | — | |
| FULL TOKENS (50K)Base Model=Mistral-7B-v0.3, Token Budget=50K2025.02 | 83.85 | — | |
| UNIFORM RANDOM (50K×0.6)Base Model=Mistral-7B-v0.3, Token Budget=50K, Selection Proportion=0.62025.02 | 83.76 | — | |
| Chinchilla2023.11 | 83.7 | — | |
| LLaMA-2Model Size=34B2023.11 | 83.7 | — | |
| NoneBackbone=Llama-3.1-8B2026.03 | 83.7 | — | |
| FLAN 137BEvaluation protocol=few-shot, k=variable, Training strategy=leave-one-category-out2022.12 | 83.6 | — | |
| DS² (10K)Base Model=Mistral-7B-v0.3, Token Budget=10K2025.02 | 83.45 | — | |
| RHOBase Model=Mistral-7B-v0.3, Selection Proportion=0.62025.02 | 83.29 | — | |
| DS² (10K)Base Model=LLaMA-3.1-8B, Token Budget=10K2025.02 | 83.26 | — | |
| MistralSize=7B, Context=16K2024.04 | 83.2 | — | |
| GemmaSize=8B, Tokens=6T, Context=8K2024.04 | 83.2 | — | |
| UNIFORM RANDOM (50K×0.6)Base Model=LLaMA-3.1-8B, Token Budget=50K, Selection Proportion=0.62025.02 | 83.11 | — | |
| Data-centricNumber of experts=42024.07 | 83.1 | — | |
| ShapLoRATunable Params=22.8M, Backbone=LLaMA-3 8B2026.01 | 82.8 | — | |
| FIXED-MODEL CLEANINGBase Model=LLaMA-3.1-8B, Selection Proportion=0.62025.02 | 82.67 | — | |
| MT-NLGSetting=One-shot2022.01 | 82.51 | — | |
| FULL TOKENS (50K)Base Model=LLaMA-3.1-8B, Token Budget=50K2025.02 | 82.49 | — | |
| SELF-EVOLVING CLEANINGBase Model=LLaMA-3.1-8B, Selection Proportion=0.62025.02 | 82.49 | — | |
| BASEBase Model=Mistral-7B-v0.32025.02 | 82.3 | — | |
| BaselineFormat=Baseline, Bit width (b)=16.00, Quantization Aware Training (QAT)=true, Backbone=Llama 3.1 8B2025.05 | 82.2 | — | |
| BASEBase Model=LLaMA-3.1-8B2025.02 | 82.18 | — | |
| LoRAModel=LLaMA3.2 3B, #Param.=48.63 M, rank=32, Evaluation Protocol=Zero-shot2025.09 | 82.14 | — | |
| LoRA-XSModel=Qwen2.5 3B, #Param.=4.13 M, rank=128, Evaluation Protocol=Zero-shot2025.09 | 82.07 | — | |
| FIXED-MODEL CLEANINGBase Model=Mistral-7B-v0.3, Selection Proportion=0.62025.02 | 82.03 | — | |
| FalconModel Size=40B2023.11 | 81.9 | — | |
| OPT-IML 175BEvaluation protocol=few-shot, k=52022.12 | 81.7 | — | |
| LLaMA-2Model Size=13B2023.11 | 81.7 | — | |
| LLAMA2Size=13B, Tokens=2T, Context=4K2024.04 | 81.7 | — | |
| DoRATunable Params=22.6M, Backbone=LLaMA-3 8B2026.01 | 81.7 | — | |
| RHOBase Model=LLaMA-3.1-8B, Selection Proportion=0.62025.02 | 81.66 | — | |
| Count-guidedNumber of experts=62024.07 | 81.5 | — | |
| ClusComp#Bit=4.09, Base Model=Llama-2-13B, Zero-shot=true2025.03 | 81.4 | — | |
| MOELoRATunable Params=29.9M, Backbone=LLaMA-3 8B2026.01 | 81.4 | — | |
| SELF-EVOLVING CLEANINGBase Model=Mistral-7B-v0.3, Selection Proportion=0.62025.02 | 81.28 | — | |
| Phi-2Base Model=Phi-2, Evaluation Protocol=zero-shot2025.05 | 81.16 | — | |
| Llama 3-8BShots=0-shot2024.12 | 81.16 | — | |
| ArrowBase Model=Phi-2, Evaluation Protocol=zero-shot2025.05 | 81.13 | — | |
| LaMDA-PT 137BEvaluation protocol=0-shot2022.12 | 81 | — | |
| Fine-tuneModel=LLaMA3.2 3B, #Param.=3266.58 M, Evaluation Protocol=Zero-shot2025.09 | 80.73 | — | |
| LoRAModel=LLaMA3.2 3B, #Param.=12.16 M, rank=8, Evaluation Protocol=Zero-shot2025.09 | 80.73 | — | |
| OPT-IML 175BEvaluation protocol=0-shot2022.12 | 80.7 | — | |
| LLaMA 7B ChatEvaluation Protocol=Zero-shot, Framework=EleutherAI LM-Eval harness, Prompt Prefix=[Text] [S], Number of Parameters=7B, Mode=Chat2023.12 | 80.7 | — |