Reading Comprehension on BoolQ (val)
97.7AccuracyST-MOE
Evaluation Results
| Method | Links | |
|---|---|---|
| ST-MOEStatus=Fine-tuned SOTA2022.06 | 97.7 | |
| PaLMStatus=Unified SOTA2022.06 | 92.2 | |
| D2-LoRABackbone=Qwen2.5-7B-Instruct, Training Epochs=1, Scoring Method=CLL2026.02 | 85.2 | |
| DoRABackbone=Qwen2.5-7B-Instruct, Training Epochs=1, Scoring Method=CLL2026.02 | 84.2 | |
| LoRABackbone=Qwen2.5-7B-Instruct, Training Epochs=1, Scoring Method=CLL2026.02 | 82 | |
| LLRCBackbone=Llama-2-13b, Param Ratio=0.80, Fine-tuning=None2025.12 | 81.3 | |
| LLRCBackbone=Llama-2-13b, Param Ratio=0.90, Fine-tuning=None2025.12 | 81.1 | |
| BaselineBackbone=Llama-2-13b, Param Ratio=1.00, Fine-tuning=None2025.12 | 80.1 | |
| SVD-LLM+FinetuneBackbone=Llama-2-13b, Param Ratio=0.90, Fine-tuning=Alpaca2025.12 | 80 | |
| UNIFIED-IO XLModel Size=XL2022.06 | 79.7 | |
| SVD-LLM+FinetuneBackbone=Llama-2-13b, Param Ratio=0.80, Fine-tuning=Alpaca2025.12 | 78.6 | |
| BaselineBackbone=Llama-2-7b, Param Ratio=1.00, Fine-tuning=None2025.12 | 77.8 | |
| LLRCBackbone=Llama-2-7b, Param Ratio=0.90, Fine-tuning=None2025.12 | 77.3 | |
| SVD-LLM (w)Backbone=Llama-2-13b, Param Ratio=0.90, Fine-tuning=None2025.12 | 76.6 | |
| LLM-Pruner+FinetuneBackbone=Llama-2-13b, Param Ratio=0.90, Fine-tuning=Alpaca2025.12 | 76.4 | |
| SVD-LLM (w)Backbone=Llama-2-13b, Param Ratio=0.80, Fine-tuning=None2025.12 | 73.9 | |
| LLM-PrunerBackbone=Llama-2-13b, Param Ratio=0.90, Fine-tuning=None2025.12 | 73.4 | |
| UNIFIED-IO LARGEModel Size=LARGE2022.06 | 73.1 | |
| LLM-Pruner+FinetuneBackbone=Llama-2-13b, Param Ratio=0.80, Fine-tuning=Alpaca2025.12 | 72.9 | |
| LLRCBackbone=Llama-2-7b, Param Ratio=0.80, Fine-tuning=None2025.12 | 71.4 | |
| LLM-Pruner+FinetuneBackbone=Llama-2-7b, Param Ratio=0.90, Fine-tuning=Alpaca2025.12 | 71.4 | |
| UNIFIED-IO BASEModel Size=BASE2022.06 | 70.8 | |
| LLM-Pruner+FinetuneBackbone=Llama-2-7b, Param Ratio=0.80, Fine-tuning=Alpaca2025.12 | 67.8 | |
| SVD-LLM+FinetuneBackbone=Llama-2-7b, Param Ratio=0.90, Fine-tuning=Alpaca2025.12 | 67.6 | |
| SVD-LLM+FinetuneBackbone=Llama-2-7b, Param Ratio=0.80, Fine-tuning=Alpaca2025.12 | 67.5 | |
| LLM-PrunerBackbone=Llama-2-13b, Param Ratio=0.80, Fine-tuning=None2025.12 | 67.2 | |
| UNIFIED-IO SMALLModel Size=SMALL2022.06 | 65.9 | |
| D2-LoRABackbone=Llama-3.2-3B-Instruct, Training Epochs=2, Scoring Method=CLL2026.02 | 65.6 | |
| LLM-PrunerBackbone=Llama-2-7b, Param Ratio=0.90, Fine-tuning=None2025.12 | 64.8 | |
| LLM-PrunerBackbone=Llama-2-7b, Param Ratio=0.80, Fine-tuning=None2025.12 | 64.6 | |
| LoRABackbone=Llama-3.2-3B-Instruct, Training Epochs=2, Scoring Method=CLL2026.02 | 62.6 | |
| DoRABackbone=Llama-3.2-3B-Instruct, Training Epochs=2, Scoring Method=CLL2026.02 | 61.8 | |
| SVD-LLM (w)Backbone=Llama-2-7b, Param Ratio=0.90, Fine-tuning=None2025.12 | 49 | |
| SVD-LLM (w)Backbone=Llama-2-7b, Param Ratio=0.80, Fine-tuning=None2025.12 | 44.4 |