Boolean Question Answering on BoolQ (Accuracy, Delta Accuracy)
92.3AccuracyLoRA
Evaluation results
| Method | Links | ||
|---|---|---|---|
| LoRABase Model=Llama-3.1-8B, Params (%)=0.26%, Rank=82026.02 | 92.3 | 0.7 | |
| Post-block steeringBase Model=Llama-3.1-8B, Params (%)=0.02%, Rank=82026.02 | 92.3 | 0.7 | |
| SFTBase Model=Llama-3.1-8B, Params (%)=100%2026.02 | 91.6 | 0 | |
| SFTBase Model=Qwen 3 4B, Params (%)=100%2026.02 | 91.4 | 0 | |
| ReFTBase Model=Llama-3.1-8B, Params (%)=0.02%, Rank=82026.02 | 91.3 | -0.3 | |
| LoRABase Model=Qwen 3 4B, Params (%)=0.41%, Rank=82026.02 | 91 | -0.4 | |
| Post-block steeringBase Model=Qwen 3 4B, Params (%)=0.04%, Rank=82026.02 | 90.7 | -0.7 | |
| ReFTBase Model=Qwen 3 4B, Params (%)=0.04%, Rank=82026.02 | 90.5 | -0.9 | |
| Count-guidedModel=GPT-oss, Scope=Local, Number of experts pruned=4e2026.04 | 89.3 | — | |
| ZipCalModel=gemma-2-9b-it, Compression Method=RTE, Compression Setting (Sparsity/Bit-width)=25%2026.03 | 89.08 | 0.35 | |
| DEKModel=GPT-oss, Scope=Local, Number of experts pruned=2e2026.04 | 89 | — | |
| GRAPEModel=GPT-oss, Scope=Global, Number of experts pruned=2e2026.04 | 89 | — | |
| GRAPEModel=GPT-oss, Scope=Global, Number of experts pruned=4e2026.04 | 89 | — | |
| Router-guidedModel=GPT-oss, Scope=Local, Number of experts pruned=2e2026.04 | 88.9 | — | |
| EnumerateModel=GPT-oss, Scope=Local, Number of experts pruned=2e2026.04 | 88.8 | — | |
| OriginalModel=GPT-oss, Scope=–2026.04 | 88.7 | — | |
| EnumerateModel=GPT-oss, Scope=Local, Number of experts pruned=4e2026.04 | 88.7 | — | |
| SFTBase Model=Llama-3.2-1B, Params (%)=100%2026.02 | 88.2 | 0 | |
| Count-guidedModel=GPT-oss, Scope=Local, Number of experts pruned=2e2026.04 | 88.2 | — | |
| DEKModel=GPT-oss, Scope=Local, Number of experts pruned=4e2026.04 | 88.1 | — | |
| Router-guidedModel=GPT-oss, Scope=Local, Number of experts pruned=4e2026.04 | 87.3 | — | |
| Post-block steeringBase Model=Llama-3.2-1B, Params (%)=0.04%, Rank=82026.02 | 86.2 | -2 | |
| ZipCalModel=Llama-3.1-8B-Instruct, Compression Method=GPTQ, Compression Setting (Sparsity/Bit-width)=W4A162026.03 | 85.38 | 1.15 | |
| LoRABase Model=Llama-3.2-1B, Params (%)=0.45%, Rank=82026.02 | 84.7 | -3.5 | |
| LoRABase Model=gemma-3-1b, Params (%)=0.45%, Rank=82026.02 | 84.7 | 1.4 | |
| ReFTBase Model=Llama-3.2-1B, Params (%)=0.04%, Rank=82026.02 | 84.1 | -4.1 | |
| SFTBase Model=gemma-3-1b, Params (%)=100%2026.02 | 83.3 | 0 | |
| Post-block steeringBase Model=gemma-3-1b, Params (%)=0.04%, Rank=82026.02 | 82 | -1.3 | |
| ReFTBase Model=gemma-3-1b, Params (%)=0.04%, Rank=82026.02 | 76.2 | -7.1 |