Instruction Following on BBH
67.1AccuracyBaseline Step 0
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Baseline Step 0Model=L3.1-8B-Magpie, Chat Template=On, k=12026.02 | 67.1 | 1 | — | |
| StaticModel=L3.1-8B-Magpie, Chat Template=On, k=12026.02 | 57.6 | 1 | — | |
| ConfAdaptModel=L3.1-8B-Magpie, Chat Template=On, tau (τ)=0.9952026.02 | 57.3 | 2.4 | — | |
| ConfAdaptModel=L3.1-8B-Magpie, Chat Template=On, tau (τ)=0.992026.02 | 56.8 | 2.6 | — | |
| ConfAdaptModel=L3.1-8B-Magpie, Chat Template=On, tau (τ)=0.982026.02 | 56.4 | 3 | — | |
| ConfAdaptModel=L3.1-8B-Magpie, Chat Template=On, tau (τ)=0.972026.02 | 56.2 | 3.2 | — | |
| ConfAdaptModel=L3.1-8B-Magpie, Chat Template=On, tau (τ)=0.962026.02 | 56 | 3.3 | — | |
| ConfAdaptModel=L3.1-8B-Magpie, Chat Template=On, tau (τ)=0.952026.02 | 55.6 | 3.4 | — | |
| ConfAdaptModel=L3.1-8B-Magpie, Chat Template=On, tau (τ)=0.92026.02 | 53.8 | 3.9 | — | |
| ConfAdaptModel=L3.1-8B-Magpie, Chat Template=On, tau (τ)=0.872026.02 | 51.9 | 4.1 | — | |
| StaticModel=L3.1-8B-Magpie, Chat Template=On, k=22026.02 | 51.4 | 2 | — | |
| ConfAdaptModel=L3.1-8B-Magpie, Chat Template=On, tau (τ)=0.852026.02 | 50.4 | 4.2 | — | |
| ConfAdaptModel=L3.1-8B-Magpie, Chat Template=On, tau (τ)=0.82026.02 | 45.5 | 4.4 | — | |
| StaticModel=L3.1-8B-Magpie, Chat Template=On, k=32026.02 | 45.1 | 3 | — | |
| ConfAdaptModel=L3.1-8B-Magpie, Chat Template=On, tau (τ)=0.752026.02 | 39.4 | 4.6 | — | |
| Baseline Step 0Model=Qwen3-4B-Inst-2507, Chat Template=Off, k=12026.02 | 34.4 | 1 | — | |
| ConfAdaptModel=L3.1-8B-Magpie, Chat Template=On, tau (τ)=0.72026.02 | 33.6 | 4.8 | — | |
| StaticModel=L3.1-8B-Magpie, Chat Template=On, k=42026.02 | 28 | 4 | — | |
| StaticModel=Qwen3-4B-Inst-2507, Chat Template=Off, k=12026.02 | 28 | 1 | — | |
| ConfAdaptModel=L3.1-8B-Magpie, Chat Template=On, tau (τ)=0.652026.02 | 27.6 | 5.1 | — | |
| ConfAdaptModel=Qwen3-4B-Inst-2507, Chat Template=Off, tau (τ)=0.9952026.02 | 27.4 | 2.1 | — | |
| ConfAdaptModel=Qwen3-4B-Inst-2507, Chat Template=Off, tau (τ)=0.992026.02 | 27.2 | 2.3 | — | |
| StaticModel=Qwen3-4B-Inst-2507, Chat Template=Off, k=22026.02 | 26.9 | 2 | — | |
| ConfAdaptModel=Qwen3-4B-Inst-2507, Chat Template=Off, tau (τ)=0.982026.02 | 26.8 | 2.5 | — | |
| ConfAdaptModel=Qwen3-4B-Inst-2507, Chat Template=Off, tau (τ)=0.972026.02 | 26.4 | 2.6 | — | |
| ConfAdaptModel=Qwen3-4B-Inst-2507, Chat Template=Off, tau (τ)=0.962026.02 | 26.2 | 2.6 | — | |
| ConfAdaptModel=Qwen3-4B-Inst-2507, Chat Template=Off, tau (τ)=0.952026.02 | 26.2 | 2.7 | — | |
| ConfAdaptModel=Qwen3-4B-Inst-2507, Chat Template=Off, tau (τ)=0.92026.02 | 26.1 | 3 | — | |
| ConfAdaptModel=Qwen3-4B-Inst-2507, Chat Template=Off, tau (τ)=0.872026.02 | 25.8 | 3.1 | — | |
| ConfAdaptModel=Qwen3-4B-Inst-2507, Chat Template=Off, tau (τ)=0.852026.02 | 25.2 | 3.1 | — | |
| ConfAdaptModel=Qwen3-4B-Inst-2507, Chat Template=Off, tau (τ)=0.82026.02 | 24.6 | 3.3 | — | |
| ConfAdaptModel=Qwen3-4B-Inst-2507, Chat Template=Off, tau (τ)=0.752026.02 | 23.5 | 3.4 | — | |
| ConfAdaptModel=Qwen3-4B-Inst-2507, Chat Template=Off, tau (τ)=0.72026.02 | 22.5 | 3.6 | — | |
| ConfAdaptModel=Qwen3-4B-Inst-2507, Chat Template=Off, tau (τ)=0.652026.02 | 21.8 | 3.7 | — | |
| ConfAdaptModel=L3.1-8B-Magpie, Chat Template=On, tau (τ)=0.62026.02 | 21.2 | 5.3 | — | |
| StaticModel=Qwen3-4B-Inst-2507, Chat Template=Off, k=32026.02 | 21 | 3 | — | |
| ConfAdaptModel=Qwen3-4B-Inst-2507, Chat Template=Off, tau (τ)=0.62026.02 | 19.7 | 4 | — | |
| StaticModel=L3.1-8B-Magpie, Chat Template=On, k=52026.02 | 16.6 | 5 | — | |
| StaticModel=Qwen3-4B-Inst-2507, Chat Template=Off, k=42026.02 | 14.9 | 4 | — | |
| StaticModel=Qwen3-4B-Inst-2507, Chat Template=Off, k=52026.02 | 10.2 | 5 | — | |
| Bi-LoRABackbone=Llama 3.1-8B, Evaluation Protocol=fine-tuning2025.08 | — | — | 43.45 | |
| Bi-LoRACost=×1, Backbone=Llama 3.1-8B, Evaluation Protocol=Fine-tuning2025.08 | — | — | 43.45 | |
| DeLoRABackbone=Llama 3.1-8B, Evaluation Protocol=fine-tuning2025.08 | — | — | 42.72 | |
| DoRABackbone=Llama 3.1-8B, Evaluation Protocol=fine-tuning2025.08 | — | — | 42.98 | |
| Flat-LoRACost=×1, Backbone=Llama 3.1-8B, Evaluation Protocol=Fine-tuning2025.08 | — | — | 43.99 | |
| Full FTCost=×1, Backbone=Llama 3.1-8B, Evaluation Protocol=Fine-tuning2025.08 | — | — | 44.78 | |
| HiRABackbone=Llama 3.1-8B, Evaluation Protocol=fine-tuning2025.08 | — | — | 42.89 | |
| LoRABackbone=Llama 3.1-8B, Evaluation Protocol=fine-tuning2025.08 | — | — | 42.82 | |
| LoRACost=×1, Backbone=Llama 3.1-8B, Evaluation Protocol=Fine-tuning2025.08 | — | — | 42.82 | |
| LoRA-ESAMCost=×1.x, Backbone=Llama 3.1-8B, Evaluation Protocol=Fine-tuning2025.08 | — | — | 43.4 | |
| LoRA-LookSAMCost=×1.x, Backbone=Llama 3.1-8B, Evaluation Protocol=Fine-tuning2025.08 | — | — | 43.28 | |
| LoRA-nBARCost=×1, Backbone=Llama 3.1-8B, Evaluation Protocol=Fine-tuning2025.08 | — | — | 43.39 | |
| LoRA-oBARCost=×1, Backbone=Llama 3.1-8B, Evaluation Protocol=Fine-tuning2025.08 | — | — | 43.44 | |
| LoRA-SAMCost=×2, Backbone=Llama 3.1-8B, Evaluation Protocol=Fine-tuning2025.08 | — | — | 43.49 | |
| PiSSABackbone=Llama 3.1-8B, Evaluation Protocol=fine-tuning2025.08 | — | — | 43.25 | |
| ROPCost=×1, Backbone=Llama 3.1-8B, Evaluation Protocol=Fine-tuning2025.08 | — | — | 43.47 | |
| RWP_fullCost=×1, Backbone=Llama 3.1-8B, Evaluation Protocol=Fine-tuning2025.08 | — | — | 43.35 | |
| RWP_LoRACost=×1, Backbone=Llama 3.1-8B, Evaluation Protocol=Fine-tuning2025.08 | — | — | 44.1 | |
| S^2-SAMBackbone=Llama 3.1-8B, Evaluation Protocol=fine-tuning2025.08 | — | — | 42.44 | |
| WSAMBackbone=Llama 3.1-8B, Evaluation Protocol=fine-tuning2025.08 | — | — | 43.16 |