Commonsense Reasoning on Commonsense-15K (test)
36.11ARC-Challenge AccuracyUB-SMoE
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| UB-SMoEBackbone=OLMoE-1B-7B, Method Category=Heterogeneous sparsity methods, Computation Budget=Averaged over all budgets2026.05 | 36.11 | 50.17 | 49.52 | 22.58 | 40.3 | 51.18 | 44.86 | 46.65 | 42.67 | |
| A3SMoEBackbone=OLMoE-1B-7B, Method Category=Heterogeneous sparsity methods, Computation Budget=Averaged over all budgets2026.05 | 33.47 | 47.24 | 43.01 | 24.48 | 39.25 | 42.19 | 41.93 | 37.33 | 38.61 | |
| SMoE-LLBBackbone=OLMoE-1B-7B, Method Category=Heterogeneous sparsity methods, Computation Budget=Averaged over all budgets2026.05 | 30.8 | 42.13 | 51.22 | 20.96 | 33.6 | 42.44 | 38.01 | 44.1 | 37.91 | |
| FlexLoRABackbone=OLMoE-1B-7B, Method Category=Heterogeneous LoRA-rank methods, Computation Budget=Averaged over all budgets2026.05 | 30.12 | 41.36 | 45.73 | 16.07 | 32.25 | 33.27 | 30.99 | 34.45 | 33.03 | |
| HetLoRABackbone=OLMoE-1B-7B, Method Category=Heterogeneous LoRA-rank methods, Computation Budget=Averaged over all budgets2026.05 | 12.84 | 15.82 | 37.09 | 7.14 | 12 | 22.78 | 14.52 | 27.7 | 18.74 | |
| FLoRISTBackbone=OLMoE-1B-7B, Method Category=Heterogeneous LoRA-rank methods, Computation Budget=Averaged over all budgets2026.05 | 9.6 | 10.97 | 34.07 | 6.05 | 13.8 | 23.19 | 8.14 | 12.57 | 14.8 | |
| FLoRABackbone=OLMoE-1B-7B, Method Category=Heterogeneous LoRA-rank methods, Computation Budget=Averaged over all budgets2026.05 | 8.68 | 10.04 | 34.72 | 6.74 | 11.6 | 22.85 | 8.85 | 17.86 | 15.17 |