Commonsense Reasoning on HellaS (Accuracy)
93.49AccuracyABBA
Evaluation Results
| Method | Links | |
|---|---|---|
| ABBAModel=Llama-3.2-3B, #Params=48.63M2025.09 | 93.49 | |
| BoHAModel=Llama-3.2-3B, #Params=48.63M2025.09 | 93.16 | |
| FFTModel=Llama-3.2-3B, #Params=3.21B2025.09 | 93.14 | |
| HiRAModel=Llama-3.2-3B, #Params=48.63M2025.09 | 92.94 | |
| GraLoRAModel=Llama-3.2-3B, #Params=48.63M2025.09 | 92.38 | |
| rsLoRAModel=Llama-3.2-3B, #Params=48.63M2025.09 | 90.45 | |
| DoRAModel=Llama-3.2-3B, #Params=49.40M2025.09 | 90.43 | |
| LoRAModel=Llama-3.2-3B, #Params=48.63M2025.09 | 89.63 | |
| PiSSAModel=Llama-3.2-3B, #Params=48.63M2025.09 | 88.05 | |
| EMoEBase Model=ERNIE-4.5-21B-A3B, Trained Activated Experts=6+2, MoE Routing Method=EMoE, Inference Budget (k')=12 + 22025.09 | 84.54 | |
| EMoEBase Model=ERNIE-4.5-21B-A3B, Trained Activated Experts=6+2, MoE Routing Method=EMoE, Inference Budget (k')=6 + 22025.09 | 84.18 | |
| EMoEBase Model=ERNIE-4.5-21B-A3B, Trained Activated Experts=6+2, MoE Routing Method=EMoE, Inference Budget (k')=3 + 22025.09 | 82.96 | |
| Top-kBase Model=ERNIE-4.5-21B-A3B, Trained Activated Experts=6+2, MoE Routing Method=Top-k, Inference Budget (k')=6 + 22025.09 | 81.56 | |
| ABBAModel=Llama-3.2-1B, #Params=22.54M2025.09 | 80.99 | |
| Top-kBase Model=ERNIE-4.5-21B-A3B, Trained Activated Experts=6+2, MoE Routing Method=Top-k, Inference Budget (k')=3 + 22025.09 | 80.86 | |
| Top-kBase Model=ERNIE-4.5-21B-A3B, Trained Activated Experts=6+2, MoE Routing Method=Top-k, Inference Budget (k')=12 + 22025.09 | 80.08 | |
| BoHAModel=Llama-3.2-1B, #Params=22.54M2025.09 | 79.82 | |
| FFTModel=Llama-3.2-1B, #Params=1.24B2025.09 | 79.63 | |
| GraLoRAModel=Llama-3.2-1B, #Params=22.54M2025.09 | 77.81 | |
| DoRAModel=Llama-3.2-1B, #Params=22.92M2025.09 | 77.48 | |
| LoRAModel=Llama-3.2-1B, #Params=22.54M2025.09 | 77.41 | |
| PiSSAModel=Llama-3.2-1B, #Params=22.54M2025.09 | 76.82 | |
| HiRAModel=Llama-3.2-1B, #Params=22.54M2025.09 | 75.74 | |
| rsLoRAModel=Llama-3.2-1B, #Params=22.54M2025.09 | 75.34 | |
| EMoEBase Model=DeepSeek-V2-Lite, Trained Activated Experts=6+2, MoE Routing Method=EMoE, Inference Budget (k')=12 + 22025.09 | 62.25 | |
| EMoEBase Model=DeepSeek-V2-Lite, Trained Activated Experts=6+2, MoE Routing Method=EMoE, Inference Budget (k')=6 + 22025.09 | 61.29 | |
| EMoEBase Model=DeepSeek-V2-Lite, Trained Activated Experts=6+2, MoE Routing Method=EMoE, Inference Budget (k')=3 + 22025.09 | 59.61 | |
| Top-kBase Model=DeepSeek-V2-Lite, Trained Activated Experts=6+2, MoE Routing Method=Top-k, Inference Budget (k')=6 + 22025.09 | 58.52 | |
| Top-kBase Model=DeepSeek-V2-Lite, Trained Activated Experts=6+2, MoE Routing Method=Top-k, Inference Budget (k')=12 + 22025.09 | 57.6 | |
| Top-kBase Model=DeepSeek-V2-Lite, Trained Activated Experts=6+2, MoE Routing Method=Top-k, Inference Budget (k')=3 + 22025.09 | 57.36 | |
| EMoEBase Model=LoRAMoE, Trained Activated Experts=2, MoE Routing Method=EMoE, Inference Budget (k')=62025.09 | 56.14 | |
| EMoEBase Model=LoRAMoE, Trained Activated Experts=2, MoE Routing Method=EMoE, Inference Budget (k')=42025.09 | 55.95 | |
| Top-kBase Model=LoRAMoE, Trained Activated Experts=2, MoE Routing Method=Top-k, Inference Budget (k')=22025.09 | 54.87 | |
| EMoEBase Model=LoRAMoE, Trained Activated Experts=2, MoE Routing Method=EMoE, Inference Budget (k')=22025.09 | 53.57 | |
| Top-kBase Model=LoRAMoE, Trained Activated Experts=2, MoE Routing Method=Top-k, Inference Budget (k')=42025.09 | 53.07 | |
| Top-kBase Model=LoRAMoE, Trained Activated Experts=2, MoE Routing Method=Top-k, Inference Budget (k')=12025.09 | 52.71 | |
| EMoEBase Model=LoRAMoE, Trained Activated Experts=2, MoE Routing Method=EMoE, Inference Budget (k')=12025.09 | 51.52 | |
| EMoEBase Model=OLMoE-1B-7B-0924, Trained Activated Experts=8, MoE Routing Method=EMoE, Inference Budget (k')=82025.09 | 51.22 | |
| Top-kBase Model=LoRAMoE, Trained Activated Experts=2, MoE Routing Method=Top-k, Inference Budget (k')=62025.09 | 50.76 | |
| EMoEBase Model=OLMoE-1B-7B-0924, Trained Activated Experts=8, MoE Routing Method=EMoE, Inference Budget (k')=162025.09 | 50.69 | |
| Top-kBase Model=OLMoE-1B-7B-0924, Trained Activated Experts=8, MoE Routing Method=Top-k, Inference Budget (k')=82025.09 | 48.68 | |
| Top-kBase Model=OLMoE-1B-7B-0924, Trained Activated Experts=8, MoE Routing Method=Top-k, Inference Budget (k')=162025.09 | 48.59 | |
| EMoEBase Model=OLMoE-1B-7B-0924, Trained Activated Experts=8, MoE Routing Method=EMoE, Inference Budget (k')=42025.09 | 43.24 | |
| Top-kBase Model=OLMoE-1B-7B-0924, Trained Activated Experts=8, MoE Routing Method=Top-k, Inference Budget (k')=42025.09 | 41.4 |