Commonsense Reasoning on HellaSwag (Accuracy, Delta, and Confidence Intervals)
89.3AccuracyMoE-Sieve (Qwen1.5-MoE-A2.7B)
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| MoE-Sieve (Qwen1.5-MoE-A2.7B)Condition=Hot (25%)2026.03 | 89.3 | 0.73 | 0.53 | — | |
| Qwen1.5-MoE-A2.7BCondition=Full LoRA2026.03 | 88.5 | — | — | — | |
| MoE-Sieve (OLMoE-1B-7B)Condition=Hot (25%)2026.03 | 80.7 | 0.17 | 0.71 | — | |
| OLMoE-1B-7BCondition=Full LoRA2026.03 | 80.5 | — | — | — | |
| NAG_Qwen3-1.7BTarget Setting=Single-Target, Backbone=Qwen3-1.7B2026.04 | 60.6 | — | — | — | |
| NAG_SmolLM3-3BTarget Setting=Single-Target, Backbone=SmolLM3-3B2026.04 | 59.8 | — | — | — | |
| NAG_Llama-3.2-3BTarget Setting=Single-Target, Backbone=Llama-3.2-3B2026.04 | 58.6 | — | — | — | |
| NAG_Qwen3-1.7BTarget Setting=Multi-Target, Backbone=Qwen3-1.7B2026.04 | 57.8 | — | — | — | |
| BETRTarget Setting=Single-Target2026.04 | 57.5 | — | — | — | |
| FineWeb-Edu2026.04 | 55.3 | — | — | — | |
| NAG_SmolLM3-3BTarget Setting=Multi-Target, Backbone=SmolLM3-3B2026.04 | 55.2 | — | — | — | |
| NAG_Llama-3.2-3BTarget Setting=Multi-Target, Backbone=Llama-3.2-3B2026.04 | 54.9 | — | — | — | |
| Random2026.04 | 51.6 | — | — | — | |
| BETRTarget Setting=Multi-Target2026.04 | 49.3 | — | — | — | |
| Teacher (Qwen3-0.6B)Role=Teacher, Model=Qwen3-0.6B, Evaluation Protocol=generation-based evaluation2026.03 | 39 | — | — | — | |
| Hybrid KDASequence Mixer=KDA, Attention Layers=7, Evaluation Protocol=generation-based evaluation2026.03 | 30 | — | — | — | |
| Hybrid MambaSequence Mixer=Mamba2, Attention Layers=7, Evaluation Protocol=generation-based evaluation2026.03 | 29 | — | — | — | |
| Pure KDASequence Mixer=KDA, Attention Layers=0, Evaluation Protocol=generation-based evaluation2026.03 | 20.3 | — | — | — | |
| Pure MambaSequence Mixer=Mamba2, Attention Layers=0, Evaluation Protocol=generation-based evaluation2026.03 | 17.3 | — | — | — | |
| FLOATDOORScenario=S1, Backbone=Qwen3-4B2026.06 | — | 0.05 | — | — | |
| FLOATDOORScenario=S2, Backbone=Qwen3-4B2026.06 | — | 0.01 | — | — | |
| FLOATDOORScenario=S3, Backbone=Qwen3-4B2026.06 | — | 0.11 | — | — | |
| FLOATDOORScenario=M1, Backbone=Qwen3-4B2026.06 | — | 0.2 | — | — | |
| FLOATDOORScenario=M2, Backbone=Qwen3-4B2026.06 | — | 0.38 | — | — |