Scientific Question Answering on ARC Challenge (Acc., F1, FCR, Time)
84.61AccuracyHard-Routed MoR-LoRA
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Hard-Routed MoR-LoRABackbone=Llama-8B, # Trainable Parameters=≈ 109M2026.06 | 84.61 | — | — | — | |
| LoRAMixer TopK=2 NormalizedBackbone=Llama-8B, # Trainable Parameters=≈ 1.133B2026.06 | 83.96 | — | — | — | |
| LoRAMixer TopK=1Backbone=Llama-8B, # Trainable Parameters=≈ 1.133B2026.06 | 83.3 | — | — | — | |
| Hard-Routed MoR-LoRABackbone=Llama-3B, # Trainable Parameters=≈ 73M2026.06 | 75.43 | — | — | — | |
| LoRAMixer TopK=2 NormalizedBackbone=Llama-3B, # Trainable Parameters=≈ 606M2026.06 | 73.98 | — | — | — | |
| LoRAMixer TopK=1Backbone=Llama-3B, # Trainable Parameters=≈ 606M2026.06 | 68.34 | — | — | — | |
| Qwen2.5-1.5BCompression=None2026.05 | 63.5 | 64.3 | 99.9 | 9.3 | |
| MedTPEBackbone=Qwen2.5-1.5B2026.05 | 60.7 | 61.2 | 99.7 | 4.7 | |
| Llama3-1BCompression=None2026.05 | 47.3 | 47.4 | 99.2 | 21.2 | |
| MedTPEBackbone=Llama3-1B2026.05 | 42.7 | 43.6 | 99.9 | 5 | |
| LLMLingua2Backbone=Qwen2.5-1.5B2026.05 | 25.9 | 23 | 98.2 | 16.7 | |
| LoopMDMS=6, Zero-shot=true2026.05 | 25.7 | — | — | — | |
| MDMZero-shot=true2026.05 | 25.5 | — | — | — | |
| LoopMDMS=12, Zero-shot=true2026.05 | 25.3 | — | — | — | |
| LoopMDMS=1, Zero-shot=true2026.05 | 23.6 | — | — | — | |
| LLMLingua2Backbone=Llama3-1B2026.05 | 16.5 | 18.1 | 84.6 | 88 |