Multi-task Language Evaluation on GSM8K, ARC-C, MedQA, BoolQ, and CoLA Average
79.87Average AccuracyPROMPT CLASSIFICATION
Evaluation Results
| Method | Links | |
|---|---|---|
| PROMPT CLASSIFICATIONComposition Type=NONE, Routing Granularity=PROMPT-LEVEL/DYNAMIC, Extra Objective=BERT CLASSIFICATION, Backbone=Llama-8B2026.06 | 79.87 | |
| HARD-ROUTED MOR-LORAComposition Type=HARD UNIT-SCALE ROUTING, Routing Granularity=TOKEN-LEVEL, Extra Objective=NONE, Backbone=Llama-8B2026.06 | 79.8 | |
| MOLE (α = 0.5)Composition Type=SOFT MOE-LORA ROUTING, Routing Granularity=TOKEN-LEVEL, Extra Objective=LOAD BALANCING, Backbone=Llama-8B2026.06 | 78.5 | |
| LORAMIXER TOPK=2 NORM.Composition Type=NORMALIZED SOFT ROUTING, Routing Granularity=TOKEN-LEVEL, Extra Objective=PRESERVATION LOSS, Backbone=Llama-8B2026.06 | 78.33 | |
| MOLE (α = 0.1)Composition Type=SOFT MOE-LORA ROUTING, Routing Granularity=TOKEN-LEVEL, Extra Objective=LOAD BALANCING, Backbone=Llama-8B2026.06 | 78.02 | |
| LORAMIXER TOPK=1Composition Type=SOFT MIXTURE ROUTING, Routing Granularity=TOKEN-LEVEL, Extra Objective=PRESERVATION LOSS, Backbone=Llama-8B2026.06 | 77.74 | |
| LORAHUBComposition Type=STATIC ADAPTER MERGING, Routing Granularity=PROMPT-LEVEL/STATIC, Extra Objective=BLACK-BOX WEIGHT SEARCH, Backbone=Llama-8B2026.06 | 75.76 | |
| PROMPT CLASSIFICATIONComposition Type=NONE, Routing Granularity=PROMPT-LEVEL/DYNAMIC, Extra Objective=BERT CLASSIFICATION, Backbone=Llama-3B2026.06 | 72.79 | |
| HARD-ROUTED MOR-LORAComposition Type=HARD UNIT-SCALE ROUTING, Routing Granularity=TOKEN-LEVEL, Extra Objective=NONE, Backbone=Llama-3B2026.06 | 72.07 | |
| LORAMIXER TOPK=2 NORM.Composition Type=NORMALIZED SOFT ROUTING, Routing Granularity=TOKEN-LEVEL, Extra Objective=PRESERVATION LOSS, Backbone=Llama-3B2026.06 | 70.81 | |
| MOLE (α = 0.1)Composition Type=SOFT MOE-LORA ROUTING, Routing Granularity=TOKEN-LEVEL, Extra Objective=LOAD BALANCING, Backbone=Llama-3B2026.06 | 68.66 | |
| MOLE (α = 0.5)Composition Type=SOFT MOE-LORA ROUTING, Routing Granularity=TOKEN-LEVEL, Extra Objective=LOAD BALANCING, Backbone=Llama-3B2026.06 | 68.5 | |
| LORAHUBComposition Type=STATIC ADAPTER MERGING, Routing Granularity=PROMPT-LEVEL/STATIC, Extra Objective=BLACK-BOX WEIGHT SEARCH, Backbone=Llama-3B2026.06 | 67.75 | |
| LORAMIXER TOPK=1Composition Type=SOFT MIXTURE ROUTING, Routing Granularity=TOKEN-LEVEL, Extra Objective=PRESERVATION LOSS, Backbone=Llama-3B2026.06 | 65.67 |