Continual Routing on MMLU
74.5AccuracyCONCUR
Evaluation results
| Method | Links | ||
|---|---|---|---|
| CONCURSetting=Setting 1: Large models only, Relative training time=1.00x2025.12 | 74.5 | 41.02 | |
| CONCURSetting=Setting 2: Large and small models, Relative training time=1.00x2025.12 | 74.4 | 40.73 | |
| Best single strategySetting=Setting 1: Large models only2025.12 | 73.7 | 44.45 | |
| Best single strategySetting=Setting 2: Large and small models2025.12 | 73.7 | 44.45 | |
| RTRSetting=Setting 2: Large and small models, Relative training time=7.66x, Training Strategy=From scratch (FS)2025.12 | 73.7 | 44.45 | |
| RTRSetting=Setting 2: Large and small models, Relative training time=3.89x, Training Strategy=Fine-tuned (75%)2025.12 | 73.6 | 44.38 | |
| RTRSetting=Setting 2: Large and small models, Relative training time=4.96x, Training Strategy=Fine-tuned (100%)2025.12 | 73.4 | 43.19 | |
| EmbedLLMSetting=Setting 1: Large models only, Relative training time=1.32x, Training Strategy=From scratch (FS)2025.12 | 73.3 | 44.29 | |
| RTRSetting=Setting 1: Large models only, Relative training time=3.93x, Training Strategy=From scratch (FS)2025.12 | 72.1 | 42.98 | |
| EmbedLLMSetting=Setting 2: Large and small models, Relative training time=3.08x, Training Strategy=From scratch (FS)2025.12 | 72.1 | 33.68 | |
| RTRSetting=Setting 2: Large and small models, Relative training time=1.69x, Training Strategy=Fine-tuned (25%)2025.12 | 71.3 | 42.27 | |
| EmbedLLMSetting=Setting 2: Large and small models, Relative training time=3.11x, Training Strategy=Fine-tuned (100%)2025.12 | 70 | 32.59 | |
| EmbedLLMSetting=Setting 2: Large and small models, Relative training time=2.82x, Training Strategy=Fine-tuned (75%)2025.12 | 69.5 | 32.69 | |
| EmbedLLMSetting=Setting 2: Large and small models, Relative training time=1.01x, Training Strategy=Fine-tuned (25%)2025.12 | 68.9 | 34.45 | |
| RouteLLMSetting=Setting 1: Large models only, Relative training time=0.18x, Training Strategy=From scratch (FS)2025.12 | 68.7 | 17.03 | |
| RTRSetting=Setting 2: Large and small models, Relative training time=2.84x, Training Strategy=Fine-tuned (50%)2025.12 | 68.4 | 13.53 | |
| EmbedLLMSetting=Setting 2: Large and small models, Relative training time=1.92x, Training Strategy=Fine-tuned (50%)2025.12 | 68.3 | 26.36 | |
| RouteLLMSetting=Setting 2: Large and small models, Relative training time=0.20x, Training Strategy=From scratch (FS)2025.12 | 54.7 | 3.2 | |
| RouteLLMSetting=Setting 2: Large and small models, Relative training time=0.23x, Training Strategy=Fine-tuned (100%)2025.12 | 54.6 | 3.28 | |
| RouteLLMSetting=Setting 2: Large and small models, Relative training time=0.14x, Training Strategy=Fine-tuned (25%)2025.12 | 53.5 | 2.8 | |
| RouteLLMSetting=Setting 2: Large and small models, Relative training time=0.20x, Training Strategy=Fine-tuned (75%)2025.12 | 53 | 2.91 | |
| RouteLLMSetting=Setting 2: Large and small models, Relative training time=0.17x, Training Strategy=Fine-tuned (50%)2025.12 | 52.9 | 2.9 |