Continual Routing on GSM8k
91.7AccuracyCONCUR
Evaluation results
| Method | Links | ||
|---|---|---|---|
| CONCURSetting=Setting 1: Large models only, Relative training time=1.00x2025.12 | 91.7 | 30.78 | |
| Best single strategySetting=Setting 1: Large models only2025.12 | 91.6 | 30.82 | |
| RTRSetting=Setting 1: Large models only, Relative training time=3.93x, Training Strategy=From scratch (FS)2025.12 | 91.6 | 30.82 | |
| Best single strategySetting=Setting 2: Large and small models2025.12 | 91.6 | 30.82 | |
| RTRSetting=Setting 2: Large and small models, Relative training time=3.89x, Training Strategy=Fine-tuned (75%)2025.12 | 91.6 | 30.82 | |
| CONCURSetting=Setting 2: Large and small models, Relative training time=1.00x2025.12 | 91.6 | 30.23 | |
| RTRSetting=Setting 2: Large and small models, Relative training time=7.66x, Training Strategy=From scratch (FS)2025.12 | 91.3 | 33.07 | |
| EmbedLLMSetting=Setting 1: Large models only, Relative training time=1.32x, Training Strategy=From scratch (FS)2025.12 | 91.2 | 33.44 | |
| EmbedLLMSetting=Setting 2: Large and small models, Relative training time=2.82x, Training Strategy=Fine-tuned (75%)2025.12 | 90.2 | 27.4 | |
| RouteLLMSetting=Setting 1: Large models only, Relative training time=0.18x, Training Strategy=From scratch (FS)2025.12 | 90 | 30.06 | |
| EmbedLLMSetting=Setting 2: Large and small models, Relative training time=3.08x, Training Strategy=From scratch (FS)2025.12 | 89.8 | 25.28 | |
| EmbedLLMSetting=Setting 2: Large and small models, Relative training time=3.11x, Training Strategy=Fine-tuned (100%)2025.12 | 89.7 | 26.45 | |
| EmbedLLMSetting=Setting 2: Large and small models, Relative training time=1.01x, Training Strategy=Fine-tuned (25%)2025.12 | 89.6 | 30.83 | |
| EmbedLLMSetting=Setting 2: Large and small models, Relative training time=1.92x, Training Strategy=Fine-tuned (50%)2025.12 | 89.5 | 30.64 | |
| RTRSetting=Setting 2: Large and small models, Relative training time=4.96x, Training Strategy=Fine-tuned (100%)2025.12 | 88.3 | 17.91 | |
| RTRSetting=Setting 2: Large and small models, Relative training time=1.69x, Training Strategy=Fine-tuned (25%)2025.12 | 88.2 | 13.49 | |
| RTRSetting=Setting 2: Large and small models, Relative training time=2.84x, Training Strategy=Fine-tuned (50%)2025.12 | 88.2 | 13.51 | |
| RouteLLMSetting=Setting 2: Large and small models, Relative training time=0.17x, Training Strategy=Fine-tuned (50%)2025.12 | 66.5 | 10.13 | |
| RouteLLMSetting=Setting 2: Large and small models, Relative training time=0.20x, Training Strategy=From scratch (FS)2025.12 | 64.1 | 9.51 | |
| RouteLLMSetting=Setting 2: Large and small models, Relative training time=0.23x, Training Strategy=Fine-tuned (100%)2025.12 | 62.5 | 9.33 | |
| RouteLLMSetting=Setting 2: Large and small models, Relative training time=0.14x, Training Strategy=Fine-tuned (25%)2025.12 | 61.5 | 9.69 | |
| RouteLLMSetting=Setting 2: Large and small models, Relative training time=0.20x, Training Strategy=Fine-tuned (75%)2025.12 | 59.1 | 9.45 |