Continual Routing on 2WikiMultiHop
59.5AccuracyCONCUR
Evaluation results
| Method | Links | ||
|---|---|---|---|
| CONCURSetting=Setting 2: Large and small models, Relative training time=1.00x2025.12 | 59.5 | 38.54 | |
| RTRSetting=Setting 1: Large models only, Relative training time=3.93x, Training Strategy=From scratch (FS)2025.12 | 59.4 | 51.12 | |
| CONCURSetting=Setting 1: Large models only, Relative training time=1.00x2025.12 | 59.3 | 50.71 | |
| EmbedLLMSetting=Setting 2: Large and small models, Relative training time=3.11x, Training Strategy=Fine-tuned (100%)2025.12 | 59.2 | 42.93 | |
| RTRSetting=Setting 2: Large and small models, Relative training time=2.84x, Training Strategy=Fine-tuned (50%)2025.12 | 59.2 | 47.79 | |
| EmbedLLMSetting=Setting 1: Large models only, Relative training time=1.32x, Training Strategy=From scratch (FS)2025.12 | 58.9 | 49.62 | |
| RTRSetting=Setting 2: Large and small models, Relative training time=3.89x, Training Strategy=Fine-tuned (75%)2025.12 | 58.9 | 47.81 | |
| RTRSetting=Setting 2: Large and small models, Relative training time=1.69x, Training Strategy=Fine-tuned (25%)2025.12 | 58.5 | 39.8 | |
| EmbedLLMSetting=Setting 2: Large and small models, Relative training time=3.08x, Training Strategy=From scratch (FS)2025.12 | 58.4 | 43.68 | |
| RTRSetting=Setting 2: Large and small models, Relative training time=7.66x, Training Strategy=From scratch (FS)2025.12 | 57.9 | 44.04 | |
| Best single strategySetting=Setting 1: Large models only, Models=Qwen2.5-7B-Instruct and Llama-3.1-8B-Instruct2025.12 | 57.6 | 49.63 | |
| Best single strategySetting=Setting 2: Large and small models, Models=Qwen2.5, Llama-3.1, and Llama-3.2 family2025.12 | 57.6 | 49.63 | |
| RTRSetting=Setting 2: Large and small models, Relative training time=4.96x, Training Strategy=Fine-tuned (100%)2025.12 | 57.6 | 44.06 | |
| EmbedLLMSetting=Setting 2: Large and small models, Relative training time=1.92x, Training Strategy=Fine-tuned (50%)2025.12 | 57.4 | 36.87 | |
| EmbedLLMSetting=Setting 2: Large and small models, Relative training time=1.01x, Training Strategy=Fine-tuned (25%)2025.12 | 56.9 | 40.12 | |
| EmbedLLMSetting=Setting 2: Large and small models, Relative training time=2.82x, Training Strategy=Fine-tuned (75%)2025.12 | 55.1 | 36.19 | |
| RouteLLMSetting=Setting 1: Large models only, Relative training time=0.18x, Training Strategy=From scratch (FS)2025.12 | 54 | 48.77 | |
| RouteLLMSetting=Setting 2: Large and small models, Relative training time=0.20x, Training Strategy=From scratch (FS)2025.12 | 41.7 | 17.9 | |
| RouteLLMSetting=Setting 2: Large and small models, Relative training time=0.14x, Training Strategy=Fine-tuned (25%)2025.12 | 41.6 | 18.21 | |
| RouteLLMSetting=Setting 2: Large and small models, Relative training time=0.17x, Training Strategy=Fine-tuned (50%)2025.12 | 41.6 | 19.96 | |
| RouteLLMSetting=Setting 2: Large and small models, Relative training time=0.20x, Training Strategy=Fine-tuned (75%)2025.12 | 41.5 | 13.73 | |
| RouteLLMSetting=Setting 2: Large and small models, Relative training time=0.23x, Training Strategy=Fine-tuned (100%)2025.12 | 41.3 | 16.62 |