# Continual Model Routing on CMRBench v1 (test) Source: [Continual Model Routing on CMRBench v1 (test)]() Collection index: [llms.txt]() Data updated: 2026-05-28T10:15:09.067Z Evaluation of Continual Model Routing performance on the test split of the CMRBench v1 dataset, measured by accuracy (M-Acc, F-Acc, D-Acc) and forgetting (M-Fgt, F-Fgt, D-Fgt) metrics. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Continual Model Routing | | Dataset (including any stated split) | CMRBench v1 (test) | | Metric | D-Acc | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 25 of 25 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [D-Acc](): Higher is better; unit: PERCENT. - [M-Acc](): Higher is better; unit: PERCENT. - [F-Acc](): Higher is better; unit: PERCENT. - [M-Fgt](): Lower is better; unit: PERCENT. - [F-Fgt](): Lower is better; unit: PERCENT. - [D-Fgt](): Lower is better; unit: PERCENT. ## Recorded results | Method | D-Acc | Result date | Report paper | | --- | --- | --- | --- | | From Scratch CARvE | 85.7 | Not recorded | 6a17cc51dbd7150506f815c6 | | Joint Training CARvE | 84.6 | Not recorded | 6a17cc51dbd7150506f815c6 | | Cumulative CARvE | 83.5 | Not recorded | 6a17cc51dbd7150506f815c6 | | CARvE | 82.9 | Not recorded | 6a17cc51dbd7150506f815c6 | | Cumulative Zero-Shot | 81.9 | Not recorded | 6a17cc51dbd7150506f815c6 | | CARvE | 81.5 | Not recorded | 6a17cc51dbd7150506f815c6 | | CARvE | 80.7 | Not recorded | 6a17cc51dbd7150506f815c6 | | CARvE EWC | 80.5 | Not recorded | 6a17cc51dbd7150506f815c6 | | From Scratch Zero-Shot | 80.1 | Not recorded | 6a17cc51dbd7150506f815c6 | | Joint Training Zero-Shot | 79.3 | Not recorded | 6a17cc51dbd7150506f815c6 | | CARvE | 78.6 | Not recorded | 6a17cc51dbd7150506f815c6 | | CARvE | 78.5 | Not recorded | 6a17cc51dbd7150506f815c6 | | Random Replay | 78.2 | Not recorded | 6a17cc51dbd7150506f815c6 | | Random Replay | 78.1 | Not recorded | 6a17cc51dbd7150506f815c6 | | Joint Training RAT | 77.3 | Not recorded | 6a17cc51dbd7150506f815c6 | | Random Replay | 77.2 | Not recorded | 6a17cc51dbd7150506f815c6 | | Random Replay | 75.9 | Not recorded | 6a17cc51dbd7150506f815c6 | | Random Replay | 70.2 | Not recorded | 6a17cc51dbd7150506f815c6 | | EWC | 66.2 | Not recorded | 6a17cc51dbd7150506f815c6 | | Sequential Finetuning | 64.3 | Not recorded | 6a17cc51dbd7150506f815c6 | | LwF | 56.4 | Not recorded | 6a17cc51dbd7150506f815c6 | | HuggingGPT | 51.7 | Not recorded | 6a17cc51dbd7150506f815c6 | | BGE-M3 | 44 | Not recorded | 6a17cc51dbd7150506f815c6 | | Gorilla RAG | 43 | Not recorded | 6a17cc51dbd7150506f815c6 | | Model Merging (TIES) | 28.6 | Not recorded | 6a17cc51dbd7150506f815c6 | ### From Scratch CARvE D-Acc: 85.7. Result ID: 6a1815a7fbc373592c1271ca. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Setting":"Upper bound (non-continual)"}. This record does not assert independent reproduction. ### Joint Training CARvE D-Acc: 84.6. Result ID: 6a1815a7fbc373592c1271c8. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Setting":"Upper bound (non-continual)"}. This record does not assert independent reproduction. ### Cumulative CARvE D-Acc: 83.5. Result ID: 6a1815a7fbc373592c1271c9. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Setting":"Upper bound (non-continual)"}. This record does not assert independent reproduction. ### CARvE D-Acc: 82.9. Result ID: 6a1815a7fbc373592c1271c3. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"LLaMA2-7B","Replay Percentage":"20%"}. This record does not assert independent reproduction. ### Cumulative Zero-Shot D-Acc: 81.9. Result ID: 6a1815a7fbc373592c1271c5. [Reporting paper]() Recorded evaluation configuration: {"Setting":"Upper bound (non-continual)"}. This record does not assert independent reproduction. ### CARvE D-Acc: 81.5. Result ID: 6a1815a7fbc373592c1271c2. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B","Replay Percentage":"10%"}. This record does not assert independent reproduction. ### CARvE D-Acc: 80.7. Result ID: 6a1815a7fbc373592c1271c0. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"LLaMA2-7B","Replay Percentage":"10%"}. This record does not assert independent reproduction. ### CARvE EWC D-Acc: 80.5. Result ID: 6a1815a7fbc373592c1271be. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"LLaMA2-7B","Replay Percentage":"10%"}. This record does not assert independent reproduction. ### From Scratch Zero-Shot D-Acc: 80.1. Result ID: 6a1815a7fbc373592c1271c6. [Reporting paper]() Recorded evaluation configuration: {"Setting":"Upper bound (non-continual)"}. This record does not assert independent reproduction. ### Joint Training Zero-Shot D-Acc: 79.3. Result ID: 6a1815a7fbc373592c1271c4. [Reporting paper]() Recorded evaluation configuration: {"Setting":"Upper bound (non-continual)"}. This record does not assert independent reproduction. ### CARvE D-Acc: 78.6. Result ID: 6a1815a7fbc373592c1271c1. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-4B","Replay Percentage":"10%"}. This record does not assert independent reproduction. ### CARvE D-Acc: 78.5. Result ID: 6a1815a7fbc373592c1271bf. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"LLaMA2-7B","Replay Percentage":"5%"}. This record does not assert independent reproduction. ### Random Replay D-Acc: 78.2. Result ID: 6a1815a7fbc373592c1271b8. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B","Replay Percentage":"10%"}. This record does not assert independent reproduction. ### Random Replay D-Acc: 78.1. Result ID: 6a1815a7fbc373592c1271b7. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"LLaMA2-7B","Replay Percentage":"20%"}. This record does not assert independent reproduction. ### Joint Training RAT D-Acc: 77.3. Result ID: 6a1815a7fbc373592c1271c7. [Reporting paper]() Recorded evaluation configuration: {"Setting":"Upper bound (non-continual)"}. This record does not assert independent reproduction. ### Random Replay D-Acc: 77.2. Result ID: 6a1815a7fbc373592c1271b9. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-4B","Replay Percentage":"10%"}. This record does not assert independent reproduction. ### Random Replay D-Acc: 75.9. Result ID: 6a1815a7fbc373592c1271b6. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"LLaMA2-7B","Replay Percentage":"10%"}. This record does not assert independent reproduction. ### Random Replay D-Acc: 70.2. Result ID: 6a1815a7fbc373592c1271b5. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"LLaMA2-7B","Replay Percentage":"5%"}. This record does not assert independent reproduction. ### EWC D-Acc: 66.2. Result ID: 6a1815a7fbc373592c1271bd. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"LLaMA2-7B"}. This record does not assert independent reproduction. ### Sequential Finetuning D-Acc: 64.3. Result ID: 6a1815a7fbc373592c1271ba. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"LLaMA2-7B"}. This record does not assert independent reproduction. ### LwF D-Acc: 56.4. Result ID: 6a1815a7fbc373592c1271bc. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"LLaMA2-7B"}. This record does not assert independent reproduction. ### HuggingGPT D-Acc: 51.7. Result ID: 6a1815a7fbc373592c1271b4. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Qwen3-32B","Setting":"LLM-controller baseline"}. This record does not assert independent reproduction. ### BGE-M3 D-Acc: 44. Result ID: 6a1815a7fbc373592c1271b2. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Setting":"Retrieval baseline"}. This record does not assert independent reproduction. ### Gorilla RAG D-Acc: 43. Result ID: 6a1815a7fbc373592c1271b3. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Setting":"Retrieval baseline"}. This record does not assert independent reproduction. ### Model Merging (TIES) D-Acc: 28.6. Result ID: 6a1815a7fbc373592c1271bb. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"LLaMA2-7B"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.