# Skill-selection on ToolBench-60 soft-0.3 preference regime (held-out pool) Source: [Skill-selection on ToolBench-60 soft-0.3 preference regime (held-out pool)]() Collection index: [llms.txt]() Data updated: 2026-06-05T09:11:26.697Z Evaluation of the model's ability to select the correct skill using the ToolBench dataset under the soft-0.3 preference regime, measured by Regret, Accuracy, and SRC. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Skill-selection | | Dataset (including any stated split) | ToolBench-60 soft-0.3 preference regime (held-out pool) | | Metric | Regret | | Unit | SCALAR | | Direction | Lower is better | | Results loaded | 18 of 18 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Regret](): Lower is better; unit: SCALAR. - [Accuracy](): Higher is better; unit: PERCENT. - [SRC](): Higher is better; unit: SCALAR. ## Recorded results | Method | Regret | Result date | Report paper | | --- | --- | --- | --- | | Bandit-as-Override | 255.9 | Not recorded | 6a2258a186173ad9c1468be8 | | Bandit-as-Override | 256.8 | Not recorded | 6a2258a186173ad9c1468be8 | | Pure-Bandit | 279.2 | Not recorded | 6a2258a186173ad9c1468be8 | | Pure-Bandit | 280.7 | Not recorded | 6a2258a186173ad9c1468be8 | | Freq-as-Override | 286.5 | Not recorded | 6a2258a186173ad9c1468be8 | | Freq-as-Override | 288 | Not recorded | 6a2258a186173ad9c1468be8 | | Bandit-as-Context | 309.6 | Not recorded | 6a2258a186173ad9c1468be8 | | Profile-Memory | 316 | Not recorded | 6a2258a186173ad9c1468be8 | | Bandit-as-Context | 327.1 | Not recorded | 6a2258a186173ad9c1468be8 | | Freq-Greedy | 327.2 | Not recorded | 6a2258a186173ad9c1468be8 | | Profile-Memory | 327.4 | Not recorded | 6a2258a186173ad9c1468be8 | | Freq-Greedy | 328 | Not recorded | 6a2258a186173ad9c1468be8 | | InCotext-Memory | 372.5 | Not recorded | 6a2258a186173ad9c1468be8 | | InCotext-Memory | 373.5 | Not recorded | 6a2258a186173ad9c1468be8 | | ZeroShot-LLM | 375.5 | Not recorded | 6a2258a186173ad9c1468be8 | | ZeroShot-LLM | 376.7 | Not recorded | 6a2258a186173ad9c1468be8 | | Random | 491.5 | Not recorded | 6a2258a186173ad9c1468be8 | | Random | 491.7 | Not recorded | 6a2258a186173ad9c1468be8 | ### Bandit-as-Override Regret: 255.9. Result ID: 6a2291ddfbc373592c20dfb0. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"DeepSeek-V4-Flash","Agent Category":"LLM with Statistical Prior"}. This record does not assert independent reproduction. ### Bandit-as-Override Regret: 256.8. Result ID: 6a2291ddfbc373592c20dfd4. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"GPT-5.2"}. This record does not assert independent reproduction. ### Pure-Bandit Regret: 279.2. Result ID: 6a2291ddfbc373592c20dfcf. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"GPT-5.2"}. This record does not assert independent reproduction. ### Pure-Bandit Regret: 280.7. Result ID: 6a2291ddfbc373592c20dfab. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"DeepSeek-V4-Flash","Agent Category":"Statistical"}. This record does not assert independent reproduction. ### Freq-as-Override Regret: 286.5. Result ID: 6a2291ddfbc373592c20dfd3. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"GPT-5.2"}. This record does not assert independent reproduction. ### Freq-as-Override Regret: 288. Result ID: 6a2291ddfbc373592c20dfaf. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"DeepSeek-V4-Flash","Agent Category":"LLM with Statistical Prior"}. This record does not assert independent reproduction. ### Bandit-as-Context Regret: 309.6. Result ID: 6a2291ddfbc373592c20dfae. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"DeepSeek-V4-Flash","Agent Category":"LLM with Statistical Prior"}. This record does not assert independent reproduction. ### Profile-Memory Regret: 316. Result ID: 6a2291ddfbc373592c20dfd1. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"GPT-5.2"}. This record does not assert independent reproduction. ### Bandit-as-Context Regret: 327.1. Result ID: 6a2291ddfbc373592c20dfd2. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"GPT-5.2"}. This record does not assert independent reproduction. ### Freq-Greedy Regret: 327.2. Result ID: 6a2291ddfbc373592c20dfce. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"GPT-5.2"}. This record does not assert independent reproduction. ### Profile-Memory Regret: 327.4. Result ID: 6a2291ddfbc373592c20dfad. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"DeepSeek-V4-Flash","Agent Category":"LLM with Memory"}. This record does not assert independent reproduction. ### Freq-Greedy Regret: 328. Result ID: 6a2291ddfbc373592c20dfaa. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"DeepSeek-V4-Flash","Agent Category":"Statistical"}. This record does not assert independent reproduction. ### InCotext-Memory Regret: 372.5. Result ID: 6a2291ddfbc373592c20dfac. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"DeepSeek-V4-Flash","Agent Category":"LLM with Memory"}. This record does not assert independent reproduction. ### InCotext-Memory Regret: 373.5. Result ID: 6a2291ddfbc373592c20dfd0. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"GPT-5.2"}. This record does not assert independent reproduction. ### ZeroShot-LLM Regret: 375.5. Result ID: 6a2291ddfbc373592c20dfcd. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"GPT-5.2"}. This record does not assert independent reproduction. ### ZeroShot-LLM Regret: 376.7. Result ID: 6a2291ddfbc373592c20dfa9. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"DeepSeek-V4-Flash","Agent Category":"No Learning"}. This record does not assert independent reproduction. ### Random Regret: 491.5. Result ID: 6a2291ddfbc373592c20dfcc. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"GPT-5.2"}. This record does not assert independent reproduction. ### Random Regret: 491.7. Result ID: 6a2291ddfbc373592c20dfa8. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"DeepSeek-V4-Flash","Agent Category":"No Learning"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.