# Skill-selection on ToolBench-60 one-hot preference regime (held-out pool) Source: [Skill-selection on ToolBench-60 one-hot preference regime (held-out pool)]() Collection index: [llms.txt]() Data updated: 2026-06-05T09:11:26.662Z Evaluation of model performance on skill selection using the specific ToolBench-60 configuration under the one-hot preference regime, measured by Regret, Accuracy, and R.R. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Skill-selection | | Dataset (including any stated split) | ToolBench-60 one-hot preference regime (held-out pool) | | Metric | Regret | | Unit | SCALAR | | Direction | Lower is better | | Results loaded | 27 of 27 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Regret](): Lower is better; unit: SCALAR. - [Accuracy](): Higher is better; unit: PERCENT. - [RR Score](): Higher is better; unit: PERCENT. ## Recorded results | Method | Regret | Result date | Report paper | | --- | --- | --- | --- | | Freq-as-Override | 103.3 | Not recorded | 6a2258a186173ad9c1468be8 | | Freq-as-Override | 107.2 | Not recorded | 6a2258a186173ad9c1468be8 | | Bandit-as-Override | 118.9 | Not recorded | 6a2258a186173ad9c1468be8 | | Bandit-as-Override | 122.4 | Not recorded | 6a2258a186173ad9c1468be8 | | Freq-as-Override | 126.3 | Not recorded | 6a2258a186173ad9c1468be8 | | Bandit-as-Override | 135.7 | Not recorded | 6a2258a186173ad9c1468be8 | | Pure-Bandit | 137.1 | Not recorded | 6a2258a186173ad9c1468be8 | | Pure-Bandit | 139.6 | Not recorded | 6a2258a186173ad9c1468be8 | | Pure-Bandit | 140.2 | Not recorded | 6a2258a186173ad9c1468be8 | | Freq-Greedy | 162.4 | Not recorded | 6a2258a186173ad9c1468be8 | | Freq-Greedy | 166 | Not recorded | 6a2258a186173ad9c1468be8 | | Freq-Greedy | 167.6 | Not recorded | 6a2258a186173ad9c1468be8 | | Profile-Memory | 208.7 | Not recorded | 6a2258a186173ad9c1468be8 | | Bandit-as-Context | 216.6 | Not recorded | 6a2258a186173ad9c1468be8 | | Profile-Memory | 245.1 | Not recorded | 6a2258a186173ad9c1468be8 | | Bandit-as-Context | 266 | Not recorded | 6a2258a186173ad9c1468be8 | | Profile-Memory | 269.5 | Not recorded | 6a2258a186173ad9c1468be8 | | Bandit-as-Context | 344.2 | Not recorded | 6a2258a186173ad9c1468be8 | | InContext-Memory | 363.9 | Not recorded | 6a2258a186173ad9c1468be8 | | InCotext-Memory | 365.4 | Not recorded | 6a2258a186173ad9c1468be8 | | InCotext-Memory | 371.5 | Not recorded | 6a2258a186173ad9c1468be8 | | ZeroShot-LLM | 376 | Not recorded | 6a2258a186173ad9c1468be8 | | ZeroShot-LLM | 377.1 | Not recorded | 6a2258a186173ad9c1468be8 | | ZeroShot-LLM | 377.2 | Not recorded | 6a2258a186173ad9c1468be8 | | Random | 491.8 | Not recorded | 6a2258a186173ad9c1468be8 | | Random | 492 | Not recorded | 6a2258a186173ad9c1468be8 | | Random | 492.1 | Not recorded | 6a2258a186173ad9c1468be8 | ### Freq-as-Override Regret: 103.3. Result ID: 6a2291ddfbc373592c20dfa6. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"DeepSeek-V4-Flash","Agent Category":"LLM with Statistical Prior"}. This record does not assert independent reproduction. ### Freq-as-Override Regret: 107.2. Result ID: 6a2291ddfbc373592c20dfca. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"GPT-5.2"}. This record does not assert independent reproduction. ### Bandit-as-Override Regret: 118.9. Result ID: 6a2291ddfbc373592c20dfa7. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"DeepSeek-V4-Flash","Agent Category":"LLM with Statistical Prior"}. This record does not assert independent reproduction. ### Bandit-as-Override Regret: 122.4. Result ID: 6a2291ddfbc373592c20dfcb. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"GPT-5.2"}. This record does not assert independent reproduction. ### Freq-as-Override Regret: 126.3. Result ID: 6a2291ddfbc373592c20dfb8. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Method Category":"LLM with Statistical Prior","Backbone":"Qwen3-30B-Instruct"}. This record does not assert independent reproduction. ### Bandit-as-Override Regret: 135.7. Result ID: 6a2291ddfbc373592c20dfb9. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Method Category":"LLM with Statistical Prior","Backbone":"Qwen3-30B-Instruct"}. This record does not assert independent reproduction. ### Pure-Bandit Regret: 137.1. Result ID: 6a2291ddfbc373592c20dfc6. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"GPT-5.2"}. This record does not assert independent reproduction. ### Pure-Bandit Regret: 139.6. Result ID: 6a2291ddfbc373592c20dfa2. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"DeepSeek-V4-Flash","Agent Category":"Statistical"}. This record does not assert independent reproduction. ### Pure-Bandit Regret: 140.2. Result ID: 6a2291ddfbc373592c20dfb4. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Method Category":"Statistical","Backbone":"Qwen3-30B-Instruct"}. This record does not assert independent reproduction. ### Freq-Greedy Regret: 162.4. Result ID: 6a2291ddfbc373592c20dfc5. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"GPT-5.2"}. This record does not assert independent reproduction. ### Freq-Greedy Regret: 166. Result ID: 6a2291ddfbc373592c20dfa1. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"DeepSeek-V4-Flash","Agent Category":"Statistical"}. This record does not assert independent reproduction. ### Freq-Greedy Regret: 167.6. Result ID: 6a2291ddfbc373592c20dfb3. [Reporting paper]() Recorded evaluation configuration: {"Method Category":"Statistical","Backbone":"Qwen3-30B-Instruct"}. This record does not assert independent reproduction. ### Profile-Memory Regret: 208.7. Result ID: 6a2291ddfbc373592c20dfc8. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"GPT-5.2"}. This record does not assert independent reproduction. ### Bandit-as-Context Regret: 216.6. Result ID: 6a2291ddfbc373592c20dfa5. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"DeepSeek-V4-Flash","Agent Category":"LLM with Statistical Prior"}. This record does not assert independent reproduction. ### Profile-Memory Regret: 245.1. Result ID: 6a2291ddfbc373592c20dfa4. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"DeepSeek-V4-Flash","Agent Category":"LLM with Memory"}. This record does not assert independent reproduction. ### Bandit-as-Context Regret: 266. Result ID: 6a2291ddfbc373592c20dfc9. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"GPT-5.2"}. This record does not assert independent reproduction. ### Profile-Memory Regret: 269.5. Result ID: 6a2291ddfbc373592c20dfb6. [Reporting paper]() Recorded evaluation configuration: {"Method Category":"LLM with Memory","Backbone":"Qwen3-30B-Instruct"}. This record does not assert independent reproduction. ### Bandit-as-Context Regret: 344.2. Result ID: 6a2291ddfbc373592c20dfb7. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Method Category":"LLM with Memory","Backbone":"Qwen3-30B-Instruct"}. This record does not assert independent reproduction. ### InContext-Memory Regret: 363.9. Result ID: 6a2291ddfbc373592c20dfb5. [Reporting paper]() Recorded evaluation configuration: {"Method Category":"LLM with Memory","Backbone":"Qwen3-30B-Instruct"}. This record does not assert independent reproduction. ### InCotext-Memory Regret: 365.4. Result ID: 6a2291ddfbc373592c20dfa3. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"DeepSeek-V4-Flash","Agent Category":"LLM with Memory"}. This record does not assert independent reproduction. ### InCotext-Memory Regret: 371.5. Result ID: 6a2291ddfbc373592c20dfc7. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"GPT-5.2"}. This record does not assert independent reproduction. ### ZeroShot-LLM Regret: 376. Result ID: 6a2291ddfbc373592c20dfc4. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"GPT-5.2"}. This record does not assert independent reproduction. ### ZeroShot-LLM Regret: 377.1. Result ID: 6a2291ddfbc373592c20dfa0. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"DeepSeek-V4-Flash","Agent Category":"No Learning"}. This record does not assert independent reproduction. ### ZeroShot-LLM Regret: 377.2. Result ID: 6a2291ddfbc373592c20dfb2. [Reporting paper]() Recorded evaluation configuration: {"Method Category":"No Learning","Backbone":"Qwen3-30B-Instruct"}. This record does not assert independent reproduction. ### Random Regret: 491.8. Result ID: 6a2291ddfbc373592c20dfc3. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"GPT-5.2"}. This record does not assert independent reproduction. ### Random Regret: 492. Result ID: 6a2291ddfbc373592c20df9f. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"DeepSeek-V4-Flash","Agent Category":"No Learning"}. This record does not assert independent reproduction. ### Random Regret: 492.1. Result ID: 6a2291ddfbc373592c20dfb1. [Reporting paper]() Recorded evaluation configuration: {"Method Category":"No Learning","Backbone":"Qwen3-30B-Instruct"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.