Skill-selection on ToolBench-60 one-hot preference regime (held-out pool)
103.3RegretFreq-as-Override
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Freq-as-OverrideBackbone=DeepSeek-V4-Flash, Agent Category=LLM with Statistical Prior2026.06 | 103.3 | 87 | 94.1 | |
| Freq-as-OverrideBackbone=GPT-5.22026.06 | 107.2 | 86.8 | 94.2 | |
| Bandit-as-OverrideBackbone=DeepSeek-V4-Flash, Agent Category=LLM with Statistical Prior2026.06 | 118.9 | 87.4 | 100 | |
| Bandit-as-OverrideBackbone=GPT-5.22026.06 | 122.4 | 86.9 | 100 | |
| Freq-as-OverrideMethod Category=LLM with Statistical Prior, Backbone=Qwen3-30B-Instruct2026.06 | 126.3 | 82.5 | 92.5 | |
| Bandit-as-OverrideMethod Category=LLM with Statistical Prior, Backbone=Qwen3-30B-Instruct2026.06 | 135.7 | 84.3 | 100 | |
| Pure-BanditBackbone=GPT-5.22026.06 | 137.1 | 81 | 99.9 | |
| Pure-BanditBackbone=DeepSeek-V4-Flash, Agent Category=Statistical2026.06 | 139.6 | 80.3 | 99.8 | |
| Pure-BanditMethod Category=Statistical, Backbone=Qwen3-30B-Instruct2026.06 | 140.2 | 80.4 | 99.9 | |
| Freq-GreedyBackbone=GPT-5.22026.06 | 162.4 | 73.6 | 87.3 | |
| Freq-GreedyBackbone=DeepSeek-V4-Flash, Agent Category=Statistical2026.06 | 166 | 73 | 86.9 | |
| Freq-GreedyMethod Category=Statistical, Backbone=Qwen3-30B-Instruct2026.06 | 167.6 | 72.1 | 86.4 | |
| Profile-MemoryBackbone=GPT-5.22026.06 | 208.7 | 64.8 | 67.5 | |
| Bandit-as-ContextBackbone=DeepSeek-V4-Flash, Agent Category=LLM with Statistical Prior2026.06 | 216.6 | 62.9 | 87.9 | |
| Profile-MemoryBackbone=DeepSeek-V4-Flash, Agent Category=LLM with Memory2026.06 | 245.1 | 57.1 | 70.9 | |
| Bandit-as-ContextBackbone=GPT-5.22026.06 | 266 | 53.1 | 94.1 | |
| Profile-MemoryMethod Category=LLM with Memory, Backbone=Qwen3-30B-Instruct2026.06 | 269.5 | 53.4 | 70.9 | |
| Bandit-as-ContextMethod Category=LLM with Memory, Backbone=Qwen3-30B-Instruct2026.06 | 344.2 | 34.1 | 82.7 | |
| InContext-MemoryMethod Category=LLM with Memory, Backbone=Qwen3-30B-Instruct2026.06 | 363.9 | 27.2 | 62.5 | |
| InCotext-MemoryBackbone=DeepSeek-V4-Flash, Agent Category=LLM with Memory2026.06 | 365.4 | 26.8 | 62.6 | |
| InCotext-MemoryBackbone=GPT-5.22026.06 | 371.5 | 25.4 | 63.5 | |
| ZeroShot-LLMBackbone=GPT-5.22026.06 | 376 | 24.3 | 15.9 | |
| ZeroShot-LLMBackbone=DeepSeek-V4-Flash, Agent Category=No Learning2026.06 | 377.1 | 23.8 | 15.9 | |
| ZeroShot-LLMMethod Category=No Learning, Backbone=Qwen3-30B-Instruct2026.06 | 377.2 | 23.9 | 15.9 | |
| RandomBackbone=GPT-5.22026.06 | 491.8 | 1.9 | 15.9 | |
| RandomBackbone=DeepSeek-V4-Flash, Agent Category=No Learning2026.06 | 492 | 1.7 | 15.9 | |
| RandomMethod Category=No Learning, Backbone=Qwen3-30B-Instruct2026.06 | 492.1 | 1.7 | 15.9 |