Skill-selection on ToolBench-60 soft-0.3 preference regime (held-out pool)
255.9RegretBandit-as-Override
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Bandit-as-OverrideBackbone=DeepSeek-V4-Flash, Agent Category=LLM with Statistical Prior2026.06 | 255.9 | 47.2 | 0.523 | |
| Bandit-as-OverrideBackbone=GPT-5.22026.06 | 256.8 | 47.3 | 0.535 | |
| Pure-BanditBackbone=GPT-5.22026.06 | 279.2 | 40.3 | 0.554 | |
| Pure-BanditBackbone=DeepSeek-V4-Flash, Agent Category=Statistical2026.06 | 280.7 | 39.8 | 0.549 | |
| Freq-as-OverrideBackbone=GPT-5.22026.06 | 286.5 | 43 | 0.286 | |
| Freq-as-OverrideBackbone=DeepSeek-V4-Flash, Agent Category=LLM with Statistical Prior2026.06 | 288 | 41.6 | 0.273 | |
| Bandit-as-ContextBackbone=DeepSeek-V4-Flash, Agent Category=LLM with Statistical Prior2026.06 | 309.6 | 37 | 0.465 | |
| Profile-MemoryBackbone=GPT-5.22026.06 | 316 | 38.6 | 0.193 | |
| Bandit-as-ContextBackbone=GPT-5.22026.06 | 327.1 | 35 | 0.473 | |
| Freq-GreedyBackbone=GPT-5.22026.06 | 327.2 | 34 | 0.404 | |
| Profile-MemoryBackbone=DeepSeek-V4-Flash, Agent Category=LLM with Memory2026.06 | 327.4 | 34.8 | 0.251 | |
| Freq-GreedyBackbone=DeepSeek-V4-Flash, Agent Category=Statistical2026.06 | 328 | 33.7 | 0.402 | |
| InCotext-MemoryBackbone=DeepSeek-V4-Flash, Agent Category=LLM with Memory2026.06 | 372.5 | 25.5 | 0.294 | |
| InCotext-MemoryBackbone=GPT-5.22026.06 | 373.5 | 25.2 | 0.304 | |
| ZeroShot-LLMBackbone=GPT-5.22026.06 | 375.5 | 24.8 | 0 | |
| ZeroShot-LLMBackbone=DeepSeek-V4-Flash, Agent Category=No Learning2026.06 | 376.7 | 24.6 | 0 | |
| RandomBackbone=GPT-5.22026.06 | 491.5 | 1.6 | 0 | |
| RandomBackbone=DeepSeek-V4-Flash, Agent Category=No Learning2026.06 | 491.7 | 1.6 | 0 |