Tool-use Execution on UltraTool first 100 queries adapted for execution (test)
82Execution Success RateEGB-Logits
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| EGB-LogitsBase Model=GPT-OSS-120b, Execution strategy=Entropy-Guided Branching (Logits)2026.04 | 82 | 375,000 | 1,790 | |
| LATS-50Base Model=GPT-OSS-120b, Search budget=502026.04 | 82 | 430,000 | 1,220 | |
| LATS-40Base Model=GPT-OSS-120b, Search budget=402026.04 | 80.3 | 364,000 | 1,040 | |
| EGB-SamplingBase Model=GPT-OSS-120b, Execution strategy=Entropy-Guided Branching (Sampling)2026.04 | 79.7 | 169,000 | 5,360 | |
| LATS-30Base Model=GPT-OSS-120b, Search budget=302026.04 | 79.7 | 296,000 | 8,410 | |
| ReflexionBase Model=GPT-OSS-120b2026.04 | 77.3 | 156,000 | 4,850 | |
| LATS-20Base Model=GPT-OSS-120b, Search budget=202026.04 | 74 | 221,000 | 6,370 | |
| LATS-10Base Model=GPT-OSS-120b, Search budget=102026.04 | 71.3 | 139,000 | 4,950 | |
| Self-ConsistencyBase Model=GPT-OSS-120b, Samples=10, Voting=plan-level majority vote2026.04 | 69.3 | 287,000 | 8,650 | |
| ReActBase Model=GPT-OSS-120b2026.04 | 69 | 287,000 | 8,680 | |
| DirectLLMBase Model=GPT-OSS-120b2026.04 | 67.3 | 265,000 | 7,170 |