Multi-hop tool-use evaluation on Multi-hop 450 tasks (test)
89AccuracyPROBE&PREFILL
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| PROBE&PREFILLModel=Qwen3-32B, N=4502026.05 | 89 | 727 | 55 | |
| DefaultModel=Qwen3-32B, N=4502026.05 | 88.9 | 1,634 | — | |
| Best BaselineModel=Qwen3-32B, N=4502026.05 | 88.6 | 1,306 | 20 | |
| DefaultModel=Qwen3-14B, N=4502026.05 | 87.5 | 1,503 | — | |
| PROBE&PREFILLModel=Qwen3-14B, N=4502026.05 | 86.2 | 996 | 34 | |
| Best BaselineModel=Qwen3-14B, N=4502026.05 | 85.7 | 789 | 47 | |
| PROBE&PREFILLModel=Qwen3-4B, N=4502026.05 | 85.3 | 437 | 75 | |
| Best BaselineModel=Qwen3-4B, N=4502026.05 | 83.9 | 636 | 63 | |
| DefaultModel=Qwen3-4B, N=4502026.05 | 82.1 | 1,719 | — | |
| PROBE&PREFILLModel=Llama-3.3-70B, N=4502026.05 | 80.3 | 1,789 | 82 | |
| Best BaselineModel=Llama-3.3-70B, N=4502026.05 | 67.6 | 347 | 65 | |
| DefaultModel=Llama-3.3-70B, N=4502026.05 | 62.4 | 985 | — | |
| Best BaselineModel=Qwen3-1.7B, N=4502026.05 | 60.6 | 175 | 85 | |
| PROBE&PREFILLModel=Llama-3.1-8B, N=4502026.05 | 60.2 | 1,361 | 35 | |
| PROBE&PREFILLModel=Qwen3-1.7B, N=4502026.05 | 59.2 | 85 | 93 | |
| Best BaselineModel=Llama-3.1-8B, N=4502026.05 | 41.3 | 595 | 41 | |
| DefaultModel=Llama-3.1-8B, N=4502026.05 | 40.2 | 1,005 | — | |
| DefaultModel=Qwen3-1.7B, N=4502026.05 | 21.2 | 1,180 | — |