Graph Reasoning on GraphWalk Synthetic Query Set 100 nodes
59Correct Answerso4-mini
Evaluation Results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| o4-miniTool use=False2026.04 | 59 | 49.17 | 68 | 67 | 66 | 248 | — | |
| gpt-4.1Tool use=True2026.04 | 35 | 29.17 | 47 | 41 | 43 | 232 | 2,483 | |
| gpt-4oTool use=True2026.04 | 34 | 28.33 | 42 | 39 | 39 | 235 | 2,548 | |
| gpt-4.1-miniTool use=True2026.04 | 32 | 26.67 | 49 | 43 | 43 | 226 | 3,561 | |
| o3-miniTool use=False2026.04 | 27 | 22.5 | 55 | 47 | 49 | 195 | — | |
| gpt-4.1Tool use=False2026.04 | 21 | 17.5 | 52 | 49 | 48 | 1,006 | — | |
| gpt-4.1-miniTool use=False2026.04 | 20 | 17.7 | 49 | 48 | 46 | 522 | — | |
| gpt-4o-miniTool use=True2026.04 | 18 | 15 | 23 | 25 | 23 | 170 | 7,362 | |
| gpt-4oTool use=False2026.04 | 18 | 15 | 50 | 44 | 44 | 300 | — | |
| gpt-4.1-nanoTool use=True2026.04 | 11 | 9.17 | 16 | 14 | 14 | 105 | 1,065 | |
| gpt-4.1-nanoTool use=False2026.04 | 7 | 5.83 | 33 | 29 | 27 | 522 | — | |
| gpt-4o-miniTool use=False2026.04 | 6 | 5 | 35 | 24 | 26 | 350 | — |