Physical Reasoning on INTERPHYRE (test)
0CliffhangerQ-7B
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Q-7BBaseline Variant=DIRECT, Prompting Strategy=One-Shot, Tool Usage=No-Tool, Reasoning Variant=no-CoT, Temperature=0.3, Max New Tokens=700, Seeds=1002026.06 | 0 | 0 | 0 | 18 | 4 | 8 | 0 | 0 | |
| Q-14BBaseline Variant=DIRECT, Prompting Strategy=One-Shot, Tool Usage=No-Tool, Reasoning Variant=no-CoT, Temperature=0.3, Max New Tokens=700, Seeds=1002026.06 | 0 | 3 | 2 | 0 | 16 | 1 | 0 | 0 | |
| O-20BBaseline Variant=DIRECT, Prompting Strategy=One-Shot, Tool Usage=No-Tool, Reasoning Variant=no-CoT, Temperature=0.3, Max New Tokens=700, Seeds=1002026.06 | 0 | 8 | 14 | 16 | 13 | 3 | 0 | 0 | |
| O-120BBaseline Variant=DIRECT, Prompting Strategy=One-Shot, Tool Usage=No-Tool, Reasoning Variant=no-CoT, Temperature=0.3, Max New Tokens=700, Seeds=1002026.06 | 0 | 2 | 13 | 3 | 1 | 29 | 3 | 0 | |
| Claude Sonnet 4.6Baseline Variant=DIRECT, Prompting Strategy=One-Shot, Tool Usage=No-Tool, Reasoning Variant=no-CoT, Temperature=0.3, Max New Tokens=700, Seeds=100 (partial for harder tier)2026.06 | 0 | 0 | 9 | 14 | 68 | 23 | 0 | 2 |