Physics-based puzzle solving on INTERPHYRE 1.0 (test)
100Cliffhanger ScoreClaude Sonnet
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Claude SonnetBackbone=Claude Sonnet, Evaluation Protocol=REACT baseline2026.06 | 100 | — | 96 | 100 | 100 | 100 | 100 | 24 | 8 | |
| O-120BBackbone=O-120B, Evaluation Protocol=REACT baseline2026.06 | 64 | — | 90.1 | 33 | 72.5 | 77.8 | 100 | 22 | 0 | |
| O-20BBackbone=O-20B, Evaluation Protocol=REACT baseline2026.06 | 30.5 | — | 80 | 14 | 35 | 26 | 58 | 16 | 0 | |
| Q-14BBackbone=Q-14B, Evaluation Protocol=REACT baseline2026.06 | 6 | — | 42 | 10 | 72 | 24 | 52 | 9.8 | 0 | |
| Q-7BBackbone=Q-7B, Evaluation Protocol=REACT baseline2026.06 | 2 | — | 24 | 8 | 62 | 18 | 20 | 7.8 | 0 | |
| Random AgentMethodology=Heuristic policy, Sampling=Uniformly from valid placement region2026.06 | 0 | — | 3 | 4 | 8 | 14 | 1 | 0 | 0 | |
| Qwen-2.5-7B InstructProtocol=DIRECT baseline, Max Interaction Turns=252026.06 | 0 | — | 0 | 0 | 18 | 4 | 8 | 0 | 0 | |
| Qwen-2.5-14B InstructProtocol=DIRECT baseline, Max Interaction Turns=252026.06 | 0 | — | 3 | 2 | 0 | 16 | 1 | 0 | 0 | |
| GPT-OSS-20BProtocol=DIRECT baseline, Max Interaction Turns=252026.06 | 0 | — | 8 | 14 | 16 | 13 | 3 | 0 | 0 | |
| GPT-OSS-120BProtocol=DIRECT baseline, Max Interaction Turns=252026.06 | 0 | — | 2 | 13 | 3 | 1 | 29 | 3 | 0 | |
| Claude SonnetProtocol=DIRECT baseline, Max Interaction Turns=252026.06 | 0 | — | 0 | 9 | 14 | 68 | 23 | 0 | 2 | |
| RandBackbone=Random, Evaluation Protocol=REACT baseline2026.06 | 0 | — | 3 | 4 | 8 | 14 | 1 | 0 | 0 |