Spatial Reasoning on ShuffleObj
44.1NL-to-Format AccuracyQwen3-1.7B
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Qwen3-1.7BShot=02026.01 | 44.1 | 43.4 | 45.4 | 45.4 | |
| Qwen3-1.7BShot=42026.01 | 35.6 | 35.3 | 46.2 | 46.3 | |
| Qwen3-1.7BShot=12026.01 | 34 | 33.7 | 48.1 | 46.2 | |
| SmolLM2-1.7BShot=02026.01 | 18.2 | 18.2 | 18 | 18 | |
| SmolLM2-1.7BShot=42026.01 | 18.1 | 16.4 | 19.7 | 19.2 | |
| SmolLM2-1.7BShot=12026.01 | 17.9 | 17.6 | 18.4 | 18.8 |