Chess Reasoning Quality Evaluation on Lichess Puzzle Database (held-out positions)
0.218WR MAEVPS
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| VPSBackbone=Qwen3-8B2026.04 | 0.218 | 37.6 | 97.8 | |
| VPSBackbone=DeepSeek-R1-Distill-Llama-8B2026.04 | 0.286 | 36.2 | 98.5 | |
| SFT onlyBackbone=Qwen3-8B2026.04 | 0.31 | 32.3 | 96.1 | |
| SFT onlyBackbone=DeepSeek-R1-Distill-Llama-8B2026.04 | 0.346 | 34.1 | 97.8 | |
| SFT + GRPOBackbone=Qwen3-8B2026.04 | 0.452 | 18.3 | 72.1 | |
| SFT + GRPOBackbone=DeepSeek-R1-Distill-Llama-8B2026.04 | 0.732 | 33 | 31.2 |