List your company
ResearchFormerly WizWand

Grounded Chess Reasoning on Chess Puzzles (test)

95Accuracy (Beginner)gpt-5
-3.821.8547.573.15Mar 20, 2026
Updated 7mo ago

Evaluation results

MethodLinks
gpt-5Model Category=Proprietary models, Reasoning Mode=Enabled where applicable, Evaluation Protocol=Pass@12026.039584543185.276.712,193
gemini-3-proModel Category=Proprietary models, Reasoning Mode=Enabled where applicable, Evaluation Protocol=Pass@12026.038886704478.275.43,182
gemini-3-flashModel Category=Proprietary models, Reasoning Mode=Enabled where applicable, Evaluation Protocol=Pass@12026.03655934193840.86,418
C1-4BModel Category=Ours, Evaluation Protocol=Pass@12026.036539392253.648.1178
C1-SFT-8BModel Category=Ours, Fine-tuning Strategy=SFT, Evaluation Protocol=Pass@12026.035736272746.642.2189
gpt-5-chatModel Category=Proprietary models, Reasoning Mode=Enabled where applicable, Evaluation Protocol=Pass@12026.035239271841.838.3925
C1-SFT-4BModel Category=Ours, Fine-tuning Strategy=SFT, Evaluation Protocol=Pass@12026.035130302646.240.9188
gemini-2.5-proModel Category=Proprietary models, Reasoning Mode=Enabled where applicable, Evaluation Protocol=Pass@12026.03373129193130.19,668
claude-sonnet-4Model Category=Proprietary models, Reasoning Mode=Enabled where applicable, Evaluation Protocol=Pass@12026.033519161026.823.88,028
claude-haiku-4.5Model Category=Proprietary models, Reasoning Mode=Enabled where applicable, Evaluation Protocol=Pass@12026.033324141125.623.38,111
claude-sonnet-4.5Model Category=Proprietary models, Reasoning Mode=Enabled where applicable, Evaluation Protocol=Pass@12026.033229151128.625.63,227
deepseek-chat-v3.1Model Category=Open-source models, Reasoning Mode=Enabled where applicable, Evaluation Protocol=Pass@12026.032721616222011,249
qwen3-next-80b-a3bModel Category=Open-source models, Reasoning Mode=Enabled where applicable, Evaluation Protocol=Pass@12026.03241414817.616.413,938
qwen3-maxModel Category=Open-source models, Reasoning Mode=Enabled where applicable, Evaluation Protocol=Pass@12026.03221531613.813.93,393
llama-4-maverickModel Category=Open-source models, Reasoning Mode=Enabled where applicable, Evaluation Protocol=Pass@12026.031285108.68.71,092
deepseek-r1-0528Model Category=Open-source models, Reasoning Mode=Enabled where applicable, Evaluation Protocol=Pass@12026.03111014161614.614,442
gemini-2.5-flashModel Category=Proprietary models, Reasoning Mode=Enabled where applicable, Evaluation Protocol=Pass@12026.0394658.27.29,991
mistral-medium-3.1Model Category=Open-source models, Reasoning Mode=Enabled where applicable, Evaluation Protocol=Pass@12026.03967487.32,818
llama-4-scoutModel Category=Open-source models, Reasoning Mode=Enabled where applicable, Evaluation Protocol=Pass@12026.0300010.40.3806
gemma-3-27bModel Category=Open-source models, Reasoning Mode=Enabled where applicable, Evaluation Protocol=Pass@12026.03000000705