List your company
ResearchFormerly WizWand

Reasoning on Checkmate-in-One

92AccuracyRoT
-3.6821.164670.84Oct 16, 2024Dec 13, 2024Feb 9, 2025Apr 8, 2025Jun 5, 2025Aug 2, 2025Sep 29, 2025
Updated 8mo ago

Evaluation results

MethodLinks
RoTBackbone=GPT-4, Shots=2-shot2024.1092
RoTBackbone=GPT-4, Shots=1-shot2024.1091.2
BOTBackbone=GPT-4, Shots=2-shot2024.1088.3
BOTBackbone=GPT-4, Shots=1-shot2024.1087.1
RoTBackbone=GPT-3.5-turbo, Shots=2-shot2024.1072.6
RoTBackbone=GPT-3.5-turbo, Shots=1-shot2024.1071.5
BOTBackbone=GPT-3.5-turbo, Shots=2-shot2024.1068.5
BOTBackbone=GPT-3.5-turbo, Shots=1-shot2024.1067.9
Meta-promptingBackbone=GPT-4, Shots=2-shot2024.1058.9
Meta-promptingBackbone=GPT-4, Shots=1-shot2024.1058.2
GOTBackbone=GPT-4, Shots=2-shot2024.1051.2
TOTBackbone=GPT-4, Shots=1-shot2024.1049.8
GOTBackbone=GPT-4, Shots=1-shot2024.1049
TOTBackbone=GPT-4, Shots=2-shot2024.1048.8
Meta-promptingBackbone=GPT-3.5-turbo, Shots=2-shot2024.1048
Meta-promptingBackbone=GPT-3.5-turbo, Shots=1-shot2024.1047.4
GOTBackbone=GPT-3.5-turbo, Shots=2-shot2024.1046
GOTBackbone=GPT-3.5-turbo, Shots=1-shot2024.1045.4
TOTBackbone=GPT-3.5-turbo, Shots=2-shot2024.1045.2
TOTBackbone=GPT-3.5-turbo, Shots=1-shot2024.1044.2
COTBackbone=GPT-4, Shots=2-shot2024.1036.7
COTBackbone=GPT-4, Shots=1-shot2024.1035.8
COTBackbone=GPT-3.5-turbo, Shots=1-shot2024.1025.7
COTBackbone=GPT-3.5-turbo, Shots=2-shot2024.1025.4
MetaTuner-JBase Model=Llama3.2-3B2025.0921.43
MetaTuner-JBase Model=Qwen2.5-7B2025.0920.86
BetterTogetherBase Model=Llama3.2-3B2025.0920.57
KTOBase Model=Llama3.2-3B2025.0920
BetterTogetherBase Model=Qwen2.5-7B2025.0919.71
PPOBase Model=Llama3.2-3B2025.0919.43
DPOBase Model=Llama3.2-3B2025.0919.14
DPOBase Model=Qwen2.5-7B2025.0918.86
SFTBase Model=Llama3.2-3B2025.0918.85
KTOBase Model=Qwen2.5-7B2025.0918.57
PPOBase Model=Qwen2.5-7B2025.0918.29
SFTBase Model=Qwen2.5-7B2025.0918
MetaTuner-JBase Model=Qwen2.5-3B2025.0917.71
BetterTogetherBase Model=Qwen2.5-3B2025.0915.14
KTOBase Model=Qwen2.5-3B2025.0914.57
PPOBase Model=Qwen2.5-3B2025.0914
SFTBase Model=Qwen2.5-3B2025.0913.71
DPOBase Model=Qwen2.5-3B2025.0913.42
BPOBase Model=Llama3.2-3B2025.091.43
CFPOBase Model=Llama3.2-3B2025.091.43
OPROBase Model=Llama3.2-3B2025.091.14
BPOBase Model=Qwen2.5-7B2025.090.86
OPROBase Model=Qwen2.5-7B2025.090.57
Base ModelBase Model=Qwen2.5-3B2025.090
Base ModelBase Model=Qwen2.5-7B2025.090
Base ModelBase Model=Llama3.2-3B2025.090
RLPromptBase Model=Qwen2.5-3B2025.090
RLPromptBase Model=Qwen2.5-7B2025.090
RLPromptBase Model=Llama3.2-3B2025.090
BPOBase Model=Qwen2.5-3B2025.090
OPROBase Model=Qwen2.5-3B2025.090
CFPOBase Model=Qwen2.5-3B2025.090
CFPOBase Model=Qwen2.5-7B2025.090