List your company
ResearchFormerly WizWand

Logical Reasoning on ProofWriter (Accuracy)

75AccuracySoftCoT
27.78440.04252.364.558Mar 21, 2026Mar 31, 2026Apr 10, 2026Apr 21, 2026May 1, 2026May 11, 2026May 22, 2026
Updated 5mo ago

Evaluation results

MethodLinks
SoftCoTBase model=Qwen2.5, Transfer cost=full retrain2026.0575
HyperGuideBase model=Qwen2.5, Transfer cost=small MLP2026.0575
Self-ConsistencyBase model=Qwen2.5, Transfer cost=N/A2026.0574
OVMBase model=Mistral, Transfer cost=full retrain2026.0572
Few-shotBase model=Qwen2.5, Transfer cost=N/A2026.0570.4
Tree of ThoughtsBase model=Qwen2.5, Transfer cost=N/A2026.0569
HyperGuideBase model=Mistral, Transfer cost=small MLP2026.0569
ORACLEBackbone=Qwen-2.5-7B-instruct, Setting=zero-shot2026.0368.2
Self-ConsistencyBase model=Mistral, Transfer cost=N/A2026.0567.6
ORACLEBackbone=Llama-3.1-8B-Instruct, Setting=zero-shot2026.0367.5
Tree of ThoughtsBase model=Mistral, Transfer cost=N/A2026.0566
ORACLEBackbone=Mistral-7B-Instruct-v0.3, Setting=zero-shot2026.0364.5
self-rewardingBackbone=Qwen-2.5-7B-instruct, Setting=zero-shot2026.0364.5
CoTBackbone=Qwen-2.5-7B-instruct, Setting=four-shot2026.0363.8
RFTBackbone=Qwen-2.5-7B-instruct, Setting=zero-shot2026.0363.5
self-rewardingBackbone=Llama-3.1-8B-Instruct, Setting=zero-shot2026.0363.4
RFTBackbone=Llama-3.1-8B-Instruct, Setting=zero-shot2026.0362
SoftCoTBase model=GPT-OSS, Transfer cost=full retrain2026.0561.4
CoTBackbone=Llama-3.1-8B-Instruct, Setting=four-shot2026.0360.3
CoTBackbone=Qwen-2.5-7B-instruct, Setting=zero-shot2026.0360
HyperGuideBase model=GPT-OSS, Transfer cost=small MLP2026.0559
self-rewardingBackbone=Mistral-7B-Instruct-v0.3, Setting=zero-shot2026.0358.8
ToT-SFTBackbone=Llama-3.1-8B-Instruct, Setting=zero-shot2026.0357
Few-shotBase model=Mistral, Transfer cost=N/A2026.0557
SoftCoTBase model=Mistral, Transfer cost=full retrain2026.0557
ToT-SFTBackbone=Mistral-7B-Instruct-v0.3, Setting=zero-shot2026.0354.8
ToT-SFTBackbone=Qwen-2.5-7B-instruct, Setting=zero-shot2026.0354.8
CoTBackbone=Llama-3.1-8B-Instruct, Setting=zero-shot2026.0353.7
RFTBackbone=Mistral-7B-Instruct-v0.3, Setting=zero-shot2026.0353.6
ToTReasoning Method=Tree-of-Thought, Evaluation Protocol=String Match (SM)2026.0351.7
CoTBackbone=Mistral-7B-Instruct-v0.3, Setting=four-shot2026.0350.5
NoTReasoning Method=Network-of-Thought, Evaluation Protocol=LLM-as-Judge2026.0350.3
NoTReasoning Method=Network-of-Thought, Evaluation Protocol=String Match (SM)2026.0349
PT-SFTBase model=Qwen2.5, Transfer cost=full retrain2026.0549
Self-ConsistencyBase model=GPT-OSS, Transfer cost=N/A2026.0547
Tree of ThoughtsBase model=GPT-OSS, Transfer cost=N/A2026.0546
CoTReasoning Method=Chain-of-Thought, Evaluation Protocol=String Match (SM)2026.0343.3
PT-SFTBase model=Mistral, Transfer cost=full retrain2026.0542.6
PT-SFTBase model=GPT-OSS, Transfer cost=full retrain2026.0542.4
OVMBase model=Qwen2.5, Transfer cost=full retrain2026.0538
CoTBackbone=Mistral-7B-Instruct-v0.3, Setting=zero-shot2026.0336.3
OVMBase model=GPT-OSS, Transfer cost=full retrain2026.0533
Few-shotBase model=GPT-OSS, Transfer cost=N/A2026.0529.6