List your company
ResearchFormerly WizWand

Reasoning on SymBench (train)

74.4AccuracyR1-CI
9.50426.35243.260.048May 27, 2025
Updated 8mo ago

Evaluation results

MethodLinks
R1-CIModel=Qwen2.5-14B-Instruct-1M, Evaluation Strategy=R1-CI2025.0574.4
R1-CIModel=Qwen2.5-14B-Instruct-1M, Evaluation Strategy=R1-CI wo IP2025.0572.2
R1-CIModel=Qwen2.5-7B-Instruct-1M, Evaluation Strategy=R1-CI2025.0572
R1-CIModel=Qwen2.5-14B-Instruct-1M, Evaluation Strategy=R1-CI wo CL2025.0571.3
R1-CIModel=Qwen2.5-3B-Instruct, Evaluation Strategy=R1-CI2025.0570
R1-CIModel=Qwen2.5-14B-Instruct-1M, Evaluation Strategy=R1-CI wo GRPO2025.0569
R1-CIModel=Qwen2.5-7B-Instruct-1M, Evaluation Strategy=R1-CI wo IP2025.0568.5
R1-CIModel=Qwen2.5-7B-Instruct-1M, Evaluation Strategy=R1-CI wo CL2025.0567.7
R1-CIModel=Qwen2.5-3B-Instruct, Evaluation Strategy=R1-CI wo IP2025.0567.1
R1-CIModel=Qwen2.5-3B-Instruct, Evaluation Strategy=R1-CI wo CL2025.0566
R1-CIModel=Qwen2.5-7B-Instruct-1M, Evaluation Strategy=R1-CI wo GRPO2025.0565.6
Qwen3-14BModel=Qwen3-14B, Evaluation Strategy=All Text2025.0565.3
Qwen3-14BModel=Qwen3-14B, Evaluation Strategy=CodeSteer2025.0565
GPT-4oModel=GPT-4o, Evaluation Strategy=Code Interpreter2025.0563.7
R1-CIModel=Qwen2.5-3B-Instruct, Evaluation Strategy=R1-CI wo GRPO2025.0562.3
GPT-4oModel=GPT-4o, Evaluation Strategy=All Text2025.0540.7
Qwen2.5-14B-Instruct-1MModel=Qwen2.5-14B-Instruct-1M, Evaluation Strategy=All Code2025.0540
Qwen2.5-14B-Instruct-1MModel=Qwen2.5-14B-Instruct-1M, Evaluation Strategy=CodeSteer2025.0536.7
Qwen2.5-7B-Instruct-1MModel=Qwen2.5-7B-Instruct-1M, Evaluation Strategy=CodeSteer2025.0529
Qwen2.5-14B-Instruct-1MModel=Qwen2.5-14B-Instruct-1M, Evaluation Strategy=Code Agent (CI wo Fine-tune)2025.0527.9
Qwen2.5-7B-Instruct-1MModel=Qwen2.5-7B-Instruct-1M, Evaluation Strategy=All Code2025.0527.5
Qwen2.5-7B-Instruct-1MModel=Qwen2.5-7B-Instruct-1M, Evaluation Strategy=Code Agent (CI wo Fine-tune)2025.0527.5
Qwen2.5-14B-Instruct-1MModel=Qwen2.5-14B-Instruct-1M, Evaluation Strategy=All Text2025.0524
Qwen2.5-3B-InstructModel=Qwen2.5-3B-Instruct, Evaluation Strategy=CodeSteer2025.0520.1
Qwen2.5-7B-Instruct-1MModel=Qwen2.5-7B-Instruct-1M, Evaluation Strategy=All Text2025.0519.5
Qwen2.5-3B-InstructModel=Qwen2.5-3B-Instruct, Evaluation Strategy=All Code2025.0518.3
Qwen2.5-3B-InstructModel=Qwen2.5-3B-Instruct, Evaluation Strategy=Code Agent (CI wo Fine-tune)2025.0513.8
Qwen2.5-3B-InstructModel=Qwen2.5-3B-Instruct, Evaluation Strategy=All Text2025.0512