List your company
ResearchFormerly WizWand

Reasoning on SymBench (test)

65.6AccuracyR1-CI
9.85624.32838.853.272May 27, 2025
Updated 8mo ago

Evaluation results

MethodLinks
R1-CIModel=Qwen2.5-14B-Instruct-1M, Evaluation Strategy=R1-CI2025.0565.6
GPT-4oModel=GPT-4o, Evaluation Strategy=Code Interpreter2025.0563.9
R1-CIModel=Qwen2.5-14B-Instruct-1M, Evaluation Strategy=R1-CI wo IP2025.0561.1
R1-CIModel=Qwen2.5-7B-Instruct-1M, Evaluation Strategy=R1-CI2025.0560.8
Qwen3-14BModel=Qwen3-14B, Evaluation Strategy=CodeSteer2025.0560.2
R1-CIModel=Qwen2.5-14B-Instruct-1M, Evaluation Strategy=R1-CI wo CL2025.0559.4
Qwen3-14BModel=Qwen3-14B, Evaluation Strategy=All Text2025.0557.4
R1-CIModel=Qwen2.5-7B-Instruct-1M, Evaluation Strategy=R1-CI wo IP2025.0557.3
R1-CIModel=Qwen2.5-14B-Instruct-1M, Evaluation Strategy=R1-CI wo GRPO2025.0556
R1-CIModel=Qwen2.5-7B-Instruct-1M, Evaluation Strategy=R1-CI wo CL2025.0555.9
R1-CIModel=Qwen2.5-7B-Instruct-1M, Evaluation Strategy=R1-CI wo GRPO2025.0555.3
R1-CIModel=Qwen2.5-3B-Instruct, Evaluation Strategy=R1-CI2025.0554.8
Qwen2.5-14B-Instruct-1MModel=Qwen2.5-14B-Instruct-1M, Evaluation Strategy=All Code2025.0553.1
R1-CIModel=Qwen2.5-3B-Instruct, Evaluation Strategy=R1-CI wo IP2025.0551
R1-CIModel=Qwen2.5-3B-Instruct, Evaluation Strategy=R1-CI wo CL2025.0549.3
R1-CIModel=Qwen2.5-3B-Instruct, Evaluation Strategy=R1-CI wo GRPO2025.0547.6
Qwen2.5-14B-Instruct-1MModel=Qwen2.5-14B-Instruct-1M, Evaluation Strategy=CodeSteer2025.0545.3
Qwen2.5-7B-Instruct-1MModel=Qwen2.5-7B-Instruct-1M, Evaluation Strategy=All Code2025.0544.1
Qwen2.5-7B-Instruct-1MModel=Qwen2.5-7B-Instruct-1M, Evaluation Strategy=CodeSteer2025.0543.8
GPT-4oModel=GPT-4o, Evaluation Strategy=All Text2025.0543.1
Qwen2.5-7B-Instruct-1MModel=Qwen2.5-7B-Instruct-1M, Evaluation Strategy=Code Agent (CI wo Fine-tune)2025.0542.6
Qwen2.5-3B-InstructModel=Qwen2.5-3B-Instruct, Evaluation Strategy=All Code2025.0533.9
Qwen2.5-14B-Instruct-1MModel=Qwen2.5-14B-Instruct-1M, Evaluation Strategy=Code Agent (CI wo Fine-tune)2025.0533.7
Qwen2.5-3B-InstructModel=Qwen2.5-3B-Instruct, Evaluation Strategy=CodeSteer2025.0532.2
Qwen2.5-3B-InstructModel=Qwen2.5-3B-Instruct, Evaluation Strategy=Code Agent (CI wo Fine-tune)2025.0525.4
Qwen2.5-14B-Instruct-1MModel=Qwen2.5-14B-Instruct-1M, Evaluation Strategy=All Text2025.0522.3
Qwen2.5-7B-Instruct-1MModel=Qwen2.5-7B-Instruct-1M, Evaluation Strategy=All Text2025.0514.6
Qwen2.5-3B-InstructModel=Qwen2.5-3B-Instruct, Evaluation Strategy=All Text2025.0512