List your company
ResearchFormerly WizWand

Mathematical Reasoning on TabMWP (Pass@1)

91.9Pass@1Qwen2.5-7B + GRPO w/ VERL.
39.3853.01566.6580.285Mar 4, 2024Jun 7, 2024Sep 11, 2024Dec 15, 2024Mar 21, 2025Jun 24, 2025Sep 28, 2025
Updated 6mo ago

Evaluation results

MethodLinks
Qwen2.5-7B + GRPO w/ VERL.Base Model=Qwen2.5-7B, RL Framework=GRPO, VERL Integration=true2025.0991.9
Qwen2.5-7B + GRPOBase Model=Qwen2.5-7B, RL Framework=GRPO, VERL Integration=false2025.0991.3
Qwen2.5-7B + PPOBase Model=Qwen2.5-7B, RL Framework=PPO, VERL Integration=false2025.0990.8
Qwen2.5-7B + PPO w/ VERL.Base Model=Qwen2.5-7B, RL Framework=PPO, VERL Integration=true2025.0990.6
Qwen2.5-7BBase Model=Qwen2.5-7B, RL Framework=None, VERL Integration=false2025.0982.8
KPMath-PlusBase=DSMath, Size=7B, Zero-shot without demonstrations=true2024.0378.7
KPMath-PlusBase=Qwen1.5, Size=72B, Zero-shot without demonstrations=true2024.0376.7
KPMath-PlusBase=Llama-2, Size=70B, Zero-shot without demonstrations=true2024.0375.1
Llama-3.2-3B-Instruct + GRPO w/ VERL.Base Model=Llama-3.2-3B-Instruct, RL Framework=GRPO, VERL Integration=true2025.0972.3
KPMath-PlusBase=Llemma, Size=34B, Zero-shot without demonstrations=true2024.0371.9
Llama-3.2-3B-Instruct + GRPOBase Model=Llama-3.2-3B-Instruct, RL Framework=GRPO, VERL Integration=false2025.0971.7
Llama-3.2-3B-Instruct + PPO w/ VERL.Base Model=Llama-3.2-3B-Instruct, RL Framework=PPO, VERL Integration=true2025.0971.3
Llama-3.2-3B-Instruct + PPOBase Model=Llama-3.2-3B-Instruct, RL Framework=PPO, VERL Integration=false2025.0971
ChatGPTZero-shot without demonstrations=false2024.0369.1
GPT-4 (0613)Zero-shot without demonstrations=false2024.0367.1
KPMath-PlusBase=Mistral, Size=7B, Zero-shot without demonstrations=true2024.0366.4
KPMath-PlusBase=Llama-2, Size=13B, Zero-shot without demonstrations=true2024.0363.9
Llama-3.2-3B-InstructBase Model=Llama-3.2-3B-Instruct, RL Framework=None, VERL Integration=false2025.0941.4