List your company
ResearchFormerly WizWand

Arithmetic Reasoning on GSM8K (ACC, PPL)

83.3ACCGRPO
17.010434.220251.4368.6398Aug 13, 2023Dec 29, 2023May 15, 2024Sep 30, 2024Feb 15, 2025Jul 3, 2025Nov 18, 2025
Updated 5mo ago

Evaluation results

MethodLinks
GRPOFramework=Agent-R1, Backbone=Qwen3-4B, Type=RL-optimized2025.1183.3—
RLOOFramework=Agent-R1, Backbone=Qwen3-4B, Type=RL-optimized2025.1181.6—
Reinforce++Framework=Agent-R1, Backbone=Qwen3-4B, Type=RL-optimized2025.1178.9—
PPOFramework=Agent-R1, Backbone=Qwen3-4B, Type=RL-optimized2025.1178.1—
ReActFramework=Agent-R1, Backbone=Qwen3-4B, Type=Training-free baseline2025.1153.1—
FP16 baselineBackbone Model=LLaMA-7B2023.0830.251.47
TSLDBackbone Model=LLaMA-7B2023.0826.231.52
Logit KDBackbone Model=LLaMA-7B2023.0825.471.52
TSLDBackbone Model=OPT-6.7B2023.0824.492.14
FP16 baselineBackbone Model=OPT-6.7B2023.0822.521.89
Logit KDBackbone Model=OPT-6.7B2023.0821.081.93
OPT-2.7B FP16Model=OPT-2.7B, Precision=FP162023.0820.392.07
TSLDModel=OPT-2.7B, Quantization=QAT-KD (tensor-wise)2023.0820.242.03
LogitModel=OPT-2.7B, Quantization=QAT-KD (tensor-wise)2023.0820.022.03
GT+LogitModel=OPT-2.7B, Quantization=QAT-KD (tensor-wise)2023.0819.562.12