Mathematical Reasoning on GSM8k, MATH, SVAMP, and MM
58.15GSM8k Accuracy+ RFT w/ TD
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| + RFT w/ TDModel Parameters=13B, Base Model=SFT LLM2024.01 | 58.15 | 13.98 | 58.8 | 41.58 | 43.13 | |
| + RLMECModel Parameters=13B, Base Model=SFT LLM2024.01 | 58.15 | 14 | 60 | 45.07 | 44.31 | |
| + RFT w/ RDModel Parameters=13B, Base Model=SFT LLM2024.01 | 57.39 | 14.34 | 56.2 | 42.81 | 42.96 | |
| + RFT w/ GTModel Parameters=13B, Base Model=SFT LLM2024.01 | 57.32 | 13.74 | 56.7 | 43.94 | 42.93 | |
| + CoHModel Parameters=13B, Base Model=SFT LLM2024.01 | 57.31 | 13.1 | 54 | 42.3 | 41.68 | |
| SFT LLMModel Parameters=13B2024.01 | 56.63 | 12.74 | 53.5 | 41.27 | 41.04 | |
| + SFTModel Parameters=13B, Base Model=SFT LLM2024.01 | 55.88 | 13.62 | 58 | 41.27 | 42.19 | |
| + RFTModel Parameters=13B, Base Model=SFT LLM2024.01 | 55.8 | 13.62 | 54.1 | 41.68 | 41.3 | |
| + Vanilla PPOModel Parameters=13B, Base Model=SFT LLM2024.01 | 53.45 | 11.76 | 55.1 | 43.12 | 40.86 | |
| + RFT w/ TDModel Parameters=7B, Base Model=SFT LLM2024.01 | 51.86 | 11.04 | 49.4 | 38.19 | 37.62 | |
| + RFT w/ RDModel Parameters=7B, Base Model=SFT LLM2024.01 | 51.78 | 11.24 | 48.7 | 40.76 | 38.12 | |
| + RLMECModel Parameters=7B, Base Model=SFT LLM2024.01 | 51.18 | 11.16 | 49.6 | 40.97 | 38.23 | |
| SFT LLMModel Parameters=7B2024.01 | 51.02 | 10.48 | 47.8 | 38.5 | 36.95 | |
| + PPO A2CModel Parameters=7B, Base Model=SFT LLM2024.01 | 50.94 | 9.38 | 46.6 | 38.5 | 36.36 | |
| + SFTModel Parameters=7B, Base Model=SFT LLM2024.01 | 50.34 | 11.04 | 47.2 | 38.4 | 36.75 | |
| + CoHModel Parameters=7B, Base Model=SFT LLM2024.01 | 50.11 | 10.94 | 48.6 | 38.5 | 37.04 | |
| + RFT w/ GTModel Parameters=7B, Base Model=SFT LLM2024.01 | 49.89 | 11.26 | 46.7 | 38.91 | 36.69 | |
| + RFTModel Parameters=7B, Base Model=SFT LLM2024.01 | 49.66 | 10.8 | 48.3 | 39.01 | 36.94 | |
| + Vanilla PPOModel Parameters=7B, Base Model=SFT LLM2024.01 | 48.97 | 10.64 | 44.9 | 38.6 | 35.78 | |
| + DPOModel Parameters=13B, Base Model=SFT LLM2024.01 | 44.2 | 4.38 | 39.7 | 32.14 | 30.11 | |
| + DPOModel Parameters=7B, Base Model=SFT LLM2024.01 | 34.19 | 5.38 | 25.8 | 32.58 | 24.49 | |
| WizardLMModel Parameters=13B2024.01 | 31.01 | 3.18 | 52 | 21.36 | 26.89 | |
| VicunaModel Parameters=13B2024.01 | 24.1 | 4.74 | 33.8 | 29.98 | 23.16 | |
| LLaMA 2Model Parameters=13B2024.01 | 21.23 | 6.58 | 34.4 | 34.39 | 24.15 | |
| WizardLMModel Parameters=7B2024.01 | 14.48 | 3.34 | 34.8 | 27.1 | 19.93 | |
| VicunaModel Parameters=7B2024.01 | 12.2 | 4.26 | 24.3 | 26.08 | 16.71 | |
| LLaMA 2Model Parameters=7B2024.01 | 11.22 | 4.8 | 29.7 | 28.44 | 18.54 |