Mathematical Reasoning on GSM8K, Minerva, MATH, and MathQA
59.24GSM8K ScoreDataAgentRL
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| DataAgentRLTarget Model=LLaMA-DCLM, GPU Hrs=1891.84, Data Reweighting Space=2-dimensional2025.07 | 59.24 | 14.8 | 22.75 | 35.3 | 47.03 | 547.66 | |
| Naive TrainingModel Backbone=LLaMA-DCLM, Data Reweighting Space=52-dimensional (Nvidia domain classifier)2025.07 | 59.21 | 16.16 | 22.85 | 32.96 | 32.8 | — | |
| Naive TrainingTarget Model=LLaMA-DCLM2025.07 | 59.21 | 16.16 | 22.85 | 32.96 | 32.8 | — | |
| Naive TrainingTarget Model=LLaMA-DCLM, GPU Hrs=874.88, Data Reweighting Space=2-dimensional2025.07 | 59.21 | 16.16 | 22.85 | 32.96 | 38.51 | — | |
| Naive TrainingModel Backbone=LLaMA-Nemo, Data Reweighting Space=52-dimensional (Nvidia domain classifier)2025.07 | 59.05 | 12.3 | 24.52 | 32.53 | 32.1 | — | |
| Naive TrainingTarget Model=LLaMA-Nemo2025.07 | 59.05 | 12.3 | 24.52 | 32.53 | 32.1 | — | |
| Naive TrainingTarget Model=LLaMA-Nemo, GPU Hrs=870.4, Data Reweighting Space=2-dimensional2025.07 | 59.05 | 12.3 | 24.52 | 32.53 | 37.86 | — | |
| DataAgentRLModel Backbone=LLaMA-DCLM, Data Reweighting Space=52-dimensional (Nvidia domain classifier)2025.07 | 59.04 | 16.48 | 22.9 | 31.72 | 32.54 | — | |
| DataAgentRLTarget Model=LLaMA-DCLM2025.07 | 59.04 | 16.48 | 22.9 | 31.72 | 32.54 | — | |
| DataAgentRLModel Backbone=LLaMA-FWE, Data Reweighting Space=52-dimensional (Nvidia domain classifier)2025.07 | 58.96 | 14.68 | 23.62 | 33.32 | 32.65 | — | |
| DataAgentRLTarget Model=LLaMA-FWE2025.07 | 58.96 | 14.68 | 23.62 | 33.32 | 32.65 | — | |
| Naive TrainingModel Backbone=LLaMA-FWE, Data Reweighting Space=52-dimensional (Nvidia domain classifier)2025.07 | 58.91 | 12.58 | 24.7 | 33.94 | 32.53 | — | |
| Naive TrainingTarget Model=LLaMA-FWE2025.07 | 58.91 | 12.58 | 24.7 | 33.94 | 32.53 | — | |
| Naive TrainingTarget Model=LLaMA-FWE, GPU Hrs=852.1, Data Reweighting Space=2-dimensional2025.07 | 58.91 | 12.58 | 24.7 | 33.94 | 38.51 | — | |
| DataAgentRLTarget Model=LLaMA-FWE, GPU Hrs=1838.2, Data Reweighting Space=2-dimensional2025.07 | 58.07 | 13.46 | 23.96 | 33.28 | 45.48 | 540.83 | |
| DataAgentSFTTarget Model=LLaMA-DCLM, GPU Hrs=1880.32, Data Reweighting Space=2-dimensional2025.07 | 57.84 | 12.7 | 21.3 | 32.73 | 44.95 | 563.27 | |
| DataAgentRLModel Backbone=LLaMA-Nemo, Data Reweighting Space=52-dimensional (Nvidia domain classifier)2025.07 | 56.78 | 11.82 | 23.61 | 32.03 | 31.06 | — | |
| DataAgentRLTarget Model=LLaMA-Nemo2025.07 | 56.78 | 11.82 | 23.61 | 32.03 | 31.06 | — | |
| DataAgentSFTModel Backbone=LLaMA-DCLM, Data Reweighting Space=52-dimensional (Nvidia domain classifier)2025.07 | 56.77 | 11.12 | 18.76 | 32.3 | 29.74 | — | |
| DataAgentSFTTarget Model=LLaMA-DCLM2025.07 | 56.77 | 11.12 | 18.76 | 32.3 | 29.74 | — | |
| DataAgentSFTTarget Model=LLaMA-FWE, GPU Hrs=1887.82, Data Reweighting Space=2-dimensional2025.07 | 56.26 | 12.19 | 21.92 | 31.32 | 45.23 | 552.61 | |
| DBLTarget Model=LLaMA-DCLM, GPU Hrs=1906.8, Data Reweighting Space=2-dimensional2025.07 | 56.22 | 12.2 | 18.24 | 30.14 | 43.5 | 575.91 | |
| RegMixTarget Model=LLaMA-FWE, GPU Hrs=1948.26, Data Reweighting Space=2-dimensional2025.07 | 55.9 | 12.2 | 20.35 | 30.1 | 43.83 | 557.18 | |
| RegMixTarget Model=LLaMA-DCLM, GPU Hrs=1987.84, Data Reweighting Space=2-dimensional2025.07 | 55.87 | 11.5 | 17.7 | 32.16 | 44.01 | 559.83 | |
| RegMixModel Backbone=LLaMA-DCLM, Data Reweighting Space=52-dimensional (Nvidia domain classifier)2025.07 | 55.78 | 10.36 | 15.75 | 28.67 | 27.64 | — | |
| RegMixTarget Model=LLaMA-DCLM2025.07 | 55.78 | 10.36 | 15.75 | 28.67 | 27.64 | — | |
| DataAgentSFTModel Backbone=LLaMA-FWE, Data Reweighting Space=52-dimensional (Nvidia domain classifier)2025.07 | 54.5 | 13.27 | 22.17 | 31.1 | 30.26 | — | |
| DataAgentSFTTarget Model=LLaMA-FWE2025.07 | 54.5 | 13.27 | 22.17 | 31.1 | 30.26 | — | |
| DataAgentRLTarget Model=LLaMA-Nemo, GPU Hrs=1812.31, Data Reweighting Space=2-dimensional2025.07 | 54.28 | 10.83 | 22.94 | 31.85 | 45.8 | 520.71 | |
| DBLTarget Model=LLaMA-FWE, GPU Hrs=1883.58, Data Reweighting Space=2-dimensional2025.07 | 54.2 | 12.05 | 20.72 | 31.52 | 43.92 | 566.47 | |
| RegMixModel Backbone=LLaMA-FWE, Data Reweighting Space=52-dimensional (Nvidia domain classifier)2025.07 | 53.68 | 10.84 | 20.35 | 30 | 28.72 | — | |
| RegMixTarget Model=LLaMA-FWE2025.07 | 53.68 | 10.84 | 20.35 | 30 | 28.72 | — | |
| DataAgentSFTModel Backbone=LLaMA-Nemo, Data Reweighting Space=52-dimensional (Nvidia domain classifier)2025.07 | 52.5 | 10.7 | 20.77 | 32.71 | 29.17 | — | |
| DataAgentSFTTarget Model=LLaMA-Nemo2025.07 | 52.5 | 10.7 | 20.77 | 32.71 | 29.17 | — | |
| RegMixTarget Model=LLaMA-Nemo, GPU Hrs=1925.23, Data Reweighting Space=2-dimensional2025.07 | 49.39 | 10.7 | 19.23 | 30 | 44.13 | 546.24 | |
| Naive TrainingModel Backbone=Pythia-1.4B, Data Reweighting Space=52-dimensional (Nvidia domain classifier)2025.07 | 48.98 | 10.16 | 14.64 | 27.52 | 25.33 | — | |
| RegMixModel Backbone=LLaMA-Nemo, Data Reweighting Space=52-dimensional (Nvidia domain classifier)2025.07 | 48.17 | 10.91 | 19.03 | 30.04 | 27.04 | — | |
| RegMixTarget Model=LLaMA-Nemo2025.07 | 48.17 | 10.91 | 19.03 | 30.04 | 27.04 | — | |
| DataAgentSFTTarget Model=LLaMA-Nemo, GPU Hrs=1887.25, Data Reweighting Space=2-dimensional2025.07 | 47.81 | 9.07 | 17.8 | 28.86 | 44.12 | 538.44 | |
| DBLTarget Model=LLaMA-Nemo, GPU Hrs=1920.21, Data Reweighting Space=2-dimensional2025.07 | 45.91 | 10.74 | 20.01 | 28.74 | 42.63 | 522.88 | |
| DataAgentRLModel Backbone=Pythia-1.4B, Data Reweighting Space=52-dimensional (Nvidia domain classifier)2025.07 | 45.44 | 8.1 | 10.01 | 25.38 | 22.23 | — | |
| DataAgentSFTModel Backbone=Pythia-1.4B, Data Reweighting Space=52-dimensional (Nvidia domain classifier)2025.07 | 38.84 | 7.19 | 7.3 | 20.68 | 18.5 | — | |
| RegMixModel Backbone=Pythia-1.4B, Data Reweighting Space=52-dimensional (Nvidia domain classifier)2025.07 | 38.28 | 7.24 | 7.51 | 21.1 | 18.53 | — | |
| Base ModelModel Backbone=LLaMA-DCLM, Data Reweighting Space=52-dimensional (Nvidia domain classifier)2025.07 | 2.55 | 4.1 | 4.22 | 24.52 | 8.85 | — | |
| Base ModelTarget Model=LLaMA-DCLM2025.07 | 2.55 | 4.1 | 4.22 | 24.52 | 8.85 | — | |
| Base ModelTarget Model=LLaMA-DCLM, Data Reweighting Space=2-dimensional2025.07 | 2.55 | 4.1 | 4.22 | 24.52 | 38.15 | 780.62 | |
| Base ModelModel Backbone=LLaMA-FWE, Data Reweighting Space=52-dimensional (Nvidia domain classifier)2025.07 | 2.5 | 2.52 | 4.06 | 23.72 | 8.2 | — | |
| Base ModelModel Backbone=LLaMA-Nemo, Data Reweighting Space=52-dimensional (Nvidia domain classifier)2025.07 | 2.5 | 4.85 | 5.3 | 23.62 | 9.07 | — | |
| Base ModelTarget Model=LLaMA-FWE2025.07 | 2.5 | 2.52 | 4.06 | 23.72 | 8.2 | — | |
| Base ModelTarget Model=LLaMA-Nemo2025.07 | 2.5 | 4.85 | 5.3 | 23.62 | 9.07 | — | |
| Base ModelTarget Model=LLaMA-FWE, Data Reweighting Space=2-dimensional2025.07 | 2.5 | 2.52 | 4.06 | 23.72 | 37.65 | 781.34 | |
| Base ModelTarget Model=LLaMA-Nemo, Data Reweighting Space=2-dimensional2025.07 | 2.5 | 4.85 | 5.3 | 23.62 | 38.22 | 782.95 | |
| Base ModelModel Backbone=Pythia-1.4B, Data Reweighting Space=52-dimensional (Nvidia domain classifier)2025.07 | 1.67 | 4.39 | 2.1 | 14.16 | 5.58 | — |