Knowledge Reasoning on MATH, GPQA, MMLU, and GSM8K (test)
89.6MATH AccuracyPDA-GAM (3∆)
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| PDA-GAM (3∆)Student Model=Qwen3-8B, Delta Setting=Multiple Preference Deltas2026.05 | 89.6 | 48.8 | 83.1 | 95.5 | 79.3 | |
| PDA-GAM (2∆)Student Model=Qwen3-8B, Delta Setting=Multiple Preference Deltas2026.05 | 88.5 | 47.9 | 82.4 | 94.2 | 78.3 | |
| PDA-TIES (3∆)Student Model=Qwen3-8B, Delta Setting=Multiple Preference Deltas2026.05 | 88.1 | 47.3 | 81.5 | 93.8 | 77.7 | |
| PDA-KnOTS (3∆)Student Model=Qwen3-8B, Delta Setting=Multiple Preference Deltas2026.05 | 87.8 | 47.1 | 81.4 | 93.7 | 77.5 | |
| PDA-TIES (2∆)Student Model=Qwen3-8B, Delta Setting=Multiple Preference Deltas2026.05 | 87.6 | 46.8 | 80.4 | 93.7 | 77.1 | |
| PDA-GAM (3∆)Student Model=Tülu3-8B, Delta Setting=Multiple Preference Deltas2026.05 | 87.4 | 46 | 81.2 | 93.5 | 77 | |
| DeepSeek-R1 (7B over 1.5B)Student Model=Qwen3-8B, Delta Setting=Single Preference Delta2026.05 | 87.2 | 46.5 | 81.8 | 93.4 | 77.2 | |
| PDA-Averaging (3∆)Student Model=Qwen3-8B, Delta Setting=Multiple Preference Deltas2026.05 | 86.8 | 46.2 | 80.5 | 92.5 | 76.5 | |
| PDA-GAM (2∆)Student Model=Tülu3-8B, Delta Setting=Multiple Preference Deltas2026.05 | 86.2 | 44.8 | 80.5 | 92.2 | 75.9 | |
| PDA-TIES (3∆)Student Model=Tülu3-8B, Delta Setting=Multiple Preference Deltas2026.05 | 85.8 | 44.5 | 79.8 | 91.5 | 75.4 | |
| PDA-KnOTS (3∆)Student Model=Tülu3-8B, Delta Setting=Multiple Preference Deltas2026.05 | 85.5 | 44.2 | 79.7 | 91.4 | 75.2 | |
| PDA-TIES (2∆)Student Model=Tülu3-8B, Delta Setting=Multiple Preference Deltas2026.05 | 85.2 | 43.9 | 79.5 | 91.2 | 75 | |
| Joint DPO (3∆)Student Model=Qwen3-8B, Delta Setting=Multiple Preference Deltas2026.05 | 85.1 | 44.3 | 80.6 | 89.6 | 74.9 | |
| DeepSeek-R1 (7B over 1.5B)Student Model=Tülu3-8B, Delta Setting=Single Preference Delta2026.05 | 84.6 | 43.1 | 79.2 | 91 | 74.5 | |
| Qwen3 (4B over 1.7B)Student Model=Qwen3-8B, Delta Setting=Single Preference Delta2026.05 | 84.5 | 44 | 80.2 | 90.2 | 74.7 | |
| PDA-Averaging (3∆)Student Model=Tülu3-8B, Delta Setting=Multiple Preference Deltas2026.05 | 83.5 | 42.8 | 78.4 | 90.1 | 73.7 | |
| Llama-3.2 (3B over 1B)Student Model=Qwen3-8B, Delta Setting=Single Preference Delta2026.05 | 82.8 | 42.6 | 79.1 | 88.5 | 73.3 | |
| Joint DPO (3∆)Student Model=Tülu3-8B, Delta Setting=Multiple Preference Deltas2026.05 | 82.7 | 40.2 | 78.4 | 89.5 | 72.7 | |
| Student BaselineStudent Model=Qwen3-8B2026.05 | 81.5 | 42.1 | 78.5 | 88 | 72.5 | |
| Qwen3 (4B over 1.7B)Student Model=Tülu3-8B, Delta Setting=Single Preference Delta2026.05 | 81.2 | 40.5 | 77.5 | 88.2 | 71.9 | |
| Llama-3.2 (3B over 1B)Student Model=Tülu3-8B, Delta Setting=Single Preference Delta2026.05 | 79.5 | 39.2 | 76 | 86 | 70.2 | |
| Sequential DPO (2∆)Student Model=Qwen3-8B, Delta Setting=Multiple Preference Deltas2026.05 | 79.2 | 39.8 | 76.9 | 88.2 | 71 | |
| Student BaselineStudent Model=Tülu3-8B2026.05 | 78.2 | 38.5 | 75.2 | 85.1 | 69.3 | |
| Sequential DPO (2∆)Student Model=Tülu3-8B, Delta Setting=Multiple Preference Deltas2026.05 | 76.3 | 34.6 | 72.3 | 82.2 | 66.4 |