Generalization Evaluation on AM-Thinking Distilled-math&code v1 (Unseen)
49.55Average Unseen ScoreEqual
Evaluation Results
| Method | Links | |
|---|---|---|
| EqualBackbone=Qwen2.5-7B, Samples=800K, Mixture Source=tulu-3-sft-mixture2026.07 | 49.55 | |
| CAUSALMIX-ABackbone=Qwen2.5-7B, Samples=800K, Mixture Source=tulu-3-sft-mixture2026.07 | 49.09 | |
| DMOBackbone=Qwen2.5-7B, Samples=800K, Mixture Source=tulu-3-sft-mixture2026.07 | 48.98 | |
| RegMixBackbone=Qwen2.5-7B, Samples=800K, Mixture Source=tulu-3-sft-mixture2026.07 | 48.12 | |
| ODMBackbone=Qwen2.5-7B, Samples=800K, Mixture Source=tulu-3-sft-mixture2026.07 | 48 | |
| CAUSALMIX-SBackbone=Qwen2.5-7B, Samples=800K, Mixture Source=tulu-3-sft-mixture2026.07 | 47.98 | |
| Qwen2.5-7B-InstructModel=Qwen2.5-7B-Instruct2026.07 | 46.94 | |
| GridBackbone=Qwen2.5-7B, Samples=800K, Mixture Source=tulu-3-sft-mixture2026.07 | 46.37 | |
| DoReMiBackbone=Qwen2.5-7B, Samples=800K, Mixture Source=tulu-3-sft-mixture2026.07 | 46.19 | |
| Llama-3.1-Tulu-3-8B-SFTModel=Llama-3.1-Tulu-3-8B-SFT2026.07 | 41.46 | |
| Qwen2.5-0.5B-InstructModel=Qwen2.5-0.5B-Instruct2026.07 | 28.6 | |
| GridBackbone=Qwen2.5-0.5B, Samples=400K, Mixture Source=tulu-3-sft-mixture2026.07 | 27.08 | |
| CAUSALMIX-SBackbone=Qwen2.5-0.5B, Samples=400K, Mixture Source=tulu-3-sft-mixture2026.07 | 26.93 | |
| DMOBackbone=Qwen2.5-0.5B, Samples=800K, Mixture Source=tulu-3-sft-mixture2026.07 | 26.49 | |
| RegMixBackbone=Qwen2.5-0.5B, Samples=100K, Mixture Source=tulu-3-sft-mixture2026.07 | 26.24 | |
| DMOBackbone=Qwen2.5-0.5B, Samples=100K, Mixture Source=tulu-3-sft-mixture2026.07 | 26.08 | |
| RegMixBackbone=Qwen2.5-0.5B, Samples=400K, Mixture Source=tulu-3-sft-mixture2026.07 | 26.07 | |
| CAUSALMIX-SBackbone=Qwen2.5-0.5B, Samples=100K, Mixture Source=tulu-3-sft-mixture2026.07 | 25.9 | |
| DMOBackbone=Qwen2.5-0.5B, Samples=400K, Mixture Source=tulu-3-sft-mixture2026.07 | 25.84 | |
| DoReMiBackbone=Qwen2.5-0.5B, Samples=400K, Mixture Source=tulu-3-sft-mixture2026.07 | 25.69 | |
| EqualBackbone=Qwen2.5-0.5B, Samples=800K, Mixture Source=tulu-3-sft-mixture2026.07 | 25.64 | |
| ODMBackbone=Qwen2.5-0.5B, Samples=400K, Mixture Source=tulu-3-sft-mixture2026.07 | 25.51 | |
| ODMBackbone=Qwen2.5-0.5B, Samples=800K, Mixture Source=tulu-3-sft-mixture2026.07 | 25.36 | |
| CAUSALMIX-SBackbone=Qwen2.5-0.5B, Samples=800K, Mixture Source=tulu-3-sft-mixture2026.07 | 25.04 | |
| CAUSALMIX-ABackbone=Qwen2.5-0.5B, Samples=800K, Mixture Source=tulu-3-sft-mixture2026.07 | 25.02 | |
| DoReMiBackbone=Qwen2.5-0.5B, Samples=800K, Mixture Source=tulu-3-sft-mixture2026.07 | 24.88 | |
| EqualBackbone=Qwen2.5-0.5B, Samples=400K, Mixture Source=tulu-3-sft-mixture2026.07 | 24.71 | |
| EqualBackbone=Qwen2.5-0.5B, Samples=100K, Mixture Source=tulu-3-sft-mixture2026.07 | 24.43 | |
| CAUSALMIX-ABackbone=Qwen2.5-0.5B, Samples=400K, Mixture Source=tulu-3-sft-mixture2026.07 | 24.26 | |
| DoReMiBackbone=Qwen2.5-0.5B, Samples=100K, Mixture Source=tulu-3-sft-mixture2026.07 | 24.12 | |
| ODMBackbone=Qwen2.5-0.5B, Samples=100K, Mixture Source=tulu-3-sft-mixture2026.07 | 23.59 | |
| GridBackbone=Qwen2.5-0.5B, Samples=100K, Mixture Source=tulu-3-sft-mixture2026.07 | 23.45 | |
| CAUSALMIX-ABackbone=Qwen2.5-0.5B, Samples=100K, Mixture Source=tulu-3-sft-mixture2026.07 | 23.42 | |
| RegMixBackbone=Qwen2.5-0.5B, Samples=800K, Mixture Source=tulu-3-sft-mixture2026.07 | 22.82 | |
| GridBackbone=Qwen2.5-0.5B, Samples=800K, Mixture Source=tulu-3-sft-mixture2026.07 | 22.53 |