Language Model Evaluation on Tulu-3 (dev)
60.85Knowledge ScoreEqual
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| EqualBackbone=Qwen2.5-7B, Samples=800K, Mixture Source=tulu-3-sft-mixture2026.07 | 60.85 | 64.55 | 59.03 | 53.61 | 68.58 | 53.49 | 60.02 | |
| ODMBackbone=Qwen2.5-7B, Samples=800K, Mixture Source=tulu-3-sft-mixture2026.07 | 60.25 | 65.69 | 59.42 | 44.22 | 63.59 | 54.83 | 58 | |
| Qwen2.5-7B-InstructModel=Qwen2.5-7B-Instruct2026.07 | 60.24 | 52.36 | 54.7 | 68.67 | 44.36 | 60.47 | 56.8 | |
| RegMixBackbone=Qwen2.5-7B, Samples=800K, Mixture Source=tulu-3-sft-mixture2026.07 | 59.6 | 68.12 | 63.37 | 55.76 | 58.04 | 55.94 | 60.14 | |
| CAUSALMIX-SBackbone=Qwen2.5-7B, Samples=800K, Mixture Source=tulu-3-sft-mixture2026.07 | 59.35 | 62.88 | 58.63 | 64.43 | 67.47 | 60.95 | 62.28 | |
| DMOBackbone=Qwen2.5-7B, Samples=800K, Mixture Source=tulu-3-sft-mixture2026.07 | 59.15 | 63.7 | 60.62 | 54.05 | 70.24 | 54.35 | 60.35 | |
| GridBackbone=Qwen2.5-7B, Samples=800K, Mixture Source=tulu-3-sft-mixture2026.07 | 59.08 | 68.77 | 65.99 | 61.55 | 44.92 | 56.43 | 59.46 | |
| DoReMiBackbone=Qwen2.5-7B, Samples=800K, Mixture Source=tulu-3-sft-mixture2026.07 | 58.02 | 63.2 | 57.35 | 57.28 | 68.21 | 52.75 | 59.47 | |
| CAUSALMIX-ABackbone=Qwen2.5-7B, Samples=800K, Mixture Source=tulu-3-sft-mixture2026.07 | 57.14 | 64.03 | 58.51 | 65.52 | 68.21 | 57.65 | 61.84 | |
| Llama-3.1-Tulu-3-8B-SFTModel=Llama-3.1-Tulu-3-8B-SFT2026.07 | 51.63 | 67.37 | 56.14 | 57.91 | 68.39 | 46.39 | 57.97 | |
| Qwen2.5-0.5B-InstructModel=Qwen2.5-0.5B-Instruct2026.07 | 29.9 | 32.14 | 35.84 | 34.1 | 30.31 | 11.14 | 28.9 | |
| DoReMiBackbone=Qwen2.5-0.5B, Samples=100K, Mixture Source=tulu-3-sft-mixture2026.07 | 29.81 | 31.7 | 22.9 | 33.4 | 35.3 | 25.34 | 29.74 | |
| CAUSALMIX-SBackbone=Qwen2.5-0.5B, Samples=400K, Mixture Source=tulu-3-sft-mixture2026.07 | 29.45 | 31.3 | 26.37 | 32.4 | 38.82 | 25.95 | 30.71 | |
| GridBackbone=Qwen2.5-0.5B, Samples=400K, Mixture Source=tulu-3-sft-mixture2026.07 | 29.34 | 31.84 | 26.68 | 35.02 | 16.64 | 17.26 | 26.13 | |
| DMOBackbone=Qwen2.5-0.5B, Samples=100K, Mixture Source=tulu-3-sft-mixture2026.07 | 28.86 | 30.91 | 22.51 | 30.68 | 35.86 | 26.19 | 29.17 | |
| CAUSALMIX-ABackbone=Qwen2.5-0.5B, Samples=400K, Mixture Source=tulu-3-sft-mixture2026.07 | 28.73 | 30.63 | 24.67 | 30.48 | 42.51 | 43.45 | 33.41 | |
| ODMBackbone=Qwen2.5-0.5B, Samples=400K, Mixture Source=tulu-3-sft-mixture2026.07 | 28.65 | 30.74 | 24.35 | 34.62 | 30.5 | 12.61 | 26.91 | |
| ODMBackbone=Qwen2.5-0.5B, Samples=800K, Mixture Source=tulu-3-sft-mixture2026.07 | 28.59 | 30.5 | 25.18 | 36.06 | 33.64 | 10.28 | 27.37 | |
| DoReMiBackbone=Qwen2.5-0.5B, Samples=400K, Mixture Source=tulu-3-sft-mixture2026.07 | 28.58 | 31.12 | 22.14 | 33.61 | 39.74 | 26.56 | 30.29 | |
| RegMixBackbone=Qwen2.5-0.5B, Samples=400K, Mixture Source=tulu-3-sft-mixture2026.07 | 28.57 | 30.19 | 27.02 | 34.32 | 24.21 | 16.65 | 26.82 | |
| GridBackbone=Qwen2.5-0.5B, Samples=100K, Mixture Source=tulu-3-sft-mixture2026.07 | 28.46 | 31.98 | 30.41 | 33.5 | 20.15 | 23.38 | 27.98 | |
| EqualBackbone=Qwen2.5-0.5B, Samples=400K, Mixture Source=tulu-3-sft-mixture2026.07 | 28.39 | 32.98 | 23.96 | 32.1 | 37.89 | 27.78 | 30.51 | |
| CAUSALMIX-ABackbone=Qwen2.5-0.5B, Samples=100K, Mixture Source=tulu-3-sft-mixture2026.07 | 28.37 | 31.31 | 24.67 | 29.57 | 38.63 | 26.93 | 29.91 | |
| CAUSALMIX-SBackbone=Qwen2.5-0.5B, Samples=800K, Mixture Source=tulu-3-sft-mixture2026.07 | 28.31 | 30.96 | 27.64 | 30.97 | 42.51 | 36.47 | 32.81 | |
| EqualBackbone=Qwen2.5-0.5B, Samples=800K, Mixture Source=tulu-3-sft-mixture2026.07 | 28.07 | 29.28 | 21.59 | 36.06 | 46.95 | 25.09 | 31.78 | |
| DMOBackbone=Qwen2.5-0.5B, Samples=800K, Mixture Source=tulu-3-sft-mixture2026.07 | 28.07 | 31.88 | 22.42 | 26.9 | 41.59 | 41.37 | 32.04 | |
| CAUSALMIX-ABackbone=Qwen2.5-0.5B, Samples=800K, Mixture Source=tulu-3-sft-mixture2026.07 | 27.93 | 29.68 | 23.56 | 27.76 | 43.81 | 50.92 | 33.94 | |
| DMOBackbone=Qwen2.5-0.5B, Samples=400K, Mixture Source=tulu-3-sft-mixture2026.07 | 27.74 | 31.75 | 24.25 | 31.8 | 38.63 | 41.62 | 32.63 | |
| DoReMiBackbone=Qwen2.5-0.5B, Samples=800K, Mixture Source=tulu-3-sft-mixture2026.07 | 27.7 | 29.66 | 20.58 | 34.53 | 42.88 | 32.93 | 31.38 | |
| EqualBackbone=Qwen2.5-0.5B, Samples=100K, Mixture Source=tulu-3-sft-mixture2026.07 | 27.65 | 29.8 | 23.32 | 33 | 33.83 | 14.57 | 27.03 | |
| ODMBackbone=Qwen2.5-0.5B, Samples=100K, Mixture Source=tulu-3-sft-mixture2026.07 | 27.5 | 30.53 | 25.41 | 33.1 | 26.06 | 23.62 | 27.7 | |
| RegMixBackbone=Qwen2.5-0.5B, Samples=100K, Mixture Source=tulu-3-sft-mixture2026.07 | 27.29 | 30.04 | 25.53 | 31.28 | 17.01 | 32.56 | 27.28 | |
| CAUSALMIX-SBackbone=Qwen2.5-0.5B, Samples=100K, Mixture Source=tulu-3-sft-mixture2026.07 | 27.27 | 30.23 | 27.77 | 34.94 | 34.94 | 14.81 | 27.85 | |
| RegMixBackbone=Qwen2.5-0.5B, Samples=800K, Mixture Source=tulu-3-sft-mixture2026.07 | 24.9 | 29.94 | 30.25 | 25.46 | 25.32 | 22.52 | 26.4 | |
| GridBackbone=Qwen2.5-0.5B, Samples=800K, Mixture Source=tulu-3-sft-mixture2026.07 | 23.94 | 24.4 | 29.67 | 23.81 | 17.93 | 22.64 | 31.17 |