Multitask Language Understanding on MMLU (pass@1)
71.9pass@1Tulu 3-SFT
Evaluation results
| Method | Links | |
|---|---|---|
| Tulu 3-SFTBase model=Qwen-2.5-7B2024.10 | 71.9 | |
| MATRIX-Gen-SFTBase model=Qwen-2.5-7B2024.10 | 71.9 | |
| OpenHermesBase model=Qwen-2.5-7B2024.10 | 69.6 | |
| WildChatBase model=Qwen-2.5-7B2024.10 | 69.3 | |
| Tulu v2 MixBase model=Qwen-2.5-7B2024.10 | 69 | |
| Evol-InstructBase model=Qwen-2.5-7B2024.10 | 68.3 | |
| ShareGPTBase model=Qwen-2.5-7B2024.10 | 67.8 | |
| Magpie-SFTBase model=Qwen-2.5-7B2024.10 | 67 | |
| Magpie-SFTBase model=Meta-Llama-3-8B2024.10 | 61.2 | |
| MATRIX-Gen-SFTBase model=Meta-Llama-3-8B2024.10 | 59.9 | |
| ShareGPTBase model=Meta-Llama-3-8B2024.10 | 58.4 | |
| Tulu v2 MixBase model=Meta-Llama-3-8B2024.10 | 58.4 | |
| Tulu 3-SFTBase model=Meta-Llama-3-8B2024.10 | 57.8 | |
| Dare Ties MergingModel=Fused2026.02 | 55.64 | |
| Task ArithmeticModel=Fused2026.02 | 55.58 | |
| w/o MergingModel=Meta-Llama-3-8B-Instruct (base0)2026.02 | 55.11 | |
| OpenHermesBase model=Meta-Llama-3-8B2024.10 | 52.8 | |
| Ties MergingModel=Fused2026.02 | 52.04 | |
| Evol-InstructBase model=Meta-Llama-3-8B2024.10 | 48 | |
| SCF-RKLModel=Fused2026.02 | 46.17 | |
| Dare Task ArithmeticModel=Fused2026.02 | 45.54 | |
| WildChatBase model=Meta-Llama-3-8B2024.10 | 43.1 | |
| SCEModel=Fused2026.02 | 42.85 | |
| w/o MergingModel=MAmmoTH2-8B-Plus (base1)2026.02 | 38.68 |