Instruction Following Evaluation Suite
25.39Self-Instruct ScoreSFT
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| SFTModel=Phi3-mini-3.8B2025.02 | 25.39 | — | — | 21.81 | — | — | 31.09 | 29.19 | 37.97 | 41.07 | |
| SFTModel=Qwen2-7B2025.02 | 22.69 | — | — | 21.42 | — | — | 30.31 | 29.07 | 37.31 | 41.04 | |
| SFTModel=Llama-3-8B2025.02 | 21.9 | — | — | 17 | — | — | 26.36 | 27.01 | 30.66 | 35.23 | |
| FKLStudent=Qwen2-0.5B, Teacher=Qwen2-7B, Category=Same Tokenizer KD2025.02 | 19.68 | — | — | 19.24 | — | — | 27.29 | 27.41 | 32.67 | 37.46 | |
| SFTModel=Gemma-2-9B2025.02 | 18.01 | — | — | 18.85 | — | — | 25.23 | 26.72 | 27.74 | 34.83 | |
| CDMStudent=Qwen2-0.5B, Teacher=Phi3-mini-3.8B, Category=Cross Tokenizer KD2025.02 | 16.55 | — | — | 16.38 | — | — | 25.1 | 25.45 | 30.66 | 36.47 | |
| MinEDStudent=Qwen2-0.5B, Teacher=Phi3-mini-3.8B, Category=Cross Tokenizer KD2025.02 | 16.26 | — | — | 15.37 | — | — | 24.72 | 25.55 | 30.76 | 35.69 | |
| MinEDStudent=Gemma-2-2B, Teacher=Llama-3-8B, Category=Cross Tokenizer KD2025.02 | 16.16 | — | — | 16.4 | — | — | 22.6 | 25.83 | 25.99 | 28.6 | |
| RKLStudent=Qwen2-0.5B, Teacher=Qwen2-7B, Category=Same Tokenizer KD2025.02 | 16.15 | — | — | 16.62 | — | — | 25.35 | 26.15 | 30.32 | 37.53 | |
| ULDStudent=Qwen2-0.5B, Teacher=Phi3-mini-3.8B, Category=Cross Tokenizer KD2025.02 | 16.15 | — | — | 15.34 | — | — | 24.93 | 26.43 | 30.63 | 36.07 | |
| SFTModel=OPT-6.7B2025.02 | 15.71 | — | — | 15.82 | — | — | 24.07 | 28.4 | 26.87 | 33.56 | |
| FKLStudent=OPT-1.3B, Teacher=OPT-6.7B, Category=Same Tokenizer KD2025.02 | 15.24 | — | — | 16.16 | — | — | 22.92 | 25.36 | 26.47 | 31.38 | |
| SFTModel=Qwen2-0.5B2025.02 | 15.17 | — | — | 15.22 | — | — | 24.07 | 24.66 | 30.31 | 35 | |
| SFTModel=Gemma-2-2B2025.02 | 14.94 | — | — | 16.89 | — | — | 22.46 | 25.12 | 25.29 | 30.07 | |
| CDMStudent=Gemma-2-2B, Teacher=Llama-3-8B, Category=Cross Tokenizer KD2025.02 | 14.89 | — | — | 18.33 | — | — | 23.55 | 26.13 | 26.4 | 32 | |
| ULDStudent=Gemma-2-2B, Teacher=Llama-3-8B, Category=Cross Tokenizer KD2025.02 | 14.58 | — | — | 17.25 | — | — | 23.23 | 26.11 | 27.69 | 30.53 | |
| CDMStudent=OPT-1.3B, Teacher=Llama-3-8B, Category=Cross Tokenizer KD2025.02 | 14.39 | — | — | 15.77 | — | — | 23 | 26.15 | 26.33 | 32.33 | |
| FKLStudent=Gemma-2-2B, Teacher=Gemma-2-9B, Category=Same Tokenizer KD2025.02 | 14.3 | — | — | 18.64 | — | — | 23.82 | 26.51 | 27.61 | 32.06 | |
| SFTModel=OPT-1.3B2025.02 | 14.26 | — | — | 14.81 | — | — | 22.47 | 25.48 | 25.88 | 31.93 | |
| RKLStudent=Gemma-2-2B, Teacher=Gemma-2-9B, Category=Same Tokenizer KD2025.02 | 13.8 | — | — | 18.64 | — | — | 22.24 | 25.26 | 23.7 | 29.79 | |
| ULDStudent=OPT-1.3B, Teacher=Llama-3-8B, Category=Cross Tokenizer KD2025.02 | 13.69 | — | — | 15.88 | — | — | 22.18 | 25.45 | 25.82 | 30.07 | |
| RKLStudent=OPT-1.3B, Teacher=OPT-6.7B, Category=Same Tokenizer KD2025.02 | 13.24 | — | — | 15.42 | — | — | 21.77 | 25.03 | 23.86 | 31.27 | |
| MinEDStudent=OPT-1.3B, Teacher=Llama-3-8B, Category=Cross Tokenizer KD2025.02 | 12.6 | — | — | 15.6 | — | — | 21.69 | 25.21 | 24.51 | 30.52 | |
| Phased IFTBase Model=Llama-3 8B, Data=Alpaca-cleaned2024.06 | 5.75 | 3.9 | 1.15 | 0.62 | 4.79 | 5.43 | 3.97 | — | — | — | |
| Phased IFTBase Model=Llama-2 7B, Data=Alpaca-cleaned2024.06 | 4.76 | 1.15 | 4.44 | 2.5 | 5.85 | 1.16 | 3.53 | — | — | — | |
| Phased IFTBase Model=Llama-2 13B, Data=Alpaca-cleaned2024.06 | 4.56 | 2.98 | 5.83 | 15.62 | 6.91 | 10.85 | 6.49 | — | — | — | |
| Phased IFTBase Model=Llama-2 7B, Data=Alpaca2024.06 | 4.17 | 6.42 | 5.56 | 6.88 | 9.31 | 14.34 | 7.26 | — | — | — | |
| Phased IFTBase Model=Llama-3 8B, Data=Alpaca2024.06 | 3.05 | 9.67 | 9.52 | 7.5 | 6.32 | 12.6 | 7.64 | — | — | — | |
| Phased IFTBase Model=Llama-2 13B, Data=Alpaca2024.06 | 1.98 | 2.75 | 7.78 | 11.88 | 11.44 | 16.28 | 7.35 | — | — | — | |
| Phased IFTBase Model=Llama-3 70B, Data=Alpaca2024.06 | 1.65 | 10.1 | 5.23 | 5.84 | 2.34 | 7.87 | 5.23 | — | — | — | |
| Phased IFTBase Model=Llama-2 70B, Data=Alpaca-cleaned2024.06 | 0.4 | 6.42 | 8.33 | 12.5 | 10.9 | 14.73 | 7.59 | — | — | — | |
| Phased IFTBase Model=Mistral 7B, Data=Alpaca2024.06 | -0.6 | 5.96 | 6.11 | 16.25 | 7.98 | 12.02 | 6.3 | — | — | — | |
| Phased IFTBase Model=Llama-2 70B, Data=Alpaca2024.06 | -3.04 | 2.57 | 7.27 | 13.46 | 4.1 | 0 | 2.82 | — | — | — |