Instruction Following on Instruction-Following Suite
52IFEval ScoreGranite 3.1 8B-CP
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Granite 3.1 8B-CPBackbone=Granite 3.1 8B, Strategy=Code-Prompted, Zero-shot=true, Greedy decoding=true2025.05 | 52 | 11 | 45 | 16 | 39 | |
| Qwen2.5 7B-PCBackbone=Qwen2.5 7B, Strategy=Pseudo-code tuned, Zero-shot=true, Greedy decoding=true2025.05 | 48 | 16 | 50 | 47 | 42 | |
| Granite 3.1 8B-PCBackbone=Granite 3.1 8B, Strategy=Pseudo-code tuned, Zero-shot=true, Greedy decoding=true2025.05 | 48 | 13 | 50 | 49 | 41 | |
| Llama 3.1 8B-PCBackbone=Llama 3.1 8B, Strategy=Pseudo-code tuned, Zero-shot=true, Greedy decoding=true2025.05 | 46 | 13 | 46 | 47 | 42 | |
| Qwen2.5 7B-CPBackbone=Qwen2.5 7B, Strategy=Code-Prompted, Zero-shot=true, Greedy decoding=true2025.05 | 46 | 2 | 42 | 33 | 46 | |
| Olmo 3 7B-CPBackbone=Olmo 3 7B, Strategy=Code-Prompted, Zero-shot=true, Greedy decoding=true2025.05 | 46 | 9 | 40 | 40 | 42 | |
| Granite 8B Code-CPBackbone=Granite 8B Code, Strategy=Code-Prompted, Zero-shot=true, Greedy decoding=true2025.05 | 45 | 8 | 25 | 11 | 33 | |
| Mistral 7B v0.3-CPBackbone=Mistral 7B v0.3, Strategy=Code-Prompted, Zero-shot=true, Greedy decoding=true2025.05 | 44 | 9 | 27 | 15 | 37 | |
| Qwen2.5 7B-NLBackbone=Qwen2.5 7B, Strategy=NL-tuned, Zero-shot=true, Greedy decoding=true2025.05 | 44 | 16 | 45 | 40 | 31 | |
| Granite 8B Code-PCBackbone=Granite 8B Code, Strategy=Pseudo-code tuned, Zero-shot=true, Greedy decoding=true2025.05 | 44 | 15 | 50 | 31 | 38 | |
| Granite 3.1 8B-NLBackbone=Granite 3.1 8B, Strategy=NL-tuned, Zero-shot=true, Greedy decoding=true2025.05 | 44 | 11 | 45 | 54 | 31 | |
| Mistral 7B v0.3-PCBackbone=Mistral 7B v0.3, Strategy=Pseudo-code tuned, Zero-shot=true, Greedy decoding=true2025.05 | 41 | 14 | 40 | 45 | 38 | |
| Llama 3.1 8B-NLBackbone=Llama 3.1 8B, Strategy=NL-tuned, Zero-shot=true, Greedy decoding=true2025.05 | 39 | 8 | 42 | 40 | 32 | |
| Granite 8B Code-NLBackbone=Granite 8B Code, Strategy=NL-tuned, Zero-shot=true, Greedy decoding=true2025.05 | 38 | 7 | 37 | 38 | 30 | |
| Olmo 3 7B-PCBackbone=Olmo 3 7B, Strategy=Pseudo-code tuned, Zero-shot=true, Greedy decoding=true2025.05 | 36 | 5 | 48 | 52 | 44 | |
| Mistral 7B v0.3-NLBackbone=Mistral 7B v0.3, Strategy=NL-tuned, Zero-shot=true, Greedy decoding=true2025.05 | 35 | 8 | 35 | 36 | 32 | |
| Olmo 3 7B-NLBackbone=Olmo 3 7B, Strategy=NL-tuned, Zero-shot=true, Greedy decoding=true2025.05 | 34 | 6 | 40 | 45 | 27 | |
| Llama 3.1 8B-CPBackbone=Llama 3.1 8B, Strategy=Code-Prompted, Zero-shot=true, Greedy decoding=true2025.05 | 32 | 9 | 25 | 20 | 37 |