Instruction Following on IFEval (IFScore)
84.63IFScoreGemini2.5-Flash
Evaluation Results
| Method | Links | |
|---|---|---|
| Gemini2.5-FlashModel Category=Proprietary Audio Language Models2025.09 | 84.63 | |
| Qwen3-Omni-30B-A3B-ThinkingModel Category=Large Sized Large Audio Language Models, Parameter Count=>20B parameters2025.09 | 80.39 | |
| Whisper-Large-v3 + GPT-oSS-20BModel Category=Cascaded Systems2025.09 | 73.72 | |
| GPT-4o-mini-audioModel Category=Proprietary Audio Language Models2025.09 | 70.47 | |
| Voxtral-Small-24BModel Category=Large Sized Large Audio Language Models, Parameter Count=>20B parameters2025.09 | 66.83 | |
| GPT-4o-transcribe + GPT-4.1-miniModel Category=Cascaded Systems2025.09 | 66.69 | |
| Kimi-AudioModel Category=Medium Sized Large Audio Language Models, Parameter Count=5B-20B parameters2025.09 | 61.29 | |
| Zero-shotBackbone=Llama-3.1-8B-Instruct2026.05 | 56 | |
| MANSUBackbone=Llama-3.1-8B-Instruct, Unlearning Task=WMDP-bio2026.05 | 55.1 | |
| MANSUBackbone=Llama-3.1-8B-Instruct, Unlearning Task=WMDP-chem2026.05 | 54.9 | |
| MANSUBackbone=Llama-3.1-8B-Instruct, Unlearning Task=WMDP-cyber2026.05 | 54.9 | |
| Zero-shotBackbone=Qwen-3-8B2026.05 | 54.8 | |
| MANSUBackbone=Llama-3.1-8B-Instruct, Unlearning Task=MUSE2026.05 | 54.7 | |
| MANSUBackbone=Qwen-3-8B, Unlearning Task=WMDP-cyber2026.05 | 54.2 | |
| MANSUBackbone=Qwen-3-8B, Unlearning Task=WMDP-bio2026.05 | 54.1 | |
| MANSUBackbone=Qwen-3-8B, Unlearning Task=WMDP-chem2026.05 | 53.9 | |
| Qwen2.5-Omni-7BModel Category=Medium Sized Large Audio Language Models, Parameter Count=5B-20B parameters2025.09 | 50.83 | |
| Phi-4-Multi-modalModel Category=Medium Sized Large Audio Language Models, Parameter Count=5B-20B parameters2025.09 | 44.51 | |
| MANSUBackbone=Qwen-3-8B, Unlearning Task=MUSE2026.05 | 43.6 | |
| Voxtral-Mini-3BModel Category=Small-sized Audio Language Models, Parameter Count=<5B parameters2025.09 | 40.02 | |
| Qwen2.5-Omni-3BModel Category=Small-sized Audio Language Models, Parameter Count=<5B parameters2025.09 | 35.82 |