# Open-ended Instruction Following on AlpacaEval Audio Source: [Open-ended Instruction Following on AlpacaEval Audio]() Collection index: [llms.txt]() Data updated: 2026-03-10T08:30:42.598Z Evaluation of model performance on open-ended instruction following tasks using the AlpacaEval Audio dataset, measured using language-specific scores (ZH/EN). A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Open-ended Instruction Following | | Dataset (including any stated split) | AlpacaEval Audio | | Metric | EN Score | | Unit | SCALAR | | Direction | Higher is better | | Results loaded | 10 of 10 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [EN Score](): Higher is better; unit: SCALAR. - [ZH Score](): Higher is better; unit: SCALAR. ## Recorded results | Method | EN Score | Result date | Report paper | | --- | --- | --- | --- | | SEA-LION-v3-8B-IT | 4.59 | Not recorded | 69afa5662d603e6de3e43875 | | MERaLiON-2-10B | 4.59 | Not recorded | 69afa5662d603e6de3e43875 | | Ours (soft-gating) | 4.58 | Not recorded | 69afa5662d603e6de3e43875 | | Ours (hard-gating) | 4.39 | Not recorded | 69afa5662d603e6de3e43875 | | ML-DiVA | 4.28 | Not recorded | 69afa5662d603e6de3e43875 | | Whisper-v3 + SEA-LION-v3-8B-IT | 3.99 | Not recorded | 69afa5662d603e6de3e43875 | | SeaLLMs-Audio | 3.93 | Not recorded | 69afa5662d603e6de3e43875 | | Qwen2-Audio | 3.77 | Not recorded | 69afa5662d603e6de3e43875 | | Glm-4-voice | 3.62 | Not recorded | 69afa5662d603e6de3e43875 | | EN-DiVA | 3.57 | Not recorded | 69afa5662d603e6de3e43875 | ### SEA-LION-v3-8B-IT EN Score: 4.59. Result ID: 69afd6656dbb7075b9582b31. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Approach category":"Text-only reference"}. This record does not assert independent reproduction. ### MERaLiON-2-10B EN Score: 4.59. Result ID: 69afd6656dbb7075b9582b34. [Reporting paper]() Recorded evaluation configuration: {"Approach category":"Zero-shot end-to-end"}. This record does not assert independent reproduction. ### Ours (soft-gating) EN Score: 4.58. Result ID: 69afd6656dbb7075b9582b39. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Approach category":"Distilled end-to-end","Gating strategy":"soft-gating"}. This record does not assert independent reproduction. ### Ours (hard-gating) EN Score: 4.39. Result ID: 69afd6656dbb7075b9582b3a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Approach category":"Distilled end-to-end","Gating strategy":"hard-gating"}. This record does not assert independent reproduction. ### ML-DiVA EN Score: 4.28. Result ID: 69afd6656dbb7075b9582b38. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Approach category":"Distilled end-to-end","Training protocol":"re-trained baseline"}. This record does not assert independent reproduction. ### Whisper-v3 + SEA-LION-v3-8B-IT EN Score: 3.99. Result ID: 69afd6656dbb7075b9582b32. [Reporting paper]() Recorded evaluation configuration: {"Approach category":"Cascaded"}. This record does not assert independent reproduction. ### SeaLLMs-Audio EN Score: 3.93. Result ID: 69afd6656dbb7075b9582b35. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Approach category":"Zero-shot end-to-end"}. This record does not assert independent reproduction. ### Qwen2-Audio EN Score: 3.77. Result ID: 69afd6656dbb7075b9582b36. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Approach category":"Zero-shot end-to-end"}. This record does not assert independent reproduction. ### Glm-4-voice EN Score: 3.62. Result ID: 69afd6656dbb7075b9582b33. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Approach category":"Zero-shot end-to-end"}. This record does not assert independent reproduction. ### EN-DiVA EN Score: 3.57. Result ID: 69afd6656dbb7075b9582b37. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Approach category":"Distilled end-to-end","Training protocol":"re-trained baseline"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.