# Open-ended Instruction Following on Audio-OpenHermes Source: [Open-ended Instruction Following on Audio-OpenHermes]() Collection index: [llms.txt]() Data updated: 2026-03-10T08:30:42.638Z Evaluation of model performance on open-ended instruction following tasks using the Audio-OpenHermes dataset, measured by metrics such as EN and ID scores. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Open-ended Instruction Following | | Dataset (including any stated split) | Audio-OpenHermes | | Metric | EN Score | | Unit | SCALAR | | Direction | Higher is better | | Results loaded | 10 of 10 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [EN Score](): Higher is better; unit: SCALAR. - [ID Score](): Higher is better; unit: SCALAR. ## Recorded results | Method | EN Score | Result date | Report paper | | --- | --- | --- | --- | | SEA-LION-v3-8B-IT | 4.44 | Not recorded | 69afa5662d603e6de3e43875 | | Ours (hard-gating) | 4.42 | Not recorded | 69afa5662d603e6de3e43875 | | Ours (soft-gating) | 4.29 | Not recorded | 69afa5662d603e6de3e43875 | | Whisper-v3 + SEA-LION-v3-8B-IT | 4.17 | Not recorded | 69afa5662d603e6de3e43875 | | MERaLiON-2-10B | 4.06 | Not recorded | 69afa5662d603e6de3e43875 | | ML-DiVA | 4.03 | Not recorded | 69afa5662d603e6de3e43875 | | Glm-4-voice | 3.72 | Not recorded | 69afa5662d603e6de3e43875 | | Qwen2-Audio | 3.65 | Not recorded | 69afa5662d603e6de3e43875 | | SeaLLMs-Audio | 3.37 | Not recorded | 69afa5662d603e6de3e43875 | | EN-DiVA | 3.28 | Not recorded | 69afa5662d603e6de3e43875 | ### SEA-LION-v3-8B-IT EN Score: 4.44. Result ID: 69afd6656dbb7075b9582b3b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Approach category":"Text-only reference"}. This record does not assert independent reproduction. ### Ours (hard-gating) EN Score: 4.42. Result ID: 69afd6656dbb7075b9582b44. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Approach category":"Distilled end-to-end","Gating strategy":"hard-gating"}. This record does not assert independent reproduction. ### Ours (soft-gating) EN Score: 4.29. Result ID: 69afd6656dbb7075b9582b43. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Approach category":"Distilled end-to-end","Gating strategy":"soft-gating"}. This record does not assert independent reproduction. ### Whisper-v3 + SEA-LION-v3-8B-IT EN Score: 4.17. Result ID: 69afd6656dbb7075b9582b3c. [Reporting paper]() Recorded evaluation configuration: {"Approach category":"Cascaded"}. This record does not assert independent reproduction. ### MERaLiON-2-10B EN Score: 4.06. Result ID: 69afd6656dbb7075b9582b3e. [Reporting paper]() Recorded evaluation configuration: {"Approach category":"Zero-shot end-to-end"}. This record does not assert independent reproduction. ### ML-DiVA EN Score: 4.03. Result ID: 69afd6656dbb7075b9582b42. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Approach category":"Distilled end-to-end","Training protocol":"re-trained baseline"}. This record does not assert independent reproduction. ### Glm-4-voice EN Score: 3.72. Result ID: 69afd6656dbb7075b9582b3d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Approach category":"Zero-shot end-to-end"}. This record does not assert independent reproduction. ### Qwen2-Audio EN Score: 3.65. Result ID: 69afd6656dbb7075b9582b40. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Approach category":"Zero-shot end-to-end"}. This record does not assert independent reproduction. ### SeaLLMs-Audio EN Score: 3.37. Result ID: 69afd6656dbb7075b9582b3f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Approach category":"Zero-shot end-to-end"}. This record does not assert independent reproduction. ### EN-DiVA EN Score: 3.28. Result ID: 69afd6656dbb7075b9582b41. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Approach category":"Distilled end-to-end","Training protocol":"re-trained baseline"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.