# Language Modeling on Open LLM Leaderboard Unified General Ability Suite (test) Source: [Language Modeling on Open LLM Leaderboard Unified General Ability Suite (test)]() Collection index: [llms.txt]() Data updated: 2026-02-26T05:50:02.487Z Comprehensive evaluation aggregating performance across a wide array of reasoning, knowledge, and general ability benchmarks (including MMLU-P, GPQA, BBH, MATH, IFEval, ARC, PIQA, BoolQ, etc.) as reported on the unified test set of the Open LLM Leaderboard. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Language Modeling | | Dataset (including any stated split) | Open LLM Leaderboard & General Ability Benchmarks (MMLU-P, GPQA, BBH, MATH, MuSR, IFEval, ARC, Hellaswag, PIQA, BoolQ, WinoGrande, COPA, OpenBookQA, SciQ) unified (test) | | Metric | MMLU-P Accuracy | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 16 of 16 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [MMLU-P Accuracy](): Higher is better; unit: PERCENT. - [GPQA Accuracy](): Higher is better; unit: PERCENT. - [BBH Accuracy](): Higher is better; unit: PERCENT. - [MATH Exact Match (EM)](): Higher is better; unit: PERCENT. - [MuSR Accuracy](): Higher is better; unit: PERCENT. - [IFEval Instruction (Strict) Accuracy](): Higher is better; unit: PERCENT. - [IFEval Prompt (Strict) Accuracy](): Higher is better; unit: PERCENT. - [ARC Challenge Accuracy](): Higher is better; unit: PERCENT. - [ARC Easy Accuracy](): Higher is better; unit: PERCENT. - [HellaSwag Accuracy](): Higher is better; unit: PERCENT. - [PIQA Accuracy](): Higher is better; unit: PERCENT. - [BoolQ Accuracy](): Higher is better; unit: PERCENT. - [WinoGrande Accuracy](): Higher is better; unit: PERCENT. - [COPA Accuracy](): Higher is better; unit: PERCENT. - [OpenBookQA Accuracy](): Higher is better; unit: PERCENT. - [SciQ Accuracy](): Higher is better; unit: PERCENT. - [Average Rank](): Lower is better; unit: SCALAR. - [Top-1 Count](): Higher is better; unit: SCALAR. ## Recorded results | Method | MMLU-P Accuracy | Result date | Report paper | | --- | --- | --- | --- | | HyperGLU | 12 | Not recorded | 6992a4599ebc499c97406aa5 | | HyperMLP | 11.9 | Not recorded | 6992a4599ebc499c97406aa5 | | HyperGLU | 11.7 | Not recorded | 6992a4599ebc499c97406aa5 | | HGRN2 | 11.5 | Not recorded | 6992a4599ebc499c97406aa5 | | HGRN | 11.4 | Not recorded | 6992a4599ebc499c97406aa5 | | SoftmaxAttn | 11.4 | Not recorded | 6992a4599ebc499c97406aa5 | | GLA | 11.4 | Not recorded | 6992a4599ebc499c97406aa5 | | GatedDeltaNet | 11.3 | Not recorded | 6992a4599ebc499c97406aa5 | | DeltaNet | 11.2 | Not recorded | 6992a4599ebc499c97406aa5 | | ReLUAttn+KVConv | 11.1 | Not recorded | 6992a4599ebc499c97406aa5 | | GSA | 11 | Not recorded | 6992a4599ebc499c97406aa5 | | RetNet | 11 | Not recorded | 6992a4599ebc499c97406aa5 | | GLA | 11 | Not recorded | 6992a4599ebc499c97406aa5 | | DeltaNet | 10.9 | Not recorded | 6992a4599ebc499c97406aa5 | | DiffTrans | 10.9 | Not recorded | 6992a4599ebc499c97406aa5 | | SoftmaxAttn | 10.6 | Not recorded | 6992a4599ebc499c97406aa5 | ### HyperGLU MMLU-P Accuracy: 12. Result ID: 699a50a75698665ec4d8378a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Parameters":"1.3B","Training Tokens":"100B","Shots":"MMLU-P: 5, GPQA: 0, BBH: 3, MATH: 4, MuSR: 0; others 0-shot"}. This record does not assert independent reproduction. ### HyperMLP MMLU-P Accuracy: 11.9. Result ID: 699a50a75698665ec4d83791. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Parameters":"340M","Training Tokens":"15B","Shots":"MMLU-P: 5, GPQA: 0, BBH: 3, MATH: 4, MuSR: 0; others 0-shot"}. This record does not assert independent reproduction. ### HyperGLU MMLU-P Accuracy: 11.7. Result ID: 699a50a75698665ec4d83792. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Parameters":"340M","Training Tokens":"15B","Shots":"MMLU-P: 5, GPQA: 0, BBH: 3, MATH: 4, MuSR: 0; others 0-shot"}. This record does not assert independent reproduction. ### HGRN2 MMLU-P Accuracy: 11.5. Result ID: 699a50a75698665ec4d83787. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Parameters":"1.3B","Training Tokens":"100B","Shots":"MMLU-P: 5, GPQA: 0, BBH: 3, MATH: 4, MuSR: 0; others 0-shot"}. This record does not assert independent reproduction. ### HGRN MMLU-P Accuracy: 11.4. Result ID: 699a50a75698665ec4d83786. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model Parameters":"1.3B","Training Tokens":"100B","Shots":"MMLU-P: 5, GPQA: 0, BBH: 3, MATH: 4, MuSR: 0; others 0-shot"}. This record does not assert independent reproduction. ### SoftmaxAttn MMLU-P Accuracy: 11.4. Result ID: 699a50a75698665ec4d83788. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model Parameters":"1.3B","Training Tokens":"100B","Shots":"MMLU-P: 5, GPQA: 0, BBH: 3, MATH: 4, MuSR: 0; others 0-shot"}. This record does not assert independent reproduction. ### GLA MMLU-P Accuracy: 11.4. Result ID: 699a50a75698665ec4d83789. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Parameters":"1.3B","Training Tokens":"100B","Shots":"MMLU-P: 5, GPQA: 0, BBH: 3, MATH: 4, MuSR: 0; others 0-shot"}. This record does not assert independent reproduction. ### GatedDeltaNet MMLU-P Accuracy: 11.3. Result ID: 699a50a75698665ec4d8378c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Parameters":"340M","Training Tokens":"15B","Shots":"MMLU-P: 5, GPQA: 0, BBH: 3, MATH: 4, MuSR: 0; others 0-shot"}. This record does not assert independent reproduction. ### DeltaNet MMLU-P Accuracy: 11.2. Result ID: 699a50a75698665ec4d8378d. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model Parameters":"340M","Training Tokens":"15B","Shots":"MMLU-P: 5, GPQA: 0, BBH: 3, MATH: 4, MuSR: 0; others 0-shot"}. This record does not assert independent reproduction. ### ReLUAttn+KVConv MMLU-P Accuracy: 11.1. Result ID: 699a50a75698665ec4d83790. [Reporting paper]() Recorded evaluation configuration: {"Model Parameters":"340M","Training Tokens":"15B","Shots":"MMLU-P: 5, GPQA: 0, BBH: 3, MATH: 4, MuSR: 0; others 0-shot"}. This record does not assert independent reproduction. ### GSA MMLU-P Accuracy: 11. Result ID: 699a50a75698665ec4d83784. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model Parameters":"1.3B","Training Tokens":"100B","Shots":"MMLU-P: 5, GPQA: 0, BBH: 3, MATH: 4, MuSR: 0; others 0-shot"}. This record does not assert independent reproduction. ### RetNet MMLU-P Accuracy: 11. Result ID: 699a50a75698665ec4d83785. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Parameters":"1.3B","Training Tokens":"100B","Shots":"MMLU-P: 5, GPQA: 0, BBH: 3, MATH: 4, MuSR: 0; others 0-shot"}. This record does not assert independent reproduction. ### GLA MMLU-P Accuracy: 11. Result ID: 699a50a75698665ec4d8378e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Parameters":"340M","Training Tokens":"15B","Shots":"MMLU-P: 5, GPQA: 0, BBH: 3, MATH: 4, MuSR: 0; others 0-shot"}. This record does not assert independent reproduction. ### DeltaNet MMLU-P Accuracy: 10.9. Result ID: 699a50a75698665ec4d83783. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model Parameters":"1.3B","Training Tokens":"100B","Shots":"MMLU-P: 5, GPQA: 0, BBH: 3, MATH: 4, MuSR: 0; others 0-shot"}. This record does not assert independent reproduction. ### DiffTrans MMLU-P Accuracy: 10.9. Result ID: 699a50a75698665ec4d8378b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Parameters":"340M","Training Tokens":"15B","Shots":"MMLU-P: 5, GPQA: 0, BBH: 3, MATH: 4, MuSR: 0; others 0-shot"}. This record does not assert independent reproduction. ### SoftmaxAttn MMLU-P Accuracy: 10.6. Result ID: 699a50a75698665ec4d8378f. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model Parameters":"340M","Training Tokens":"15B","Shots":"MMLU-P: 5, GPQA: 0, BBH: 3, MATH: 4, MuSR: 0; others 0-shot"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.