# Word Prediction on LAMBADA OpenAI Source: [Word Prediction on LAMBADA OpenAI]() Collection index: [llms.txt]() Data updated: 2026-05-12T09:26:25.301Z Evaluation of language model performance on the LAMBADA OpenAI dataset, measured using prediction accuracy. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Word Prediction | | Dataset (including any stated split) | LAMBADA OpenAI | | Metric | Accuracy | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 29 of 29 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Accuracy](): Higher is better; unit: PERCENT. ## Recorded results | Method | Accuracy | Result date | Report paper | | --- | --- | --- | --- | | Baseline | 71.4 | Not recorded | 69d732b87d0172140ec76cef | | MoBiE | 71.05 | Not recorded | 69d732b87d0172140ec76cef | | Baseline | 70.83 | Not recorded | 69d732b87d0172140ec76cef | | MxMoE | 68.68 | Not recorded | 69d732b87d0172140ec76cef | | GPTQ | 62.39 | Not recorded | 69d732b87d0172140ec76cef | | MxMoE | 56.43 | Not recorded | 69d732b87d0172140ec76cef | | NoWag | 50.46 | Not recorded | 69d732b87d0172140ec76cef | | MoBiE | 50.18 | Not recorded | 69d732b87d0172140ec76cef | | Qwen | 49.31 | Not recorded | 6992a4b59ebc499c97406d91 | | SD-MoE | 47.29 | Not recorded | 6992a4b59ebc499c97406d91 | | GPTQ | 46.18 | Not recorded | 69d732b87d0172140ec76cef | | SD-MoE | 42.97 | Not recorded | 6992a4b59ebc499c97406d91 | | SD-MoE | 40.21 | Not recorded | 6992a4b59ebc499c97406d91 | | DeepSeek | 39.63 | Not recorded | 6992a4b59ebc499c97406d91 | | Qwen | 35.09 | Not recorded | 6992a4b59ebc499c97406d91 | | QuantMoe-Bench | 34.74 | Not recorded | 69d732b87d0172140ec76cef | | NoWag | 28.9 | Not recorded | 69d732b87d0172140ec76cef | | MoEQuant | 22.98 | Not recorded | 69d732b87d0172140ec76cef | | QuantMoe-Bench | 20.12 | Not recorded | 69d732b87d0172140ec76cef | | Transformer | 15.13 | Not recorded | 6a02b435dbc836b87d824839 | | MoEQuant | 12.03 | Not recorded | 69d732b87d0172140ec76cef | | AWQ | 5.21 | Not recorded | 69d732b87d0172140ec76cef | | AWQ | 3.29 | Not recorded | 69d732b87d0172140ec76cef | | SRM | 3.05 | Not recorded | 6a02b435dbc836b87d824839 | | Mamba | 1.01 | Not recorded | 6a02b435dbc836b87d824839 | | AWQ | 0 | Not recorded | 69d732b87d0172140ec76cef | | GPTQ | 0 | Not recorded | 69d732b87d0172140ec76cef | | AWQ | 0 | Not recorded | 69d732b87d0172140ec76cef | | GPTQ | 0 | Not recorded | 69d732b87d0172140ec76cef | ### Baseline Accuracy: 71.4. Result ID: 69d8432547f34ce924c054b8. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Mixtral","Weight Bitwidth":"16"}. This record does not assert independent reproduction. ### MoBiE Accuracy: 71.05. Result ID: 69d8432547f34ce924c054c1. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Mixtral","Weight Bitwidth":"1.60"}. This record does not assert independent reproduction. ### Baseline Accuracy: 70.83. Result ID: 69d8432547f34ce924c054ae. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen1.5-MoE","Weight Bitwidth":"16"}. This record does not assert independent reproduction. ### MxMoE Accuracy: 68.68. Result ID: 69d8432547f34ce924c054c0. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Mixtral","Weight Bitwidth":"2.25"}. This record does not assert independent reproduction. ### GPTQ Accuracy: 62.39. Result ID: 69d8432547f34ce924c054ba. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Mixtral","Weight Bitwidth":"3"}. This record does not assert independent reproduction. ### MxMoE Accuracy: 56.43. Result ID: 69d8432547f34ce924c054b6. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen1.5-MoE","Weight Bitwidth":"2.25"}. This record does not assert independent reproduction. ### NoWag Accuracy: 50.46. Result ID: 69d8432547f34ce924c054be. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Mixtral","Weight Bitwidth":"2.08"}. This record does not assert independent reproduction. ### MoBiE Accuracy: 50.18. Result ID: 69d8432547f34ce924c054b7. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen1.5-MoE","Weight Bitwidth":"1.35"}. This record does not assert independent reproduction. ### Qwen Accuracy: 49.31. Result ID: 699a502d5698665ec4d7be75. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Scale":"Large (7B-1.5B)","Base Model":"Qwen","Method Configuration":"Baseline"}. This record does not assert independent reproduction. ### SD-MoE Accuracy: 47.29. Result ID: 699a502d5698665ec4d7be76. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Scale":"Large (7B-1.5B)","Base Model":"Qwen","Method Configuration":"SD-MoE"}. This record does not assert independent reproduction. ### GPTQ Accuracy: 46.18. Result ID: 69d8432547f34ce924c054b0. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen1.5-MoE","Weight Bitwidth":"3"}. This record does not assert independent reproduction. ### SD-MoE Accuracy: 42.97. Result ID: 699a502d5698665ec4d7be74. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Scale":"Small (2B-0.8B)","Base Model":"Qwen","Method Configuration":"SD-MoE"}. This record does not assert independent reproduction. ### SD-MoE Accuracy: 40.21. Result ID: 699a502d5698665ec4d7be72. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Scale":"Small (2B-0.8B)","Base Model":"DeepSeek","Method Configuration":"SD-MoE"}. This record does not assert independent reproduction. ### DeepSeek Accuracy: 39.63. Result ID: 699a502d5698665ec4d7be71. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model Scale":"Small (2B-0.8B)","Base Model":"DeepSeek","Method Configuration":"Baseline"}. This record does not assert independent reproduction. ### Qwen Accuracy: 35.09. Result ID: 699a502d5698665ec4d7be73. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Scale":"Small (2B-0.8B)","Base Model":"Qwen","Method Configuration":"Baseline"}. This record does not assert independent reproduction. ### QuantMoe-Bench Accuracy: 34.74. Result ID: 69d8432547f34ce924c054bf. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Mixtral","Weight Bitwidth":"2.3"}. This record does not assert independent reproduction. ### NoWag Accuracy: 28.9. Result ID: 69d8432547f34ce924c054b4. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen1.5-MoE","Weight Bitwidth":"2.08"}. This record does not assert independent reproduction. ### MoEQuant Accuracy: 22.98. Result ID: 69d8432547f34ce924c054bd. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Mixtral","Weight Bitwidth":"2"}. This record does not assert independent reproduction. ### QuantMoe-Bench Accuracy: 20.12. Result ID: 69d8432547f34ce924c054b5. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen1.5-MoE","Weight Bitwidth":"2.3"}. This record does not assert independent reproduction. ### Transformer Accuracy: 15.13. Result ID: 6a02f22547f34ce924e5ff8c. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Training":"compute-matched","Training dataset":"FineWeb-edu"}. This record does not assert independent reproduction. ### MoEQuant Accuracy: 12.03. Result ID: 69d8432547f34ce924c054b3. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen1.5-MoE","Weight Bitwidth":"2"}. This record does not assert independent reproduction. ### AWQ Accuracy: 5.21. Result ID: 69d8432547f34ce924c054b9. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Mixtral","Weight Bitwidth":"3"}. This record does not assert independent reproduction. ### AWQ Accuracy: 3.29. Result ID: 69d8432547f34ce924c054af. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen1.5-MoE","Weight Bitwidth":"3"}. This record does not assert independent reproduction. ### SRM Accuracy: 3.05. Result ID: 6a02f22547f34ce924e5ff8e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Training":"compute-matched","Training dataset":"FineWeb-edu"}. This record does not assert independent reproduction. ### Mamba Accuracy: 1.01. Result ID: 6a02f22547f34ce924e5ff8d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Training":"compute-matched","Training dataset":"FineWeb-edu"}. This record does not assert independent reproduction. ### AWQ Accuracy: 0. Result ID: 69d8432547f34ce924c054b1. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen1.5-MoE","Weight Bitwidth":"2"}. This record does not assert independent reproduction. ### GPTQ Accuracy: 0. Result ID: 69d8432547f34ce924c054b2. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen1.5-MoE","Weight Bitwidth":"2"}. This record does not assert independent reproduction. ### AWQ Accuracy: 0. Result ID: 69d8432547f34ce924c054bb. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Mixtral","Weight Bitwidth":"2"}. This record does not assert independent reproduction. ### GPTQ Accuracy: 0. Result ID: 69d8432547f34ce924c054bc. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Mixtral","Weight Bitwidth":"2"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.