# Multitask Evaluation on Aggregate (MMLU, GSM8K, SQuAD V2, Amazon, 20News) Source: [Multitask Evaluation on Aggregate (MMLU, GSM8K, SQuAD V2, Amazon, 20News)]() Collection index: [llms.txt]() Data updated: 2026-04-23T08:20:08.548Z Aggregate performance evaluation across a suite of knowledge, reasoning, and QA benchmarks including MMLU, GSM8K, SQuAD V2, Amazon reviews, and 20News. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Multitask Evaluation | | Dataset (including any stated split) | Aggregate (MMLU, GSM8K, SQuAD V2, Amazon, 20News) | | Metric | Average Score | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 16 of 16 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Average Score](): Higher is better; unit: PERCENT. ## Recorded results | Method | Average Score | Result date | Report paper | | --- | --- | --- | --- | | ShadowPEFT w/ pretrained shadow | 77.11 | Not recorded | 69e85bae7d0172140ec80fff | | ShadowPEFT | 76.92 | Not recorded | 69e85bae7d0172140ec80fff | | ShadowPEFT w/ random shadow | 76.6 | Not recorded | 69e85bae7d0172140ec80fff | | LoRA | 76.51 | Not recorded | 69e85bae7d0172140ec80fff | | DoRA | 75.99 | Not recorded | 69e85bae7d0172140ec80fff | | ShadowPEFT | 75.43 | Not recorded | 69e85bae7d0172140ec80fff | | DoRA | 74.85 | Not recorded | 69e85bae7d0172140ec80fff | | LoRA | 74.55 | Not recorded | 69e85bae7d0172140ec80fff | | ShadowPEFT | 62.27 | Not recorded | 69e85bae7d0172140ec80fff | | Detached Shadow Only (pretrained) | 62.11 | Not recorded | 69e85bae7d0172140ec80fff | | DoRA | 62.08 | Not recorded | 69e85bae7d0172140ec80fff | | LoRA | 61.81 | Not recorded | 69e85bae7d0172140ec80fff | | Detached Shadow Only | 38.4 | Not recorded | 69e85bae7d0172140ec80fff | | Detached Shadow Only | 36.79 | Not recorded | 69e85bae7d0172140ec80fff | | Detached Shadow Only | 36.09 | Not recorded | 69e85bae7d0172140ec80fff | | Detached Shadow Only (random) | 35.11 | Not recorded | 69e85bae7d0172140ec80fff | ### ShadowPEFT w/ pretrained shadow Average Score: 77.11. Result ID: 69e9d63147f34ce924d0eaa7. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Trainable Params":"454,991,872","Backbone Model":"Qwen3 8B + 0.5B Shadow","Inference Mode":"Shadow-attached","Shadow Model Initialization":"pretrained"}. This record does not assert independent reproduction. ### ShadowPEFT Average Score: 76.92. Result ID: 69e9d63147f34ce924d0eaa3. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Trainable Params":"29,118,720","Backbone Model":"Qwen3 8B","Inference Mode":"Shadow-attached"}. This record does not assert independent reproduction. ### ShadowPEFT w/ random shadow Average Score: 76.6. Result ID: 69e9d63147f34ce924d0eaa5. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Trainable Params":"454,991,872","Backbone Model":"Qwen3 8B + 0.5B Shadow","Inference Mode":"Shadow-attached","Shadow Model Initialization":"random"}. This record does not assert independent reproduction. ### LoRA Average Score: 76.51. Result ID: 69e9d63147f34ce924d0eaa1. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Trainable Params":"30,670,848","Backbone Model":"Qwen3 8B"}. This record does not assert independent reproduction. ### DoRA Average Score: 75.99. Result ID: 69e9d63147f34ce924d0eaa2. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Trainable Params":"31,039,488","Backbone Model":"Qwen3 8B"}. This record does not assert independent reproduction. ### ShadowPEFT Average Score: 75.43. Result ID: 69e9d63147f34ce924d0ea9f. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Trainable Params":"23,442,176","Backbone Model":"Qwen3 4B","Inference Mode":"Shadow-attached"}. This record does not assert independent reproduction. ### DoRA Average Score: 74.85. Result ID: 69e9d63147f34ce924d0ea9e. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Trainable Params":"23,906,304","Backbone Model":"Qwen3 4B"}. This record does not assert independent reproduction. ### LoRA Average Score: 74.55. Result ID: 69e9d63147f34ce924d0ea9d. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Trainable Params":"23,592,960","Backbone Model":"Qwen3 4B"}. This record does not assert independent reproduction. ### ShadowPEFT Average Score: 62.27. Result ID: 69e9d63147f34ce924d0ea9b. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Trainable Params":"9,073,920","Backbone Model":"Qwen3 0.6B","Inference Mode":"Shadow-attached"}. This record does not assert independent reproduction. ### Detached Shadow Only (pretrained) Average Score: 62.11. Result ID: 69e9d63147f34ce924d0eaa8. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Trainable Params":"454,991,872","Backbone Model":"Qwen3 8B + 0.5B Shadow","Inference Mode":"Detached","Shadow Model Initialization":"pretrained"}. This record does not assert independent reproduction. ### DoRA Average Score: 62.08. Result ID: 69e9d63147f34ce924d0ea9a. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Trainable Params":"9,318,400","Backbone Model":"Qwen3 0.6B"}. This record does not assert independent reproduction. ### LoRA Average Score: 61.81. Result ID: 69e9d63147f34ce924d0ea99. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Trainable Params":"9,175,040","Backbone Model":"Qwen3 0.6B"}. This record does not assert independent reproduction. ### Detached Shadow Only Average Score: 38.4. Result ID: 69e9d63147f34ce924d0eaa0. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Trainable Params":"23,442,176","Backbone Model":"Qwen3 4B","Inference Mode":"Detached"}. This record does not assert independent reproduction. ### Detached Shadow Only Average Score: 36.79. Result ID: 69e9d63147f34ce924d0ea9c. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Trainable Params":"9,073,920","Backbone Model":"Qwen3 0.6B","Inference Mode":"Detached"}. This record does not assert independent reproduction. ### Detached Shadow Only Average Score: 36.09. Result ID: 69e9d63147f34ce924d0eaa4. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Trainable Params":"29,118,720","Backbone Model":"Qwen3 8B","Inference Mode":"Detached"}. This record does not assert independent reproduction. ### Detached Shadow Only (random) Average Score: 35.11. Result ID: 69e9d63147f34ce924d0eaa6. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Trainable Params":"454,991,872","Backbone Model":"Qwen3 8B + 0.5B Shadow","Inference Mode":"Detached","Shadow Model Initialization":"random"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.