# Classification on MMLU (test) Source: [Classification on MMLU (test)]() Collection index: [llms.txt]() Data updated: 2026-02-26T05:34:02.404Z Evaluation of classification performance on the test split of the MMLU benchmark, reporting 0-shot and 5-shot accuracy results. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Classification | | Dataset (including any stated split) | MMLU (test) | | Metric | Accuracy (5-shot) | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 31 of 31 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Accuracy (5-shot)](): Higher is better; unit: PERCENT. - [Accuracy (0-shot)](): Higher is better; unit: PERCENT. - [Accuracy](): Higher is better; unit: PERCENT. ## Recorded results | Method | Accuracy (5-shot) | Result date | Report paper | | --- | --- | --- | --- | | OpenAI code-davinci-002 | 77.4 | Not recorded | 68fa6e791e209131cea401cb | | OpenAI text-davinci-003 | 74.2 | Not recorded | 68fa6e791e209131cea401cb | | FLAN-PaLM 540B | 73.5 | Not recorded | 68fa6e791e209131cea401cb | | PaLM 540B | 71.3 | Not recorded | 68fa6e791e209131cea401cb | | OpenAI text-davinci-002 | 64.5 | Not recorded | 68fa6e791e209131cea401cb | | FLAN-PaLM 62B | 59.6 | Not recorded | 68fa6e791e209131cea401cb | | PaLM 62B | 55.1 | Not recorded | 68fa6e791e209131cea401cb | | FLAN-T5 11B | 54.9 | Not recorded | 68fa6e791e209131cea401cb | | OPT-IML-Max 175B | 47.1 | Not recorded | 68fa6e791e209131cea401cb | | OPT-IML-Max 30B | 43.2 | Not recorded | 68fa6e791e209131cea401cb | | OPT 175B | 34.2 | Not recorded | 68fa6e791e209131cea401cb | | T0pp 11B | 33.7 | Not recorded | 68fa6e791e209131cea401cb | | OpenAI davinci | 32.3 | Not recorded | 68fa6e791e209131cea401cb | | OPT-IML-Max 1.3B | 29.5 | Not recorded | 68fa6e791e209131cea401cb | | OPT 30B | 26.1 | Not recorded | 68fa6e791e209131cea401cb | | OPT 1.3B | 25.9 | Not recorded | 68fa6e791e209131cea401cb | | T5 11B | 25.9 | Not recorded | 68fa6e791e209131cea401cb | ### OpenAI code-davinci-002 Accuracy (5-shot): 77.4. Result ID: 699023f73a6c314849fe64ff. [Reporting paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### OpenAI text-davinci-003 Accuracy (5-shot): 74.2. Result ID: 699023f73a6c314849fe64fe. [Reporting paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### FLAN-PaLM 540B Accuracy (5-shot): 73.5. Result ID: 699023f73a6c314849fe64fc. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### PaLM 540B Accuracy (5-shot): 71.3. Result ID: 699023f73a6c314849fe64f4. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### OpenAI text-davinci-002 Accuracy (5-shot): 64.5. Result ID: 699023f73a6c314849fe64fd. [Reporting paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### FLAN-PaLM 62B Accuracy (5-shot): 59.6. Result ID: 699023f73a6c314849fe64fb. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### PaLM 62B Accuracy (5-shot): 55.1. Result ID: 699023f73a6c314849fe64f3. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### FLAN-T5 11B Accuracy (5-shot): 54.9. Result ID: 699023f73a6c314849fe64fa. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### OPT-IML-Max 175B Accuracy (5-shot): 47.1. Result ID: 699023f73a6c314849fe64f8. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### OPT-IML-Max 30B Accuracy (5-shot): 43.2. Result ID: 699023f73a6c314849fe64f7. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### OPT 175B Accuracy (5-shot): 34.2. Result ID: 699023f73a6c314849fe64f1. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### T0pp 11B Accuracy (5-shot): 33.7. Result ID: 699023f73a6c314849fe64f9. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### OpenAI davinci Accuracy (5-shot): 32.3. Result ID: 699023f73a6c314849fe64f5. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### OPT-IML-Max 1.3B Accuracy (5-shot): 29.5. Result ID: 699023f73a6c314849fe64f6. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### OPT 30B Accuracy (5-shot): 26.1. Result ID: 699023f73a6c314849fe64f0. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### OPT 1.3B Accuracy (5-shot): 25.9. Result ID: 699023f73a6c314849fe64ef. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### T5 11B Accuracy (5-shot): 25.9. Result ID: 699023f73a6c314849fe64f2. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.