# Multi-discipline Evaluation on MMLU-Pro

Source: [Multi-discipline Evaluation on MMLU-Pro](<https://www.sota2.com/research/sota/multi-discipline-evaluation-on-mmlu-pro>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-06-08T06:44:00.959Z

Evaluation of model performance across 13 specific disciplines (Math, Physics, Chemistry, Law, etc.) within the MMLU-Pro benchmark.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Multi-discipline Evaluation |
| Dataset (including any stated split) | MMLU-Pro |
| Metric | Math Accuracy |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 6 of 6 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Math Accuracy](<https://www.sota2.com/research/sota/multi-discipline-evaluation-on-mmlu-pro.md?metric=Math%20Accuracy>): Higher is better; unit: PERCENT.

- [Physics Accuracy](<https://www.sota2.com/research/sota/multi-discipline-evaluation-on-mmlu-pro.md?metric=Physics%20Accuracy>): Higher is better; unit: PERCENT.

- [Chemistry Accuracy](<https://www.sota2.com/research/sota/multi-discipline-evaluation-on-mmlu-pro.md?metric=Chemistry%20Accuracy>): Higher is better; unit: PERCENT.

- [Law Accuracy](<https://www.sota2.com/research/sota/multi-discipline-evaluation-on-mmlu-pro.md?metric=Law%20Accuracy>): Higher is better; unit: PERCENT.

- [Engineering Accuracy](<https://www.sota2.com/research/sota/multi-discipline-evaluation-on-mmlu-pro.md?metric=Engineering%20Accuracy>): Higher is better; unit: PERCENT.

- [Other Disciplines Accuracy](<https://www.sota2.com/research/sota/multi-discipline-evaluation-on-mmlu-pro.md?metric=Other%20Disciplines%20Accuracy>): Higher is better; unit: PERCENT.

- [Economics Accuracy](<https://www.sota2.com/research/sota/multi-discipline-evaluation-on-mmlu-pro.md?metric=Economics%20Accuracy>): Higher is better; unit: PERCENT.

- [Health Sciences Accuracy](<https://www.sota2.com/research/sota/multi-discipline-evaluation-on-mmlu-pro.md?metric=Health%20Sciences%20Accuracy>): Higher is better; unit: PERCENT.

- [Psychology Accuracy](<https://www.sota2.com/research/sota/multi-discipline-evaluation-on-mmlu-pro.md?metric=Psychology%20Accuracy>): Higher is better; unit: PERCENT.

- [Business Accuracy](<https://www.sota2.com/research/sota/multi-discipline-evaluation-on-mmlu-pro.md?metric=Business%20Accuracy>): Higher is better; unit: PERCENT.

- [Biology Accuracy](<https://www.sota2.com/research/sota/multi-discipline-evaluation-on-mmlu-pro.md?metric=Biology%20Accuracy>): Higher is better; unit: PERCENT.

- [Philosophy Accuracy](<https://www.sota2.com/research/sota/multi-discipline-evaluation-on-mmlu-pro.md?metric=Philosophy%20Accuracy>): Higher is better; unit: PERCENT.

- [History Accuracy](<https://www.sota2.com/research/sota/multi-discipline-evaluation-on-mmlu-pro.md?metric=History%20Accuracy>): Higher is better; unit: PERCENT.

## Recorded results

| Method | Math Accuracy | Result date | Report paper |
| --- | --- | --- | --- |
| POLAR | 32.1 | Not recorded | 6a264c8490966ade919be94a |
| Base (τ=0) | 26.3 | Not recorded | 6a264c8490966ade919be94a |
| FlexiDepth | 9.8 | Not recorded | 6a264c8490966ade919be94a |
| ShortGPT | 8.9 | Not recorded | 6a264c8490966ade919be94a |
| MindSkip | 7.6 | Not recorded | 6a264c8490966ade919be94a |
| DR.LLM | 4 | Not recorded | 6a264c8490966ade919be94a |

### POLAR

Math Accuracy: 32.1. Result ID: 6a2664abfbc373592c228357.

[Reporting paper](<https://www.sota2.com/research/paper/6a264c8490966ade919be94a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a264c8490966ade919be94a.md>)

[Implementation](<https://github.com/tianyi-lab/PoLar>)

Recorded evaluation configuration: {"Backbone":"Qwen2.5-3B-Instruct","Evaluation Protocol":"pass@1"}. This record does not assert independent reproduction.

### Base (τ=0)

Math Accuracy: 26.3. Result ID: 6a2664abfbc373592c228352.

[Reporting paper](<https://www.sota2.com/research/paper/6a264c8490966ade919be94a.md>)

Recorded evaluation configuration: {"Backbone":"Qwen2.5-3B-Instruct","Evaluation Protocol":"pass@1"}. This record does not assert independent reproduction.

### FlexiDepth

Math Accuracy: 9.8. Result ID: 6a2664abfbc373592c228355.

[Reporting paper](<https://www.sota2.com/research/paper/6a264c8490966ade919be94a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7c001e209131cea8bd0b.md>)

Recorded evaluation configuration: {"Backbone":"Qwen2.5-3B-Instruct","Evaluation Protocol":"pass@1"}. This record does not assert independent reproduction.

### ShortGPT

Math Accuracy: 8.9. Result ID: 6a2664abfbc373592c228353.

[Reporting paper](<https://www.sota2.com/research/paper/6a264c8490966ade919be94a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa75391e209131cea668f7.md>)

Recorded evaluation configuration: {"Backbone":"Qwen2.5-3B-Instruct","Evaluation Protocol":"pass@1"}. This record does not assert independent reproduction.

### MindSkip

Math Accuracy: 7.6. Result ID: 6a2664abfbc373592c228354.

[Reporting paper](<https://www.sota2.com/research/paper/6a264c8490966ade919be94a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7b3e1e209131cea87448.md>)

Recorded evaluation configuration: {"Backbone":"Qwen2.5-3B-Instruct","Evaluation Protocol":"pass@1"}. This record does not assert independent reproduction.

### DR.LLM

Math Accuracy: 4. Result ID: 6a2664abfbc373592c228356.

[Reporting paper](<https://www.sota2.com/research/paper/6a264c8490966ade919be94a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7c1c1e209131cea8c690.md>)

[Implementation](<https://github.com/parameterlab/dr-llm>)

Recorded evaluation configuration: {"Backbone":"Qwen2.5-3B-Instruct","Evaluation Protocol":"pass@1"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.
