# General Language Model Capability on MMLU, GSM8K, HumanEval, and BBH Aggregate

Source: [General Language Model Capability on MMLU, GSM8K, HumanEval, and BBH Aggregate](<https://www.sota2.com/research/sota/general-language-model-capability-on-mmlu-gsm8k-humaneval-and-bbh-aggregate>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-03-24T12:34:37.723Z

Aggregate evaluation assessing general language model capabilities across MMLU, GSM8K, HumanEval, and BigBench Hard (BBH).

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | General Language Model Capability |
| Dataset (including any stated split) | MMLU, GSM8K, HumanEval, BBH Combined |
| Metric | Average Score |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 8 of 8 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Average Score](<https://www.sota2.com/research/sota/general-language-model-capability-on-mmlu-gsm8k-humaneval-and-bbh-aggregate.md?metric=Average%20Score>): Higher is better; unit: PERCENT.

## Recorded results

| Method | Average Score | Result date | Report paper |
| --- | --- | --- | --- |
| VAR | 68.42 | Not recorded | 68fa770d1e209131cea70cd6 |
| ALoL | 62.4 | Not recorded | 68fa770d1e209131cea70cd6 |
| Base | 61.8 | Not recorded | 68fa770d1e209131cea70cd6 |
| DPO | 57.44 | Not recorded | 68fa770d1e209131cea70cd6 |
| VAR | 24.2 | Not recorded | 68fa770d1e209131cea70cd6 |
| ALoL | 22.71 | Not recorded | 68fa770d1e209131cea70cd6 |
| DPO | 20.86 | Not recorded | 68fa770d1e209131cea70cd6 |
| Base | 13.79 | Not recorded | 68fa770d1e209131cea70cd6 |

### VAR

Average Score: 68.42. Result ID: 69c283a547f34ce924aa7492.

[Reporting paper](<https://www.sota2.com/research/paper/68fa770d1e209131cea70cd6.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa770d1e209131cea70cd6.md>)

Recorded evaluation configuration: {"Base Model":"Qwen2.5-7B"}. This record does not assert independent reproduction.

### ALoL

Average Score: 62.4. Result ID: 69c283a547f34ce924aa7491.

[Reporting paper](<https://www.sota2.com/research/paper/68fa770d1e209131cea70cd6.md>)

Recorded evaluation configuration: {"Base Model":"Qwen2.5-7B"}. This record does not assert independent reproduction.

### Base

Average Score: 61.8. Result ID: 69c283a547f34ce924aa748f.

[Reporting paper](<https://www.sota2.com/research/paper/68fa770d1e209131cea70cd6.md>)

Recorded evaluation configuration: {"Base Model":"Qwen2.5-7B"}. This record does not assert independent reproduction.

### DPO

Average Score: 57.44. Result ID: 69c283a547f34ce924aa7490.

[Reporting paper](<https://www.sota2.com/research/paper/68fa770d1e209131cea70cd6.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa71081e209131cea4ebc1.md>)

[Implementation](<https://github.com/huggingface/alignment-handbook>)

Recorded evaluation configuration: {"Base Model":"Qwen2.5-7B"}. This record does not assert independent reproduction.

### VAR

Average Score: 24.2. Result ID: 69c283a547f34ce924aa748e.

[Reporting paper](<https://www.sota2.com/research/paper/68fa770d1e209131cea70cd6.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa770d1e209131cea70cd6.md>)

Recorded evaluation configuration: {"Base Model":"Llama2-7B"}. This record does not assert independent reproduction.

### ALoL

Average Score: 22.71. Result ID: 69c283a547f34ce924aa748d.

[Reporting paper](<https://www.sota2.com/research/paper/68fa770d1e209131cea70cd6.md>)

Recorded evaluation configuration: {"Base Model":"Llama2-7B"}. This record does not assert independent reproduction.

### DPO

Average Score: 20.86. Result ID: 69c283a547f34ce924aa748c.

[Reporting paper](<https://www.sota2.com/research/paper/68fa770d1e209131cea70cd6.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa71081e209131cea4ebc1.md>)

[Implementation](<https://github.com/huggingface/alignment-handbook>)

Recorded evaluation configuration: {"Base Model":"Llama2-7B"}. This record does not assert independent reproduction.

### Base

Average Score: 13.79. Result ID: 69c283a547f34ce924aa748b.

[Reporting paper](<https://www.sota2.com/research/paper/68fa770d1e209131cea70cd6.md>)

Recorded evaluation configuration: {"Base Model":"Llama2-7B"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.