# Large Language Model Evaluation on ARC, TruthfulQA, Winogrande, GSM8K, HellaSwag, and MMLU

Source: [Large Language Model Evaluation on ARC, TruthfulQA, Winogrande, GSM8K, HellaSwag, and MMLU](<https://www.sota2.com/research/sota/large-language-model-evaluation-on-arc-truthfulqa-winogrande-gsm8k-hellaswag-and-mmlu>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-03-17T09:12:09.908Z

Comprehensive evaluation of large language models covering reasoning (GSM8K, ARC), common sense (HellaSwag, Winogrande), knowledge (MMLU), and truthfulness (TruthfulQA).

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Large Language Model Evaluation |
| Dataset (including any stated split) | ARC, TruthfulQA, Winogrande, GSM8K, HellaSwag, MMLU |
| Metric | ARC Accuracy |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 16 of 16 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [ARC Accuracy](<https://www.sota2.com/research/sota/large-language-model-evaluation-on-arc-truthfulqa-winogrande-gsm8k-hellaswag-and-mmlu.md?metric=ARC%20Accuracy>): Higher is better; unit: PERCENT.

- [TruthfulQA Score](<https://www.sota2.com/research/sota/large-language-model-evaluation-on-arc-truthfulqa-winogrande-gsm8k-hellaswag-and-mmlu.md?metric=TruthfulQA%20Score>): Higher is better; unit: PERCENT.

- [Winogrande Score](<https://www.sota2.com/research/sota/large-language-model-evaluation-on-arc-truthfulqa-winogrande-gsm8k-hellaswag-and-mmlu.md?metric=Winogrande%20Score>): Higher is better; unit: PERCENT.

- [GSM8K Accuracy](<https://www.sota2.com/research/sota/large-language-model-evaluation-on-arc-truthfulqa-winogrande-gsm8k-hellaswag-and-mmlu.md?metric=GSM8K%20Accuracy>): Higher is better; unit: PERCENT.

- [HellaSwag Score](<https://www.sota2.com/research/sota/large-language-model-evaluation-on-arc-truthfulqa-winogrande-gsm8k-hellaswag-and-mmlu.md?metric=HellaSwag%20Score>): Higher is better; unit: PERCENT.

- [MMLU Accuracy](<https://www.sota2.com/research/sota/large-language-model-evaluation-on-arc-truthfulqa-winogrande-gsm8k-hellaswag-and-mmlu.md?metric=MMLU%20Accuracy>): Higher is better; unit: PERCENT.

- [Average Score](<https://www.sota2.com/research/sota/large-language-model-evaluation-on-arc-truthfulqa-winogrande-gsm8k-hellaswag-and-mmlu.md?metric=Average%20Score>): Higher is better; unit: PERCENT.

## Recorded results

| Method | ARC Accuracy | Result date | Report paper |
| --- | --- | --- | --- |
| DNPO | 73.7 | Not recorded | 68fa77991e209131cea7445c |
| DNPO | 73.5 | Not recorded | 68fa77991e209131cea7445c |
| DNPO | 73.4 | Not recorded | 68fa77991e209131cea7445c |
| SPIN | 70.9 | Not recorded | 68fa77991e209131cea7445c |
| SPIN | 70.7 | Not recorded | 68fa77991e209131cea7445c |
| SFT | 70.4 | Not recorded | 68fa77991e209131cea7445c |
| SPIN | 70.3 | Not recorded | 68fa77991e209131cea7445c |
| SPIN | 70.2 | Not recorded | 68fa77991e209131cea7445c |
| DNPO | 64.6 | Not recorded | 68fa77991e209131cea7445c |
| DNPO | 64.6 | Not recorded | 68fa77991e209131cea7445c |
| DNPO | 64.3 | Not recorded | 68fa77991e209131cea7445c |
| SPIN | 63.8 | Not recorded | 68fa77991e209131cea7445c |
| SPIN | 63.7 | Not recorded | 68fa77991e209131cea7445c |
| SPIN | 63.6 | Not recorded | 68fa77991e209131cea7445c |
| SPIN | 63.5 | Not recorded | 68fa77991e209131cea7445c |
| SFT | 63.4 | Not recorded | 68fa77991e209131cea7445c |

### DNPO

ARC Accuracy: 73.7. Result ID: 69b91ada47f34ce924a1ed56.

[Reporting paper](<https://www.sota2.com/research/paper/68fa77991e209131cea7445c.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa77991e209131cea7445c.md>)

Recorded evaluation configuration: {"Base Model":"Zephyr-7B","Iteration":"3"}. This record does not assert independent reproduction.

### DNPO

ARC Accuracy: 73.5. Result ID: 69b91ada47f34ce924a1ed54.

[Reporting paper](<https://www.sota2.com/research/paper/68fa77991e209131cea7445c.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa77991e209131cea7445c.md>)

Recorded evaluation configuration: {"Base Model":"Zephyr-7B","Iteration":"2"}. This record does not assert independent reproduction.

### DNPO

ARC Accuracy: 73.4. Result ID: 69b91ada47f34ce924a1ed52.

[Reporting paper](<https://www.sota2.com/research/paper/68fa77991e209131cea7445c.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa77991e209131cea7445c.md>)

Recorded evaluation configuration: {"Base Model":"Zephyr-7B","Iteration":"1"}. This record does not assert independent reproduction.

### SPIN

ARC Accuracy: 70.9. Result ID: 69b91ada47f34ce924a1ed50.

[Reporting paper](<https://www.sota2.com/research/paper/68fa77991e209131cea7445c.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73b11e209131cea5defb.md>)

Recorded evaluation configuration: {"Base Model":"Zephyr-7B","Iteration":"0"}. This record does not assert independent reproduction.

### SPIN

ARC Accuracy: 70.7. Result ID: 69b91ada47f34ce924a1ed53.

[Reporting paper](<https://www.sota2.com/research/paper/68fa77991e209131cea7445c.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73b11e209131cea5defb.md>)

Recorded evaluation configuration: {"Base Model":"Zephyr-7B","Iteration":"2"}. This record does not assert independent reproduction.

### SFT

ARC Accuracy: 70.4. Result ID: 69b91ada47f34ce924a1ed4f.

[Reporting paper](<https://www.sota2.com/research/paper/68fa77991e209131cea7445c.md>)

Recorded evaluation configuration: {"Base Model":"Zephyr-7B","Iteration":"SFT"}. This record does not assert independent reproduction.

### SPIN

ARC Accuracy: 70.3. Result ID: 69b91ada47f34ce924a1ed55.

[Reporting paper](<https://www.sota2.com/research/paper/68fa77991e209131cea7445c.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73b11e209131cea5defb.md>)

Recorded evaluation configuration: {"Base Model":"Zephyr-7B","Iteration":"3"}. This record does not assert independent reproduction.

### SPIN

ARC Accuracy: 70.2. Result ID: 69b91ada47f34ce924a1ed51.

[Reporting paper](<https://www.sota2.com/research/paper/68fa77991e209131cea7445c.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73b11e209131cea5defb.md>)

Recorded evaluation configuration: {"Base Model":"Zephyr-7B","Iteration":"1"}. This record does not assert independent reproduction.

### DNPO

ARC Accuracy: 64.6. Result ID: 69b91ada47f34ce924a1ed4c.

[Reporting paper](<https://www.sota2.com/research/paper/68fa77991e209131cea7445c.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa77991e209131cea7445c.md>)

Recorded evaluation configuration: {"Base Model":"Llama-3.2-3B","Iteration":"2"}. This record does not assert independent reproduction.

### DNPO

ARC Accuracy: 64.6. Result ID: 69b91ada47f34ce924a1ed4e.

[Reporting paper](<https://www.sota2.com/research/paper/68fa77991e209131cea7445c.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa77991e209131cea7445c.md>)

Recorded evaluation configuration: {"Base Model":"Llama-3.2-3B","Iteration":"3"}. This record does not assert independent reproduction.

### DNPO

ARC Accuracy: 64.3. Result ID: 69b91ada47f34ce924a1ed4a.

[Reporting paper](<https://www.sota2.com/research/paper/68fa77991e209131cea7445c.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa77991e209131cea7445c.md>)

Recorded evaluation configuration: {"Base Model":"Llama-3.2-3B","Iteration":"1"}. This record does not assert independent reproduction.

### SPIN

ARC Accuracy: 63.8. Result ID: 69b91ada47f34ce924a1ed4d.

[Reporting paper](<https://www.sota2.com/research/paper/68fa77991e209131cea7445c.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73b11e209131cea5defb.md>)

Recorded evaluation configuration: {"Base Model":"Llama-3.2-3B","Iteration":"3"}. This record does not assert independent reproduction.

### SPIN

ARC Accuracy: 63.7. Result ID: 69b91ada47f34ce924a1ed48.

[Reporting paper](<https://www.sota2.com/research/paper/68fa77991e209131cea7445c.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73b11e209131cea5defb.md>)

Recorded evaluation configuration: {"Base Model":"Llama-3.2-3B","Iteration":"0"}. This record does not assert independent reproduction.

### SPIN

ARC Accuracy: 63.6. Result ID: 69b91ada47f34ce924a1ed49.

[Reporting paper](<https://www.sota2.com/research/paper/68fa77991e209131cea7445c.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73b11e209131cea5defb.md>)

Recorded evaluation configuration: {"Base Model":"Llama-3.2-3B","Iteration":"1"}. This record does not assert independent reproduction.

### SPIN

ARC Accuracy: 63.5. Result ID: 69b91ada47f34ce924a1ed4b.

[Reporting paper](<https://www.sota2.com/research/paper/68fa77991e209131cea7445c.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73b11e209131cea5defb.md>)

Recorded evaluation configuration: {"Base Model":"Llama-3.2-3B","Iteration":"2"}. This record does not assert independent reproduction.

### SFT

ARC Accuracy: 63.4. Result ID: 69b91ada47f34ce924a1ed47.

[Reporting paper](<https://www.sota2.com/research/paper/68fa77991e209131cea7445c.md>)

Recorded evaluation configuration: {"Base Model":"Llama-3.2-3B","Iteration":"SFT"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.