# World Knowledge and Reading Comprehension on NQ, MMLU STEM, ARC, SciQ, LogiQA, BoolQ

Source: [World Knowledge and Reading Comprehension on NQ, MMLU STEM, ARC, SciQ, LogiQA, BoolQ](<https://www.sota2.com/research/sota/world-knowledge-and-reading-comprehension-on-nq-mmlu-stem-arc-sciq-logiqa-boolq>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-02-26T05:53:29.025Z

Evaluation covering world knowledge and reading comprehension across several key benchmarks: NQ, MMLU STEM subset, ARC (Easy/Challenge), SciQ, LogiQA, and BoolQ.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | World Knowledge and Reading Comprehension |
| Dataset (including any stated split) | LM Evaluation Harness NQ, MMLU STEM, ARC, SciQ, LogiQA, BoolQ |
| Metric | NQ Accuracy |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 15 of 15 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [NQ Accuracy](<https://www.sota2.com/research/sota/world-knowledge-and-reading-comprehension-on-nq-mmlu-stem-arc-sciq-logiqa-boolq.md?metric=NQ%20Accuracy>): Higher is better; unit: PERCENT.

- [MMLU STEM Accuracy](<https://www.sota2.com/research/sota/world-knowledge-and-reading-comprehension-on-nq-mmlu-stem-arc-sciq-logiqa-boolq.md?metric=MMLU%20STEM%20Accuracy>): Higher is better; unit: PERCENT.

- [ARC Easy Accuracy](<https://www.sota2.com/research/sota/world-knowledge-and-reading-comprehension-on-nq-mmlu-stem-arc-sciq-logiqa-boolq.md?metric=ARC%20Easy%20Accuracy>): Higher is better; unit: PERCENT.

- [ARC Challenge Accuracy](<https://www.sota2.com/research/sota/world-knowledge-and-reading-comprehension-on-nq-mmlu-stem-arc-sciq-logiqa-boolq.md?metric=ARC%20Challenge%20Accuracy>): Higher is better; unit: PERCENT.

- [SciQ Accuracy](<https://www.sota2.com/research/sota/world-knowledge-and-reading-comprehension-on-nq-mmlu-stem-arc-sciq-logiqa-boolq.md?metric=SciQ%20Accuracy>): Higher is better; unit: PERCENT.

- [LogiQA Accuracy](<https://www.sota2.com/research/sota/world-knowledge-and-reading-comprehension-on-nq-mmlu-stem-arc-sciq-logiqa-boolq.md?metric=LogiQA%20Accuracy>): Higher is better; unit: PERCENT.

- [BoolQ Accuracy](<https://www.sota2.com/research/sota/world-knowledge-and-reading-comprehension-on-nq-mmlu-stem-arc-sciq-logiqa-boolq.md?metric=BoolQ%20Accuracy>): Higher is better; unit: PERCENT.

- [Average Accuracy](<https://www.sota2.com/research/sota/world-knowledge-and-reading-comprehension-on-nq-mmlu-stem-arc-sciq-logiqa-boolq.md?metric=Average%20Accuracy>): Higher is better; unit: PERCENT.

## Recorded results

| Method | NQ Accuracy | Result date | Report paper |
| --- | --- | --- | --- |
| Mistral-7B Base | 29.81 | Not recorded | 68fa7a2c1e209131cea81877 |
| DSIR | 29.22 | Not recorded | 68fa7a2c1e209131cea81877 |
| Uniform (OpenWebMath) | 29.17 | Not recorded | 68fa7a2c1e209131cea81877 |
| AutoDS | 29.06 | Not recorded | 68fa7a2c1e209131cea81877 |
| QuRating | 28.89 | Not recorded | 68fa7a2c1e209131cea81877 |
| Uniform (OpenWebMath) | 26.07 | Not recorded | 68fa7a2c1e209131cea81877 |
| LLaMA2-7B Base | 26.01 | Not recorded | 68fa7a2c1e209131cea81877 |
| QuRating | 25.96 | Not recorded | 68fa7a2c1e209131cea81877 |
| AutoDS | 25.84 | Not recorded | 68fa7a2c1e209131cea81877 |
| DSIR | 25.76 | Not recorded | 68fa7a2c1e209131cea81877 |
| Gemma-2B Base | 14.88 | Not recorded | 68fa7a2c1e209131cea81877 |
| QuRating | 14.32 | Not recorded | 68fa7a2c1e209131cea81877 |
| Uniform (OpenWebMath) | 13.8 | Not recorded | 68fa7a2c1e209131cea81877 |
| DSIR | 13.27 | Not recorded | 68fa7a2c1e209131cea81877 |
| AutoDS | 13.27 | Not recorded | 68fa7a2c1e209131cea81877 |

### Mistral-7B Base

NQ Accuracy: 29.81. Result ID: 698b6fe97a967eddb76e92ba.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7a2c1e209131cea81877.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa70d61e209131cea4d909.md>)

[Implementation](<https://github.com/mistralai/mistral-src>)

Recorded evaluation configuration: {"Backbone":"Mistral-7B","Selection Method":"None (Base Model)","Few-shot settings":"NQ: 5, MMLU STEM: 5, ARC: 25, SciQ: 2, LogiQA: 2, BoolQ: 0"}. This record does not assert independent reproduction.

### DSIR

NQ Accuracy: 29.22. Result ID: 698b6fe97a967eddb76e92bc.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7a2c1e209131cea81877.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa71411e209131cea5025c.md>)

[Implementation](<https://github.com/p-lambda/dsir>)

Recorded evaluation configuration: {"Backbone":"Mistral-7B","Selection Method":"DSIR","Few-shot settings":"NQ: 5, MMLU STEM: 5, ARC: 25, SciQ: 2, LogiQA: 2, BoolQ: 0"}. This record does not assert independent reproduction.

### Uniform (OpenWebMath)

NQ Accuracy: 29.17. Result ID: 698b6fe97a967eddb76e92bb.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7a2c1e209131cea81877.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa70d61e209131cea4d902.md>)

Recorded evaluation configuration: {"Backbone":"Mistral-7B","Selection Method":"Uniform Sampling","Few-shot settings":"NQ: 5, MMLU STEM: 5, ARC: 25, SciQ: 2, LogiQA: 2, BoolQ: 0"}. This record does not assert independent reproduction.

### AutoDS

NQ Accuracy: 29.06. Result ID: 698b6fe97a967eddb76e92be.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7a2c1e209131cea81877.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7a2c1e209131cea81877.md>)

[Implementation](<https://github.com/yifanzhang-pro/AutoMathText>)

Recorded evaluation configuration: {"Backbone":"Mistral-7B","Selection Method":"AutoDS","Few-shot settings":"NQ: 5, MMLU STEM: 5, ARC: 25, SciQ: 2, LogiQA: 2, BoolQ: 0"}. This record does not assert independent reproduction.

### QuRating

NQ Accuracy: 28.89. Result ID: 698b6fe97a967eddb76e92bd.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7a2c1e209131cea81877.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa74241e209131cea5ff51.md>)

Recorded evaluation configuration: {"Backbone":"Mistral-7B","Selection Method":"QuRating","Few-shot settings":"NQ: 5, MMLU STEM: 5, ARC: 25, SciQ: 2, LogiQA: 2, BoolQ: 0"}. This record does not assert independent reproduction.

### Uniform (OpenWebMath)

NQ Accuracy: 26.07. Result ID: 698b6fe97a967eddb76e92b6.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7a2c1e209131cea81877.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa70d61e209131cea4d902.md>)

Recorded evaluation configuration: {"Backbone":"LLaMA2-7B","Selection Method":"Uniform Sampling","Few-shot settings":"NQ: 5, MMLU STEM: 5, ARC: 25, SciQ: 2, LogiQA: 2, BoolQ: 0"}. This record does not assert independent reproduction.

### LLaMA2-7B Base

NQ Accuracy: 26.01. Result ID: 698b6fe97a967eddb76e92b5.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7a2c1e209131cea81877.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa701f1e209131cea492fe.md>)

[Implementation](<https://github.com/facebookresearch/llama>)

Recorded evaluation configuration: {"Backbone":"LLaMA2-7B","Selection Method":"None (Base Model)","Few-shot settings":"NQ: 5, MMLU STEM: 5, ARC: 25, SciQ: 2, LogiQA: 2, BoolQ: 0"}. This record does not assert independent reproduction.

### QuRating

NQ Accuracy: 25.96. Result ID: 698b6fe97a967eddb76e92b8.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7a2c1e209131cea81877.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa74241e209131cea5ff51.md>)

Recorded evaluation configuration: {"Backbone":"LLaMA2-7B","Selection Method":"QuRating","Few-shot settings":"NQ: 5, MMLU STEM: 5, ARC: 25, SciQ: 2, LogiQA: 2, BoolQ: 0"}. This record does not assert independent reproduction.

### AutoDS

NQ Accuracy: 25.84. Result ID: 698b6fe97a967eddb76e92b9.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7a2c1e209131cea81877.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7a2c1e209131cea81877.md>)

[Implementation](<https://github.com/yifanzhang-pro/AutoMathText>)

Recorded evaluation configuration: {"Backbone":"LLaMA2-7B","Selection Method":"AutoDS","Few-shot settings":"NQ: 5, MMLU STEM: 5, ARC: 25, SciQ: 2, LogiQA: 2, BoolQ: 0"}. This record does not assert independent reproduction.

### DSIR

NQ Accuracy: 25.76. Result ID: 698b6fe97a967eddb76e92b7.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7a2c1e209131cea81877.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa71411e209131cea5025c.md>)

[Implementation](<https://github.com/p-lambda/dsir>)

Recorded evaluation configuration: {"Backbone":"LLaMA2-7B","Selection Method":"DSIR","Few-shot settings":"NQ: 5, MMLU STEM: 5, ARC: 25, SciQ: 2, LogiQA: 2, BoolQ: 0"}. This record does not assert independent reproduction.

### Gemma-2B Base

NQ Accuracy: 14.88. Result ID: 698b6fe97a967eddb76e92b0.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7a2c1e209131cea81877.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa72f11e209131cea5973c.md>)

Recorded evaluation configuration: {"Backbone":"Gemma-2B","Selection Method":"None (Base Model)","Few-shot settings":"NQ: 5, MMLU STEM: 5, ARC: 25, SciQ: 2, LogiQA: 2, BoolQ: 0"}. This record does not assert independent reproduction.

### QuRating

NQ Accuracy: 14.32. Result ID: 698b6fe97a967eddb76e92b3.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7a2c1e209131cea81877.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa74241e209131cea5ff51.md>)

Recorded evaluation configuration: {"Backbone":"Gemma-2B","Selection Method":"QuRating","Few-shot settings":"NQ: 5, MMLU STEM: 5, ARC: 25, SciQ: 2, LogiQA: 2, BoolQ: 0"}. This record does not assert independent reproduction.

### Uniform (OpenWebMath)

NQ Accuracy: 13.8. Result ID: 698b6fe97a967eddb76e92b1.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7a2c1e209131cea81877.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa70d61e209131cea4d902.md>)

Recorded evaluation configuration: {"Backbone":"Gemma-2B","Selection Method":"Uniform Sampling","Few-shot settings":"NQ: 5, MMLU STEM: 5, ARC: 25, SciQ: 2, LogiQA: 2, BoolQ: 0"}. This record does not assert independent reproduction.

### DSIR

NQ Accuracy: 13.27. Result ID: 698b6fe97a967eddb76e92b2.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7a2c1e209131cea81877.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa71411e209131cea5025c.md>)

[Implementation](<https://github.com/p-lambda/dsir>)

Recorded evaluation configuration: {"Backbone":"Gemma-2B","Selection Method":"DSIR","Few-shot settings":"NQ: 5, MMLU STEM: 5, ARC: 25, SciQ: 2, LogiQA: 2, BoolQ: 0"}. This record does not assert independent reproduction.

### AutoDS

NQ Accuracy: 13.27. Result ID: 698b6fe97a967eddb76e92b4.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7a2c1e209131cea81877.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7a2c1e209131cea81877.md>)

[Implementation](<https://github.com/yifanzhang-pro/AutoMathText>)

Recorded evaluation configuration: {"Backbone":"Gemma-2B","Selection Method":"AutoDS","Few-shot settings":"NQ: 5, MMLU STEM: 5, ARC: 25, SciQ: 2, LogiQA: 2, BoolQ: 0"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.