# Reading Comprehension on DROP (DROP metric)

Source: [Reading Comprehension on DROP (DROP metric)](<https://www.sota2.com/research/sota/reading-comprehension-on-drop-drop-metric>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-05-27T09:20:27.601Z

Evaluation of model performance on the DROP dataset for Reading Comprehension, reporting the metric specifically keyed as 'DROP'.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Reading Comprehension |
| Dataset (including any stated split) | DROP |
| Metric | DROP Score |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 25 of 25 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [DROP Score](<https://www.sota2.com/research/sota/reading-comprehension-on-drop-drop-metric.md?metric=DROP%20Score>): Higher is better; unit: PERCENT.

## Recorded results

| Method | DROP Score | Result date | Report paper |
| --- | --- | --- | --- |
| MONA | 48.68 | Not recorded | 6a167a8bdbd7150506f7ffbc |
| Muon | 47.57 | Not recorded | 6a167a8bdbd7150506f7ffbc |
| AdamW | 45.6 | Not recorded | 6a167a8bdbd7150506f7ffbc |
| Warmup-Stable-Only (WSO) | 36.4 | Not recorded | 69ba31ab0636e5bf4afc09be |
| WSD | 32.6 | Not recorded | 69ba31ab0636e5bf4afc09be |
| Linear | 30.5 | Not recorded | 69ba31ab0636e5bf4afc09be |
| WSD | 29.9 | Not recorded | 69ba31ab0636e5bf4afc09be |
| Cosine | 28.8 | Not recorded | 69ba31ab0636e5bf4afc09be |
| WSD | 28.3 | Not recorded | 69ba31ab0636e5bf4afc09be |
| Linear | 28.3 | Not recorded | 69ba31ab0636e5bf4afc09be |
| Cosine | 26.9 | Not recorded | 69ba31ab0636e5bf4afc09be |
| HYPERCONNECTIONS | 24.86 | Not recorded | 69facb4ac0f56ed84308a61d |
| WSD | 23.7 | Not recorded | 69ba31ab0636e5bf4afc09be |
| RESFORMER | 22.8 | Not recorded | 69facb4ac0f56ed84308a61d |
| Warmup-Stable-Only (WSO) | 22.7 | Not recorded | 69ba31ab0636e5bf4afc09be |
| DENSEFORMER | 22.66 | Not recorded | 69facb4ac0f56ed84308a61d |
| WSD | 22 | Not recorded | 69ba31ab0636e5bf4afc09be |
| WSD | 22 | Not recorded | 69ba31ab0636e5bf4afc09be |
| SATFORMER | 21.41 | Not recorded | 69facb4ac0f56ed84308a61d |
| MUDDFORMER | 20.84 | Not recorded | 69facb4ac0f56ed84308a61d |
| TRANSFORMER | 20.07 | Not recorded | 69facb4ac0f56ed84308a61d |
| Cosine | 17.6 | Not recorded | 69ba31ab0636e5bf4afc09be |
| Cosine | 17.1 | Not recorded | 69ba31ab0636e5bf4afc09be |
| Linear | 16.3 | Not recorded | 69ba31ab0636e5bf4afc09be |
| Linear | 16.2 | Not recorded | 69ba31ab0636e5bf4afc09be |

### MONA

DROP Score: 48.68. Result ID: 6a16b74bfbc373592c10234e.

[Reporting paper](<https://www.sota2.com/research/paper/6a167a8bdbd7150506f7ffbc.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a167a8bdbd7150506f7ffbc.md>)

Recorded evaluation configuration: {"Model":"MOE-68B-A3B","Training Tokens":"700B","Evaluation Mode":"Chain of Thought (CoT)"}. This record does not assert independent reproduction.

### Muon

DROP Score: 47.57. Result ID: 6a16b74bfbc373592c10234d.

[Reporting paper](<https://www.sota2.com/research/paper/6a167a8bdbd7150506f7ffbc.md>)

Recorded evaluation configuration: {"Model":"MOE-68B-A3B","Training Tokens":"700B","Evaluation Mode":"Chain of Thought (CoT)"}. This record does not assert independent reproduction.

### AdamW

DROP Score: 45.6. Result ID: 6a16b74bfbc373592c10234c.

[Reporting paper](<https://www.sota2.com/research/paper/6a167a8bdbd7150506f7ffbc.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa55761e209131ce9946db.md>)

Recorded evaluation configuration: {"Model":"MOE-68B-A3B","Training Tokens":"700B","Evaluation Mode":"Chain of Thought (CoT)"}. This record does not assert independent reproduction.

### Warmup-Stable-Only (WSO)

DROP Score: 36.4. Result ID: 69ba5e5b47f34ce924a41fde.

[Reporting paper](<https://www.sota2.com/research/paper/69ba31ab0636e5bf4afc09be.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69ba31ab0636e5bf4afc09be.md>)

Recorded evaluation configuration: {"Model":"8B","αpre":"1.0","αmid":"1.0"}. This record does not assert independent reproduction.

### WSD

DROP Score: 32.6. Result ID: 69ba5e5b47f34ce924a41fdf.

[Reporting paper](<https://www.sota2.com/research/paper/69ba31ab0636e5bf4afc09be.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73671e209131cea5c3ab.md>)

Recorded evaluation configuration: {"Model":"8B","αpre":"1.0","αmid":"0.0"}. This record does not assert independent reproduction.

### Linear

DROP Score: 30.5. Result ID: 69ba5e5b47f34ce924a41fe5.

[Reporting paper](<https://www.sota2.com/research/paper/69ba31ab0636e5bf4afc09be.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa772c1e209131cea71903.md>)

Recorded evaluation configuration: {"Model":"8B","αpre":"0.1","αmid":"1.0"}. This record does not assert independent reproduction.

### WSD

DROP Score: 29.9. Result ID: 69ba5e5b47f34ce924a41fe1.

[Reporting paper](<https://www.sota2.com/research/paper/69ba31ab0636e5bf4afc09be.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73671e209131cea5c3ab.md>)

Recorded evaluation configuration: {"Model":"8B","αpre":"0.1","αmid":"1.0"}. This record does not assert independent reproduction.

### Cosine

DROP Score: 28.8. Result ID: 69ba5e5b47f34ce924a41fe3.

[Reporting paper](<https://www.sota2.com/research/paper/69ba31ab0636e5bf4afc09be.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa680c1e209131cea19d53.md>)

[Implementation](<https://github.com/openai/gpt-3/blob/master/overlap_frequency.md>)

Recorded evaluation configuration: {"Model":"8B","αpre":"0.1","αmid":"1.0"}. This record does not assert independent reproduction.

### WSD

DROP Score: 28.3. Result ID: 69ba5e5b47f34ce924a41fe0.

[Reporting paper](<https://www.sota2.com/research/paper/69ba31ab0636e5bf4afc09be.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73671e209131cea5c3ab.md>)

Recorded evaluation configuration: {"Model":"8B","αpre":"0.1","αmid":"0.0"}. This record does not assert independent reproduction.

### Linear

DROP Score: 28.3. Result ID: 69ba5e5b47f34ce924a41fe4.

[Reporting paper](<https://www.sota2.com/research/paper/69ba31ab0636e5bf4afc09be.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa772c1e209131cea71903.md>)

Recorded evaluation configuration: {"Model":"8B","αpre":"0.1","αmid":"0.0"}. This record does not assert independent reproduction.

### Cosine

DROP Score: 26.9. Result ID: 69ba5e5b47f34ce924a41fe2.

[Reporting paper](<https://www.sota2.com/research/paper/69ba31ab0636e5bf4afc09be.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa680c1e209131cea19d53.md>)

[Implementation](<https://github.com/openai/gpt-3/blob/master/overlap_frequency.md>)

Recorded evaluation configuration: {"Model":"8B","αpre":"0.1","αmid":"0.0"}. This record does not assert independent reproduction.

### HYPERCONNECTIONS

DROP Score: 24.86. Result ID: 69fb03c947f34ce924de18f9.

[Reporting paper](<https://www.sota2.com/research/paper/69facb4ac0f56ed84308a61d.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa77771e209131cea734fa.md>)

Recorded evaluation configuration: {"Input length":"2k tokens","Evaluation protocol":"Arora et al. \[2024\]"}. This record does not assert independent reproduction.

### WSD

DROP Score: 23.7. Result ID: 69ba5e5b47f34ce924a41fd9.

[Reporting paper](<https://www.sota2.com/research/paper/69ba31ab0636e5bf4afc09be.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73671e209131cea5c3ab.md>)

Recorded evaluation configuration: {"Model":"1B","αpre":"0.1","αmid":"1.0"}. This record does not assert independent reproduction.

### RESFORMER

DROP Score: 22.8. Result ID: 69fb03c947f34ce924de18fa.

[Reporting paper](<https://www.sota2.com/research/paper/69facb4ac0f56ed84308a61d.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa79301e209131cea7c81b.md>)

[Implementation](<https://github.com/Zcchill/Value-Residual-Learning>)

Recorded evaluation configuration: {"Input length":"2k tokens","Evaluation protocol":"Arora et al. \[2024\]"}. This record does not assert independent reproduction.

### Warmup-Stable-Only (WSO)

DROP Score: 22.7. Result ID: 69ba5e5b47f34ce924a41fd6.

[Reporting paper](<https://www.sota2.com/research/paper/69ba31ab0636e5bf4afc09be.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69ba31ab0636e5bf4afc09be.md>)

Recorded evaluation configuration: {"Model":"1B","αpre":"1.0","αmid":"1.0"}. This record does not assert independent reproduction.

### DENSEFORMER

DROP Score: 22.66. Result ID: 69fb03c947f34ce924de18f7.

[Reporting paper](<https://www.sota2.com/research/paper/69facb4ac0f56ed84308a61d.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa729e1e209131cea578df.md>)

[Implementation](<https://github.com/epfml/DenseFormer>)

Recorded evaluation configuration: {"Input length":"2k tokens","Evaluation protocol":"Arora et al. \[2024\]"}. This record does not assert independent reproduction.

### WSD

DROP Score: 22. Result ID: 69ba5e5b47f34ce924a41fd7.

[Reporting paper](<https://www.sota2.com/research/paper/69ba31ab0636e5bf4afc09be.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73671e209131cea5c3ab.md>)

Recorded evaluation configuration: {"Model":"1B","αpre":"1.0","αmid":"0.0"}. This record does not assert independent reproduction.

### WSD

DROP Score: 22. Result ID: 69ba5e5b47f34ce924a41fd8.

[Reporting paper](<https://www.sota2.com/research/paper/69ba31ab0636e5bf4afc09be.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73671e209131cea5c3ab.md>)

Recorded evaluation configuration: {"Model":"1B","αpre":"0.1","αmid":"0.0"}. This record does not assert independent reproduction.

### SATFORMER

DROP Score: 21.41. Result ID: 69fb03c947f34ce924de18fb.

[Reporting paper](<https://www.sota2.com/research/paper/69facb4ac0f56ed84308a61d.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69facb4ac0f56ed84308a61d.md>)

Recorded evaluation configuration: {"Input length":"2k tokens","Evaluation protocol":"Arora et al. \[2024\]"}. This record does not assert independent reproduction.

### MUDDFORMER

DROP Score: 20.84. Result ID: 69fb03c947f34ce924de18f8.

[Reporting paper](<https://www.sota2.com/research/paper/69facb4ac0f56ed84308a61d.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa78d41e209131cea7aa07.md>)

Recorded evaluation configuration: {"Input length":"2k tokens","Evaluation protocol":"Arora et al. \[2024\]"}. This record does not assert independent reproduction.

### TRANSFORMER

DROP Score: 20.07. Result ID: 69fb03c947f34ce924de18f6.

[Reporting paper](<https://www.sota2.com/research/paper/69facb4ac0f56ed84308a61d.md>)

Recorded evaluation configuration: {"Input length":"2k tokens","Evaluation protocol":"Arora et al. \[2024\]"}. This record does not assert independent reproduction.

### Cosine

DROP Score: 17.6. Result ID: 69ba5e5b47f34ce924a41fda.

[Reporting paper](<https://www.sota2.com/research/paper/69ba31ab0636e5bf4afc09be.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa680c1e209131cea19d53.md>)

[Implementation](<https://github.com/openai/gpt-3/blob/master/overlap_frequency.md>)

Recorded evaluation configuration: {"Model":"1B","αpre":"0.1","αmid":"0.0"}. This record does not assert independent reproduction.

### Cosine

DROP Score: 17.1. Result ID: 69ba5e5b47f34ce924a41fdb.

[Reporting paper](<https://www.sota2.com/research/paper/69ba31ab0636e5bf4afc09be.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa680c1e209131cea19d53.md>)

[Implementation](<https://github.com/openai/gpt-3/blob/master/overlap_frequency.md>)

Recorded evaluation configuration: {"Model":"1B","αpre":"0.1","αmid":"1.0"}. This record does not assert independent reproduction.

### Linear

DROP Score: 16.3. Result ID: 69ba5e5b47f34ce924a41fdd.

[Reporting paper](<https://www.sota2.com/research/paper/69ba31ab0636e5bf4afc09be.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa772c1e209131cea71903.md>)

Recorded evaluation configuration: {"Model":"1B","αpre":"0.1","αmid":"1.0"}. This record does not assert independent reproduction.

### Linear

DROP Score: 16.2. Result ID: 69ba5e5b47f34ce924a41fdc.

[Reporting paper](<https://www.sota2.com/research/paper/69ba31ab0636e5bf4afc09be.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa772c1e209131cea71903.md>)

Recorded evaluation configuration: {"Model":"1B","αpre":"0.1","αmid":"0.0"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.