# Machine Reading Comprehension on CMRC Simplified Chinese 2018 (dev)

Source: [Machine Reading Comprehension on CMRC Simplified Chinese 2018 (dev)](<https://www.sota2.com/research/sota/machine-reading-comprehension-on-cmrc-simplified-chinese-2018-dev>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-02-26T05:41:33.608Z

Evaluation of Machine Reading Comprehension performance on the development split of the CMRC Simplified Chinese 2018 dataset, typically measured using Exact Match (EM) and F1 scores.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Machine Reading Comprehension |
| Dataset (including any stated split) | CMRC Simplified Chinese 2018 (dev) |
| Metric | EM |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 18 of 18 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [EM](<https://www.sota2.com/research/sota/machine-reading-comprehension-on-cmrc-simplified-chinese-2018-dev.md?metric=EM>): Higher is better; unit: PERCENT.

- [F1](<https://www.sota2.com/research/sota/machine-reading-comprehension-on-cmrc-simplified-chinese-2018-dev.md?metric=F1>): Higher is better; unit: PERCENT.

## Recorded results

| Method | EM | Result date | Report paper |
| --- | --- | --- | --- |
| PERT | 72.2 | Not recorded | 68fa6c001e209131cea3193a |
| MacBERT-large | 70.7 | Not recorded | 68fa6b4f1e209131cea2d827 |
| MacBERT | 70.7 | Not recorded | 68fa6c001e209131cea3193a |
| ELECTRA-large | 69.1 | Not recorded | 68fa6b4f1e209131cea2d827 |
| ELECTRA | 69.1 | Not recorded | 68fa6c001e209131cea3193a |
| MacBERT-base | 68.5 | Not recorded | 68fa6b4f1e209131cea2d827 |
| ROBERTa-wwm-ext-large | 68.5 | Not recorded | 68fa6b4f1e209131cea2d827 |
| MacBERT | 68.5 | Not recorded | 68fa6c001e209131cea3193a |
| PERT | 68.5 | Not recorded | 68fa6c001e209131cea3193a |
| RoBERTa | 68.5 | Not recorded | 68fa6c001e209131cea3193a |
| ELECTRA-base | 68.4 | Not recorded | 68fa6b4f1e209131cea2d827 |
| ELECTRA | 68.4 | Not recorded | 68fa6c001e209131cea3193a |
| ROBERTa-wwm-ext | 67.4 | Not recorded | 68fa6b4f1e209131cea2d827 |
| RoBERTa | 67.4 | Not recorded | 68fa6c001e209131cea3193a |
| BERT-wwm-ext | 67.1 | Not recorded | 68fa6b4f1e209131cea2d827 |
| BERT | 67.1 | Not recorded | 68fa6c001e209131cea3193a |
| BERT-wwm | 66.3 | Not recorded | 68fa6b4f1e209131cea2d827 |
| BERT | 65.5 | Not recorded | 68fa6b4f1e209131cea2d827 |

### PERT

EM: 72.2. Result ID: 698eb866ffc3901de0a0147b.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6c001e209131cea3193a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6c001e209131cea3193a.md>)

[Implementation](<https://github.com/ymcui/PERT>)

Recorded evaluation configuration: {"Model Size":"large"}. This record does not assert independent reproduction.

### MacBERT-large

EM: 70.7. Result ID: 698e7cb3ffc3901de09dffe2.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6b4f1e209131cea2d827.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6b4f1e209131cea2d827.md>)

[Implementation](<https://github.com/ymcui/Chinese-BERT-wwm>)

Recorded evaluation configuration: {"Model Size":"large"}. This record does not assert independent reproduction.

### MacBERT

EM: 70.7. Result ID: 698eb866ffc3901de0a0147a.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6c001e209131cea3193a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6b4f1e209131cea2d827.md>)

[Implementation](<https://github.com/ymcui/Chinese-BERT-wwm>)

Recorded evaluation configuration: {"Model Size":"large"}. This record does not assert independent reproduction.

### ELECTRA-large

EM: 69.1. Result ID: 698e7cb3ffc3901de09dffe0.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6b4f1e209131cea2d827.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa673f1e209131cea15373.md>)

[Implementation](<https://github.com/google-research/electra>)

Recorded evaluation configuration: {"Model Size":"large"}. This record does not assert independent reproduction.

### ELECTRA

EM: 69.1. Result ID: 698eb866ffc3901de0a01479.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6c001e209131cea3193a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa673f1e209131cea15373.md>)

[Implementation](<https://github.com/google-research/electra>)

Recorded evaluation configuration: {"Model Size":"large"}. This record does not assert independent reproduction.

### MacBERT-base

EM: 68.5. Result ID: 698e7cb3ffc3901de09dffdf.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6b4f1e209131cea2d827.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6b4f1e209131cea2d827.md>)

[Implementation](<https://github.com/ymcui/Chinese-BERT-wwm>)

Recorded evaluation configuration: {"Model Size":"base"}. This record does not assert independent reproduction.

### ROBERTa-wwm-ext-large

EM: 68.5. Result ID: 698e7cb3ffc3901de09dffe1.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6b4f1e209131cea2d827.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6b4f1e209131cea2d827.md>)

[Implementation](<https://github.com/ymcui/Chinese-BERT-wwm>)

Recorded evaluation configuration: {"Model Size":"large"}. This record does not assert independent reproduction.

### MacBERT

EM: 68.5. Result ID: 698eb866ffc3901de0a01476.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6c001e209131cea3193a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6b4f1e209131cea2d827.md>)

[Implementation](<https://github.com/ymcui/Chinese-BERT-wwm>)

Recorded evaluation configuration: {"Model Size":"base"}. This record does not assert independent reproduction.

### PERT

EM: 68.5. Result ID: 698eb866ffc3901de0a01477.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6c001e209131cea3193a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6c001e209131cea3193a.md>)

[Implementation](<https://github.com/ymcui/PERT>)

Recorded evaluation configuration: {"Model Size":"base"}. This record does not assert independent reproduction.

### RoBERTa

EM: 68.5. Result ID: 698eb866ffc3901de0a01478.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6c001e209131cea3193a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa66131e209131cea0e18b.md>)

[Implementation](<https://github.com/pytorch/fairseq>)

Recorded evaluation configuration: {"Model Size":"large"}. This record does not assert independent reproduction.

### ELECTRA-base

EM: 68.4. Result ID: 698e7cb3ffc3901de09dffde.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6b4f1e209131cea2d827.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa673f1e209131cea15373.md>)

[Implementation](<https://github.com/google-research/electra>)

Recorded evaluation configuration: {"Model Size":"base"}. This record does not assert independent reproduction.

### ELECTRA

EM: 68.4. Result ID: 698eb866ffc3901de0a01475.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6c001e209131cea3193a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa673f1e209131cea15373.md>)

[Implementation](<https://github.com/google-research/electra>)

Recorded evaluation configuration: {"Model Size":"base"}. This record does not assert independent reproduction.

### ROBERTa-wwm-ext

EM: 67.4. Result ID: 698e7cb3ffc3901de09dffdd.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6b4f1e209131cea2d827.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6b4f1e209131cea2d827.md>)

[Implementation](<https://github.com/ymcui/Chinese-BERT-wwm>)

Recorded evaluation configuration: {"Model Size":"base"}. This record does not assert independent reproduction.

### RoBERTa

EM: 67.4. Result ID: 698eb866ffc3901de0a01474.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6c001e209131cea3193a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa66131e209131cea0e18b.md>)

[Implementation](<https://github.com/pytorch/fairseq>)

Recorded evaluation configuration: {"Model Size":"base"}. This record does not assert independent reproduction.

### BERT-wwm-ext

EM: 67.1. Result ID: 698e7cb3ffc3901de09dffdc.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6b4f1e209131cea2d827.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6b4f1e209131cea2d827.md>)

[Implementation](<https://github.com/ymcui/Chinese-BERT-wwm>)

Recorded evaluation configuration: {"Model Size":"base"}. This record does not assert independent reproduction.

### BERT

EM: 67.1. Result ID: 698eb866ffc3901de0a01473.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6c001e209131cea3193a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa65c31e209131cea0c4ed.md>)

[Implementation](<https://github.com/google-research/bert>)

Recorded evaluation configuration: {"Model Size":"base"}. This record does not assert independent reproduction.

### BERT-wwm

EM: 66.3. Result ID: 698e7cb3ffc3901de09dffdb.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6b4f1e209131cea2d827.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6b4f1e209131cea2d827.md>)

[Implementation](<https://github.com/ymcui/Chinese-BERT-wwm>)

Recorded evaluation configuration: {"Model Size":"base"}. This record does not assert independent reproduction.

### BERT

EM: 65.5. Result ID: 698e7cb3ffc3901de09dffda.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6b4f1e209131cea2d827.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa65c31e209131cea0c4ed.md>)

[Implementation](<https://github.com/google-research/bert>)

Recorded evaluation configuration: {"Model Size":"base"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.