# Question Answering on SQuAD-Shifts New Wiki

Source: [Question Answering on SQuAD-Shifts New Wiki](<https://www.sota2.com/research/sota/question-answering-on-squad-shifts-new-wiki>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-02-27T09:37:00.223Z

Evaluation of Question Answering performance specifically on the New Wiki subset of the SQuAD-Shifts dataset, reporting MAPE.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Question Answering |
| Dataset (including any stated split) | SQuAD-Shifts New Wiki |
| Metric | MAPE |
| Unit | PERCENT |
| Direction | Lower is better |
| Results loaded | 6 of 6 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [MAPE](<https://www.sota2.com/research/sota/question-answering-on-squad-shifts-new-wiki.md?metric=MAPE>): Lower is better; unit: PERCENT.

## Recorded results

| Method | MAPE | Result date | Report paper |
| --- | --- | --- | --- |
| ATC | 5.18 | Not recorded | 68fa755b1e209131cea67726 |
| ALine-D | 6.33 | Not recorded | 68fa755b1e209131cea67726 |
| ALine-S | 6.58 | Not recorded | 68fa755b1e209131cea67726 |
| DOC-Feat | 7.5 | Not recorded | 68fa755b1e209131cea67726 |
| Naive Agr | 25.24 | Not recorded | 68fa755b1e209131cea67726 |
| AC | 25.96 | Not recorded | 68fa755b1e209131cea67726 |

### ATC

MAPE: 5.18. Result ID: 69a165af35dfcabb00e6cbbb.

[Reporting paper](<https://www.sota2.com/research/paper/68fa755b1e209131cea67726.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6dab1e209131cea3b57f.md>)

Recorded evaluation configuration: {"Model Ensemble":"GPT, OPT, and Llama models individually finetuned"}. This record does not assert independent reproduction.

### ALine-D

MAPE: 6.33. Result ID: 69a165af35dfcabb00e6cbb8.

[Reporting paper](<https://www.sota2.com/research/paper/68fa755b1e209131cea67726.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa755b1e209131cea67726.md>)

Recorded evaluation configuration: {"Model Ensemble":"GPT, OPT, and Llama models individually finetuned"}. This record does not assert independent reproduction.

### ALine-S

MAPE: 6.58. Result ID: 69a165af35dfcabb00e6cbb9.

[Reporting paper](<https://www.sota2.com/research/paper/68fa755b1e209131cea67726.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa755b1e209131cea67726.md>)

Recorded evaluation configuration: {"Model Ensemble":"GPT, OPT, and Llama models individually finetuned"}. This record does not assert independent reproduction.

### DOC-Feat

MAPE: 7.5. Result ID: 69a165af35dfcabb00e6cbbd.

[Reporting paper](<https://www.sota2.com/research/paper/68fa755b1e209131cea67726.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6a9b1e209131cea29564.md>)

Recorded evaluation configuration: {"Model Ensemble":"GPT, OPT, and Llama models individually finetuned"}. This record does not assert independent reproduction.

### Naive Agr

MAPE: 25.24. Result ID: 69a165af35dfcabb00e6cbba.

[Reporting paper](<https://www.sota2.com/research/paper/68fa755b1e209131cea67726.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6c811e209131cea34627.md>)

Recorded evaluation configuration: {"Model Ensemble":"GPT, OPT, and Llama models individually finetuned"}. This record does not assert independent reproduction.

### AC

MAPE: 25.96. Result ID: 69a165af35dfcabb00e6cbbc.

[Reporting paper](<https://www.sota2.com/research/paper/68fa755b1e209131cea67726.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa64e41e209131cea0731a.md>)

[Implementation](<https://www.sota2.com/github.com/hendrycks/error-detection>)

Recorded evaluation configuration: {"Model Ensemble":"GPT, OPT, and Llama models individually finetuned"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.