# Question Answering on HotpotQA full-wiki first 200 examples (val)

Source: [Question Answering on HotpotQA full-wiki first 200 examples (val)](<https://www.sota2.com/research/sota/question-answering-on-hotpotqa-full-wiki-first-200-examples-val>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-06-19T07:23:12.345Z

Evaluation of Question Answering performance on the first 200 examples of the full Wikipedia context split of the HotpotQA validation set, measured using Mean Jaccard and Time To First Token (TTFT) metrics.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Question Answering |
| Dataset (including any stated split) | HotpotQA full-wiki first 200 examples (val) |
| Metric | Mean Jaccard |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 2 of 2 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Mean Jaccard](<https://www.sota2.com/research/sota/question-answering-on-hotpotqa-full-wiki-first-200-examples-val.md?metric=Mean%20Jaccard>): Higher is better; unit: PERCENT.

- [Median TTFT (Retrieval Order)](<https://www.sota2.com/research/sota/question-answering-on-hotpotqa-full-wiki-first-200-examples-val.md?metric=Median%20TTFT%20(Retrieval%20Order)>): Lower is better; unit: TIME.

- [Median TTFT (Optimized Order)](<https://www.sota2.com/research/sota/question-answering-on-hotpotqa-full-wiki-first-200-examples-val.md?metric=Median%20TTFT%20(Optimized%20Order)>): Lower is better; unit: TIME.

## Recorded results

| Method | Mean Jaccard | Result date | Report paper |
| --- | --- | --- | --- |
| Retrieval Order | 0 | Not recorded | 6a34ccf2206542251d7c5e33 |
| CacheWeaver | 0 | Not recorded | 6a34ccf2206542251d7c5e33 |

### Retrieval Order

Mean Jaccard: 0. Result ID: 6a34edf5fbc373592c31d047.

[Reporting paper](<https://www.sota2.com/research/paper/6a34ccf2206542251d7c5e33.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa69b91e209131cea23c26.md>)

[Implementation](<https://github.com/huggingface/transformers/blob/master/examples/rag/>)

Recorded evaluation configuration: {"Locality source":"Dataset order","Config":"C"}. This record does not assert independent reproduction.

### CacheWeaver

Mean Jaccard: 0. Result ID: 6a34edf5fbc373592c31d048.

[Reporting paper](<https://www.sota2.com/research/paper/6a34ccf2206542251d7c5e33.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a34ccf2206542251d7c5e33.md>)

Recorded evaluation configuration: {"Locality source":"Dataset order","Config":"C"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.