# Multi-hop Question Answering on HotpotQA N=60 (test)

Source: [Multi-hop Question Answering on HotpotQA N=60 (test)](<https://www.sota2.com/research/sota/multi-hop-question-answering-on-hotpotqa-n-60-test>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-06-26T08:50:29.311Z

Evaluation of model performance on the specific N=60 subset of the HotpotQA test set for multi-hop question answering, focusing on efficiency and statistical comparison metrics.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Multi-hop Question Answering |
| Dataset (including any stated split) | HotpotQA N=60 (test) |
| Metric | Average Rounds |
| Unit | SCALAR |
| Direction | Lower is better |
| Results loaded | 7 of 7 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Average Rounds](<https://www.sota2.com/research/sota/multi-hop-question-answering-on-hotpotqa-n-60-test.md?metric=Average%20Rounds>): Lower is better; unit: SCALAR.

- [Token Saving (%)](<https://www.sota2.com/research/sota/multi-hop-question-answering-on-hotpotqa-n-60-test.md?metric=Token%20Saving%20(%25)>): Higher is better; unit: PERCENT.

- [Delta IS](<https://www.sota2.com/research/sota/multi-hop-question-answering-on-hotpotqa-n-60-test.md?metric=Delta%20IS>): Higher is better; unit: SCALAR.

- [P-Value](<https://www.sota2.com/research/sota/multi-hop-question-answering-on-hotpotqa-n-60-test.md?metric=P-Value>): Lower is better; unit: SCALAR.

- [Non-inferiority Verdict](<https://www.sota2.com/research/sota/multi-hop-question-answering-on-hotpotqa-n-60-test.md?metric=Non-inferiority%20Verdict>): Higher is better; unit: OTHER.

## Recorded results

| Method | Average Rounds | Result date | Report paper |
| --- | --- | --- | --- |
| fixed\_k1 | 1 | Not recorded | 6a3e0796206542251d7cb4d4 |
| shp (full) | 2.4 | Not recorded | 6a3e0796206542251d7cb4d4 |
| oracle\_is | 2.73 | Not recorded | 6a3e0796206542251d7cb4d4 |
| fixed\_k3 | 3 | Not recorded | 6a3e0796206542251d7cb4d4 |
| entropy\_only | 3.92 | Not recorded | 6a3e0796206542251d7cb4d4 |
| fixed\_k6 (base) | 6 | Not recorded | 6a3e0796206542251d7cb4d4 |
| critic\_only | 6 | Not recorded | 6a3e0796206542251d7cb4d4 |

### fixed\_k1

Average Rounds: 1. Result ID: 6a3e3d52fbc373592c608c63.

[Reporting paper](<https://www.sota2.com/research/paper/6a3e0796206542251d7cb4d4.md>)

Recorded evaluation configuration: {"Policy":"fixed\_k1"}. This record does not assert independent reproduction.

### shp (full)

Average Rounds: 2.4. Result ID: 6a3e3d52fbc373592c608c64.

[Reporting paper](<https://www.sota2.com/research/paper/6a3e0796206542251d7cb4d4.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a3e0796206542251d7cb4d4.md>)

Recorded evaluation configuration: {"Policy":"shp (full)"}. This record does not assert independent reproduction.

### oracle\_is

Average Rounds: 2.73. Result ID: 6a3e3d52fbc373592c608c65.

[Reporting paper](<https://www.sota2.com/research/paper/6a3e0796206542251d7cb4d4.md>)

Recorded evaluation configuration: {"Policy":"oracle\_is"}. This record does not assert independent reproduction.

### fixed\_k3

Average Rounds: 3. Result ID: 6a3e3d52fbc373592c608c62.

[Reporting paper](<https://www.sota2.com/research/paper/6a3e0796206542251d7cb4d4.md>)

Recorded evaluation configuration: {"Policy":"fixed\_k3"}. This record does not assert independent reproduction.

### entropy\_only

Average Rounds: 3.92. Result ID: 6a3e3d52fbc373592c608c60.

[Reporting paper](<https://www.sota2.com/research/paper/6a3e0796206542251d7cb4d4.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a3e0796206542251d7cb4d4.md>)

Recorded evaluation configuration: {"Policy":"entropy\_only"}. This record does not assert independent reproduction.

### fixed\_k6 (base)

Average Rounds: 6. Result ID: 6a3e3d52fbc373592c608c5f.

[Reporting paper](<https://www.sota2.com/research/paper/6a3e0796206542251d7cb4d4.md>)

Recorded evaluation configuration: {"Policy":"fixed\_k6"}. This record does not assert independent reproduction.

### critic\_only

Average Rounds: 6. Result ID: 6a3e3d52fbc373592c608c61.

[Reporting paper](<https://www.sota2.com/research/paper/6a3e0796206542251d7cb4d4.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a3e0796206542251d7cb4d4.md>)

Recorded evaluation configuration: {"Policy":"critic\_only"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.