# Information Retrieval on QUEST (test)

Source: [Information Retrieval on QUEST (test)](<https://www.sota2.com/research/sota/information-retrieval-on-quest-test>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-02-26T05:54:46.130Z

Evaluation of model performance on the test split of the QUEST dataset for Information Retrieval tasks, measured using ranking metrics such as P@K, R@K, F1@K, NDCG@K, and MRR.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Information Retrieval |
| Dataset (including any stated split) | QUEST (test) |
| Metric | P@1 |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 14 of 14 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [P@1](<https://www.sota2.com/research/sota/information-retrieval-on-quest-test.md?metric=P%401>): Higher is better; unit: PERCENT.

- [P@10](<https://www.sota2.com/research/sota/information-retrieval-on-quest-test.md?metric=P%4010>): Higher is better; unit: PERCENT.

- [R@1](<https://www.sota2.com/research/sota/information-retrieval-on-quest-test.md?metric=R%401>): Higher is better; unit: PERCENT.

- [R@10](<https://www.sota2.com/research/sota/information-retrieval-on-quest-test.md?metric=R%4010>): Higher is better; unit: PERCENT.

- [F1@1](<https://www.sota2.com/research/sota/information-retrieval-on-quest-test.md?metric=F1%401>): Higher is better; unit: PERCENT.

- [F1@10](<https://www.sota2.com/research/sota/information-retrieval-on-quest-test.md?metric=F1%4010>): Higher is better; unit: PERCENT.

- [NDCG@1](<https://www.sota2.com/research/sota/information-retrieval-on-quest-test.md?metric=NDCG%401>): Higher is better; unit: PERCENT.

- [NDCG@10](<https://www.sota2.com/research/sota/information-retrieval-on-quest-test.md?metric=NDCG%4010>): Higher is better; unit: PERCENT.

- [MRR](<https://www.sota2.com/research/sota/information-retrieval-on-quest-test.md?metric=MRR>): Higher is better; unit: PERCENT.

## Recorded results

| Method | P@1 | Result date | Report paper |
| --- | --- | --- | --- |
| OrLog | 30.8 | Not recorded | 69802fd3a8b01e76cc8e17d3 |
| LLM | 26.3 | Not recorded | 69802fd3a8b01e76cc8e17d3 |
| LLM\* | 24.1 | Not recorded | 69802fd3a8b01e76cc8e17d3 |
| OrLog | 22.1 | Not recorded | 69802fd3a8b01e76cc8e17d3 |
| LLM\* | 20.4 | Not recorded | 69802fd3a8b01e76cc8e17d3 |
| LLM | 19.2 | Not recorded | 69802fd3a8b01e76cc8e17d3 |
| LLM | 18.5 | Not recorded | 69802fd3a8b01e76cc8e17d3 |
| E5 | 18.2 | Not recorded | 69802fd3a8b01e76cc8e17d3 |
| OrLog | 16.1 | Not recorded | 69802fd3a8b01e76cc8e17d3 |
| LLM\* | 15.8 | Not recorded | 69802fd3a8b01e76cc8e17d3 |
| LLM | 15.5 | Not recorded | 69802fd3a8b01e76cc8e17d3 |
| LLM\* | 14.4 | Not recorded | 69802fd3a8b01e76cc8e17d3 |
| OrLog | 13.8 | Not recorded | 69802fd3a8b01e76cc8e17d3 |
| BM25 | 11.6 | Not recorded | 69802fd3a8b01e76cc8e17d3 |

### OrLog

P@1: 30.8. Result ID: 6998a7395698665ec409db24.

[Reporting paper](<https://www.sota2.com/research/paper/69802fd3a8b01e76cc8e17d3.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69802fd3a8b01e76cc8e17d3.md>)

Recorded evaluation configuration: {"Base Retriever":"E5","Reasoning Strategy":"OrLog","Knowledge Setting":"Parametric+","Backbone Model":"Llama-3.3-70B-Instruct"}. This record does not assert independent reproduction.

### LLM

P@1: 26.3. Result ID: 6998a7395698665ec409db23.

[Reporting paper](<https://www.sota2.com/research/paper/69802fd3a8b01e76cc8e17d3.md>)

Recorded evaluation configuration: {"Base Retriever":"E5","Reasoning Strategy":"LLM-as-reasoner","Knowledge Setting":"Parametric+","Backbone Model":"Llama-3.3-70B-Instruct"}. This record does not assert independent reproduction.

### LLM\*

P@1: 24.1. Result ID: 6998a7395698665ec409db22.

[Reporting paper](<https://www.sota2.com/research/paper/69802fd3a8b01e76cc8e17d3.md>)

Recorded evaluation configuration: {"Base Retriever":"E5","Reasoning Strategy":"Logic-augmented LLM","Knowledge Setting":"Parametric+","Backbone Model":"Llama-3.3-70B-Instruct"}. This record does not assert independent reproduction.

### OrLog

P@1: 22.1. Result ID: 6998a7395698665ec409db1d.

[Reporting paper](<https://www.sota2.com/research/paper/69802fd3a8b01e76cc8e17d3.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69802fd3a8b01e76cc8e17d3.md>)

Recorded evaluation configuration: {"Base Retriever":"BM25","Reasoning Strategy":"OrLog","Knowledge Setting":"Parametric+","Backbone Model":"Llama-3.3-70B-Instruct"}. This record does not assert independent reproduction.

### LLM\*

P@1: 20.4. Result ID: 6998a7395698665ec409db1f.

[Reporting paper](<https://www.sota2.com/research/paper/69802fd3a8b01e76cc8e17d3.md>)

Recorded evaluation configuration: {"Base Retriever":"E5","Reasoning Strategy":"Logic-augmented LLM","Knowledge Setting":"Parametric","Backbone Model":"Llama-3.3-70B-Instruct"}. This record does not assert independent reproduction.

### LLM

P@1: 19.2. Result ID: 6998a7395698665ec409db20.

[Reporting paper](<https://www.sota2.com/research/paper/69802fd3a8b01e76cc8e17d3.md>)

Recorded evaluation configuration: {"Base Retriever":"E5","Reasoning Strategy":"LLM-as-reasoner","Knowledge Setting":"Parametric","Backbone Model":"Llama-3.3-70B-Instruct"}. This record does not assert independent reproduction.

### LLM

P@1: 18.5. Result ID: 6998a7395698665ec409db1c.

[Reporting paper](<https://www.sota2.com/research/paper/69802fd3a8b01e76cc8e17d3.md>)

Recorded evaluation configuration: {"Base Retriever":"BM25","Reasoning Strategy":"LLM-as-reasoner","Knowledge Setting":"Parametric+","Backbone Model":"Llama-3.3-70B-Instruct"}. This record does not assert independent reproduction.

### E5

P@1: 18.2. Result ID: 6998a7395698665ec409db1e.

[Reporting paper](<https://www.sota2.com/research/paper/69802fd3a8b01e76cc8e17d3.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa72491e209131cea5595a.md>)

[Implementation](<https://github.com/microsoft/unilm>)

Recorded evaluation configuration: {"Base Retriever":"E5","Reasoning Strategy":"Original Ranking"}. This record does not assert independent reproduction.

### OrLog

P@1: 16.1. Result ID: 6998a7395698665ec409db21.

[Reporting paper](<https://www.sota2.com/research/paper/69802fd3a8b01e76cc8e17d3.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69802fd3a8b01e76cc8e17d3.md>)

Recorded evaluation configuration: {"Base Retriever":"E5","Reasoning Strategy":"OrLog","Knowledge Setting":"Parametric","Backbone Model":"Llama-3.3-70B-Instruct"}. This record does not assert independent reproduction.

### LLM\*

P@1: 15.8. Result ID: 6998a7395698665ec409db18.

[Reporting paper](<https://www.sota2.com/research/paper/69802fd3a8b01e76cc8e17d3.md>)

Recorded evaluation configuration: {"Base Retriever":"BM25","Reasoning Strategy":"Logic-augmented LLM","Knowledge Setting":"Parametric","Backbone Model":"Llama-3.3-70B-Instruct"}. This record does not assert independent reproduction.

### LLM

P@1: 15.5. Result ID: 6998a7395698665ec409db19.

[Reporting paper](<https://www.sota2.com/research/paper/69802fd3a8b01e76cc8e17d3.md>)

Recorded evaluation configuration: {"Base Retriever":"BM25","Reasoning Strategy":"LLM-as-reasoner","Knowledge Setting":"Parametric","Backbone Model":"Llama-3.3-70B-Instruct"}. This record does not assert independent reproduction.

### LLM\*

P@1: 14.4. Result ID: 6998a7395698665ec409db1b.

[Reporting paper](<https://www.sota2.com/research/paper/69802fd3a8b01e76cc8e17d3.md>)

Recorded evaluation configuration: {"Base Retriever":"BM25","Reasoning Strategy":"Logic-augmented LLM","Knowledge Setting":"Parametric+","Backbone Model":"Llama-3.3-70B-Instruct"}. This record does not assert independent reproduction.

### OrLog

P@1: 13.8. Result ID: 6998a7395698665ec409db1a.

[Reporting paper](<https://www.sota2.com/research/paper/69802fd3a8b01e76cc8e17d3.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69802fd3a8b01e76cc8e17d3.md>)

Recorded evaluation configuration: {"Base Retriever":"BM25","Reasoning Strategy":"OrLog","Knowledge Setting":"Parametric","Backbone Model":"Llama-3.3-70B-Instruct"}. This record does not assert independent reproduction.

### BM25

P@1: 11.6. Result ID: 6998a7395698665ec409db17.

[Reporting paper](<https://www.sota2.com/research/paper/69802fd3a8b01e76cc8e17d3.md>)

Recorded evaluation configuration: {"Base Retriever":"BM25","Reasoning Strategy":"Original Ranking"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.
