# Conversational Search on TREC CAsT 2021

Source: [Conversational Search on TREC CAsT 2021](<https://www.sota2.com/research/sota/conversational-search-on-trec-cast-2021>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-04-10T00:19:53.924Z

Evaluation of model performance on the TREC CAsT 2021 dataset for Conversational Search, measured using ranking metrics like MRR, NDCG@3, and Recall@10.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Conversational Search |
| Dataset (including any stated split) | TREC CAsT 2021 |
| Metric | MRR |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 8 of 8 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [MRR](<https://www.sota2.com/research/sota/conversational-search-on-trec-cast-2021.md?metric=MRR>): Higher is better; unit: PERCENT.

- [NDCG@3](<https://www.sota2.com/research/sota/conversational-search-on-trec-cast-2021.md?metric=NDCG%403>): Higher is better; unit: PERCENT.

- [Recall@10](<https://www.sota2.com/research/sota/conversational-search-on-trec-cast-2021.md?metric=Recall%4010>): Higher is better; unit: PERCENT.

## Recorded results

| Method | MRR | Result date | Report paper |
| --- | --- | --- | --- |
| MSPA-CQR | 67.4 | Not recorded | 69d732b67d0172140ec76cd4 |
| LLM4CS | 66.1 | Not recorded | 69d732b67d0172140ec76cd4 |
| AdaCQR | 64.2 | Not recorded | 69d732b67d0172140ec76cd4 |
| CHIQ | 62.9 | Not recorded | 69d732b67d0172140ec76cd4 |
| E5-Mistral | 48.2 | Not recorded | 69d732b67d0172140ec76cd4 |
| RepLLaMA | 47.4 | Not recorded | 69d732b67d0172140ec76cd4 |
| LLM-Embedder | 46.8 | Not recorded | 69d732b67d0172140ec76cd4 |
| InstructorR | 46.7 | Not recorded | 69d732b67d0172140ec76cd4 |

### MSPA-CQR

MRR: 67.4. Result ID: 69d8421447f34ce924c04b22.

[Reporting paper](<https://www.sota2.com/research/paper/69d732b67d0172140ec76cd4.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69d732b67d0172140ec76cd4.md>)

Recorded evaluation configuration: {"Backbone":"LLaMA2-7B","Zero-shot":"true","Dense retrieval (ANCE)":"true"}. This record does not assert independent reproduction.

### LLM4CS

MRR: 66.1. Result ID: 69d8421447f34ce924c04b1d.

[Reporting paper](<https://www.sota2.com/research/paper/69d732b67d0172140ec76cd4.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa70e81e209131cea4de25.md>)

Recorded evaluation configuration: {"Backbone":"ChatGPT-3.5","Zero-shot":"true","Dense retrieval (ANCE)":"true"}. This record does not assert independent reproduction.

### AdaCQR

MRR: 64.2. Result ID: 69d8421447f34ce924c04b21.

[Reporting paper](<https://www.sota2.com/research/paper/69d732b67d0172140ec76cd4.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa767b1e209131cea6d4cb.md>)

Recorded evaluation configuration: {"Backbone":"T5-base","Zero-shot":"true","Dense retrieval (ANCE)":"true"}. This record does not assert independent reproduction.

### CHIQ

MRR: 62.9. Result ID: 69d8421447f34ce924c04b20.

[Reporting paper](<https://www.sota2.com/research/paper/69d732b67d0172140ec76cd4.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa74fc1e209131cea65141.md>)

Recorded evaluation configuration: {"Backbone":"LLaMA2-7B","Zero-shot":"true","Dense retrieval (ANCE)":"true"}. This record does not assert independent reproduction.

### E5-Mistral

MRR: 48.2. Result ID: 69d8421447f34ce924c04b1b.

[Reporting paper](<https://www.sota2.com/research/paper/69d732b67d0172140ec76cd4.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73611e209131cea5c103.md>)

Recorded evaluation configuration: {"Backbone":"Mistral-7B","Zero-shot":"true","Dense retrieval (ANCE)":"true"}. This record does not assert independent reproduction.

### RepLLaMA

MRR: 47.4. Result ID: 69d8421447f34ce924c04b1e.

[Reporting paper](<https://www.sota2.com/research/paper/69d732b67d0172140ec76cd4.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa70db1e209131cea4dba5.md>)

Recorded evaluation configuration: {"Backbone":"LLaMA2-7B","Zero-shot":"true","Dense retrieval (ANCE)":"true"}. This record does not assert independent reproduction.

### LLM-Embedder

MRR: 46.8. Result ID: 69d8421447f34ce924c04b1f.

[Reporting paper](<https://www.sota2.com/research/paper/69d732b67d0172140ec76cd4.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa70f21e209131cea4e3a5.md>)

Recorded evaluation configuration: {"Backbone":"LLaMA2-7B","Zero-shot":"true","Dense retrieval (ANCE)":"true"}. This record does not assert independent reproduction.

### InstructorR

MRR: 46.7. Result ID: 69d8421447f34ce924c04b1c.

[Reporting paper](<https://www.sota2.com/research/paper/69d732b67d0172140ec76cd4.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6bf71e209131cea314c0.md>)

[Implementation](<https://github.com/openai/following-instructions-human-feedback>)

Recorded evaluation configuration: {"Backbone":"ChatGPT-3.5","Zero-shot":"true","Dense retrieval (ANCE)":"true"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.