# Information Retrieval on 18 datasets (BEIR, LoTTE, etc.)

Source: [Information Retrieval on 18 datasets (BEIR, LoTTE, etc.)](<https://www.sota2.com/research/sota/information-retrieval-on-18-datasets-beir-lotte-etc>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-03-25T11:06:10.846Z

Evaluation of Information Retrieval performance across a collection of 18 datasets, including BEIR and LoTTE, focusing on the trade-off between latency reduction and precision loss at various recall levels (P@1, P@5, P@10).

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Information Retrieval |
| Dataset (including any stated split) | 18 datasets (BEIR, LoTTE, etc.) |
| Metric | P@1 Precision Loss Mean |
| Unit | SCALAR |
| Direction | Lower is better |
| Results loaded | 15 of 15 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [P@1 Precision Loss Mean](<https://www.sota2.com/research/sota/information-retrieval-on-18-datasets-beir-lotte-etc.md?metric=P%401%20Precision%20Loss%20Mean>): Lower is better; unit: SCALAR.

- [P@1 Latency Reduction Range](<https://www.sota2.com/research/sota/information-retrieval-on-18-datasets-beir-lotte-etc.md?metric=P%401%20Latency%20Reduction%20Range>): Higher is better; unit: PERCENT.

- [P@1 Latency Reduction Mean](<https://www.sota2.com/research/sota/information-retrieval-on-18-datasets-beir-lotte-etc.md?metric=P%401%20Latency%20Reduction%20Mean>): Higher is better; unit: PERCENT.

- [P@1 Precision Loss Max](<https://www.sota2.com/research/sota/information-retrieval-on-18-datasets-beir-lotte-etc.md?metric=P%401%20Precision%20Loss%20Max>): Lower is better; unit: SCALAR.

- [P@5 Latency Reduction Range](<https://www.sota2.com/research/sota/information-retrieval-on-18-datasets-beir-lotte-etc.md?metric=P%405%20Latency%20Reduction%20Range>): Higher is better; unit: PERCENT.

- [P@5 Latency Reduction Mean](<https://www.sota2.com/research/sota/information-retrieval-on-18-datasets-beir-lotte-etc.md?metric=P%405%20Latency%20Reduction%20Mean>): Higher is better; unit: PERCENT.

- [P@5 Precision Loss Mean](<https://www.sota2.com/research/sota/information-retrieval-on-18-datasets-beir-lotte-etc.md?metric=P%405%20Precision%20Loss%20Mean>): Lower is better; unit: SCALAR.

- [P@5 Precision Loss Max](<https://www.sota2.com/research/sota/information-retrieval-on-18-datasets-beir-lotte-etc.md?metric=P%405%20Precision%20Loss%20Max>): Lower is better; unit: SCALAR.

- [P@10 Latency Reduction Range](<https://www.sota2.com/research/sota/information-retrieval-on-18-datasets-beir-lotte-etc.md?metric=P%4010%20Latency%20Reduction%20Range>): Higher is better; unit: PERCENT.

- [P@10 Latency Reduction Mean](<https://www.sota2.com/research/sota/information-retrieval-on-18-datasets-beir-lotte-etc.md?metric=P%4010%20Latency%20Reduction%20Mean>): Higher is better; unit: PERCENT.

- [P@10 Precision Loss Mean](<https://www.sota2.com/research/sota/information-retrieval-on-18-datasets-beir-lotte-etc.md?metric=P%4010%20Precision%20Loss%20Mean>): Lower is better; unit: SCALAR.

- [P@10 Precision Loss Max](<https://www.sota2.com/research/sota/information-retrieval-on-18-datasets-beir-lotte-etc.md?metric=P%4010%20Precision%20Loss%20Max>): Lower is better; unit: SCALAR.

## Recorded results

| Method | P@1 Precision Loss Mean | Result date | Report paper |
| --- | --- | --- | --- |
| PRISM | -0.001 | Not recorded | 68fa7c471e209131cea8d5cd |
| PRISM | -0.001 | Not recorded | 68fa7c471e209131cea8d5cd |
| PRISM Quant | 0 | Not recorded | 68fa7c471e209131cea8d5cd |
| PRISM | 0 | Not recorded | 68fa7c471e209131cea8d5cd |
| PRISM | 0 | Not recorded | 68fa7c471e209131cea8d5cd |
| PRISM | 0 | Not recorded | 68fa7c471e209131cea8d5cd |
| PRISM | 0 | Not recorded | 68fa7c471e209131cea8d5cd |
| PRISM Quant | 0 | Not recorded | 68fa7c471e209131cea8d5cd |
| PRISM Quant | 0.001 | Not recorded | 68fa7c471e209131cea8d5cd |
| PRISM Quant | 0.002 | Not recorded | 68fa7c471e209131cea8d5cd |
| PRISM | 0.003 | Not recorded | 68fa7c471e209131cea8d5cd |
| PRISM | 0.003 | Not recorded | 68fa7c471e209131cea8d5cd |
| PRISM | 0.039 | Not recorded | 68fa7c471e209131cea8d5cd |
| PRISM | 0.039 | Not recorded | 68fa7c471e209131cea8d5cd |
| PRISM Quant | 0.04 | Not recorded | 68fa7c471e209131cea8d5cd |

### PRISM

P@1 Precision Loss Mean: -0.001. Result ID: 69c3c07b47f34ce924ad03c7.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7c471e209131cea8d5cd.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7c471e209131cea8d5cd.md>)

Recorded evaluation configuration: {"Model":"Qwen3 4B","Baseline":"HF"}. This record does not assert independent reproduction.

### PRISM

P@1 Precision Loss Mean: -0.001. Result ID: 69c3c07b47f34ce924ad03c8.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7c471e209131cea8d5cd.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7c471e209131cea8d5cd.md>)

Recorded evaluation configuration: {"Model":"Qwen3 4B","Baseline":"HF Offload"}. This record does not assert independent reproduction.

### PRISM Quant

P@1 Precision Loss Mean: 0. Result ID: 69c3c07b47f34ce924ad03c9.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7c471e209131cea8d5cd.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7c471e209131cea8d5cd.md>)

Recorded evaluation configuration: {"Model":"Qwen3 4B","Baseline":"HF Quant"}. This record does not assert independent reproduction.

### PRISM

P@1 Precision Loss Mean: 0. Result ID: 69c3c07b47f34ce924ad03cd.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7c471e209131cea8d5cd.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7c471e209131cea8d5cd.md>)

Recorded evaluation configuration: {"Model":"Bge M3","Baseline":"HF"}. This record does not assert independent reproduction.

### PRISM

P@1 Precision Loss Mean: 0. Result ID: 69c3c07b47f34ce924ad03ce.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7c471e209131cea8d5cd.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7c471e209131cea8d5cd.md>)

Recorded evaluation configuration: {"Model":"Bge M3","Baseline":"HF Offload"}. This record does not assert independent reproduction.

### PRISM

P@1 Precision Loss Mean: 0. Result ID: 69c3c07b47f34ce924ad03d0.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7c471e209131cea8d5cd.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7c471e209131cea8d5cd.md>)

Recorded evaluation configuration: {"Model":"Bge MiniCPM","Baseline":"HF"}. This record does not assert independent reproduction.

### PRISM

P@1 Precision Loss Mean: 0. Result ID: 69c3c07b47f34ce924ad03d1.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7c471e209131cea8d5cd.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7c471e209131cea8d5cd.md>)

Recorded evaluation configuration: {"Model":"Bge MiniCPM","Baseline":"HF Offload"}. This record does not assert independent reproduction.

### PRISM Quant

P@1 Precision Loss Mean: 0. Result ID: 69c3c07b47f34ce924ad03d2.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7c471e209131cea8d5cd.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7c471e209131cea8d5cd.md>)

Recorded evaluation configuration: {"Model":"Bge MiniCPM","Baseline":"HF Quant"}. This record does not assert independent reproduction.

### PRISM Quant

P@1 Precision Loss Mean: 0.001. Result ID: 69c3c07b47f34ce924ad03c6.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7c471e209131cea8d5cd.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7c471e209131cea8d5cd.md>)

Recorded evaluation configuration: {"Model":"Qwen3 0.6B","Baseline":"HF Quant"}. This record does not assert independent reproduction.

### PRISM Quant

P@1 Precision Loss Mean: 0.002. Result ID: 69c3c07b47f34ce924ad03cf.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7c471e209131cea8d5cd.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7c471e209131cea8d5cd.md>)

Recorded evaluation configuration: {"Model":"Bge M3","Baseline":"HF Quant"}. This record does not assert independent reproduction.

### PRISM

P@1 Precision Loss Mean: 0.003. Result ID: 69c3c07b47f34ce924ad03c4.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7c471e209131cea8d5cd.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7c471e209131cea8d5cd.md>)

Recorded evaluation configuration: {"Model":"Qwen3 0.6B","Baseline":"HF"}. This record does not assert independent reproduction.

### PRISM

P@1 Precision Loss Mean: 0.003. Result ID: 69c3c07b47f34ce924ad03c5.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7c471e209131cea8d5cd.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7c471e209131cea8d5cd.md>)

Recorded evaluation configuration: {"Model":"Qwen3 0.6B","Baseline":"HF Offload"}. This record does not assert independent reproduction.

### PRISM

P@1 Precision Loss Mean: 0.039. Result ID: 69c3c07b47f34ce924ad03ca.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7c471e209131cea8d5cd.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7c471e209131cea8d5cd.md>)

Recorded evaluation configuration: {"Model":"Qwen3 8B","Baseline":"HF"}. This record does not assert independent reproduction.

### PRISM

P@1 Precision Loss Mean: 0.039. Result ID: 69c3c07b47f34ce924ad03cb.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7c471e209131cea8d5cd.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7c471e209131cea8d5cd.md>)

Recorded evaluation configuration: {"Model":"Qwen3 8B","Baseline":"HF Offload"}. This record does not assert independent reproduction.

### PRISM Quant

P@1 Precision Loss Mean: 0.04. Result ID: 69c3c07b47f34ce924ad03cc.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7c471e209131cea8d5cd.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7c471e209131cea8d5cd.md>)

Recorded evaluation configuration: {"Model":"Qwen3 8B","Baseline":"HF Quant"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.
