Zero-shot Information Retrieval on BEIR

39.1NFCorpus NDCG@10 (Zero-shot)SSR-CLS

Updated 2mo ago

Evaluation Results

Method
SSR-CLS2026.05	39.1	—	—	—	61.2	69.9	40	45.5	—	—	46.6	33.7	88.7	48.4	17.9	82.8	23.5	73.3	76.8	53.4	—	—
SSR-tok2026.05	38.8	—	—	—	60.7	69.5	39.4	45.2	—	—	46.1	33.1	88.3	48.2	17.5	81.9	22.8	72.9	76.4	52.9	—	—
Splade-v32026.05	35.8	—	—	—	58.6	69.2	37.5	43.9	—	—	50.8	29.6	81.4	45.2	15.8	79.8	23.3	71	74.8	51.2	—	—
XTR2023.04	35.3	—	78.9	—	60.9	66.2	43.8	—	—	—	44.2	—	88.1	44.3	17.1	77	24.5	74.3	—	52.7	—	30.9
SPAR2021.10	34	—	73.7	—	57	66.2	39.8	—	—	—	52.6	30.3	89.6	41.9	17.2	66.5	21.4	62.8	39.1	49.4	—	—
XTR2023.04	34	—	73.6	—	53	64.7	34.7	—	—	—	40.7	—	86.1	40.9	14.5	73.7	20.7	71	—	49.1	—	31.3
AligneR2026.05	34	—	—	—	52.4	61.5	33.5	38.8	—	—	28.9	34.8	82.3	41.6	14.3	72.4	18.2	70.3	68.8	46.6	—	—
XTR2026.05	34	—	—	—	52.8	64.7	34.8	45	—	—	40.7	31.3	85.9	40.8	14.6	73.7	20.8	71.1	73.6	48.8	—	—
SPAR2021.10	33.8	—	73.5	—	53.1	67.6	33.7	—	—	—	48.8	27.5	87	42.2	17.1	81.2	24.5	68.3	36.5	49.6	—	—
PLAID2026.05	33.8	—	—	—	56.4	66.7	36.3	39.7	—	—	46.1	26.3	85	44.7	15.4	78.8	17.4	69.2	73.7	49.3	—	—
SPAR2021.10	33.7	—	63	—	51.3	66.4	34.1	—	—	—	45.9	24.9	87.5	42.8	16.9	76.9	24.8	69.5	35.8	48.1	—	—
CITADEL2026.05	33.7	—	—	—	54	66.3	33	39.9	—	—	51.1	34.2	85.3	42.2	15.9	76.6	18.3	69	71.5	49.4	—	—
ColBERTv22026.05	33.5	—	—	—	56.3	66.6	36.1	39.8	—	—	46.3	26.1	85.3	44.9	15.2	78.7	17.3	69.1	73.5	49.2	—	—
Splade-v22026.05	33.5	—	—	—	52.1	68.4	33.8	43.3	—	—	47.6	36.3	83.8	43.4	15.9	78.7	23.5	69.4	71.5	50.1	—	—
COIL2026.05	33.2	—	—	—	52.1	71.3	31.3	35.3	—	—	29.5	28.1	83.8	39.9	15.5	83.8	21.7	70.7	66.8	47.4	—	—
GTR-large2021.10	32.9	—	55.7	—	54.7	57.9	42.4	—	—	—	52.5	21.9	89	39.1	15.8	71.2	26.2	63.9	38.4	47.2	—	—
docT5query2021.10	32.8	—	71.3	—	39.9	58	29.1	—	—	—	34.9	34.7	80.2	33.1	16.2	71.4	20.1	67.5	32.5	44.4	—	—
Contriever2021.10	32.8	—	59.6	—	49.8	63.8	32.9	—	—	—	44.6	23	86.5	41.3	16.5	75.8	23.7	67.7	34.5	46.6	—	—
BM252021.10	32.5	—	65.6	—	32.9	60.3	23.6	—	—	—	31.5	36.7	78.9	31.3	15.8	75.3	21.3	66.5	29.9	43	—	—
SPAR2021.10	32.3	—	60.7	—	55.7	60.9	40.9	—	—	—	51.4	23.5	89.3	41.1	16.3	58	20.5	57.3	37.4	46.1	—	—
GenQ2023.04	31.9	—	61.9	—	35.8	53.4	30.8	—	—	—	49.3	—	83	32.8	14.3	66.9	17.5	64.4	—	43.1	—	18.2
SPAR2021.10	31.8	—	76.4	—	51.1	63	32.3	—	—	—	47.2	30.3	86.9	36.1	15.5	80.2	23.5	62.6	35.8	48.1	—	—
GTR-large2021.10	31.4	—	56.3	—	55.1	57.8	41.1	—	—	—	50.9	22	88.5	39.5	15.5	56.6	19.8	54	36.2	44.6	—	—
ColBERT2026.05	30.7	—	—	—	51.8	59.2	31.7	36	—	—	23.6	20.5	85.7	39.2	14.6	76.9	18.1	66.8	67.9	41.5	—	—
ColBERT2021.10	30.5	—	67.7	—	52.4	59.3	31.7	—	—	—	23.3	20.2	85.4	39.2	14.5	77.1	18.4	67.1	35	44.4	—	—
SPAR2021.10	27.7	—	68.4	—	47.3	53.6	32.1	—	—	—	45	28.1	86.7	32.1	14.1	72.6	23.2	59.6	33.2	44.5	—	—
ANCE2021.10	23.7	—	65.4	—	44.6	45.6	29.5	—	—	—	41.5	24	85.2	28.1	12.2	66.9	19.8	50.7	29.6	40.5	—	—
GTR-XL2021.12	0.343	—	0.584	0.317	0.559	0.591	0.444	0.268	0.35	0.479	0.531	0.23	0.89	0.396	0.159	0.717	0.27	0.635	0.388	0.452	0.453	—
GTR-XXL2021.12	0.342	—	0.501	0.324	0.568	0.599	0.467	0.273	0.346	0.506	0.54	0.256	0.892	0.408	0.161	0.74	0.267	0.662	0.399	0.457	0.458	—
GTR-Large2021.12	0.329	—	0.557	0.32	0.547	0.579	0.424	0.265	0.343	0.47	0.525	0.219	0.89	0.391	0.158	0.712	0.262	0.639	0.384	0.444	0.445	—
docT5query2021.12	0.328	—	0.713	0.431	0.399	0.58	0.291	0.307	0.42	0.437	0.349	0.347	0.802	0.331	0.162	0.714	0.201	0.675	0.325	0.429	0.434	—
BM252021.12	0.325	—	0.656	0.465	0.329	0.603	0.236	0.33	0.398	0.408	0.315	0.367	0.789	0.313	0.158	0.753	0.213	0.665	0.299	0.413	0.423	—
TAS-B2021.12	0.319	—	0.481	0.383	0.463	0.584	0.3	0.289	0.377	0.427	0.429	0.162	0.835	0.384	0.149	0.7	0.228	0.643	0.314	0.414	0.415	—
GenQ2021.12	0.319	—	0.619	0.398	0.358	0.534	0.308	0.281	0.396	0.362	0.493	0.182	0.83	0.328	0.143	0.669	0.175	0.644	0.347	0.41	0.41	—
GTR-Base2021.12	0.308	—	0.539	0.271	0.495	0.535	0.349	0.261	0.337	0.437	0.511	0.205	0.881	0.347	0.149	0.66	0.241	0.6	0.357	0.416	0.416	—
ColBERT2021.12	0.305	—	0.677	0.474	0.524	0.593	0.317	0.274	0.393	0.391	0.233	0.202	0.854	0.392	0.145	0.771	0.184	0.671	0.35	0.429	0.431	—
ANCE2021.12	0.237	—	0.654	0.306	0.446	0.456	0.295	0.249	0.382	0.392	0.415	0.24	0.852	0.281	0.122	0.669	0.198	0.507	0.296	0.389	0.389	—
DPR2021.12	0.189	—	0.332	0.127	0.474	0.391	0.112	0.155	0.161	0.252	0.175	0.131	0.248	0.263	0.077	0.562	0.148	0.318	0.153	0.234	0.237	—