Multilingual Retrieval on Mr. TyDi 1.0 (test) benchmark leaderboard · SOTA2 Research

Multilingual Retrieval on Mr. TyDi 1.0 (test)

72.4MRR@100 (ar)mContriever + Mr. Tydi

Evaluation Results

Method	Links
mContriever + Mr. Tydi2021.12		72.4	67.2	56.6	60.2	63	54.9	55.3	59.7	70.7	90.3	67.3	65.2	94	98.6	92.2	92.7	94.5	88.8	88.9	92.4	93.7	98.9	95.2	93.6
Hybrid2021.12		49.1	53.5	28.4	36.5	45.5	35.5	36.2	42.7	40.5	42	49.2	41.7	86.3	93.7	69.6	78.8	88.7	77.8	70.6	76	78.6	82.7	87.5	80.9
mContriever + MS MARCO2021.12		43.4	42.3	27.1	35.1	42.6	32.4	34.2	36.1	51.2	37.4	40.2	38.4	88.7	91.4	77.2	88.1	89.8	81.7	78.2	83.8	91.4	96.6	90.5	87
BM252021.12		36.7	41.3	15.1	28.8	38.2	21.7	28.1	32.9	39.6	42.4	41.7	33.3	80	87.4	55.1	72.5	84.6	65.6	79.7	66	76.4	81.3	85.3	74.3
XLM-R + MS MARCO2021.12		36.5	41.7	23	32.7	39.2	24.8	32.2	29.3	35.1	54.7	38.5	35.2	79.9	84.2	73.1	81.6	87.4	70.9	71.1	74.1	73.9	91.2	89.5	79.7
mBERT + MS MARCO2021.12		34.8	35.1	25.7	29.6	36.3	27.1	28.1	30	37.4	39.6	20.3	31.3	81.1	88.7	77.8	74.2	81	76.1	66.7	77.6	74.1	89.5	57.8	76.8
mContriever2021.12		27.3	36.3	9.2	21.1	23.5	19.5	22.3	17.5	38.3	22.5	37.2	25	82	89.6	48.8	79.6	81.4	72.8	66.2	68.5	88.7	80.8	90.3	77.2
mDPR2021.12		26	25.8	16.2	11.3	14.6	18.1	21.9	18.5	7.3	10.6	13.5	16.7	62	67.1	47.5	37.5	46.6	53.5	49	49.8	26.4	35.2	45.5	47.3