Multilingual Long-Doc Retrieval on MLDR (test)
65Average Retrieval ScoreM3-Embedding (All)
Evaluation Results
| Method | Links | |||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| M3-Embedding (All)Max Length=81922024.02 | 65 | 64.7 | 57.9 | 63.8 | 86.8 | 83.9 | 52.2 | 75.5 | 60.1 | 55.7 | 85.4 | 73.8 | 44.7 | 40 | — | |
| M3-Embedding (Dense+Sparse)Max Length=81922024.02 | 64.8 | 63 | 56.4 | 64.2 | 88.7 | 84.2 | 52.3 | 75.8 | 58.5 | 53.1 | 86 | 75.6 | 42.9 | 42 | — | |
| M3-Embedding (Sparse)Max Length=81922024.02 | 62.2 | 58.7 | 53 | 62.1 | 87.4 | 82.7 | 49.6 | 74.7 | 53.9 | 47.9 | 85.2 | 72.9 | 40.3 | 40.5 | — | |
| M3-Embedding (Multi-vec)Max Length=81922024.02 | 57.6 | 56.6 | 50.4 | 55.8 | 79.5 | 77.2 | 46.6 | 66.8 | 52.8 | 48.8 | 77.5 | 64.2 | 39.4 | 32.7 | — | |
| BM25Max Length=81922024.02 | 53.6 | 45.1 | 52.6 | 57 | 78 | 75.7 | 43.7 | 70.9 | 36.2 | 25.7 | 82.6 | 61.3 | 33.6 | 34.6 | — | |
| M3-Embedding (Dense)Max Length=81922024.02 | 52.5 | 47.6 | 46.1 | 48.9 | 74.8 | 73.8 | 40.7 | 62.7 | 50.9 | 42.9 | 74.4 | 59.5 | 33.6 | 26 | — | |
| M3-Embedding (Dense-w.o.long (MCLS))Max Length=81922024.02 | 45 | 37.9 | 43.3 | 41.2 | 67.7 | 64.6 | 32 | 55.8 | 43.4 | 33.1 | 67.8 | 52.8 | 27.2 | 18.2 | — | |
| E5mistral-7bMax Length=81922024.02 | 42.6 | 29.6 | 40.6 | 43.3 | 70.2 | 60.5 | 23.2 | 55.3 | 41.6 | 32.7 | 69.5 | 52.4 | 18.2 | 16.8 | — | |
| M3-Embedding (Dense-w.o.long)Max Length=81922024.02 | 41.2 | 35.4 | 35.2 | 37.5 | 64 | 59.3 | 28.8 | 53.1 | 41.7 | 29.8 | 63.5 | 51.1 | 19.5 | 16.5 | — | |
| jina-embeddings-v2-base-enMax Length=81922024.02 | 37 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| mE5largeMax Length=5122024.02 | 34.2 | 33 | 26.9 | 33 | 51.1 | 49.5 | 21 | 43.1 | 29.9 | 27.1 | 58.7 | 42.4 | 15.9 | 13.2 | — | |
| text-embedding-ada-002Max Length=81912024.02 | 32.5 | 16.3 | 34.4 | 38.7 | 59.8 | 53.9 | 8 | 46.5 | 28.6 | 20.7 | 60.6 | 34.8 | 9 | 11.2 | — | |
| mContrieverMax Length=5122024.02 | 31 | 25.4 | 24.2 | 28.7 | 44.6 | 50.3 | 17.2 | 43.2 | 27.3 | 23.6 | 56.6 | 37.7 | 9 | 15.3 | — | |
| mDPRMax Length=5122024.02 | 23.5 | 15.6 | 17.1 | 23.9 | 34.1 | 39.6 | 14.6 | 35.4 | 23.7 | 16.5 | 43.3 | 28.8 | 3.4 | 9.5 | — | |
| CLSFinetuning Pool=CLS, Finetuning Granularity=-, Evaluation Pool=CLS, Evaluation Granularity=-2026.01 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 29.1 | |
| LMK poolingFinetuning Pool=LMK, Finetuning Granularity=128, Evaluation Pool=LMK, Evaluation Granularity=1282026.01 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 30 | |
| LMK poolingFinetuning Pool=LMK, Finetuning Granularity=Variable, Evaluation Pool=LMK, Evaluation Granularity=322026.01 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 31.4 | |
| LMK poolingFinetuning Pool=LMK, Finetuning Granularity=Variable, Evaluation Pool=LMK, Evaluation Granularity=642026.01 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 31.1 | |
| LMK poolingFinetuning Pool=LMK, Finetuning Granularity=Variable, Evaluation Pool=LMK, Evaluation Granularity=1282026.01 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 30.3 | |
| LMK poolingFinetuning Pool=LMK, Finetuning Granularity=Variable, Evaluation Pool=LMK, Evaluation Granularity=2562026.01 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 29.7 | |
| MeanFinetuning Pool=Mean, Finetuning Granularity=-, Evaluation Pool=Mean, Evaluation Granularity=-2026.01 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 30 |