Text Embedding on MTEB English v1 (Task Averages)
67.6Classification ScoreCrossBERT
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| CrossBERTParams=279M, FLOPS=4.1e19, tps=207k, Evaluation Protocol=Full Contrastive Finetuning, Finetuning Dataset=MS-MARCO2026.07 | 67.6 | 31.5 | 80.5 | 52.2 | 42.5 | 75.3 | 31.6 | 54.5 | 54.1 | |
| OptiBERTParams=239M, FLOPS=7.0e19, Evaluation Protocol=Full Contrastive Finetuning, Finetuning Dataset=AllNLI only2026.07 | 67.5 | 36.1 | 75.8 | 48.1 | 23.3 | 79.7 | 30 | 51.5 | — | |
| CrossBERTParams=279M, FLOPS=4.1e19, tps=207k, Evaluation Protocol=Contrastive Finetuning over frozen features, Finetuning Dataset=MS-MARCO2026.07 | 65.7 | 37.1 | 78 | 51.4 | 40.7 | 72.3 | 30.7 | 53.7 | 53.6 | |
| BERTParams=239M, FLOPS=6.8e19, tps=123k, Evaluation Protocol=Full Contrastive Finetuning, Finetuning Dataset=MS-MARCO2026.07 | 62.7 | 35.1 | 80.7 | 51.5 | 43 | 75.8 | 29.8 | 54.1 | 53.9 | |
| ModernBERTParams=352M, FLOPS=4.9e21, Evaluation Protocol=Full Contrastive Finetuning, Finetuning Dataset=MSMARCO, StackOverFlowDupQuestion, Fever, STS12, STSBenchmark, AllNLI2026.07 | 62.4 | 38.7 | 65.5 | 50.1 | 23.1 | 68.3 | 27.8 | 46.9 | — | |
| ElectraParams=258M, FLOPS=7.5e19, tps=116k, Evaluation Protocol=Full Contrastive Finetuning, Finetuning Dataset=MS-MARCO2026.07 | 62.1 | 27.7 | 77.9 | 48.8 | 34.3 | 72.4 | 29.1 | 50.3 | 49 | |
| NeoBERTParams=198M, FLOPS=2.9e21, Evaluation Protocol=Full Contrastive Finetuning, Finetuning Dataset=MSMARCO, StackOverFlowDupQuestion, Fever, STS12, STSBenchmark, AllNLI2026.07 | 61.6 | 40.8 | 76.2 | 51.2 | 31.6 | 74.8 | 30.7 | 51.3 | — | |
| BERTParams=239M, FLOPS=6.8e19, tps=123k, Evaluation Protocol=Contrastive Finetuning over frozen features, Finetuning Dataset=MS-MARCO2026.07 | 60.1 | 29.7 | 64.6 | 43.7 | 19.2 | 62.7 | 30.5 | 44.3 | 42.1 | |
| DeBERTaV3Params=304M, FLOPS=>2e20, Evaluation Protocol=Full Contrastive Finetuning, Finetuning Dataset=MSMARCO, StackOverFlowDupQuestion, Fever, STS12, STSBenchmark, AllNLI2026.07 | 45.9 | 16.4 | 45 | 40.8 | 4 | 40.1 | 29.9 | 27.1 | — | |
| ElectraParams=258M, FLOPS=7.5e19, tps=116k, Evaluation Protocol=Contrastive Finetuning over frozen features, Finetuning Dataset=MS-MARCO2026.07 | 41.3 | 12.8 | 35.4 | 32.4 | 0 | 32.8 | 26.9 | 26.2 | 22.4 |