Code Retrieval on MTEB Code (test)
98.1Apps ScoreCodeR-all-1.5B
Evaluation results
| Method | Links | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| CodeR-all-1.5BPruning=(500, 1000)2026.03 | 98.1 | 46.7 | 64.4 | 90.5 | 89.5 | 98.3 | 70.8 | 94.4 | 46.1 | 95.4 | 90.6 | 94.4 | 81.8 | 81.1 | |
| C2-LLM-7BPruning=(500, 1000)2026.03 | 86.7 | 39.8 | 75.8 | 91.1 | 89.8 | 97.9 | 81.5 | 92.5 | 34.1 | 94.9 | 90.7 | 94.3 | 79.7 | 80.7 | |
| SPLADE-Code-8BPruning=(500, 1000)2026.03 | 86.7 | 32.5 | 75.7 | 92.1 | 88.9 | 94.6 | 77.1 | 94.6 | 28.1 | 96.5 | 87.6 | 93.9 | 77.9 | 79 | |
| CodeXEmb-7BPruning=(500, 1000)2026.03 | 85.4 | 42.5 | 78.9 | — | 89.7 | 98 | — | 94.5 | 40.5 | 96.3 | 87.5 | 68.8 | 78.2 | — | |
| Dense-8BPruning=(500, 1000)2026.03 | 82.1 | 32.2 | 72.2 | 78.9 | 88.9 | 95.2 | 68.6 | 93.1 | 28.2 | 95.8 | 80.6 | 91.6 | 76 | 75.6 | |
| CodeR-coir-1.5BPruning=(500, 1000)2026.03 | 81.5 | 46.6 | 64.2 | — | 89.1 | 97.8 | — | 94.4 | 46.3 | 95.1 | 89.6 | 93.2 | 79.8 | — | |
| CodeXEmb-2BPruning=(500, 1000)2026.03 | 76.9 | 40.5 | 78.4 | — | 87.9 | 97.7 | — | 90.3 | 38.6 | 94.5 | 86.4 | 65.5 | 75.7 | — | |
| SPLADE-Code-0.6BPruning=(500, 1000)2026.03 | 66.5 | 35.4 | 74.3 | 90.5 | 86.2 | 91.4 | 68.4 | 90.9 | 31.4 | 93.3 | 84.7 | 92 | 74.6 | 75.4 | |
| C2-LLM-0.5BPruning=(500, 1000)2026.03 | 61 | 38.3 | 74.1 | 89.2 | 86.7 | 96.3 | 71.4 | 84.3 | 34 | 89.4 | 88.6 | 92.3 | 72.6 | 75.4 | |
| Dense-0.6BPruning=(500, 1000)2026.03 | 51.8 | 33.3 | 71.1 | 77.5 | 85.8 | 92 | 58.7 | 86.2 | 32.6 | 91.5 | 80.2 | 89.4 | 71.4 | 70.8 | |
| Jina-v2-codePruning=(500, 1000)2026.03 | 16.3 | 41 | 44.2 | — | 84 | 82.7 | — | 86.6 | 30.5 | 89.4 | 69 | 52.1 | 59.6 | — | |
| BM25Pruning=(500, 1000)2026.03 | 4.7 | 18.7 | 24.9 | 63.5 | 70.3 | 59.3 | 53.3 | 47.7 | 34.4 | 70.2 | 68.1 | 59.1 | 45.8 | 47.9 |