Retrieval on CodeRAG-Bench
100HumanEval ScoreBM25
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| BM25pruning=(1000, 1000)2026.03 | 100 | 98.6 | 5.2 | 6.7 | 93.2 | 43 | 57.7 | |
| Dense-0.6Bpruning=(1000, 1000)2026.03 | 100 | 99.1 | 29 | 16.9 | 92.1 | 46.4 | 63.9 | |
| Dense-8Bpruning=(1000, 1000)2026.03 | 100 | 98.9 | 30.3 | 26.4 | 89 | 50.4 | 65.8 | |
| CodeR-all-1.5Bpruning=(1000, 1000)2026.03 | 100 | 99.2 | 40.8 | 36.1 | 93.1 | 67.4 | 72.8 | |
| CodeR-coir-1.5Bpruning=(1000, 1000)2026.03 | 100 | 98.9 | 37.2 | 32.5 | 92.2 | 64.6 | 70.9 | |
| CodeXEmbed-2Bpruning=(1000, 1000)2026.03 | 100 | 97.4 | 25.4 | 23.9 | 88.7 | 52.4 | 64.6 | |
| SFR-mistral-7Bpruning=(1000, 1000)2026.03 | 100 | 99 | 19.3 | 37.1 | 83.8 | 62.7 | 67 | |
| C2-LLM-7Bpruning=(1000, 1000)2026.03 | 100 | 99.1 | 34.7 | 31.2 | 72.3 | 53.2 | 65.1 | |
| SPLADE-Code-0.6Bpruning=(1000, 1000)2026.03 | 100 | 99.2 | 21.1 | 29.1 | 95.8 | 39.5 | 64.1 | |
| SPLADE-Code-8Bpruning=(1000, 1000)2026.03 | 100 | 99.4 | 42 | 38.8 | 94.2 | 52.6 | 71.2 | |
| C2-LLM-0.5Bpruning=(1000, 1000)2026.03 | 99.8 | 98.6 | 18.1 | 21 | 67.6 | 44.6 | 58.3 |