Code Retrieval on MBPP (val)
92nDCG@5Retriever Fine-tuning
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Retriever Fine-tuningRetriever=Qwen3-Emb-4B2026.04 | 92 | 1 | |
| Policy OptimizationRetriever=Qwen3-Emb-4B2026.04 | 91.6 | 0.6 | |
| Joint AdaptationRetriever=Qwen3-Emb-4B2026.04 | 91.1 | 0.1 | |
| Direct RetrievalRetriever=Qwen3-Emb-4B2026.04 | 91 | 0 | |
| Zero-shot TransformationRetriever=Qwen3-Emb-4B2026.04 | 90.5 | -0.5 | |
| Policy OptimizationRetriever=Qwen3-Emb-0.6B2026.04 | 90.2 | 3.1 | |
| Zero-shot TransformationRetriever=Qwen3-Emb-0.6B2026.04 | 89.9 | 2.8 | |
| Joint AdaptationRetriever=Qwen3-Emb-0.6B2026.04 | 89.3 | 2.2 | |
| Joint AdaptationRetriever=Jina-Colbert-V22026.04 | 88.4 | 12.2 | |
| Retriever Fine-tuningRetriever=Qwen3-Emb-0.6B2026.04 | 87.6 | 0.5 | |
| Zero-shot TransformationRetriever=OpenAI text-emb-large2026.04 | 87.2 | 2.4 | |
| Policy OptimizationRetriever=OpenAI text-emb-large2026.04 | 87.2 | 2.4 | |
| Policy OptimizationRetriever=Jina-Colbert-V22026.04 | 87.2 | 11 | |
| Direct RetrievalRetriever=Qwen3-Emb-0.6B2026.04 | 87.1 | 0 | |
| Retriever Fine-tuningRetriever=Jina-Colbert-V22026.04 | 86.4 | 10.2 | |
| Policy OptimizationRetriever=OpenAI text-emb-small2026.04 | 85.2 | 10.6 | |
| Zero-shot TransformationRetriever=Jina-Colbert-V22026.04 | 85.2 | 9 | |
| Direct RetrievalRetriever=OpenAI text-emb-large2026.04 | 84.8 | 0 | |
| Zero-shot TransformationRetriever=OpenAI text-emb-small2026.04 | 84 | 9.4 | |
| Direct RetrievalRetriever=Jina-Colbert-V22026.04 | 76.2 | 0 | |
| Direct RetrievalRetriever=OpenAI text-emb-small2026.04 | 74.6 | 0 | |
| Policy OptimizationRetriever=BM252026.04 | 57.7 | 55.3 | |
| Zero-shot TransformationRetriever=BM252026.04 | 51.2 | 48.8 | |
| Direct RetrievalRetriever=BM252026.04 | 2.4 | 0 |