Code Retrieval on HumanEval (val)
98.5nDCG@5Policy Optimization
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Policy OptimizationRetriever=Qwen3-Emb-4B2026.04 | 98.5 | 0.4 | |
| Retriever Fine-tuningRetriever=Qwen3-Emb-4B2026.04 | 98.3 | 0.2 | |
| Joint AdaptationRetriever=Qwen3-Emb-0.6B2026.04 | 98.1 | 4.4 | |
| Direct RetrievalRetriever=Qwen3-Emb-4B2026.04 | 98.1 | 0 | |
| Joint AdaptationRetriever=Qwen3-Emb-4B2026.04 | 97.9 | -0.2 | |
| Policy OptimizationRetriever=Qwen3-Emb-0.6B2026.04 | 97.6 | 3.9 | |
| Zero-shot TransformationRetriever=Qwen3-Emb-4B2026.04 | 97.3 | -0.8 | |
| Zero-shot TransformationRetriever=Qwen3-Emb-0.6B2026.04 | 96.3 | 2.6 | |
| Retriever Fine-tuningRetriever=Qwen3-Emb-0.6B2026.04 | 96.1 | 2.4 | |
| Policy OptimizationRetriever=OpenAI text-emb-large2026.04 | 95.1 | 2.1 | |
| Zero-shot TransformationRetriever=OpenAI text-emb-large2026.04 | 94.8 | 1.8 | |
| Policy OptimizationRetriever=OpenAI text-emb-small2026.04 | 94.6 | 10.5 | |
| Direct RetrievalRetriever=Qwen3-Emb-0.6B2026.04 | 93.7 | 0 | |
| Zero-shot TransformationRetriever=OpenAI text-emb-small2026.04 | 93.6 | 9.5 | |
| Direct RetrievalRetriever=OpenAI text-emb-large2026.04 | 93 | 0 | |
| Joint AdaptationRetriever=Jina-Colbert-V22026.04 | 92.9 | 28.6 | |
| Policy OptimizationRetriever=Jina-Colbert-V22026.04 | 91.7 | 27.4 | |
| Zero-shot TransformationRetriever=Jina-Colbert-V22026.04 | 87.9 | 23.6 | |
| Retriever Fine-tuningRetriever=Jina-Colbert-V22026.04 | 87.2 | 22.9 | |
| Direct RetrievalRetriever=OpenAI text-emb-small2026.04 | 84.1 | 0 | |
| Policy OptimizationRetriever=BM252026.04 | 80.5 | 60.1 | |
| Zero-shot TransformationRetriever=BM252026.04 | 72.1 | 51.7 | |
| Direct RetrievalRetriever=Jina-Colbert-V22026.04 | 64.3 | 0 | |
| Direct RetrievalRetriever=BM252026.04 | 20.4 | 0 |