Code Retrieval on FreshStack (val)
43.8nDCG@5Retriever Fine-tuning
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Retriever Fine-tuningRetriever=Qwen3-Emb-4B2026.04 | 43.8 | 1.4 | |
| Direct RetrievalRetriever=Qwen3-Emb-4B2026.04 | 42.4 | 0 | |
| Joint AdaptationRetriever=Qwen3-Emb-4B2026.04 | 40.3 | -2.1 | |
| Policy OptimizationRetriever=Qwen3-Emb-4B2026.04 | 40.1 | -2.3 | |
| Joint AdaptationRetriever=Qwen3-Emb-0.6B2026.04 | 38.6 | 1.2 | |
| Zero-shot TransformationRetriever=Qwen3-Emb-4B2026.04 | 38.4 | -4 | |
| Direct RetrievalRetriever=Qwen3-Emb-0.6B2026.04 | 37.4 | 0 | |
| Retriever Fine-tuningRetriever=Qwen3-Emb-0.6B2026.04 | 37 | -0.4 | |
| Policy OptimizationRetriever=Qwen3-Emb-0.6B2026.04 | 36.7 | -0.7 | |
| Zero-shot TransformationRetriever=Qwen3-Emb-0.6B2026.04 | 35.6 | -1.8 | |
| Policy OptimizationRetriever=OpenAI text-emb-large2026.04 | 34.6 | 2.3 | |
| Zero-shot TransformationRetriever=OpenAI text-emb-large2026.04 | 34 | 1.7 | |
| Zero-shot TransformationRetriever=OpenAI text-emb-small2026.04 | 32.7 | 3.5 | |
| Policy OptimizationRetriever=OpenAI text-emb-small2026.04 | 32.7 | 3.5 | |
| Direct RetrievalRetriever=OpenAI text-emb-large2026.04 | 32.3 | 0 | |
| Direct RetrievalRetriever=OpenAI text-emb-small2026.04 | 29.2 | 0 | |
| Joint AdaptationRetriever=Jina-Colbert-V22026.04 | 22.3 | 4.3 | |
| Retriever Fine-tuningRetriever=Jina-Colbert-V22026.04 | 21.8 | 3.8 | |
| Policy OptimizationRetriever=Jina-Colbert-V22026.04 | 20.5 | 2.5 | |
| Zero-shot TransformationRetriever=Jina-Colbert-V22026.04 | 20 | 2 | |
| Direct RetrievalRetriever=Jina-Colbert-V22026.04 | 18 | 0 | |
| Direct RetrievalRetriever=BM252026.04 | 16.8 | 0 | |
| Zero-shot TransformationRetriever=BM252026.04 | 12.2 | -4.6 | |
| Policy OptimizationRetriever=BM252026.04 | 11.9 | -4.9 |