Information Retrieval on FollowIR
0.49MAPQwen3-Embedding-8B
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Qwen3-Embedding-8BSetting=Optimized2026.05 | 0.49 | — | — | — | — | — | — | — | — | — | |
| Qwen3-Embedding-8BSetting=Optimized2026.05 | 0.49 | — | — | — | — | — | — | — | — | — | |
| Qwen3-Embedding-8BSetting=Optimized2026.05 | 0.49 | — | — | — | — | — | — | — | — | — | |
| Qwen3-Embedding-8BSetting=HyDE Optimized2026.05 | 0.48 | — | — | — | — | — | — | — | — | — | |
| Qwen3-Embedding-8BSetting=Rerank-only2026.05 | 0.47 | — | — | — | — | — | — | — | — | — | |
| Llama-Embed-Nemotron-8BSetting=Optimized2026.05 | 0.47 | — | — | — | — | — | — | — | — | — | |
| Llama-Embed-Nemotron-8BSetting=HyDE Optimized2026.05 | 0.47 | — | — | — | — | — | — | — | — | — | |
| Qwen3-Embedding-8BSetting=Rerank-only2026.05 | 0.47 | — | — | — | — | — | — | — | — | — | |
| Llama-Embed-Nemotron-8BSetting=Optimized2026.05 | 0.47 | — | — | — | — | — | — | — | — | — | |
| Qwen3-Embedding-8BSetting=Rerank-only2026.05 | 0.47 | — | — | — | — | — | — | — | — | — | |
| Llama-Embed-Nemotron-8BSetting=Optimized2026.05 | 0.47 | — | — | — | — | — | — | — | — | — | |
| Qwen3-Embedding-0.6BSetting=Optimized2026.05 | 0.46 | — | — | — | — | — | — | — | — | — | |
| Qwen3-Embedding-0.6BSetting=HyDE Optimized2026.05 | 0.46 | — | — | — | — | — | — | — | — | — | |
| Llama-Embed-Nemotron-8BSetting=Rerank-only2026.05 | 0.46 | — | — | — | — | — | — | — | — | — | |
| Linq-Embed-MistralSetting=Rerank-only2026.05 | 0.46 | — | — | — | — | — | — | — | — | — | |
| E5-Mistral-7BSetting=Optimized2026.05 | 0.46 | — | — | — | — | — | — | — | — | — | |
| E5-Mistral-7BSetting=HyDE Optimized2026.05 | 0.46 | — | — | — | — | — | — | — | — | — | |
| Qwen3-Embedding-0.6BSetting=Optimized2026.05 | 0.46 | — | — | — | — | — | — | — | — | — | |
| Llama-Embed-Nemotron-8BSetting=Rerank-only2026.05 | 0.46 | — | — | — | — | — | — | — | — | — | |
| Linq-Embed-MistralSetting=Rerank-only2026.05 | 0.46 | — | — | — | — | — | — | — | — | — | |
| E5-Mistral-7BSetting=Optimized2026.05 | 0.46 | — | — | — | — | — | — | — | — | — | |
| Qwen3-Embedding-0.6BSetting=Optimized2026.05 | 0.46 | — | — | — | — | — | — | — | — | — | |
| Llama-Embed-Nemotron-8BSetting=Rerank-only2026.05 | 0.46 | — | — | — | — | — | — | — | — | — | |
| Linq-Embed-MistralSetting=Rerank-only2026.05 | 0.46 | — | — | — | — | — | — | — | — | — | |
| E5-Mistral-7BSetting=Optimized2026.05 | 0.46 | — | — | — | — | — | — | — | — | — | |
| Qwen3-Embedding-0.6BSetting=Rerank-only2026.05 | 0.45 | — | — | — | — | — | — | — | — | — | |
| Qwen3-Embedding-8BSetting=Original2026.05 | 0.45 | — | — | — | — | — | — | — | — | — | |
| Llama-Embed-Nemotron-8BSetting=Original2026.05 | 0.45 | — | — | — | — | — | — | — | — | — | |
| Qwen3-Embedding-0.6BSetting=Rerank-only2026.05 | 0.45 | — | — | — | — | — | — | — | — | — | |
| Qwen3-Embedding-8BSetting=Original2026.05 | 0.45 | — | — | — | — | — | — | — | — | — | |
| Llama-Embed-Nemotron-8BSetting=Original2026.05 | 0.45 | — | — | — | — | — | — | — | — | — | |
| Qwen3-Embedding-0.6BSetting=Rerank-only2026.05 | 0.45 | — | — | — | — | — | — | — | — | — | |
| Qwen3-Embedding-8BSetting=Original2026.05 | 0.45 | — | — | — | — | — | — | — | — | — | |
| Llama-Embed-Nemotron-8BSetting=Original2026.05 | 0.45 | — | — | — | — | — | — | — | — | — | |
| Qwen3-Embedding-8BSetting=HyDE2026.05 | 0.44 | — | — | — | — | — | — | — | — | — | |
| E5-Mistral-7BSetting=Rerank-only2026.05 | 0.44 | — | — | — | — | — | — | — | — | — | |
| E5-Mistral-7BSetting=Rerank-only2026.05 | 0.44 | — | — | — | — | — | — | — | — | — | |
| E5-Mistral-7BSetting=Rerank-only2026.05 | 0.44 | — | — | — | — | — | — | — | — | — | |
| Linq-Embed-MistralSetting=Original2026.05 | 0.43 | — | — | — | — | — | — | — | — | — | |
| Linq-Embed-MistralSetting=Optimized2026.05 | 0.43 | — | — | — | — | — | — | — | — | — | |
| Linq-Embed-MistralSetting=Original2026.05 | 0.43 | — | — | — | — | — | — | — | — | — | |
| Linq-Embed-MistralSetting=Optimized2026.05 | 0.43 | — | — | — | — | — | — | — | — | — | |
| Linq-Embed-MistralSetting=Original2026.05 | 0.43 | — | — | — | — | — | — | — | — | — | |
| Linq-Embed-MistralSetting=Optimized2026.05 | 0.43 | — | — | — | — | — | — | — | — | — | |
| Llama-Embed-Nemotron-8BSetting=HyDE2026.05 | 0.42 | — | — | — | — | — | — | — | — | — | |
| Linq-Embed-MistralSetting=HyDE2026.05 | 0.42 | — | — | — | — | — | — | — | — | — | |
| Linq-Embed-MistralSetting=HyDE Optimized2026.05 | 0.42 | — | — | — | — | — | — | — | — | — | |
| Qwen3-Embedding-0.6BSetting=Original2026.05 | 0.41 | — | — | — | — | — | — | — | — | — | |
| Qwen3-Embedding-0.6BSetting=HyDE2026.05 | 0.41 | — | — | — | — | — | — | — | — | — | |
| E5-Mistral-7BSetting=Original2026.05 | 0.41 | — | — | — | — | — | — | — | — | — | |
| E5-Mistral-7BSetting=HyDE2026.05 | 0.41 | — | — | — | — | — | — | — | — | — | |
| Qwen3-Embedding-0.6BSetting=Original2026.05 | 0.41 | — | — | — | — | — | — | — | — | — | |
| E5-Mistral-7BSetting=Original2026.05 | 0.41 | — | — | — | — | — | — | — | — | — | |
| Qwen3-Embedding-0.6BSetting=Original2026.05 | 0.41 | — | — | — | — | — | — | — | — | — | |
| E5-Mistral-7BSetting=Original2026.05 | 0.41 | — | — | — | — | — | — | — | — | — | |
| BGE-M3Size=0.6B2026.03 | — | 1.5 | -1.6 | 21.4 | -1.3 | 7.5 | -8.8 | 10.1 | -3.9 | — | |
| DICE-256Backbone=Dream, Chunk Size=2562026.06 | — | — | — | — | — | — | — | — | — | 39.03 | |
| DICE-256+ov128Backbone=Dream, Chunk Size=256, Overlap=1282026.06 | — | — | — | — | — | — | — | — | — | 38.8 | |
| DICE-256+ov32Backbone=Dream, Chunk Size=256, Overlap=322026.06 | — | — | — | — | — | — | — | — | — | 42.25 | |
| DICE-512Backbone=Dream, Chunk Size=5122026.06 | — | — | — | — | — | — | — | — | — | 35.42 | |
| E5-Mistral-7B-InstructSize=7B2026.03 | — | 2.7 | 2.5 | 28.8 | 0.8 | 13.7 | -7.8 | 15.1 | -1.5 | — | |
| Fair Baseline (LLaMA3.1-8B)Size=8B2026.03 | — | 2.5 | 2.8 | 18.9 | 0.1 | 8.1 | -2.6 | 9.8 | 0.1 | — | |
| Fair Baseline (Qwen3-0.6B)Size=0.6B2026.03 | — | 2.1 | 3.5 | 13.4 | -0.6 | 6.7 | -3.4 | 7.4 | -0.2 | — | |
| Fair Baseline (Qwen3-8B)Size=8B2026.03 | — | 2.8 | 4.4 | 18.9 | 2 | 11.2 | -1.3 | 11 | 1.7 | — | |
| GIRCSE (LLaMA3.1-8B)Size=8B2026.03 | — | 2.9 | 1 | 19.3 | -1.3 | 12.2 | -1.2 | 11.5 | -0.5 | — | |
| GIRCSE (Qwen3-0.6B)Size=0.6B2026.03 | — | 3 | 3.9 | 12.8 | -0.9 | 5.7 | -7.7 | 7.2 | -1.6 | — | |
| GIRCSE (Qwen3-8B)Size=8B2026.03 | — | 3 | 4.2 | 22.6 | 0.7 | 8.5 | 1 | 11.4 | 2 | — | |
| Inbedder (LLaMA2-7B)Size=7B2026.03 | — | 2.6 | 3.2 | 8.5 | 3.1 | 1.6 | -6.4 | 4.2 | -0.1 | — | |
| LaSER (LLaMA3.1-8B)Size=8B2026.03 | — | 3.6 | 2.7 | 22 | 1.6 | 11.1 | -1.9 | 12.2 | 0.8 | — | |
| LaSER (Qwen3-0.6B)Size=0.6B2026.03 | — | 2 | 3.4 | 25 | 1 | 7.4 | -4.5 | 11.5 | 0 | — | |
| LaSER (Qwen3-8B)Size=8B2026.03 | — | 4.1 | 5.8 | 21.8 | 0.2 | 11.4 | 1.3 | 12.5 | 2.4 | — | |
| Multilingual-E5-Large-InstructSize=0.6B2026.03 | — | 2.4 | 2.7 | 25 | 1.2 | 8.5 | -6 | 12 | -0.7 | — | |
| Qwen3-Embedding-0.6BSize=0.6B2026.03 | — | 2.8 | 8.9 | 27.3 | 3.6 | 11.4 | 2.7 | 13.8 | 5.1 | — | |
| Qwen3-Embedding-4BSize=4B2026.03 | — | 5 | 11 | 26.7 | 6.9 | 11.2 | 11.1 | 14.3 | 9.7 | — | |
| Qwen3-Embedding-8BSize=8B2026.03 | — | 3.1 | 6.2 | 25.5 | 8.8 | 10.1 | 6.6 | 12.9 | 7.2 | — | |
| Search-R3 (Qwen2.5-1.5B)Size=1.5B2026.03 | — | 3.2 | 7.1 | 26.2 | -0.1 | 10.1 | 2.7 | 13.2 | 3.2 | — | |
| SINGLEBackbone=Dream2026.06 | — | — | — | — | — | — | — | — | — | 36.66 |