Reasoning-based Retrieval on BRIGHT 1.0 (test)
61.77NDCG@10 (Bio.)HGPO
Evaluation results
| Method | Links | |||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| HGPOBackbone=Qwen3-4B-Thinking-2507, Retriever=Diver-4B+BM252026.05 | 61.77 | 59.53 | 27.87 | 41.93 | 26.7 | 33.99 | 32.72 | 30.81 | 19.72 | 16.07 | 43.02 | 45.16 | 36.61 | |
| Tree-GRPOBackbone=Qwen3-4B-Thinking-2507, Retriever=Diver-4B+BM252026.05 | 60.19 | 56.29 | 27.91 | 43.47 | 27.6 | 31.09 | 30.09 | 35.91 | 16.92 | 15.59 | 43.12 | 48.47 | 36.39 | |
| Diver2026.05 | 60 | 55.9 | 31.8 | 47.9 | 27.1 | 33.9 | 31.9 | 35.1 | 23.1 | 16.8 | 36.9 | 46.6 | 37.2 | |
| RICE-POBackbone=Qwen3-4B-Thinking-2507, Retriever=Diver-4B+BM252026.05 | 59.32 | 59.15 | 28.49 | 42 | 28.18 | 35 | 30.73 | 35.37 | 17.14 | 18.97 | 45.39 | 49.37 | 37.76 | |
| GRPOBackbone=Qwen3-4B-Thinking-2507, Retriever=Diver-4B+BM252026.05 | 58.03 | 56.82 | 29.08 | 42.57 | 27.97 | 34.52 | 27.28 | 36.61 | 17.44 | 12.44 | 44.92 | 49.29 | 36.42 | |
| GiGPOBackbone=Qwen3-4B-Thinking-2507, Retriever=Diver-4B+BM252026.05 | 57.82 | 57.8 | 27.91 | 41.32 | 27.94 | 32.75 | 29.35 | 35.69 | 17.91 | 16.15 | 45.94 | 45.19 | 36.31 | |
| Prompting agentsBackbone=Qwen3-4B-Thinking-2507, Retriever=Diver-4B+BM252026.05 | 54.36 | 54.89 | 27.04 | 40.54 | 26.12 | 32.44 | 29.38 | 30.8 | 15.36 | 12.99 | 40.38 | 47.27 | 34.3 | |
| TongSearch-QR2026.05 | 46.2 | 45.1 | 31.2 | 39.6 | 25.3 | 28.7 | 28.4 | 31.2 | 16.3 | 10.8 | 40 | 39.3 | 31.9 | |
| RICE-POBackbone=DeepSeek-R1-Distill-Qwen-1.5B, Retriever=Diver-4B+BM252026.05 | 40.17 | 43.79 | 18.35 | 32.13 | 17.47 | 22.61 | 22.54 | 34.13 | 10.79 | 10.77 | 40.06 | 37 | 27.49 | |
| Tree-GRPOBackbone=DeepSeek-R1-Distill-Qwen-1.5B, Retriever=Diver-4B+BM252026.05 | 37.24 | 44.09 | 19.42 | 28.73 | 18.99 | 22.99 | 21.65 | 29.25 | 11.73 | 10.79 | 30.68 | 36.49 | 26 | |
| GiGPOBackbone=DeepSeek-R1-Distill-Qwen-1.5B, Retriever=Diver-4B+BM252026.05 | 37 | 41.58 | 23.41 | 29.1 | 16.81 | 22.28 | 21.41 | 33.53 | 11.83 | 7.96 | 31.14 | 34.46 | 25.88 | |
| HGPOBackbone=DeepSeek-R1-Distill-Qwen-1.5B, Retriever=Diver-4B+BM252026.05 | 36.36 | 41.73 | 20.02 | 26.94 | 15 | 23.62 | 20.46 | 29.16 | 11.65 | 7.81 | 34.58 | 38.01 | 25.45 | |
| RaDeR2026.05 | 36.1 | 42.9 | 25.2 | 37.9 | 16.6 | 27.4 | 25 | 34.8 | 11.9 | 12 | 37.7 | 43.4 | 29.2 | |
| GRPOBackbone=DeepSeek-R1-Distill-Qwen-1.5B, Retriever=Diver-4B+BM252026.05 | 34.5 | 45.61 | 18.71 | 31.68 | 17.86 | 19.88 | 22.38 | 27.98 | 11.25 | 5.42 | 31.08 | 33.48 | 24.98 | |
| DeepRetrieval2026.05 | 22.48 | 25.6 | 17.06 | 21.13 | 13.46 | 16.75 | 16.93 | 24.99 | 8.13 | 3.88 | 27.9 | 22.01 | 18.36 | |
| Prompting agentsBackbone=DeepSeek-R1-Distill-Qwen-1.5B, Retriever=Diver-4B+BM252026.05 | 20.41 | 25.14 | 9.74 | 13.77 | 7.99 | 6.81 | 8.79 | 13.08 | 7.01 | 3.03 | 13.48 | 11.74 | 11.75 |