Retrieval on MAIR (test)
59.38nDCG@10 (Avg)ERA
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| ERAModel=Qwen3-8B, Training label percentage=20%2026.04 | 59.38 | 59.13 | 63.03 | 71.83 | 58.58 | 53.5 | 58.63 | |
| Zero-shotModel=Qwen3-8B, Training label percentage=20%2026.04 | 57.91 | 56.94 | 61.85 | 69.38 | 57.89 | 53.75 | 56.65 | |
| ERAModel=Qwen3-0.6B, Training label percentage=20%2026.04 | 52.19 | 49.41 | 58.49 | 63.57 | 49.99 | 46.28 | 51.76 | |
| ERAModel=OpenAI-large, Training label percentage=20%2026.04 | 51.31 | 50.16 | 56.88 | 61.26 | 42.53 | 44.63 | 52.46 | |
| Zero-shotModel=Qwen3-0.6B, Training label percentage=20%2026.04 | 50.37 | 48.48 | 58.34 | 60.56 | 48.58 | 44.55 | 49.17 | |
| Zero-shotModel=OpenAI-large, Training label percentage=20%2026.04 | 46.64 | 44.05 | 49.25 | 57.58 | 36.21 | 42.29 | 48.57 | |
| Embedding AdapterModel=Qwen3-8B, Training label percentage=20%2026.04 | 46.3 | 43.23 | 53.44 | 58.26 | 42.75 | 37.12 | 47.01 | |
| ERAModel=OpenAI-small, Training label percentage=20%2026.04 | 44.95 | 42.34 | 48.33 | 56.88 | 37.16 | 38.02 | 46.87 | |
| ERAModel=BGE-M3, Training label percentage=20%2026.04 | 42.14 | 40.28 | 40.25 | 60.77 | 41.78 | 32.35 | 44.08 | |
| Embedding AdapterModel=Qwen3-0.6B, Training label percentage=20%2026.04 | 38.94 | 34.49 | 48.68 | 51.02 | 35.15 | 32.51 | 38.26 | |
| Embedding AdapterModel=OpenAI-large, Training label percentage=20%2026.04 | 36.96 | 33.72 | 38.41 | 44.82 | 26.33 | 35.6 | 38.91 | |
| Zero-shotModel=OpenAI-small, Training label percentage=20%2026.04 | 36.74 | 35.62 | 41.35 | 50.36 | 31.77 | 30.64 | 36.67 | |
| Zero-shotModel=BGE-M3, Training label percentage=20%2026.04 | 34.03 | 34.81 | 34.85 | 52.44 | 37.01 | 25.25 | 33.28 | |
| Embedding AdapterModel=OpenAI-small, Training label percentage=20%2026.04 | 29.67 | 23.68 | 34.1 | 38.17 | 25.34 | 26.64 | 30.66 | |
| Embedding AdapterModel=BGE-M3, Training label percentage=20%2026.04 | 26.18 | 21.66 | 30.44 | 42.09 | 25.24 | 18.76 | 26.55 |