Search Relevance on Germanic language family Qwen2.5 series benchmark (test)
87.56NDCG@1SERM
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| SERMBase Model Scale=Qwen2.5-7B, Iteration/Distillation Stage=Iteration 32026.01 | 87.56 | 87.71 | 57.79 | |
| SERMBase Model Scale=Qwen2.5-1.5B, Iteration/Distillation Stage=Iteration 32026.01 | 87.3 | 87.62 | 56.78 | |
| SERMBase Model Scale=Qwen2.5-7B, Iteration/Distillation Stage=Iteration 22026.01 | 87.27 | 87.6 | 57.4 | |
| SERMBase Model Scale=Qwen2.5-7B, Iteration/Distillation Stage=Iteration 12026.01 | 87.04 | 87.53 | 56.5 | |
| SERMBase Model Scale=Qwen2.5-1.5B, Iteration/Distillation Stage=Iteration 22026.01 | 87.03 | 87.52 | 56.57 | |
| SERMBase Model Scale=Qwen2.5-7B, Iteration/Distillation Stage=Distillation (0.5B)2026.01 | 86.78 | 87.27 | 56.92 | |
| SERMBase Model Scale=Qwen2.5-1.5B, Iteration/Distillation Stage=Iteration 12026.01 | 86.64 | 87.38 | 55.57 | |
| SERMBase Model Scale=Qwen2.5-1.5B, Iteration/Distillation Stage=Distillation (0.5B)2026.01 | 86.54 | 86.97 | 56.06 | |
| Self-TrainingBase Model Scale=Qwen2.5-1.5B, Iteration/Distillation Stage=Iteration 32026.01 | 85.04 | 86.79 | 55.66 | |
| Self-TrainingBase Model Scale=Qwen2.5-7B, Iteration/Distillation Stage=Iteration 22026.01 | 84.95 | 86.78 | 55.85 | |
| Self-TrainingBase Model Scale=Qwen2.5-1.5B, Iteration/Distillation Stage=Iteration 22026.01 | 84.93 | 86.75 | 55.49 | |
| Self-TrainingBase Model Scale=Qwen2.5-1.5B, Iteration/Distillation Stage=Iteration 12026.01 | 84.91 | 86.77 | 55.22 | |
| Self-TrainingBase Model Scale=Qwen2.5-7B, Iteration/Distillation Stage=Iteration 12026.01 | 84.87 | 86.75 | 55.74 | |
| Self-TrainingBase Model Scale=Qwen2.5-7B, Iteration/Distillation Stage=Iteration 32026.01 | 84.78 | 86.72 | 55.63 | |
| CT+SFTBase Model Scale=Qwen2.5-7B2026.01 | 84.74 | 86.68 | 55.3 | |
| CT+SFTBase Model Scale=Qwen2.5-1.5B2026.01 | 84.59 | 86.63 | 54.75 | |
| Self-TrainingBase Model Scale=Qwen2.5-1.5B, Iteration/Distillation Stage=Distillation (0.5B)2026.01 | 84.17 | 86.34 | 54.77 | |
| Self-TrainingBase Model Scale=Qwen2.5-7B, Iteration/Distillation Stage=Distillation (0.5B)2026.01 | 83.86 | 86.21 | 54.88 |