Generative Retrieval on JD.com industrial dataset (test)
5.75Hallucination RateSFT Baseline
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| SFT BaselineTraining Data=Full 7-day dataset2026.02 | 5.75 | 29.58 | 54.33 | 66.7 | 24.15 | 37.55 | 61.39 | 67.75 | 28.59 | |
| RAD-DPO (w/o TLGD)Ablation=without Token-level Gradient Distribution, Training Data=30M search sessions subset2026.02 | 6.47 | 30.25 | 55.86 | 67.83 | 24.7 | 38.72 | 62.37 | 68.64 | 29.39 | |
| RAD-DPOTraining Data=30M search sessions subset2026.02 | 6.52 | 30.48 | 55.93 | 67.94 | 24.81 | 38.88 | 62.47 | 68.64 | 29.47 | |
| RAD-DPO (w/o Multi-label SFT)Ablation=without Multi-label training strategy, Training Data=30M search sessions subset2026.02 | 6.54 | 30.31 | 55.72 | 67.73 | 24.55 | 38.48 | 62.21 | 68.49 | 29.28 | |
| RAD-DPO (w/o RDRW)Ablation=without Representation-based Dynamic Reward Weighting, Training Data=30M search sessions subset2026.02 | 6.57 | 30.4 | 55.7 | 67.92 | 24.63 | 38.81 | 62.26 | 68.59 | 29.35 | |
| ORPOTraining Data=30M search sessions subset2026.02 | 6.85 | 30.24 | 55.79 | 67.5 | 24.79 | 38.44 | 62.31 | 68.25 | 29.4 | |
| SimPOTraining Data=30M search sessions subset2026.02 | 7.08 | 30.15 | 55.5 | 67.47 | 24.63 | 38.39 | 62.14 | 68.24 | 29.3 | |
| DPOTraining Data=30M search sessions subset2026.02 | 8.38 | 27.31 | 52.22 | 65.24 | 22.86 | 36.48 | 60.01 | 66.42 | 27.47 |