E-commerce Relevance Classification on Taobao dataset (offline)
67.34F1 Score (Class 1)TaoSR1
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| TaoSR1Training Stage=CoT & DPO & GRPO, Paradigm=post-CoT (respond-then-think)2025.08 | 67.34 | 73.15 | 44.94 | 83.06 | 67.12 | 76.86 | |
| TaoSR1Training Stage=CoT & DPO & GRPO, Paradigm=think-then-respond2025.08 | 66.82 | 72.15 | 39.4 | 82.27 | 65.16 | 75.92 | |
| BERTLayers=242025.08 | 65.74 | 69.63 | 33.87 | 76.06 | 61.33 | 69.36 | |
| TaoSR1Training Stage=CoT & DPO, Paradigm=post-CoT (respond-then-think)2025.08 | 65.74 | 71.95 | 39.43 | 83 | 65.03 | 76.49 | |
| LLM-base2025.08 | 65.19 | 68.86 | 32.91 | 81.9 | 62.22 | 75.04 | |
| Qwen3-30B-A3BParameters=30B-A3B2025.08 | 65.09 | 68.8 | 32.47 | 81.68 | 62.01 | 74.42 | |
| TaoSR1Training Stage=CoT & DPO, Paradigm=think-then-respond2025.08 | 62.9 | 71.2 | 37.96 | 82.25 | 63.58 | 75.54 | |
| TaoSR1Training Stage=CoT, Paradigm=post-CoT (respond-then-think)2025.08 | 57.63 | 72.64 | 27.91 | 81.88 | 60.01 | 75.12 | |
| TaoSR1Training Stage=CoT, Paradigm=think-then-respond2025.08 | 43.3 | 67.54 | 19.68 | 75.62 | 51.54 | 68.22 | |
| Qwen3-0.6BParameters=0.6B2025.08 | 42.02 | 68.13 | 23.5 | 78.14 | 52.95 | 70.29 |