Conversion Rate (CVR) Prediction on Industrial-scale Recommender Dataset
91.56AUCSORT(Transformer+ST+LA+QP+AG+QKN+MoE)
Evaluation results
| Method | Links | ||
|---|---|---|---|
| SORT(Transformer+ST+LA+QP+AG+QKN+MoE)Scale=Large, Params=685M, FLOPs=188G2026.03 | 91.56 | 0.34 | |
| TransformerScale=Large, Params=144M, FLOPs=322G2026.03 | 91.22 | — | |
| SORT(Transformer+ST+LA+QP+AG+QKN+MoE)Scale=Base, Params=85M, FLOPs=24G2026.03 | 91.18 | 0.25 | |
| Transformer+special token (ST)Scale=Base, Params=18M, FLOPs=43G2026.03 | 91.13 | 0.2 | |
| Transformer+query pruning (QP)Scale=Base, Params=18M, FLOPs=24G2026.03 | 91.12 | 0.19 | |
| Transformer+QKNorm (QKN)Scale=Base, Params=18M, FLOPs=43G2026.03 | 91.04 | 0.11 | |
| OneTransScale=Base, Params=54M, FLOPs=24G2026.03 | 91.03 | 0.1 | |
| Transformer+local attention (LA)Scale=Base, Params=18M, FLOPs=40G2026.03 | 90.99 | 0.06 | |
| Transformer+MoEScale=Base, Params=83M, FLOPs=43G2026.03 | 90.99 | 0.06 | |
| Transformer+attention gate (AG)Scale=Base, Params=20M, FLOPs=47G2026.03 | 90.97 | 0.04 | |
| TransformerScale=Base, Params=18M, FLOPs=43G2026.03 | 90.93 | — | |
| SORT(Transformer+ST+LA+QP+AG+QKN+MoE)Scale=Small, Params=14M, FLOPs=4G2026.03 | 90.82 | 0.33 | |
| HSTUScale=Base, Params=18M, FLOPs=43G2026.03 | 90.7 | -0.23 | |
| TransformerScale=Small, Params=3M, FLOPs=8G2026.03 | 90.49 | — |