Conversational Recommendation on REDIAL (R@K/NDCG@K)
0.2691Recall@5ConvRec-R1 (Llama-3.2-3B)
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| ConvRec-R1 (Llama-3.2-3B)Backbone=Llama-3.2-3B, Protocol=SFT+Rank-GRPO (exp∞)2025.10 | 0.2691 | 0.374 | 0.4361 | 0.4648 | 0.182 | 0.2167 | 0.2336 | 0.2405 | |
| Rank-GRPO (exp∞)Backbone=Llama-3.2-3B, Training Strategy=SFT+Rank-GRPO (exp∞), Evaluation Setting=off-policy, mu=22025.10 | 0.2648 | 0.3713 | 0.4332 | 0.461 | 0.1792 | 0.2148 | 0.2322 | 0.2385 | |
| ConvRec-R1 (Llama-3.2-1B)Backbone=Llama-3.2-1B, Protocol=SFT+Rank-GRPO (exp∞)2025.10 | 0.2516 | 0.3651 | 0.4255 | 0.4526 | 0.1714 | 0.2092 | 0.2257 | 0.2323 | |
| Rank-GRPO (exp∞)Backbone=Llama-3.2-1B, Training Strategy=SFT+Rank-GRPO (exp∞), Evaluation Setting=off-policy, mu=22025.10 | 0.2479 | 0.3562 | 0.4225 | 0.4429 | 0.1659 | 0.202 | 0.22 | 0.2251 | |
| ConvRec-R1 (Qwen2.5-0.5B)Backbone=Qwen2.5-0.5B, Protocol=SFT+Rank-GRPO (exp∞)2025.10 | 0.2385 | 0.3542 | 0.4153 | 0.4431 | 0.164 | 0.2018 | 0.2184 | 0.2252 | |
| ConvRec-R1 (Llama-3.2-3B)Backbone=Llama-3.2-3B, Protocol=SFT+Rank-GRPO (log)2025.10 | 0.2378 | 0.3391 | 0.4084 | 0.4471 | 0.163 | 0.1964 | 0.2153 | 0.2247 | |
| ConvRec-R1 (Llama-3.2-1B)Backbone=Llama-3.2-1B, Protocol=SFT+Rank-GRPO (log)2025.10 | 0.2365 | 0.3392 | 0.4059 | 0.4472 | 0.1605 | 0.1942 | 0.2125 | 0.2226 | |
| Rank-GRPO (exp∞)Backbone=Qwen2.5-0.5B, Training Strategy=SFT+Rank-GRPO (exp∞), Evaluation Setting=off-policy, mu=22025.10 | 0.2363 | 0.3399 | 0.3993 | 0.435 | 0.1614 | 0.1956 | 0.2118 | 0.2205 | |
| Rank-GRPO (log)Backbone=Llama-3.2-3B, Training Strategy=SFT+Rank-GRPO (log), Evaluation Setting=off-policy, mu=22025.10 | 0.2327 | 0.3404 | 0.3997 | 0.4424 | 0.162 | 0.1976 | 0.2137 | 0.224 | |
| Llama-3.2-3BBackbone=Llama-3.2-3B, Protocol=SFT+Vanilla GRPO2025.10 | 0.2282 | 0.3265 | 0.3936 | 0.43 | 0.1574 | 0.1898 | 0.208 | 0.2168 | |
| ConvRec-R1 (Qwen2.5-0.5B)Backbone=Qwen2.5-0.5B, Protocol=SFT+Rank-GRPO (log)2025.10 | 0.2251 | 0.3253 | 0.3885 | 0.4272 | 0.1531 | 0.1857 | 0.2029 | 0.2123 | |
| Rank-GRPO (log)Backbone=Llama-3.2-1B, Training Strategy=SFT+Rank-GRPO (log), Evaluation Setting=off-policy, mu=22025.10 | 0.2249 | 0.3277 | 0.391 | 0.4345 | 0.1535 | 0.1874 | 0.2046 | 0.2153 | |
| GSPOBackbone=Llama-3.2-3B, Training Strategy=SFT+GSPO, Evaluation Setting=off-policy, mu=22025.10 | 0.2242 | 0.3198 | 0.3781 | 0.4188 | 0.1535 | 0.1853 | 0.2012 | 0.2111 | |
| Vanilla GRPOBackbone=Llama-3.2-3B, Training Strategy=SFT+Vanilla GRPO, Evaluation Setting=off-policy, mu=22025.10 | 0.2237 | 0.3228 | 0.3895 | 0.4268 | 0.1546 | 0.1872 | 0.2053 | 0.2144 | |
| Llama-3.2-1BBackbone=Llama-3.2-1B, Protocol=SFT+Vanilla GRPO2025.10 | 0.2236 | 0.3296 | 0.3924 | 0.4196 | 0.1526 | 0.1876 | 0.2047 | 0.2114 | |
| Vanilla GRPOBackbone=Llama-3.2-1B, Training Strategy=SFT+Vanilla GRPO, Evaluation Setting=off-policy, mu=22025.10 | 0.2218 | 0.3255 | 0.379 | 0.4143 | 0.1495 | 0.1835 | 0.1981 | 0.2066 | |
| GSPOBackbone=Llama-3.2-1B, Training Strategy=SFT+GSPO, Evaluation Setting=off-policy, mu=22025.10 | 0.2156 | 0.3102 | 0.3705 | 0.4157 | 0.1438 | 0.1748 | 0.1911 | 0.2022 | |
| Rank-GRPO (log)Backbone=Qwen2.5-0.5B, Training Strategy=SFT+Rank-GRPO (log), Evaluation Setting=off-policy, mu=22025.10 | 0.214 | 0.3143 | 0.3694 | 0.4038 | 0.147 | 0.1801 | 0.195 | 0.2034 | |
| Qwen2.5-0.5BBackbone=Qwen2.5-0.5B, Protocol=SFT+Vanilla GRPO2025.10 | 0.2114 | 0.3095 | 0.3633 | 0.3917 | 0.1431 | 0.1756 | 0.1902 | 0.1971 | |
| CRAG2025.10 | 0.2097 | 0.3211 | 0.381 | 0.4201 | 0.13 | 0.172 | 0.1889 | 0.1976 | |
| Vanilla GRPOBackbone=Qwen2.5-0.5B, Training Strategy=SFT+Vanilla GRPO, Evaluation Setting=off-policy, mu=22025.10 | 0.2095 | 0.3005 | 0.3506 | 0.3829 | 0.1422 | 0.172 | 0.1857 | 0.1936 | |
| Claude-3.7mode=zero-shot2025.10 | 0.2088 | 0.3135 | 0.3717 | 0.4097 | 0.1346 | 0.1691 | 0.1849 | 0.1941 | |
| GSPOBackbone=Qwen2.5-0.5B, Training Strategy=SFT+GSPO, Evaluation Setting=off-policy, mu=22025.10 | 0.2056 | 0.3005 | 0.3536 | 0.3841 | 0.1379 | 0.1692 | 0.1837 | 0.1911 | |
| NBCRS2025.10 | 0.1848 | 0.2874 | 0.3687 | 0.4108 | 0.1228 | 0.1568 | 0.1788 | 0.1891 | |
| SFTBackbone=Llama-3.2-3B, Training Strategy=SFT, Evaluation Setting=off-policy, mu=22025.10 | 0.1756 | 0.275 | 0.3392 | 0.3793 | 0.1114 | 0.1441 | 0.1615 | 0.1712 | |
| Llama-3.2-3BBackbone=Llama-3.2-3B, Protocol=SFT2025.10 | 0.1756 | 0.275 | 0.3392 | 0.3793 | 0.1114 | 0.1441 | 0.1615 | 0.1712 | |
| SFTBackbone=Llama-3.2-1B, Training Strategy=SFT, Evaluation Setting=off-policy, mu=22025.10 | 0.1714 | 0.2731 | 0.3345 | 0.3715 | 0.1078 | 0.1414 | 0.158 | 0.1671 | |
| Llama-3.2-1BBackbone=Llama-3.2-1B, Protocol=SFT2025.10 | 0.1714 | 0.2731 | 0.3345 | 0.3715 | 0.1078 | 0.1414 | 0.158 | 0.1671 | |
| SFTBackbone=Qwen2.5-0.5B, Training Strategy=SFT, Evaluation Setting=off-policy, mu=22025.10 | 0.1634 | 0.257 | 0.3145 | 0.3512 | 0.1036 | 0.1345 | 0.1502 | 0.159 | |
| Qwen2.5-0.5BBackbone=Qwen2.5-0.5B, Protocol=SFT2025.10 | 0.1634 | 0.257 | 0.3145 | 0.3512 | 0.1036 | 0.1345 | 0.1502 | 0.159 | |
| Llama-3.2-3B (zero-shot)Backbone=Llama-3.2-3B, Training Strategy=zero-shot, Evaluation Setting=off-policy, mu=22025.10 | 0.1473 | 0.203 | 0.2284 | 0.2309 | 0.0972 | 0.1158 | 0.1228 | 0.1234 | |
| Llama-3.2-3BBackbone=Llama-3.2-3B, Protocol=zero-shot2025.10 | 0.1473 | 0.203 | 0.2284 | 0.2309 | 0.0972 | 0.1158 | 0.1228 | 0.1234 | |
| UniCRS2025.10 | 0.1241 | 0.1756 | 0.2156 | 0.233 | 0.0851 | 0.105 | 0.1153 | 0.1199 | |
| Llama-3.2-1B (zero-shot)Backbone=Llama-3.2-1B, Training Strategy=zero-shot, Evaluation Setting=off-policy, mu=22025.10 | 0.1036 | 0.1522 | 0.1672 | 0.1704 | 0.0657 | 0.0818 | 0.0859 | 0.0867 | |
| Llama-3.2-1BBackbone=Llama-3.2-1B, Protocol=zero-shot2025.10 | 0.1036 | 0.1522 | 0.1672 | 0.1704 | 0.0657 | 0.0818 | 0.0859 | 0.0867 | |
| KGSF2025.10 | 0.0995 | 0.1535 | 0.1999 | 0.2235 | 0.063 | 0.0826 | 0.0951 | 0.102 | |
| KBRD2025.10 | 0.0763 | 0.1356 | 0.1697 | 0.1907 | 0.0474 | 0.0686 | 0.0778 | 0.084 | |
| Redial2025.10 | 0.0178 | 0.038 | 0.044 | 0.0502 | 0.0114 | 0.0187 | 0.0203 | 0.0221 | |
| Qwen2.5-0.5B (zero-shot)Backbone=Qwen2.5-0.5B, Training Strategy=zero-shot, Evaluation Setting=off-policy, mu=22025.10 | 0.005 | 0.005 | 0.005 | 0.005 | 0.0042 | 0.0042 | 0.0042 | 0.0042 | |
| Qwen2.5-0.5BBackbone=Qwen2.5-0.5B, Protocol=zero-shot2025.10 | 0.005 | 0.005 | 0.005 | 0.005 | 0.0042 | 0.0042 | 0.0042 | 0.0042 |