Conversational Recommendation on REDDIT V2
11.78Recall@5Llama-3.2-3B + SFT + Rank-GRPO (exp∞)
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Llama-3.2-3B + SFT + Rank-GRPO (exp∞)Backbone=Llama-3.2-3B, SFT=true, RL Training=Rank-GRPO, Objective=exp∞2025.10 | 11.78 | 17.56 | 21.5 | 23.68 | 9.19 | 11.07 | 12.23 | 12.83 | |
| CRAGType=LLM Prompting-Based2025.10 | 11.46 | 17.15 | 20.3 | 22.12 | 8.85 | 10.65 | 11.64 | 12.27 | |
| GPT-4oMode=zero-shot, Type=LLM Prompting-Based2025.10 | 11.06 | 16.25 | 19.92 | 21.47 | 8.61 | 10.28 | 11.36 | 11.97 | |
| Llama-3.2-3B + SFT + Rank-GRPO (log)Backbone=Llama-3.2-3B, SFT=true, RL Training=Rank-GRPO, Objective=log2025.10 | 11.03 | 16.38 | 20.33 | 23.02 | 8.62 | 10.35 | 11.69 | 12.39 | |
| Llama-3.2-1B + SFT + Rank-GRPO (exp∞)Backbone=Llama-3.2-1B, SFT=true, RL Training=Rank-GRPO, Objective=exp∞2025.10 | 10.39 | 15.69 | 19.37 | 21.65 | 8.13 | 9.85 | 10.92 | 11.53 | |
| Llama-3.2-3B + SFT + Vanilla GRPOBackbone=Llama-3.2-3B, SFT=true, RL Training=Vanilla GRPO2025.10 | 10.34 | 15.68 | 19.46 | 21.86 | 8.24 | 9.96 | 11.06 | 11.7 | |
| Llama-3.2-1B + SFT + Rank-GRPO (log)Backbone=Llama-3.2-1B, SFT=true, RL Training=Rank-GRPO, Objective=log2025.10 | 10.14 | 15.27 | 19.07 | 21.59 | 7.92 | 9.56 | 10.67 | 11.34 | |
| Llama-3.2-1B + SFT + Vanilla GRPOBackbone=Llama-3.2-1B, SFT=true, RL Training=Vanilla GRPO2025.10 | 9.79 | 14.74 | 18.06 | 20.37 | 7.62 | 9.2 | 10.26 | 10.97 | |
| GPT-4o-miniMode=zero-shot, Type=LLM Prompting-Based2025.10 | 9.49 | 13.48 | 16 | 16.87 | 7.47 | 8.77 | 9.5 | 9.73 | |
| Qwen2.5-0.5B + SFT + Rank-GRPO (exp∞)Backbone=Qwen2.5-0.5B, SFT=true, RL Training=Rank-GRPO, Objective=exp∞2025.10 | 9.46 | 14.68 | 18.13 | 20.47 | 7.44 | 9.15 | 10.16 | 10.79 | |
| Qwen2.5-0.5B + SFT + Rank-GRPO (log)Backbone=Qwen2.5-0.5B, SFT=true, RL Training=Rank-GRPO, Objective=log2025.10 | 8.92 | 13.53 | 17.2 | 19.46 | 7.01 | 8.49 | 9.57 | 10.17 | |
| Qwen2.5-0.5B + SFT + Vanilla GRPOBackbone=Qwen2.5-0.5B, SFT=true, RL Training=Vanilla GRPO2025.10 | 8.34 | 12.98 | 16.17 | 18.03 | 6.51 | 8.01 | 8.95 | 9.45 | |
| NBCRSArchitecture=Traditional and Transformer-based2025.10 | 8.01 | 12.9 | 16.55 | 20.19 | 6.61 | 8.33 | 9.54 | 10.48 | |
| Llama-3.2-3B + SFTBackbone=Llama-3.2-3B, SFT=true2025.10 | 7.88 | 11.91 | 14.1 | 15.41 | 6.15 | 7.44 | 8.07 | 8.42 | |
| Llama-3.2-1B + SFTBackbone=Llama-3.2-1B, SFT=true2025.10 | 7.54 | 11.48 | 13.54 | 14.98 | 5.95 | 7.23 | 7.82 | 8.19 | |
| UniCRSArchitecture=Traditional and Transformer-based2025.10 | 7.22 | 10.53 | 13.44 | 14.94 | 5.48 | 6.4 | 7.26 | 7.78 | |
| Qwen2.5-0.5B + SFTBackbone=Qwen2.5-0.5B, SFT=true2025.10 | 6.42 | 10.27 | 12.12 | 13.08 | 5.02 | 6.25 | 6.78 | 7.04 | |
| KGSFArchitecture=Traditional and Transformer-based2025.10 | 5.79 | 9.21 | 12.46 | 14.33 | 4.05 | 5.03 | 5.99 | 6.62 | |
| Llama-3.2-3BBackbone=Llama-3.2-3B, Mode=zero-shot2025.10 | 5.74 | 8.05 | 9.12 | 9.61 | 4.63 | 5.35 | 5.66 | 5.8 | |
| KBRDArchitecture=Traditional and Transformer-based2025.10 | 4.44 | 8.13 | 10.58 | 12.23 | 3.05 | 4.18 | 4.9 | 5.45 | |
| Llama-3.2-1BBackbone=Llama-3.2-1B, Mode=zero-shot2025.10 | 1.87 | 2.4 | 2.57 | 2.63 | 1.57 | 1.7 | 1.75 | 1.76 | |
| RedialArchitecture=Traditional and Transformer-based2025.10 | 1.03 | 2.28 | 2.74 | 3.22 | 0.73 | 1.13 | 1.28 | 1.43 | |
| Qwen2.5-0.5BBackbone=Qwen2.5-0.5B, Mode=zero-shot2025.10 | 0.21 | 0.21 | 0.21 | 0.21 | 0.23 | 0.22 | 0.21 | 0.21 |