Recommendation on Steam (HR/NDCG/Repeat/OOD Metrics)
8.68HR@10RecLM-cgen
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| RecLM-cgenMethod Type=LLMs (ours)2025.05 | 8.68 | 4.56 | 5.79 | 3.61 | 0 | 0 | |
| CtrlRecMethod Type=LLMs (finetuned)2025.05 | 7.56 | 3.67 | 5.07 | 3.18 | 1.08 | 2.4 | |
| PALRMethod Type=LLMs (finetuned)2025.05 | 7.39 | 4.08 | 4.88 | 3.05 | 1.05 | 2.46 | |
| RecLM-tokenMethod Type=LLMs (ours)2025.05 | 7.33 | 3.88 | 5.67 | 3.31 | 0 | 0 | |
| LC-RecMethod Type=LLMs (item tokenizer)2025.05 | 7.25 | 3.9 | 4.73 | 3.09 | 0 | 0 | |
| SASRecMethod Type=Traditional Recommenders2025.05 | 6.94 | 3.08 | 4.28 | 2.24 | — | — | |
| IDGenRecMethod Type=LLMs (item tokenizer)2025.05 | 6.82 | 3.44 | 4.16 | 2.48 | 0 | 0 | |
| SETRecMethod Type=LLMs (item tokenizer)2025.05 | 6.26 | 3.03 | 3.46 | 2.13 | 0 | 0 | |
| RecLM-retMethod Type=LLMs (ours)2025.05 | 6 | 2.91 | 3.59 | 2.14 | 0 | 0 | |
| GRU4RecMethod Type=Traditional Recommenders2025.05 | 5.99 | 2.81 | 3.23 | 1.93 | — | — | |
| LETTERMethod Type=LLMs (item tokenizer)2025.05 | 5.69 | 2.87 | 3.35 | 2.12 | 0 | 0 | |
| BIGRecMethod Type=LLMs (finetuned)2025.05 | 3.96 | 2.44 | 2.91 | 2.01 | 0 | 0 | |
| GPT-4oMethod Type=LLMs (frozen)2025.05 | 3.83 | 1.94 | 2.34 | 1.47 | 1.07 | 16.08 | |
| Llama3-cgenMethod Type=LLMs (frozen)2025.05 | 2.61 | 1.25 | 1.47 | 0.88 | 0 | 2.59 | |
| Llama3Method Type=LLMs (frozen)2025.05 | 2.3 | 1.2 | 1.36 | 0.9 | 2.06 | 15.26 |