Recommendation on Movies (HR/NDCG/OOD Metrics)
0.17HR@10RecLM-token
Evaluation Results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| RecLM-tokenMethod Type=LLMs (ours)2025.05 | 0.17 | 0.1622 | 0.1667 | 0.1611 | 1 | 0 | |
| LC-RecMethod Type=LLMs (item tokenizer)2025.05 | 0.1607 | 0.1458 | 0.1532 | 0.1434 | 0 | 0 | |
| SASRecMethod Type=Traditional Recommenders2025.05 | 0.151 | 0.1351 | 0.1422 | 0.1323 | — | — | |
| RecLM-cgenMethod Type=LLMs (ours)2025.05 | 0.1467 | 0.1311 | 0.14 | 0.129 | 0 | 0 | |
| CtrlRecMethod Type=LLMs (finetuned)2025.05 | 0.1347 | 0.1248 | 0.1304 | 0.1234 | 9.02 | 8.13 | |
| PALRMethod Type=LLMs (finetuned)2025.05 | 0.1335 | 0.1244 | 0.1294 | 0.123 | 34.69 | 14.85 | |
| IDGenRecMethod Type=LLMs (item tokenizer)2025.05 | 0.1243 | 0.1064 | 0.1107 | 0.1098 | 0 | 0 | |
| LETTERMethod Type=LLMs (item tokenizer)2025.05 | 0.1205 | 0.1097 | 0.1136 | 0.1002 | 0 | 0 | |
| RecLM-retMethod Type=LLMs (ours)2025.05 | 0.1145 | 0.1052 | 0.1038 | 0.097 | 0 | 0 | |
| SETRecMethod Type=LLMs (item tokenizer)2025.05 | 0.0929 | 0.0823 | 0.0867 | 0.0803 | 0 | 0 | |
| BIGRecMethod Type=LLMs (finetuned)2025.05 | 0.0861 | 0.076 | 0.0823 | 0.0747 | 0 | 0 | |
| GRU4RecMethod Type=Traditional Recommenders2025.05 | 0.0722 | 0.0556 | 0.0625 | 0.0525 | — | — | |
| Llama3-cgenMethod Type=LLMs (frozen)2025.05 | 0.0246 | 0.0106 | 0.0123 | 0.0064 | 0 | 11.91 | |
| Llama3Method Type=LLMs (frozen)2025.05 | 0.0049 | 0.0025 | 0.0029 | 0.0019 | 3.15 | 52.52 | |
| GPT-4oMethod Type=LLMs (frozen)2025.05 | 0.0046 | 0.0028 | 0.0027 | 0.0022 | 0.89 | 61.21 |