Rating Prediction on Amazon Beauty (test)
1.6Multi. RMSEOur Method (zero-shot CoT)
Evaluation Results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Our Method (zero-shot CoT)Backbone=PaLM 2-M, Learning Setting=Zero-shot, Reasoning strategy=Chain-of-thought2024.07 | 1.6 | 56 | 62 | 37 | 1.14 | 0.236 | 0.503 | 0.339 | 0.665 | |
| No Item DescriptionBackbone=PaLM 2-M, Learning Setting=Zero-shot, Input Feature=Excluding item descriptions2024.07 | 1.66 | 48 | 57 | 21 | 1.33 | 0.235 | 0.504 | 0.34 | 0.667 | |
| No ReviewBackbone=PaLM 2-M, Learning Setting=Zero-shot, Input Feature=Excluding user reviews2024.07 | 1.69 | 48 | 57 | 21 | 1.35 | 0.237 | 0.507 | 0.337 | 0.667 | |
| No Reasoning OutputsBackbone=PaLM 2-M, Learning Setting=Zero-shot, Task type=Rating prediction-only2024.07 | 1.7 | 49 | 57 | 23 | 1.35 | — | — | — | — | |
| Naive Baseline (Avg.)Strategy=Historical rating average of the user2024.07 | 1.75 | 52 | 60 | 25 | 1.35 | — | — | — | — | |
| No Review, No RatingBackbone=PaLM 2-M, Learning Setting=Zero-shot, Input Feature=Excluding user reviews and ratings2024.07 | 1.75 | 43 | 53 | 19 | 1.42 | 0.215 | 0.494 | 0.331 | 0.66 | |
| TAI2Vec-ContEvaluation Setup=Grayscale2026.04 | 1.7782 | — | — | — | — | — | — | — | — | |
| TAI2Vec-DiscEvaluation Setup=Grayscale2026.04 | 1.7805 | — | — | — | — | — | — | — | — | |
| SeqI2VEvaluation Setup=Grayscale2026.04 | 1.8123 | — | — | — | — | — | — | — | — | |
| Item2VecEvaluation Setup=Grayscale2026.04 | 1.8255 | — | — | — | — | — | — | — | — | |
| One-shotBackbone=PaLM 2-M, Learning Setting=One-shot2024.07 | 1.97 | 43 | 57 | 26 | 1.52 | 0.225 | 0.502 | 0.335 | 0.664 |