Rating Prediction on Amazon Movies TV (test)
62Binary AccuracyOur Method (zero-shot CoT)
Evaluation Results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Our Method (zero-shot CoT)Backbone=PaLM 2-M, Learning Setting=Zero-shot, Reasoning strategy=Chain-of-thought2024.07 | 62 | 66 | 40 | 1.06 | 1.53 | 19.4 | 0.465 | 0.296 | 64.7 | |
| Naive Baseline (Avg.)Strategy=Historical rating average of the user2024.07 | 59 | 63 | 30 | 1.21 | 1.63 | — | — | — | — | |
| No Reasoning OutputBackbone=PaLM 2-M, Learning Setting=Zero-shot, Task type=Rating prediction-only2024.07 | 59 | 63 | 29 | 1.18 | 1.56 | — | — | — | — | |
| No ReviewBackbone=PaLM 2-M, Learning Setting=Zero-shot, Input Feature=Excluding user reviews2024.07 | 58 | 63 | 28 | 1.2 | 1.58 | 17.3 | 0.452 | 0.291 | 64.1 | |
| No Item DescriptionBackbone=PaLM 2-M, Learning Setting=Zero-shot, Input Feature=Excluding item descriptions2024.07 | 54 | 62 | 28 | 1.22 | 1.6 | 18.3 | 0.46 | 0.296 | 64.7 | |
| One-shotBackbone=PaLM 2-M, Learning Setting=One-shot2024.07 | 47 | 59 | 23 | 1.32 | 1.68 | 18.2 | 0.452 | 0.276 | 64.1 | |
| No Review, No RatingBackbone=PaLM 2-M, Learning Setting=Zero-shot, Input Feature=Excluding user reviews and ratings2024.07 | 43 | 54 | 20 | 1.42 | 1.75 | 15 | 0.434 | 0.283 | 63.3 |