Opinion Summarization on AmaSum product reviews (sports shoes) (test)
86CoverageAspect-aware decomposition
Evaluation Results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Aspect-aware decompositionBackbone=GPT-4o2025.01 | 86 | 0.87 | 0.79 | 0.17 | 16.1 | |
| Aspect-aware decompositionBackbone=Llama 70B2025.01 | 83 | 0.86 | 0.74 | 0.16 | 16.4 | |
| Aspect-aware decompositionBackbone=Llama 8B2025.01 | 77 | 0.78 | 0.69 | 0.09 | 16.44 | |
| Chunk-wise decompositionBackbone=Llama 8B2025.01 | 58 | 0.8 | 0.66 | 0.08 | 16.59 | |
| Chunk-wise decompositionBackbone=Llama 70B2025.01 | 57 | 0.88 | 0.54 | 0.07 | 15.28 | |
| HIRO-absBackbone=Not specified2025.01 | 54 | 0.35 | 0.78 | 0.13 | 14.9 | |
| Naive aspect-aware promptingBackbone=Llama 8B2025.01 | 54 | 0.29 | 0.5 | 0.07 | 8.8 | |
| Naive aspect-aware promptingBackbone=Llama 70B2025.01 | 49 | 0.48 | 0.6 | 0.09 | 7.35 | |
| FT-Llama 8BBackbone=Llama 8B2025.01 | 45 | 0.12 | 0.43 | 0.16 | 9.9 | |
| Automatic decompositionBackbone=Llama 8B2025.01 | 39 | 0.11 | 0.47 | 0.13 | 9.23 | |
| Automatic decompositionBackbone=Llama 70B2025.01 | 31 | 0.28 | 0.68 | 0.14 | 7.74 |