Story Premise Diversity Evaluation on Story Premise Human Evaluation Set 600 premises 1.0 (test)
3.875Average ScoreMoPS
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| MoPS2024.06 | 3.875 | 4 | 5 | 3 | 4 | 4 | 3 | 4 | 4 | |
| WritingPrompts (WP)source=Reddit2024.06 | 3.75 | 2 | 5 | 5 | 2 | 5 | 5 | 4 | 2 | |
| DOCbackbone=llama2-13b-chat2024.06 | 3.5 | 3 | 2 | 4 | 5 | 5 | 4 | 3 | 4 | |
| Storium (STM)source=RPG platform2024.06 | 3.125 | 3 | 3 | 3 | 3 | 4 | 4 | 2 | 2 | |
| Vanilla (VIL)backbone=gpt-3.5-turbo, temperature=0.6, deduplication=cosine similarity threshold 0.852024.06 | 2.625 | 3 | 2 | 4 | 3 | 3 | 2 | 2 | 2 | |
| Complex (CPX)mode=few-shot, shots=32024.06 | 2.25 | 4 | 2 | 2 | 2 | 3 | 1 | 1 | 3 |