Narrative Detective Stories Human Subjective Evaluation (test)
3.96Surprise ScoreLlama 3.1-8B
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Llama 3.1-8Bnumber of stories=242025.07 | 3.96 | 2.75 | 2.67 | 2.5 | |
| Gemini 2.5 flash TB 0number of stories=302025.07 | 3.8 | 2.67 | 3.27 | 3 | |
| Poirotnumber of stories=372025.07 | 3.7 | 4.03 | 4.51 | 4.11 | |
| GPT-4o mininumber of stories=232025.07 | 3.61 | 2.57 | 2.74 | 2.35 | |
| Gemini 2.5-pronumber of stories=292025.07 | 3.5 | 3.11 | 3.72 | 3.41 | |
| Gemini 2.5 flash TB -1number of stories=302025.07 | 3.47 | 2.6 | 2.9 | 2.53 | |
| Avg. generatednumber of stories=2902025.07 | 3.44 | 2.77 | 2.99 | 2.65 | |
| GPT-4onumber of stories=262025.07 | 3.42 | 3.23 | 3.73 | 3.39 | |
| Llama 3.1-70Bnumber of stories=262025.07 | 3.39 | 2.58 | 2.69 | 2.58 | |
| Llama 3.3-70Bnumber of stories=552025.07 | 3.38 | 2.4 | 2.47 | 2.29 | |
| Avg. realnumber of stories=702025.07 | 3.32 | 3.91 | 4.39 | 4.21 | |
| Gemini 1.5-pronumber of stories=192025.07 | 3.11 | 3.16 | 3.42 | 2.42 | |
| Sherlocknumber of stories=332025.07 | 3.09 | 3.85 | 4.21 | 4.3 | |
| Llama 3.2-3Bnumber of stories=282025.07 | 2.79 | 3.18 | 2.68 | 2.39 |