# Story Evaluation on HANNA (test) Source: [Story Evaluation on HANNA (test)]() Collection index: [llms.txt]() Data updated: 2026-04-23T08:21:09.780Z Statistical evaluation of story generation models on the test split of the HANNA dataset, using correlation coefficients (Pearson, Spearman, Kendall) and error metrics (MSE, F1-Score). A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Story Evaluation | | Dataset (including any stated split) | HANNA (test) | | Metric | Kendall Correlation | | Unit | SCALAR | | Direction | Higher is better | | Results loaded | 26 of 26 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Kendall Correlation](): Higher is better; unit: SCALAR. - [Pearson Correlation](): Higher is better; unit: SCALAR. - [Spearman Correlation](): Higher is better; unit: SCALAR. - [Mean Squared Error](): Lower is better; unit: SCALAR. - [F1 Score](): Higher is better; unit: PERCENT. ## Recorded results | Method | Kendall Correlation | Result date | Report paper | | --- | --- | --- | --- | | EvolvR | 0.5429 | Not recorded | 68fa7a871e209131cea83b76 | | Qwen2.5-7B-Instruct + Point CoT + GRPO | 0.4606 | Not recorded | 68fa7a871e209131cea83b76 | | Qwen2.5-7B-Instruct + Point CoT | 0.438 | Not recorded | 68fa7a871e209131cea83b76 | | Gemini-2.5-Pro-06-17 | 0.4041 | Not recorded | 68fa7a871e209131cea83b76 | | Claude-4-opus4 | 0.3702 | Not recorded | 68fa7a871e209131cea83b76 | | GPT-o3-0416-global | 0.3543 | Not recorded | 68fa7a871e209131cea83b76 | | Differential Polarity Index | 0.345 | Not recorded | 69e85baf7d0172140ec81006 | | Themis-8B | 0.3295 | Not recorded | 68fa7a871e209131cea83b76 | | Claude-4-Sonnet | 0.3273 | Not recorded | 68fa7a871e209131cea83b76 | | AutoJ-13B | 0.3223 | Not recorded | 68fa7a871e209131cea83b76 | | Gemini-2.5-flash-06-17 | 0.3111 | Not recorded | 68fa7a871e209131cea83b76 | | GPT-4o | 0.2867 | Not recorded | 68fa7a871e209131cea83b76 | | GPT-4.1 | 0.2612 | Not recorded | 68fa7a871e209131cea83b76 | | Beluga-13B | 0.25 | Not recorded | 69e85baf7d0172140ec81006 | | Mistral-7B | 0.2 | Not recorded | 69e85baf7d0172140ec81006 | | TIGERScore-13B | 0.1879 | Not recorded | 68fa7a871e209131cea83b76 | | ChatGPT | 0.18 | Not recorded | 69e85baf7d0172140ec81006 | | ROUGE-1 | 0.18 | Not recorded | 69e85baf7d0172140ec81006 | | ChrF | 0.18 | Not recorded | 69e85baf7d0172140ec81006 | | Llama-13B | 0.16 | Not recorded | 69e85baf7d0172140ec81006 | | BERTScore | 0.16 | Not recorded | 69e85baf7d0172140ec81006 | | Qwen2.5-7B-Instruct + GRPO | 0.1518 | Not recorded | 68fa7a871e209131cea83b76 | | Qwen2.5-7B-Instruct | 0.1339 | Not recorded | 68fa7a871e209131cea83b76 | | BLEU | 0.12 | Not recorded | 69e85baf7d0172140ec81006 | | InstructScore-7B | 0.0989 | Not recorded | 68fa7a871e209131cea83b76 | | BARTScore | 0.06 | Not recorded | 69e85baf7d0172140ec81006 | ### EvolvR Kendall Correlation: 0.5429. Result ID: 69b8fd2c47f34ce924a0f57f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Category":"Story Evalution Finetune Open-source Models"}. This record does not assert independent reproduction. ### Qwen2.5-7B-Instruct + Point CoT + GRPO Kendall Correlation: 0.4606. Result ID: 69b8fd2c47f34ce924a0f57e. [Reporting paper]() Recorded evaluation configuration: {"Model Category":"Story Evalution Finetune Open-source Models","Point CoT":"true","GRPO":"true"}. This record does not assert independent reproduction. ### Qwen2.5-7B-Instruct + Point CoT Kendall Correlation: 0.438. Result ID: 69b8fd2c47f34ce924a0f57c. [Reporting paper]() Recorded evaluation configuration: {"Model Category":"Story Evalution Finetune Open-source Models","Point CoT":"true"}. This record does not assert independent reproduction. ### Gemini-2.5-Pro-06-17 Kendall Correlation: 0.4041. Result ID: 69b8fd2c47f34ce924a0f574. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Category":"General Closed-source Models"}. This record does not assert independent reproduction. ### Claude-4-opus4 Kendall Correlation: 0.3702. Result ID: 69b8fd2c47f34ce924a0f576. [Reporting paper]() Recorded evaluation configuration: {"Model Category":"General Closed-source Models"}. This record does not assert independent reproduction. ### GPT-o3-0416-global Kendall Correlation: 0.3543. Result ID: 69b8fd2c47f34ce924a0f572. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Category":"General Closed-source Models"}. This record does not assert independent reproduction. ### Differential Polarity Index Kendall Correlation: 0.345. Result ID: 69e9d66e47f34ce924d0eb91. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Abbreviation":"DPI"}. This record does not assert independent reproduction. ### Themis-8B Kendall Correlation: 0.3295. Result ID: 69b8fd2c47f34ce924a0f578. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Category":"NLG Evalution Open-Source Models"}. This record does not assert independent reproduction. ### Claude-4-Sonnet Kendall Correlation: 0.3273. Result ID: 69b8fd2c47f34ce924a0f575. [Reporting paper]() Recorded evaluation configuration: {"Model Category":"General Closed-source Models"}. This record does not assert independent reproduction. ### AutoJ-13B Kendall Correlation: 0.3223. Result ID: 69b8fd2c47f34ce924a0f57a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Category":"NLG Evalution Open-Source Models"}. This record does not assert independent reproduction. ### Gemini-2.5-flash-06-17 Kendall Correlation: 0.3111. Result ID: 69b8fd2c47f34ce924a0f573. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Category":"General Closed-source Models"}. This record does not assert independent reproduction. ### GPT-4o Kendall Correlation: 0.2867. Result ID: 69b8fd2c47f34ce924a0f570. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Category":"General Closed-source Models"}. This record does not assert independent reproduction. ### GPT-4.1 Kendall Correlation: 0.2612. Result ID: 69b8fd2c47f34ce924a0f571. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Category":"General Closed-source Models"}. This record does not assert independent reproduction. ### Beluga-13B Kendall Correlation: 0.25. Result ID: 69e9d66e47f34ce924d0eb92. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"setting":"1"}. This record does not assert independent reproduction. ### Mistral-7B Kendall Correlation: 0.2. Result ID: 69e9d66e47f34ce924d0eb94. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"setting":"1"}. This record does not assert independent reproduction. ### TIGERScore-13B Kendall Correlation: 0.1879. Result ID: 69b8fd2c47f34ce924a0f579. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Category":"NLG Evalution Open-Source Models"}. This record does not assert independent reproduction. ### ChatGPT Kendall Correlation: 0.18. Result ID: 69e9d66e47f34ce924d0eb95. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"setting":"1"}. This record does not assert independent reproduction. ### ROUGE-1 Kendall Correlation: 0.18. Result ID: 69e9d66e47f34ce924d0eb99. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### ChrF Kendall Correlation: 0.18. Result ID: 69e9d66e47f34ce924d0eb9a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### Llama-13B Kendall Correlation: 0.16. Result ID: 69e9d66e47f34ce924d0eb93. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"setting":"1"}. This record does not assert independent reproduction. ### BERTScore Kendall Correlation: 0.16. Result ID: 69e9d66e47f34ce924d0eb97. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### Qwen2.5-7B-Instruct + GRPO Kendall Correlation: 0.1518. Result ID: 69b8fd2c47f34ce924a0f57d. [Reporting paper]() Recorded evaluation configuration: {"Model Category":"Story Evalution Finetune Open-source Models","GRPO":"true"}. This record does not assert independent reproduction. ### Qwen2.5-7B-Instruct Kendall Correlation: 0.1339. Result ID: 69b8fd2c47f34ce924a0f57b. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model Category":"Story Evalution Finetune Open-source Models"}. This record does not assert independent reproduction. ### BLEU Kendall Correlation: 0.12. Result ID: 69e9d66e47f34ce924d0eb98. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### InstructScore-7B Kendall Correlation: 0.0989. Result ID: 69b8fd2c47f34ce924a0f577. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Category":"NLG Evalution Open-Source Models"}. This record does not assert independent reproduction. ### BARTScore Kendall Correlation: 0.06. Result ID: 69e9d66e47f34ce924d0eb96. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.