# Story Evaluation on StoryER (test) Source: [Story Evaluation on StoryER (test)]() Collection index: [llms.txt]() Data updated: 2026-03-17T07:09:55.546Z Evaluation of model performance on the test split of the StoryER dataset, assessed using correlation metrics (Pearson, Spearman, Kendall) and error metrics (MSE, F1-Score). A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Story Evaluation | | Dataset (including any stated split) | StoryER (test) | | Metric | Pearson Correlation | | Unit | SCALAR | | Direction | Higher is better | | Results loaded | 17 of 17 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Pearson Correlation](): Higher is better; unit: SCALAR. - [Spearman Correlation](): Higher is better; unit: SCALAR. - [Kendall Correlation](): Higher is better; unit: SCALAR. - [Mean Squared Error](): Lower is better; unit: SCALAR. - [F1 Score](): Higher is better; unit: PERCENT. ## Recorded results | Method | Pearson Correlation | Result date | Report paper | | --- | --- | --- | --- | | EvolvR | 0.6774 | Not recorded | 68fa7a871e209131cea83b76 | | GPT-o3-0416-global | 0.5682 | Not recorded | 68fa7a871e209131cea83b76 | | Qwen2.5-7B-Instruct + Point CoT | 0.5677 | Not recorded | 68fa7a871e209131cea83b76 | | Qwen2.5-7B-Instruct + Point CoT + GRPO | 0.5646 | Not recorded | 68fa7a871e209131cea83b76 | | Gemini-2.5-Pro-06-17 | 0.5451 | Not recorded | 68fa7a871e209131cea83b76 | | Themis-8B | 0.5362 | Not recorded | 68fa7a871e209131cea83b76 | | GPT-4o | 0.4808 | Not recorded | 68fa7a871e209131cea83b76 | | Claude-4-opus4 | 0.4796 | Not recorded | 68fa7a871e209131cea83b76 | | Claude-4-Sonnet | 0.4157 | Not recorded | 68fa7a871e209131cea83b76 | | Gemini-2.5-flash-06-17 | 0.3882 | Not recorded | 68fa7a871e209131cea83b76 | | GPT-4.1 | 0.3673 | Not recorded | 68fa7a871e209131cea83b76 | | Qwen2.5-7B-Instruct + GRPO | 0.343 | Not recorded | 68fa7a871e209131cea83b76 | | AutoJ-13B | 0.3182 | Not recorded | 68fa7a871e209131cea83b76 | | Coke | 0.3142 | Not recorded | 68fa7a871e209131cea83b76 | | TIGERScore-13B | 0.2759 | Not recorded | 68fa7a871e209131cea83b76 | | Qwen2.5-7B-Instruct | 0.2206 | Not recorded | 68fa7a871e209131cea83b76 | | InstructScore-7B | 0.1902 | Not recorded | 68fa7a871e209131cea83b76 | ### EvolvR Pearson Correlation: 0.6774. Result ID: 69b8fd2c47f34ce924a0f56f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Category":"Story Evalution Finetune Open-source Models"}. This record does not assert independent reproduction. ### GPT-o3-0416-global Pearson Correlation: 0.5682. Result ID: 69b8fd2c47f34ce924a0f561. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Category":"General Closed-source Models"}. This record does not assert independent reproduction. ### Qwen2.5-7B-Instruct + Point CoT Pearson Correlation: 0.5677. Result ID: 69b8fd2c47f34ce924a0f56c. [Reporting paper]() Recorded evaluation configuration: {"Model Category":"Story Evalution Finetune Open-source Models","Point CoT":"true"}. This record does not assert independent reproduction. ### Qwen2.5-7B-Instruct + Point CoT + GRPO Pearson Correlation: 0.5646. Result ID: 69b8fd2c47f34ce924a0f56e. [Reporting paper]() Recorded evaluation configuration: {"Model Category":"Story Evalution Finetune Open-source Models","Point CoT":"true","GRPO":"true"}. This record does not assert independent reproduction. ### Gemini-2.5-Pro-06-17 Pearson Correlation: 0.5451. Result ID: 69b8fd2c47f34ce924a0f563. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Category":"General Closed-source Models"}. This record does not assert independent reproduction. ### Themis-8B Pearson Correlation: 0.5362. Result ID: 69b8fd2c47f34ce924a0f567. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Category":"NLG Evalution Open-Source Models"}. This record does not assert independent reproduction. ### GPT-4o Pearson Correlation: 0.4808. Result ID: 69b8fd2c47f34ce924a0f55f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Category":"General Closed-source Models"}. This record does not assert independent reproduction. ### Claude-4-opus4 Pearson Correlation: 0.4796. Result ID: 69b8fd2c47f34ce924a0f565. [Reporting paper]() Recorded evaluation configuration: {"Model Category":"General Closed-source Models"}. This record does not assert independent reproduction. ### Claude-4-Sonnet Pearson Correlation: 0.4157. Result ID: 69b8fd2c47f34ce924a0f564. [Reporting paper]() Recorded evaluation configuration: {"Model Category":"General Closed-source Models"}. This record does not assert independent reproduction. ### Gemini-2.5-flash-06-17 Pearson Correlation: 0.3882. Result ID: 69b8fd2c47f34ce924a0f562. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Category":"General Closed-source Models"}. This record does not assert independent reproduction. ### GPT-4.1 Pearson Correlation: 0.3673. Result ID: 69b8fd2c47f34ce924a0f560. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Category":"General Closed-source Models"}. This record does not assert independent reproduction. ### Qwen2.5-7B-Instruct + GRPO Pearson Correlation: 0.343. Result ID: 69b8fd2c47f34ce924a0f56d. [Reporting paper]() Recorded evaluation configuration: {"Model Category":"Story Evalution Finetune Open-source Models","GRPO":"true"}. This record does not assert independent reproduction. ### AutoJ-13B Pearson Correlation: 0.3182. Result ID: 69b8fd2c47f34ce924a0f569. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Category":"NLG Evalution Open-Source Models"}. This record does not assert independent reproduction. ### Coke Pearson Correlation: 0.3142. Result ID: 69b8fd2c47f34ce924a0f56a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Category":"Story Evalution Finetune Open-source Models"}. This record does not assert independent reproduction. ### TIGERScore-13B Pearson Correlation: 0.2759. Result ID: 69b8fd2c47f34ce924a0f568. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Category":"NLG Evalution Open-Source Models"}. This record does not assert independent reproduction. ### Qwen2.5-7B-Instruct Pearson Correlation: 0.2206. Result ID: 69b8fd2c47f34ce924a0f56b. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model Category":"Story Evalution Finetune Open-source Models"}. This record does not assert independent reproduction. ### InstructScore-7B Pearson Correlation: 0.1902. Result ID: 69b8fd2c47f34ce924a0f566. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Category":"NLG Evalution Open-Source Models"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.