Story Evaluation on HANNA (test)
0.5429Kendall CorrelationEvolvR
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| EvolvRModel Category=Story Evalution Finetune Open-source Models2025.08 | 0.5429 | 0.6155 | 0.6033 | 0.044 | 74.06 | |
| Qwen2.5-7B-Instruct + Point CoT + GRPOModel Category=Story Evalution Finetune Open-source Models, Point CoT=true, GRPO=true2025.08 | 0.4606 | 0.5307 | 0.5121 | 0.0483 | 68.29 | |
| Qwen2.5-7B-Instruct + Point CoTModel Category=Story Evalution Finetune Open-source Models, Point CoT=true2025.08 | 0.438 | 0.5173 | 0.4919 | 0.0521 | 67.15 | |
| Gemini-2.5-Pro-06-17Model Category=General Closed-source Models2025.08 | 0.4041 | 0.5 | 0.4586 | 0.1782 | 45.69 | |
| Claude-4-opus4Model Category=General Closed-source Models2025.08 | 0.3702 | 0.4872 | 0.4237 | 0.1814 | 38.33 | |
| GPT-o3-0416-globalModel Category=General Closed-source Models2025.08 | 0.3543 | 0.4837 | 0.4128 | 0.0854 | 40.16 | |
| Differential Polarity IndexAbbreviation=DPI2026.04 | 0.345 | — | — | — | — | |
| Themis-8BModel Category=NLG Evalution Open-Source Models2025.08 | 0.3295 | 0.4565 | 0.3696 | 0.1432 | 37.99 | |
| Claude-4-SonnetModel Category=General Closed-source Models2025.08 | 0.3273 | 0.4053 | 0.3675 | 0.1327 | 36.59 | |
| AutoJ-13BModel Category=NLG Evalution Open-Source Models2025.08 | 0.3223 | 0.4153 | 0.3689 | 0.0609 | 44.1 | |
| Gemini-2.5-flash-06-17Model Category=General Closed-source Models2025.08 | 0.3111 | 0.3883 | 0.3647 | 0.1876 | 57 | |
| GPT-4oModel Category=General Closed-source Models2025.08 | 0.2867 | 0.427 | 0.3517 | 0.0744 | 49.22 | |
| GPT-4.1Model Category=General Closed-source Models2025.08 | 0.2612 | 0.3171 | 0.2987 | 0.082 | 67.27 | |
| Beluga-13Bsetting=12026.04 | 0.25 | — | — | — | — | |
| Mistral-7Bsetting=12026.04 | 0.2 | — | — | — | — | |
| TIGERScore-13BModel Category=NLG Evalution Open-Source Models2025.08 | 0.1879 | 0.2897 | 0.2056 | 0.1309 | 51.8 | |
| ChatGPTsetting=12026.04 | 0.18 | — | — | — | — | |
| ROUGE-12026.04 | 0.18 | — | — | — | — | |
| ChrF2026.04 | 0.18 | — | — | — | — | |
| Llama-13Bsetting=12026.04 | 0.16 | — | — | — | — | |
| BERTScore2026.04 | 0.16 | — | — | — | — | |
| Qwen2.5-7B-Instruct + GRPOModel Category=Story Evalution Finetune Open-source Models, GRPO=true2025.08 | 0.1518 | 0.1773 | 0.1717 | 0.079 | 45.07 | |
| Qwen2.5-7B-InstructModel Category=Story Evalution Finetune Open-source Models2025.08 | 0.1339 | 0.1873 | 0.1605 | 0.1499 | 52.36 | |
| BLEU2026.04 | 0.12 | — | — | — | — | |
| InstructScore-7BModel Category=NLG Evalution Open-Source Models2025.08 | 0.0989 | 0.1258 | 0.1056 | 0.2071 | 57.87 | |
| BARTScore2026.04 | 0.06 | — | — | — | — |