# Narrative Generation on Narrative Constraints Dataset Source: [Narrative Generation on Narrative Constraints Dataset]() Collection index: [llms.txt]() Data updated: 2026-05-12T09:40:57.019Z Evaluation of models on the Narrative Constraints Dataset focusing on constraint satisfaction and lexical diversity, measured using metrics like Jaccard Similarity, Effective Number, and Unique Constraints. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Narrative Generation | | Dataset (including any stated split) | Narrative Constraints Dataset | | Metric | Jaccard Similarity | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 18 of 18 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Jaccard Similarity](): Higher is better; unit: PERCENT. - [Effective Number](): Higher is better; unit: SCALAR. - [Unique Constraints](): Higher is better; unit: SCALAR. ## Recorded results | Method | Jaccard Similarity | Result date | Report paper | | --- | --- | --- | --- | | gpt5 | 31.69 | Not recorded | 6a02b438dbc836b87d824867 | | gpt5 | 30.7 | Not recorded | 6a02b438dbc836b87d824867 | | gemini | 25.97 | Not recorded | 6a02b438dbc836b87d824867 | | gpt5 | 25.46 | Not recorded | 6a02b438dbc836b87d824867 | | gpt4.1 | 24.58 | Not recorded | 6a02b438dbc836b87d824867 | | o4mini | 24.2 | Not recorded | 6a02b438dbc836b87d824867 | | o4mini | 22.32 | Not recorded | 6a02b438dbc836b87d824867 | | claude | 21.66 | Not recorded | 6a02b438dbc836b87d824867 | | claude | 21.6 | Not recorded | 6a02b438dbc836b87d824867 | | gemini | 20.06 | Not recorded | 6a02b438dbc836b87d824867 | | o4mini | 20.04 | Not recorded | 6a02b438dbc836b87d824867 | | gemini | 19.36 | Not recorded | 6a02b438dbc836b87d824867 | | claude | 18.6 | Not recorded | 6a02b438dbc836b87d824867 | | gpt4.1 | 14.83 | Not recorded | 6a02b438dbc836b87d824867 | | gpt4.1 | 13.44 | Not recorded | 6a02b438dbc836b87d824867 | | qwen | 8.96 | Not recorded | 6a02b438dbc836b87d824867 | | qwen | 8.67 | Not recorded | 6a02b438dbc836b87d824867 | | qwen | 8.2 | Not recorded | 6a02b438dbc836b87d824867 | ### gpt5 Jaccard Similarity: 31.69. Result ID: 6a02f4a147f34ce924e612ac. [Reporting paper]() Recorded evaluation configuration: {"Instruction Type":"Quality-focused"}. This record does not assert independent reproduction. ### gpt5 Jaccard Similarity: 30.7. Result ID: 6a02f4a147f34ce924e612ad. [Reporting paper]() Recorded evaluation configuration: {"Instruction Type":"Basic"}. This record does not assert independent reproduction. ### gemini Jaccard Similarity: 25.97. Result ID: 6a02f4a147f34ce924e612ae. [Reporting paper]() Recorded evaluation configuration: {"Instruction Type":"Creativity-focused"}. This record does not assert independent reproduction. ### gpt5 Jaccard Similarity: 25.46. Result ID: 6a02f4a147f34ce924e612af. [Reporting paper]() Recorded evaluation configuration: {"Instruction Type":"Creativity-focused"}. This record does not assert independent reproduction. ### gpt4.1 Jaccard Similarity: 24.58. Result ID: 6a02f4a147f34ce924e612b0. [Reporting paper]() Recorded evaluation configuration: {"Instruction Type":"Creativity-focused"}. This record does not assert independent reproduction. ### o4mini Jaccard Similarity: 24.2. Result ID: 6a02f4a147f34ce924e612b1. [Reporting paper]() Recorded evaluation configuration: {"Instruction Type":"Quality-focused"}. This record does not assert independent reproduction. ### o4mini Jaccard Similarity: 22.32. Result ID: 6a02f4a147f34ce924e612b2. [Reporting paper]() Recorded evaluation configuration: {"Instruction Type":"Basic"}. This record does not assert independent reproduction. ### claude Jaccard Similarity: 21.66. Result ID: 6a02f4a147f34ce924e612b3. [Reporting paper]() Recorded evaluation configuration: {"Instruction Type":"Creativity-focused"}. This record does not assert independent reproduction. ### claude Jaccard Similarity: 21.6. Result ID: 6a02f4a147f34ce924e612b4. [Reporting paper]() Recorded evaluation configuration: {"Instruction Type":"Quality-focused"}. This record does not assert independent reproduction. ### gemini Jaccard Similarity: 20.06. Result ID: 6a02f4a147f34ce924e612b5. [Reporting paper]() Recorded evaluation configuration: {"Instruction Type":"Basic"}. This record does not assert independent reproduction. ### o4mini Jaccard Similarity: 20.04. Result ID: 6a02f4a147f34ce924e612b6. [Reporting paper]() Recorded evaluation configuration: {"Instruction Type":"Creativity-focused"}. This record does not assert independent reproduction. ### gemini Jaccard Similarity: 19.36. Result ID: 6a02f4a147f34ce924e612b7. [Reporting paper]() Recorded evaluation configuration: {"Instruction Type":"Quality-focused"}. This record does not assert independent reproduction. ### claude Jaccard Similarity: 18.6. Result ID: 6a02f4a147f34ce924e612b8. [Reporting paper]() Recorded evaluation configuration: {"Instruction Type":"Basic"}. This record does not assert independent reproduction. ### gpt4.1 Jaccard Similarity: 14.83. Result ID: 6a02f4a147f34ce924e612b9. [Reporting paper]() Recorded evaluation configuration: {"Instruction Type":"Quality-focused"}. This record does not assert independent reproduction. ### gpt4.1 Jaccard Similarity: 13.44. Result ID: 6a02f4a147f34ce924e612ba. [Reporting paper]() Recorded evaluation configuration: {"Instruction Type":"Basic"}. This record does not assert independent reproduction. ### qwen Jaccard Similarity: 8.96. Result ID: 6a02f4a147f34ce924e612bb. [Reporting paper]() Recorded evaluation configuration: {"Instruction Type":"Quality-focused"}. This record does not assert independent reproduction. ### qwen Jaccard Similarity: 8.67. Result ID: 6a02f4a147f34ce924e612bc. [Reporting paper]() Recorded evaluation configuration: {"Instruction Type":"Creativity-focused"}. This record does not assert independent reproduction. ### qwen Jaccard Similarity: 8.2. Result ID: 6a02f4a147f34ce924e612bd. [Reporting paper]() Recorded evaluation configuration: {"Instruction Type":"Basic"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.