# Human Evaluation for Fiction Generation (test)

Source: [Human Evaluation for Fiction Generation (test)](<https://www.sota2.com/research/sota/human-evaluation-for-fiction-generation-test>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-03-17T10:12:43.670Z

Human evaluation benchmark for assessing Fiction Generation quality based on 12 specific criteria (e.g., Novelty, Coherence, Fluency) evaluated on the test set.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Fiction Generation |
| Dataset (including any stated split) | Human Evaluation for Fiction Generation (test) |
| Metric | Novelty (NC) |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 3 of 3 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Novelty (NC)](<https://www.sota2.com/research/sota/human-evaluation-for-fiction-generation-test.md?metric=Novelty%20(NC)>): Higher is better; unit: PERCENT.

- [Coherence (CR)](<https://www.sota2.com/research/sota/human-evaluation-for-fiction-generation-test.md?metric=Coherence%20(CR)>): Higher is better; unit: PERCENT.

- [Engagement Rate (ER)](<https://www.sota2.com/research/sota/human-evaluation-for-fiction-generation-test.md?metric=Engagement%20Rate%20(ER)>): Higher is better; unit: PERCENT.

- [Plausibility (PS)](<https://www.sota2.com/research/sota/human-evaluation-for-fiction-generation-test.md?metric=Plausibility%20(PS)>): Higher is better; unit: PERCENT.

- [Character Development (CD)](<https://www.sota2.com/research/sota/human-evaluation-for-fiction-generation-test.md?metric=Character%20Development%20(CD)>): Higher is better; unit: PERCENT.

- [Style Consistency (SD)](<https://www.sota2.com/research/sota/human-evaluation-for-fiction-generation-test.md?metric=Style%20Consistency%20(SD)>): Higher is better; unit: PERCENT.

- [Grammar Score (GR)](<https://www.sota2.com/research/sota/human-evaluation-for-fiction-generation-test.md?metric=Grammar%20Score%20(GR)>): Higher is better; unit: PERCENT.

- [Fluency (FL)](<https://www.sota2.com/research/sota/human-evaluation-for-fiction-generation-test.md?metric=Fluency%20(FL)>): Higher is better; unit: PERCENT.

- [Diversity (DI)](<https://www.sota2.com/research/sota/human-evaluation-for-fiction-generation-test.md?metric=Diversity%20(DI)>): Higher is better; unit: PERCENT.

- [Overall Quality (OQ)](<https://www.sota2.com/research/sota/human-evaluation-for-fiction-generation-test.md?metric=Overall%20Quality%20(OQ)>): Higher is better; unit: PERCENT.

- [Thematic Depth (TH)](<https://www.sota2.com/research/sota/human-evaluation-for-fiction-generation-test.md?metric=Thematic%20Depth%20(TH)>): Higher is better; unit: PERCENT.

- [Average Score](<https://www.sota2.com/research/sota/human-evaluation-for-fiction-generation-test.md?metric=Average%20Score>): Higher is better; unit: PERCENT.

## Recorded results

| Method | Novelty (NC) | Result date | Report paper |
| --- | --- | --- | --- |
| Narrative Studio | 38.33 | Not recorded | 69b8e0170636e5bf4afbead7 |
| BiT-MCTS | 32.5 | Not recorded | 69b8e0170636e5bf4afbead7 |
| StoryWriter | 29.17 | Not recorded | 69b8e0170636e5bf4afbead7 |

### Narrative Studio

Novelty (NC): 38.33. Result ID: 69b9290a47f34ce924a26a83.

[Reporting paper](<https://www.sota2.com/research/paper/69b8e0170636e5bf4afbead7.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa77cc1e209131cea7566b.md>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

### BiT-MCTS

Novelty (NC): 32.5. Result ID: 69b9290a47f34ce924a26a84.

[Reporting paper](<https://www.sota2.com/research/paper/69b8e0170636e5bf4afbead7.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69b8e0170636e5bf4afbead7.md>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

### StoryWriter

Novelty (NC): 29.17. Result ID: 69b9290a47f34ce924a26a82.

[Reporting paper](<https://www.sota2.com/research/paper/69b8e0170636e5bf4afbead7.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa79911e209131cea7e480.md>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.
