Fiction Generation on 40 themes (test)
62.5NCBiT-MCTS
Evaluation Results
| Method | Links | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| BiT-MCTSBackbone LLM=Gemini-2.5-Flash2026.03 | 62.5 | 40 | 37.5 | 57.5 | 57.5 | 75 | 42.5 | 42.5 | 92.5 | 77.5 | 58.5 | |
| BiT-MCTSBackbone LLM=GPT-5-Mini2026.03 | 57.5 | 42.5 | 42.5 | 50 | 45 | 57.5 | 40 | 45 | 77.5 | 47.5 | 50.5 | |
| Narrative StudioBackbone LLM=DeepSeek-V32026.03 | 42.5 | 57.5 | 20 | 42.5 | 20 | 15 | 7.5 | 10 | 25 | 30 | 27 | |
| StoryWriterBackbone LLM=DeepSeek-V32026.03 | 32.5 | 17.5 | 20 | 30 | 17.5 | 42.5 | 25 | 25 | 40 | 42.5 | 27.2 | |
| StoryWriterBackbone LLM=GPT-5-Mini2026.03 | 27.5 | 10 | 40 | 40 | 40 | 32.5 | 32.5 | 30 | 17.5 | 45 | 31.5 | |
| BiT-MCTSBackbone LLM=DeepSeek-V32026.03 | 25 | 25 | 60 | 52.5 | 62.5 | 42.5 | 60 | 62.5 | 35 | 27.5 | 47.2 | |
| StoryWriterBackbone LLM=Gemini-2.5-Flash2026.03 | 25 | 22.5 | 15 | 20 | 22.5 | 17.5 | 15 | 15 | 5 | 10 | 16.8 | |
| Narrative StudioBackbone LLM=Gemini-2.5-Flash2026.03 | 12.5 | 30 | 35 | 12.5 | 15 | 0 | 10 | 10 | 2.5 | 7.5 | 13.5 | |
| Narrative StudioBackbone LLM=GPT-5-Mini2026.03 | 10 | 35 | 12.5 | 7.5 | 12.5 | 0 | 17.5 | 17.5 | 2.5 | 5 | 12 | |
| VanillaBackbone LLM=GPT-5-Mini2026.03 | 5 | 12.5 | 5 | 2.5 | 2.5 | 10 | 10 | 7.5 | 2.5 | 2.5 | 6 | |
| VanillaBackbone LLM=DeepSeek-V32026.03 | 0 | 0 | 0 | 0 | 0 | 0 | 7.5 | 2.5 | 0 | 0 | 1 | |
| VanillaBackbone LLM=Gemini-2.5-Flash2026.03 | 0 | 7.5 | 12.5 | 10 | 5 | 7.5 | 32.5 | 32.5 | 0 | 5 | 11.2 |