Narrative Reasoning on BookSum oracle timing
94Average ScoreCFPG
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| CFPGBase Model=Claude-Haiku-4.5, Model Variant=Instruct2026.01 | 94 | 96.5 | |
| CFPGBase Model=GPT-4.1-mini, Model Variant=Instruct2026.01 | 91.1 | 100 | |
| CFPGBase Model=Qwen2.5-14B, Model Variant=Instruct2026.01 | 89.8 | 100 | |
| CFPGBase Model=Llama-3.1-8B, Model Variant=Instruct2026.01 | 80.2 | 100 | |
| CFPGBase Model=Qwen2.5-7B, Model Variant=Instruct2026.01 | 79.7 | 100 | |
| CFPGBase Model=Qwen2.5-3B, Model Variant=Instruct2026.01 | 78.1 | 99.8 | |
| PromptBase Model=Claude-Haiku-4.5, Model Variant=Instruct2026.01 | 65.7 | — | |
| PromptBase Model=Qwen2.5-14B, Model Variant=Instruct2026.01 | 58.3 | — | |
| PromptBase Model=GPT-4.1-mini, Model Variant=Instruct2026.01 | 56.9 | — | |
| PromptBase Model=Llama-3.1-8B, Model Variant=Instruct2026.01 | 53 | — | |
| PromptBase Model=Qwen2.5-7B, Model Variant=Instruct2026.01 | 51.7 | — | |
| PromptBase Model=Qwen2.5-3B, Model Variant=Instruct2026.01 | 48.1 | — |