Long-form generation on LongBench Write-en (Binned Context Length Metrics)
88.31Sequence Length Success RateIS-Writer-8B
Evaluation results
| Method | Links | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| IS-Writer-8BType=Capability-enhanced LLM, Parameters=8B2026.06 | 88.31 | — | — | — | — | — | — | — | — | — | — | 88.19 | 88.25 | 85.46 | |
| Qwen3-235B-A22B-InstructType=Open-source LLM, Parameters=235B2026.06 | 85.07 | — | — | — | — | — | — | — | — | — | — | 89.24 | 87.15 | 71.91 | |
| DeepSeek-V3.2-671BType=Open-source LLM, Parameters=671B2026.06 | 80.24 | — | — | — | — | — | — | — | — | — | — | 90.1 | 85.17 | 71.03 | |
| Gemini-2.5-flashType=Proprietary LLM2026.06 | 79.35 | — | — | — | — | — | — | — | — | — | — | 87.99 | 83.67 | 82.11 | |
| LongWriter-8BType=Capability-enhanced LLM, Parameters=8B2026.06 | 77.25 | — | — | — | — | — | — | — | — | — | — | 78.68 | 77.97 | 75.37 | |
| DeepSeek-R1-671BType=Open-source LLM, Parameters=671B2026.06 | 75.31 | — | — | — | — | — | — | — | — | — | — | 88.44 | 81.87 | 70.81 | |
| Qwen-2.5-14Btuning=LongDPO2025.02 | 69.43 | — | — | — | — | — | — | — | — | — | — | 89.21 | — | — | |
| GPT-4o2025.02 | 67.8 | — | — | — | — | — | — | — | — | — | — | 90.9 | — | — | |
| Qwen-2.5-14Btuning=Base2025.02 | 65.29 | — | — | — | — | — | — | — | — | — | — | 87.88 | — | — | |
| Qwen-2.5-14Btuning=DPO2025.02 | 65.19 | — | — | — | — | — | — | — | — | — | — | 87.48 | — | — | |
| DeepSeek-V3-671BType=Open-source LLM, Parameters=671B2026.06 | 62.76 | — | — | — | — | — | — | — | — | — | — | 87.43 | 75.1 | 49.92 | |
| Claude-3.5-sonnetType=Proprietary LLM2026.06 | 60.17 | — | — | — | — | — | — | — | — | — | — | 85.63 | 72.9 | 45.74 | |
| GPT-4o-miniType=Proprietary LLM2026.06 | 59.45 | — | — | — | — | — | — | — | — | — | — | 84.98 | 72.22 | 43.6 | |
| Llama3.1-70B-instruct2025.02 | 59 | — | — | — | — | — | — | — | — | — | — | 80.3 | — | — | |
| Writing-Model-Qwen-7BType=Capability-enhanced LLM, Parameters=7B2026.06 | 58.77 | — | — | — | — | — | — | — | — | — | — | 81.7 | 70.24 | 39.77 | |
| GPT-4oType=Proprietary LLM2026.06 | 57.29 | — | — | — | — | — | — | — | — | — | — | 85.94 | 71.61 | 41.09 | |
| Llama3.1-8B-instruct2025.02 | 56.8 | — | — | — | — | — | — | — | — | — | — | 76.3 | — | — | |
| LLaMA3.3-70B-InstructType=Open-source LLM, Parameters=70B2026.06 | 55.58 | — | — | — | — | — | — | — | — | — | — | 77.33 | 66.45 | 35.45 | |
| QWQ2025.02 | 53.27 | — | — | — | — | — | — | — | — | — | — | 93.01 | — | — | |
| Suri-I-ORPO-7BType=Capability-enhanced LLM, Parameters=7B2026.06 | 42.42 | — | — | — | — | — | — | — | — | — | — | 46.08 | 44.25 | 29.67 | |
| Deepwriter-8BType=Capability-enhanced LLM, Parameters=8B2026.06 | 41.98 | — | — | — | — | — | — | — | — | — | — | 73.9 | 57.94 | 37.04 | |
| GPT-4o2025.02 | — | — | — | — | — | — | — | — | — | 92.2 | 93.5 | — | — | — | |
| Llama3.1-70B-instruct2025.02 | — | — | — | — | — | — | — | — | — | 88.6 | 84.4 | — | — | — | |
| Llama3.1-8B-instruct2025.02 | — | — | — | — | — | — | — | — | — | 78.2 | 80.6 | — | — | — | |
| LongWriter-LlamaBase Model=Llama-3-8B, Training Strategy=None2025.02 | — | 88.1 | 86 | 74.5 | 86.9 | 89.1 | 88.3 | 80.8 | 79.2 | 83.12 | 85.1 | — | — | — | |
| LongWriter-Llama w/ DPOBase Model=Llama-3-8B, Training Strategy=DPO2025.02 | — | 90.93 | 85.78 | 76.67 | 85.46 | 90.01 | 90.53 | 81.07 | 80.9 | 85.55 | 85.66 | — | — | — | |
| LongWriter-Llama w/ LongDPOBase Model=Llama-3-8B, Training Strategy=LongDPO2025.02 | — | 90.68 | 86.27 | 77.23 | 91.25 | 93.35 | 90.53 | 88.25 | 85.06 | 87.38 | 88.28 | — | — | — | |
| LongWriter-QwenBase Model=Qwen-2.5-7B, Training Strategy=None2025.02 | — | 90.8 | 87.99 | 81.27 | 84.37 | 81.21 | 84.84 | 58.69 | 78.13 | 79.51 | 85.08 | — | — | — | |
| LongWriter-Qwen w/ DPOBase Model=Qwen-2.5-7B, Training Strategy=DPO2025.02 | — | 86.32 | 88.23 | 88.71 | 89.16 | 89.28 | 84.09 | 60.89 | 78.82 | 81.3 | 85.07 | — | — | — | |
| LongWriter-Qwen w/ LongDPOBase Model=Qwen-2.5-7B, Training Strategy=LongDPO2025.02 | — | 88.93 | 91.91 | 85.47 | 91.25 | 88.63 | 85.6 | 71.14 | 85.41 | 83.54 | 88.54 | — | — | — | |
| Qwen-2.5-14Btuning=Base2025.02 | — | — | — | — | — | — | — | — | — | 81.83 | 91.91 | — | — | — | |
| Qwen-2.5-14Btuning=DPO2025.02 | — | — | — | — | — | — | — | — | — | 81.79 | 92.72 | — | — | — | |
| Qwen-2.5-14Btuning=LongDPO2025.02 | — | — | — | — | — | — | — | — | — | 85.69 | 90.53 | — | — | — | |
| QWQ2025.02 | — | — | — | — | — | — | — | — | — | 90.59 | 94.31 | — | — | — |