Character Roleplay on ARCANE validated slice 5 novels (Overall)
62.4Overall ScoreDeepSeek-V4
Evaluation results
| Method | Links | |
|---|---|---|
| DeepSeek-V4Size=Pro, Mode=Arc2026.06 | 62.4 | |
| ARCANESize=32B, Mode=Arc2026.06 | 60.4 | |
| DeepSeek-V4Size=Flash, Mode=Arc2026.06 | 59.7 | |
| DeepSeek-V4Size=Pro, Mode=LifeChoice2026.06 | 57.7 | |
| DeepSeek-V4Size=Pro, Mode=Summary2026.06 | 57.2 | |
| ARCANESize=8B, Mode=Arc2026.06 | 56.9 | |
| DeepSeek-V4Size=Pro, Mode=RAG2026.06 | 56.7 | |
| DeepSeek-V4Size=Flash, Mode=LifeChoice2026.06 | 56.1 | |
| DeepSeek-V4Size=Flash, Mode=Summary2026.06 | 55.8 | |
| DeepSeek-V4Size=Flash, Mode=RAG2026.06 | 54.9 | |
| DeepSeek-V4Size=Flash, Mode=TimeChara2026.06 | 53.1 | |
| DeepSeek-V4Size=Flash, Mode=Vanilla2026.06 | 52.9 | |
| DeepSeek-V4Size=Pro, Mode=Vanilla2026.06 | 52.4 | |
| DeepSeek-V4Size=Pro, Mode=TimeChara2026.06 | 52.1 | |
| ARCANESize=32B, Mode=RAG2026.06 | 52 | |
| ARCANESize=32B, Mode=LifeChoice2026.06 | 52 | |
| Qwen3Size=32B, Mode=Arc2026.06 | 50.1 | |
| ARCANESize=32B, Mode=Vanilla2026.06 | 49.9 | |
| ARCANESize=32B, Mode=TimeChara2026.06 | 49.9 | |
| ARCANESize=32B, Mode=Summary2026.06 | 49.3 | |
| ARCANESize=8B, Mode=RAG2026.06 | 48.5 | |
| ARCANESize=8B, Mode=LifeChoice2026.06 | 47.6 | |
| Qwen3Size=32B, Mode=LifeChoice2026.06 | 47.4 | |
| Qwen3Size=32B, Mode=RAG2026.06 | 47.2 | |
| ARCANESize=8B, Mode=Summary2026.06 | 47.2 | |
| Qwen3Size=32B, Mode=Summary2026.06 | 45.6 | |
| Qwen3Size=32B, Mode=TimeChara2026.06 | 44.4 | |
| Qwen3Size=32B, Mode=Vanilla2026.06 | 43.7 | |
| ARCANESize=8B, Mode=TimeChara2026.06 | 43.7 | |
| Qwen3Size=8B, Mode=Arc2026.06 | 43.1 | |
| ARCANESize=8B, Mode=Vanilla2026.06 | 43 | |
| Qwen3Size=8B, Mode=RAG2026.06 | 40.9 | |
| Qwen3Size=8B, Mode=LifeChoice2026.06 | 39.8 | |
| Qwen3Size=8B, Mode=Summary2026.06 | 38.7 | |
| Qwen3Size=8B, Mode=TimeChara2026.06 | 37.4 | |
| Qwen3Size=8B, Mode=Vanilla2026.06 | 37.2 |