Role-playing evaluation (Main characters)
83.88ROUGE-L (Haruhi)Codified FSM
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| Codified FSMRP Model=gpt-4.1, Discriminator=gpt-4.1-mini, #Character=5.3 (avg)2026.02 | 83.88 | 80.45 | 78.31 | 83.89 | 86.11 | 83.28 | 82.65 | |
| Character UpdatingRP Model=gpt-4.1, Discriminator=gpt-4.1-mini, #Character=5.3 (avg)2026.02 | 83.63 | 79.23 | 75.08 | 82.48 | 85.87 | 82.51 | 81.47 | |
| Plot SummaryRP Model=gpt-4.1, Discriminator=gpt-4.1-mini, #Character=5.3 (avg)2026.02 | 83.19 | 79.51 | 76.12 | 82.91 | 85.48 | 83 | 81.7 | |
| PromptTransRP Model=gpt-4.1, Discriminator=gpt-4.1-mini, #Character=5.3 (avg)2026.02 | 82.86 | 79.68 | 75.07 | 82.66 | 84.96 | 82.17 | 81.23 | |
| Codified ProfileRP Model=gpt-4.1, Discriminator=gpt-4.1-mini, #Character=5.3 (avg)2026.02 | 82.77 | 80.49 | 76.19 | 82.58 | 84.47 | 81.66 | 81.36 | |
| Textual ProfileRP Model=gpt-4.1, Discriminator=gpt-4.1-mini, #Character=5.3 (avg)2026.02 | 81.17 | 80.04 | 75.94 | 80.47 | 85.85 | 83.14 | 81.1 | |
| Vanilla (No Profile)RP Model=gpt-4.1, Discriminator=gpt-4.1-mini, #Character=5.3 (avg)2026.02 | 80.96 | 77.81 | 75.21 | 80.34 | 85.58 | 82.46 | 80.39 | |
| Plot SummaryRP Model=gpt-4.1, Discriminator Model=gpt-4.1-mini2026.02 | 21.11 | 18.22 | 17.98 | 18.93 | 20.52 | 18.51 | 19.21 | |
| Character UpdatingRP Model=gpt-4.1, Discriminator Model=gpt-4.1-mini2026.02 | 20.71 | 18.05 | 18.53 | 18.58 | 20.1 | 18.81 | 19.13 | |
| Codified FSMRP Model=gpt-4.1, Discriminator Model=gpt-4.1-mini2026.02 | 20.61 | 19.09 | 18.78 | 19.32 | 21.28 | 19.35 | 19.74 | |
| Codified ProfileRP Model=gpt-4.1, Discriminator Model=gpt-4.1-mini2026.02 | 20.57 | 18.5 | 18.82 | 18.82 | 20.81 | 18.95 | 19.41 | |
| PromptTransRP Model=gpt-4.1, Discriminator Model=gpt-4.1-mini2026.02 | 20.09 | 18.09 | 18.72 | 18.91 | 20.6 | 18.68 | 19.18 |