List your company

ResearchFormerly WizWand

Role-playing

Benchmarks

DatasetSOTA methodMetricTrendResultsLast updated

Llama-3.1-8B-Instruct

86.66LLM-as-a-Judge Score

91

Jun 15, 2026

4.043Overall Average Score

70

Jun 26, 2026

CharacterBench latest (full)

4.525Overall Score

47

May 26, 2026

RoleBench (test)

Llama-3.1-8B-Instruct

88.82LLM-as-a-Judge Score

42

Jun 15, 2026

Full System Prompt

94.46GPT-4o Win Rate

28

Feb 26, 2026

CharacterBench 1.0 (test)

Qwen2.5-7B-Instruct + Character-R1

4.444MC

28

Feb 26, 2026

RPGBench Aggregate (Overall)

-0.026Avg Score

18

Feb 26, 2026

RPGBench Dialogue Shift (Generalization)

-0.956Turn Composition

18

Feb 26, 2026

RPGBench Character Shift (Generalization)

-0.8Deviation Score (Literature)

18

Feb 26, 2026

RPGBench User Shift Generalization

-0.016RP Score (German)

18

Feb 26, 2026

RPGBench In-distribution

-0.034R-EMI

18

Feb 26, 2026

Role-playing evaluation (Main characters)

83.88ROUGE-L (Haruhi)

12

Feb 26, 2026

DynSess-Eval Human Evaluation 1.0 (test)

Doubao-1.5-pro-character

3.38Average Score

10

May 29, 2026

0.67KR

7

Feb 26, 2026

4.98Engagement (EA)

7

Feb 26, 2026

Alpaca-P 50P x 100Q scale (test)

81.97Alpaca-P Score

5

Jun 15, 2026

Alpaca-P CoSER evaluation framework (test 20 randomly sampled instances)

4.38Anthropomorphism

5

Jun 15, 2026

Role-playing evaluation (Minor characters)

Codified Profile

21.21K-On! ROUGE-L

5

Feb 26, 2026

Alpaca-P and Character Fidelity CoSER criterion (test)

Humanish-Roleplay

83.35Alpaca-P Score

4

Jun 15, 2026

CharacterEval unseen roles transfer setting

Baichuan2-7B + PCL

2.749CC

4

Feb 26, 2026

Role-playing human assessment set (test)

Baichuan2-7B+PCL

602Win Count

4

Feb 26, 2026

RoleBench Chinese (instruction generalization)

36.4Win Rate (vs GPT-4)

4

Feb 26, 2026