# Role-playing on CharacterBench 1.0 (test) Source: [Role-playing on CharacterBench 1.0 (test)]() Collection index: [llms.txt]() Data updated: 2026-02-26T05:49:43.931Z Evaluation of role-playing capabilities using the CharacterBench 1.0 test split, measured across multiple specific criteria (MC, FA, BC\_K, AC\_h, etc.) to derive an overall average score. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Role-playing | | Dataset (including any stated split) | CharacterBench 1.0 (test) | | Metric | MC | | Unit | SCALAR | | Direction | Higher is better | | Results loaded | 28 of 28 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [MC](): Higher is better; unit: SCALAR. - [FA](): Higher is better; unit: SCALAR. - [BC\_K](): Higher is better; unit: SCALAR. - [AC\_h](): Higher is better; unit: SCALAR. - [AC\_b](): Higher is better; unit: SCALAR. - [BC\_h\_P](): Higher is better; unit: SCALAR. - [BC\_b\_P](): Higher is better; unit: SCALAR. - [ES](): Higher is better; unit: SCALAR. - [ER](): Higher is better; unit: SCALAR. - [MS](): Higher is better; unit: SCALAR. - [MR](): Higher is better; unit: SCALAR. - [HL](): Higher is better; unit: SCALAR. - [EG](): Higher is better; unit: SCALAR. - [Average Score](): Higher is better; unit: SCALAR. ## Recorded results | Method | MC | Result date | Report paper | | --- | --- | --- | --- | | Qwen2.5-7B-Instruct + Character-R1 | 4.444 | Not recorded | 69608b6453fdd5c36174413b | | Llama-3.2-3B-Instruct + Character-R1 | 4.275 | Not recorded | 69608b6453fdd5c36174413b | | Qwen2.5-7B-RoleMRC | 4.175 | Not recorded | 69608b6453fdd5c36174413b | | Llama-3.1-8B-RoleMRC | 4.169 | Not recorded | 69608b6453fdd5c36174413b | | Crab | 4.131 | Not recorded | 69608b6453fdd5c36174413b | | Haruhi-Zero-7B | 4.088 | Not recorded | 69608b6453fdd5c36174413b | | Qwen2.5-72B-Instruct | 4.06 | Not recorded | 69608b6453fdd5c36174413b | | Qwen2.5-7B-Instruct + RAR | 4.059 | Not recorded | 69608b6453fdd5c36174413b | | CoSER-Llama-3.1-8B | 4.056 | Not recorded | 69608b6453fdd5c36174413b | | Qwen2.5-7B-Instruct + SFT | 4.02 | Not recorded | 69608b6453fdd5c36174413b | | Humanish-Roleplay | 4.013 | Not recorded | 69608b6453fdd5c36174413b | | Qwen2.5-7B-Instruct + Neeko | 3.997 | Not recorded | 69608b6453fdd5c36174413b | | Claude-4-opus | 3.997 | Not recorded | 69608b6453fdd5c36174413b | | Gemini-3-pro | 3.946 | Not recorded | 69608b6453fdd5c36174413b | | Llama-3-70B-Instruct | 3.94 | Not recorded | 69608b6453fdd5c36174413b | | Qwen2.5-7B-Instruct | 3.938 | Not recorded | 69608b6453fdd5c36174413b | | Llama-3.2-3B-Instruct + RAR | 3.848 | Not recorded | 69608b6453fdd5c36174413b | | Llama-3.2-3B-Instruct + Neeko | 3.796 | Not recorded | 69608b6453fdd5c36174413b | | GPT-4o | 3.793 | Not recorded | 69608b6453fdd5c36174413b | | CharacterGLM | 3.76 | Not recorded | 69608b6453fdd5c36174413b | | Baichuan-NPC | 3.672 | Not recorded | 69608b6453fdd5c36174413b | | Llama-3.2-3B-Instruct + SFT | 3.606 | Not recorded | 69608b6453fdd5c36174413b | | Llama-3.2-3B-Instruct | 3.588 | Not recorded | 69608b6453fdd5c36174413b | | GLM-4 | 3.524 | Not recorded | 69608b6453fdd5c36174413b | | GPT-3.5-turbo | 3.49 | Not recorded | 69608b6453fdd5c36174413b | | CharacterGLM-6B | 3.245 | Not recorded | 69608b6453fdd5c36174413b | | Peach-9B-8k-Roleplay | 2.931 | Not recorded | 69608b6453fdd5c36174413b | | Peach-2.0-9B-8k-Roleplay | 2.819 | Not recorded | 69608b6453fdd5c36174413b | ### Qwen2.5-7B-Instruct + Character-R1 MC: 4.444. Result ID: 699698ed5698665ec4e28520. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B","Method":"Character-R1"}. This record does not assert independent reproduction. ### Llama-3.2-3B-Instruct + Character-R1 MC: 4.275. Result ID: 699698ed5698665ec4e28525. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama-3.2-3B","Method":"Character-R1"}. This record does not assert independent reproduction. ### Qwen2.5-7B-RoleMRC MC: 4.175. Result ID: 699698ed5698665ec4e2852d. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### Llama-3.1-8B-RoleMRC MC: 4.169. Result ID: 699698ed5698665ec4e28528. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### Crab MC: 4.131. Result ID: 699698ed5698665ec4e2852a. [Reporting paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### Haruhi-Zero-7B MC: 4.088. Result ID: 699698ed5698665ec4e28529. [Reporting paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### Qwen2.5-72B-Instruct MC: 4.06. Result ID: 699698ed5698665ec4e28535. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"Large Foundation Model"}. This record does not assert independent reproduction. ### Qwen2.5-7B-Instruct + RAR MC: 4.059. Result ID: 699698ed5698665ec4e2851f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B","Framework":"RAR"}. This record does not assert independent reproduction. ### CoSER-Llama-3.1-8B MC: 4.056. Result ID: 699698ed5698665ec4e2852b. [Reporting paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### Qwen2.5-7B-Instruct + SFT MC: 4.02. Result ID: 699698ed5698665ec4e2851d. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B","Adaptation":"SFT"}. This record does not assert independent reproduction. ### Humanish-Roleplay MC: 4.013. Result ID: 699698ed5698665ec4e2852e. [Reporting paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### Qwen2.5-7B-Instruct + Neeko MC: 3.997. Result ID: 699698ed5698665ec4e2851e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B","Framework":"Neeko"}. This record does not assert independent reproduction. ### Claude-4-opus MC: 3.997. Result ID: 699698ed5698665ec4e28537. [Reporting paper]() Recorded evaluation configuration: {"Type":"Large Foundation Model"}. This record does not assert independent reproduction. ### Gemini-3-pro MC: 3.946. Result ID: 699698ed5698665ec4e28536. [Reporting paper]() Recorded evaluation configuration: {"Type":"Large Foundation Model"}. This record does not assert independent reproduction. ### Llama-3-70B-Instruct MC: 3.94. Result ID: 699698ed5698665ec4e28530. [Reporting paper]() Recorded evaluation configuration: {"Type":"Large Foundation Model"}. This record does not assert independent reproduction. ### Qwen2.5-7B-Instruct MC: 3.938. Result ID: 699698ed5698665ec4e2851c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B"}. This record does not assert independent reproduction. ### Llama-3.2-3B-Instruct + RAR MC: 3.848. Result ID: 699698ed5698665ec4e28524. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama-3.2-3B","Framework":"RAR"}. This record does not assert independent reproduction. ### Llama-3.2-3B-Instruct + Neeko MC: 3.796. Result ID: 699698ed5698665ec4e28523. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama-3.2-3B","Framework":"Neeko"}. This record does not assert independent reproduction. ### GPT-4o MC: 3.793. Result ID: 699698ed5698665ec4e28534. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"Large Foundation Model"}. This record does not assert independent reproduction. ### CharacterGLM MC: 3.76. Result ID: 699698ed5698665ec4e2852f. [Reporting paper]() Recorded evaluation configuration: {"Type":"Large Foundation Model"}. This record does not assert independent reproduction. ### Baichuan-NPC MC: 3.672. Result ID: 699698ed5698665ec4e28532. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"Large Foundation Model"}. This record does not assert independent reproduction. ### Llama-3.2-3B-Instruct + SFT MC: 3.606. Result ID: 699698ed5698665ec4e28522. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Llama-3.2-3B","Adaptation":"SFT"}. This record does not assert independent reproduction. ### Llama-3.2-3B-Instruct MC: 3.588. Result ID: 699698ed5698665ec4e28521. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Llama-3.2-3B"}. This record does not assert independent reproduction. ### GLM-4 MC: 3.524. Result ID: 699698ed5698665ec4e28531. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"Large Foundation Model"}. This record does not assert independent reproduction. ### GPT-3.5-turbo MC: 3.49. Result ID: 699698ed5698665ec4e28533. [Reporting paper]() Recorded evaluation configuration: {"Type":"Large Foundation Model"}. This record does not assert independent reproduction. ### CharacterGLM-6B MC: 3.245. Result ID: 699698ed5698665ec4e28526. [Reporting paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### Peach-9B-8k-Roleplay MC: 2.931. Result ID: 699698ed5698665ec4e28527. [Reporting paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### Peach-2.0-9B-8k-Roleplay MC: 2.819. Result ID: 699698ed5698665ec4e2852c. [Reporting paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.