# Multimodal Role-playing on MMRole (out-of-domain)

Source: [Multimodal Role-playing on MMRole (out-of-domain)](<https://www.sota2.com/research/sota/multimodal-role-playing-on-mmrole-out-of-domain>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-05-12T12:11:50.006Z

Evaluation of Multimodal Role-playing capabilities on the MMRole (out-of-domain) dataset, measured by specific metrics such as IA, Fluency (Flu), Coherence (Coh), and Overall score.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Multimodal Role-playing |
| Dataset (including any stated split) | MMRole (out-of-domain) |
| Metric | IA |
| Unit | SCALAR |
| Direction | Higher is better |
| Results loaded | 17 of 17 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [IA](<https://www.sota2.com/research/sota/multimodal-role-playing-on-mmrole-out-of-domain.md?metric=IA>): Higher is better; unit: SCALAR.

- [Fluency](<https://www.sota2.com/research/sota/multimodal-role-playing-on-mmrole-out-of-domain.md?metric=Fluency>): Higher is better; unit: SCALAR.

- [Coherence](<https://www.sota2.com/research/sota/multimodal-role-playing-on-mmrole-out-of-domain.md?metric=Coherence>): Higher is better; unit: SCALAR.

- [ITR](<https://www.sota2.com/research/sota/multimodal-role-playing-on-mmrole-out-of-domain.md?metric=ITR>): Higher is better; unit: SCALAR.

- [RA](<https://www.sota2.com/research/sota/multimodal-role-playing-on-mmrole-out-of-domain.md?metric=RA>): Higher is better; unit: SCALAR.

- [PC](<https://www.sota2.com/research/sota/multimodal-role-playing-on-mmrole-out-of-domain.md?metric=PC>): Higher is better; unit: SCALAR.

- [KC](<https://www.sota2.com/research/sota/multimodal-role-playing-on-mmrole-out-of-domain.md?metric=KC>): Higher is better; unit: SCALAR.

- [TC](<https://www.sota2.com/research/sota/multimodal-role-playing-on-mmrole-out-of-domain.md?metric=TC>): Higher is better; unit: SCALAR.

- [Overall Score](<https://www.sota2.com/research/sota/multimodal-role-playing-on-mmrole-out-of-domain.md?metric=Overall%20Score>): Higher is better; unit: SCALAR.

## Recorded results

| Method | IA | Result date | Report paper |
| --- | --- | --- | --- |
| Claude-Opus-4 | 1.164 | Not recorded | 6a02b469dbc836b87d824b21 |
| SFT | 1.136 | Not recorded | 6a02b469dbc836b87d824b21 |
| SFT | 1.12 | Not recorded | 6a02b469dbc836b87d824b21 |
| CAVI | 1.111 | Not recorded | 6a02b469dbc836b87d824b21 |
| VCD | 1.095 | Not recorded | 6a02b469dbc836b87d824b21 |
| VCD | 1.095 | Not recorded | 6a02b469dbc836b87d824b21 |
| GPT-5 | 1.092 | Not recorded | 6a02b469dbc836b87d824b21 |
| CAVI | 1.086 | Not recorded | 6a02b469dbc836b87d824b21 |
| AIR | 1.085 | Not recorded | 6a02b469dbc836b87d824b21 |
| Gemini-Pro-3 | 1.076 | Not recorded | 6a02b469dbc836b87d824b21 |
| ClearSight | 1.075 | Not recorded | 6a02b469dbc836b87d824b21 |
| MemVR | 1.073 | Not recorded | 6a02b469dbc836b87d824b21 |
| AIR | 1.073 | Not recorded | 6a02b469dbc836b87d824b21 |
| MemVR | 1.066 | Not recorded | 6a02b469dbc836b87d824b21 |
| ClearSight | 1.056 | Not recorded | 6a02b469dbc836b87d824b21 |
| Base | 1.039 | Not recorded | 6a02b469dbc836b87d824b21 |
| Base | 1.02 | Not recorded | 6a02b469dbc836b87d824b21 |

### Claude-Opus-4

IA: 1.164. Result ID: 6a03185247f34ce924e7265b.

[Reporting paper](<https://www.sota2.com/research/paper/6a02b469dbc836b87d824b21.md>)

Recorded evaluation configuration: {"Backbone":"Closed large MLLMs"}. This record does not assert independent reproduction.

### SFT

IA: 1.136. Result ID: 6a03185247f34ce924e7264c.

[Reporting paper](<https://www.sota2.com/research/paper/6a02b469dbc836b87d824b21.md>)

Recorded evaluation configuration: {"Backbone":"Qwen3-VL-8B"}. This record does not assert independent reproduction.

### SFT

IA: 1.12. Result ID: 6a03185247f34ce924e72653.

[Reporting paper](<https://www.sota2.com/research/paper/6a02b469dbc836b87d824b21.md>)

Recorded evaluation configuration: {"Backbone":"Gemma-3n-E4B"}. This record does not assert independent reproduction.

### CAVI

IA: 1.111. Result ID: 6a03185247f34ce924e72651.

[Reporting paper](<https://www.sota2.com/research/paper/6a02b469dbc836b87d824b21.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a02b469dbc836b87d824b21.md>)

Recorded evaluation configuration: {"Backbone":"Qwen3-VL-8B"}. This record does not assert independent reproduction.

### VCD

IA: 1.095. Result ID: 6a03185247f34ce924e7264f.

[Reporting paper](<https://www.sota2.com/research/paper/6a02b469dbc836b87d824b21.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa71461e209131cea5053d.md>)

Recorded evaluation configuration: {"Backbone":"Qwen3-VL-8B"}. This record does not assert independent reproduction.

### VCD

IA: 1.095. Result ID: 6a03185247f34ce924e72656.

[Reporting paper](<https://www.sota2.com/research/paper/6a02b469dbc836b87d824b21.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa71461e209131cea5053d.md>)

Recorded evaluation configuration: {"Backbone":"Gemma-3n-E4B"}. This record does not assert independent reproduction.

### GPT-5

IA: 1.092. Result ID: 6a03185247f34ce924e72659.

[Reporting paper](<https://www.sota2.com/research/paper/6a02b469dbc836b87d824b21.md>)

[Method origin paper](<https://www.sota2.com/research/paper/695f39f053fdd5c361743ecb.md>)

Recorded evaluation configuration: {"Backbone":"Closed large MLLMs"}. This record does not assert independent reproduction.

### CAVI

IA: 1.086. Result ID: 6a03185247f34ce924e72658.

[Reporting paper](<https://www.sota2.com/research/paper/6a02b469dbc836b87d824b21.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a02b469dbc836b87d824b21.md>)

Recorded evaluation configuration: {"Backbone":"Gemma-3n-E4B"}. This record does not assert independent reproduction.

### AIR

IA: 1.085. Result ID: 6a03185247f34ce924e72650.

[Reporting paper](<https://www.sota2.com/research/paper/6a02b469dbc836b87d824b21.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69a519692d603e6de3e3c851.md>)

Recorded evaluation configuration: {"Backbone":"Qwen3-VL-8B"}. This record does not assert independent reproduction.

### Gemini-Pro-3

IA: 1.076. Result ID: 6a03185247f34ce924e7265a.

[Reporting paper](<https://www.sota2.com/research/paper/6a02b469dbc836b87d824b21.md>)

Recorded evaluation configuration: {"Backbone":"Closed large MLLMs"}. This record does not assert independent reproduction.

### ClearSight

IA: 1.075. Result ID: 6a03185247f34ce924e72655.

[Reporting paper](<https://www.sota2.com/research/paper/6a02b469dbc836b87d824b21.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa78dd1e209131cea7adf7.md>)

[Implementation](<https://github.com/ustc-hyin/ClearSight>)

Recorded evaluation configuration: {"Backbone":"Gemma-3n-E4B"}. This record does not assert independent reproduction.

### MemVR

IA: 1.073. Result ID: 6a03185247f34ce924e72654.

[Reporting paper](<https://www.sota2.com/research/paper/6a02b469dbc836b87d824b21.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa78821e209131cea78bff.md>)

Recorded evaluation configuration: {"Backbone":"Gemma-3n-E4B"}. This record does not assert independent reproduction.

### AIR

IA: 1.073. Result ID: 6a03185247f34ce924e72657.

[Reporting paper](<https://www.sota2.com/research/paper/6a02b469dbc836b87d824b21.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69a519692d603e6de3e3c851.md>)

Recorded evaluation configuration: {"Backbone":"Gemma-3n-E4B"}. This record does not assert independent reproduction.

### MemVR

IA: 1.066. Result ID: 6a03185247f34ce924e7264d.

[Reporting paper](<https://www.sota2.com/research/paper/6a02b469dbc836b87d824b21.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa78821e209131cea78bff.md>)

Recorded evaluation configuration: {"Backbone":"Qwen3-VL-8B"}. This record does not assert independent reproduction.

### ClearSight

IA: 1.056. Result ID: 6a03185247f34ce924e7264e.

[Reporting paper](<https://www.sota2.com/research/paper/6a02b469dbc836b87d824b21.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa78dd1e209131cea7adf7.md>)

[Implementation](<https://github.com/ustc-hyin/ClearSight>)

Recorded evaluation configuration: {"Backbone":"Qwen3-VL-8B"}. This record does not assert independent reproduction.

### Base

IA: 1.039. Result ID: 6a03185247f34ce924e7264b.

[Reporting paper](<https://www.sota2.com/research/paper/6a02b469dbc836b87d824b21.md>)

Recorded evaluation configuration: {"Backbone":"Qwen3-VL-8B"}. This record does not assert independent reproduction.

### Base

IA: 1.02. Result ID: 6a03185247f34ce924e72652.

[Reporting paper](<https://www.sota2.com/research/paper/6a02b469dbc836b87d824b21.md>)

Recorded evaluation configuration: {"Backbone":"Gemma-3n-E4B"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.