# Open-Ended Image Editing on MadVerse (test)

Source: [Open-Ended Image Editing on MadVerse (test)](<https://www.sota2.com/research/sota/open-ended-image-editing-on-madverse-test>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-05-15T10:03:40.204Z

Evaluation of Open-Ended Image Editing performance on the test split of the MadVerse dataset, assessed using metrics such as Instruction Following, Identity Preservation, and Visual Quality.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Open-Ended Image Editing |
| Dataset (including any stated split) | MadVerse (test) |
| Metric | Instruction Following |
| Unit | SCALAR |
| Direction | Higher is better |
| Results loaded | 7 of 7 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Instruction Following](<https://www.sota2.com/research/sota/open-ended-image-editing-on-madverse-test.md?metric=Instruction%20Following>): Higher is better; unit: SCALAR.

- [Identity Preservation](<https://www.sota2.com/research/sota/open-ended-image-editing-on-madverse-test.md?metric=Identity%20Preservation>): Higher is better; unit: SCALAR.

- [Visual Quality](<https://www.sota2.com/research/sota/open-ended-image-editing-on-madverse-test.md?metric=Visual%20Quality>): Higher is better; unit: SCALAR.

## Recorded results

| Method | Instruction Following | Result date | Report paper |
| --- | --- | --- | --- |
| Plan2Pix | 4.196 | Not recorded | 6a06a8cbdbc836b87d82a390 |
| Plan2Pix | 4.196 | Not recorded | 6a06a8cbdbc836b87d82a390 |
| Qwen-Image-Edit-2511 | 3.807 | Not recorded | 6a06a8cbdbc836b87d82a390 |
| Qwen-Image-Edit-2511 | 3.355 | Not recorded | 6a06a8cbdbc836b87d82a390 |
| FLUX.1-Kontext-dev | 3.33 | Not recorded | 6a06a8cbdbc836b87d82a390 |
| FLUX.1-Kontext-dev | 2.32 | Not recorded | 6a06a8cbdbc836b87d82a390 |
| GenArtist | 1.252 | Not recorded | 6a06a8cbdbc836b87d82a390 |

### Plan2Pix

Instruction Following: 4.196. Result ID: 6a06ef6547f34ce924ef22e2.

[Reporting paper](<https://www.sota2.com/research/paper/6a06a8cbdbc836b87d82a390.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a06a8cbdbc836b87d82a390.md>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

### Plan2Pix

Instruction Following: 4.196. Result ID: 6a06ef6547f34ce924ef22f5.

[Reporting paper](<https://www.sota2.com/research/paper/6a06a8cbdbc836b87d82a390.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a06a8cbdbc836b87d82a390.md>)

Recorded evaluation configuration: {"Evaluation Judge":"Gemini-3-Pro"}. This record does not assert independent reproduction.

### Qwen-Image-Edit-2511

Instruction Following: 3.807. Result ID: 6a06ef6547f34ce924ef22f4.

[Reporting paper](<https://www.sota2.com/research/paper/6a06a8cbdbc836b87d82a390.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7a6a1e209131cea83137.md>)

Recorded evaluation configuration: {"Input Strategy":"Qwen3-VL-8B Plan","Evaluation Judge":"Gemini-3-Pro"}. This record does not assert independent reproduction.

### Qwen-Image-Edit-2511

Instruction Following: 3.355. Result ID: 6a06ef6547f34ce924ef22f3.

[Reporting paper](<https://www.sota2.com/research/paper/6a06a8cbdbc836b87d82a390.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7a6a1e209131cea83137.md>)

Recorded evaluation configuration: {"Input Strategy":"High-Level Instruction","Evaluation Judge":"Gemini-3-Pro"}. This record does not assert independent reproduction.

### FLUX.1-Kontext-dev

Instruction Following: 3.33. Result ID: 6a06ef6547f34ce924ef22f2.

[Reporting paper](<https://www.sota2.com/research/paper/6a06a8cbdbc836b87d82a390.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa79a81e209131cea7eb79.md>)

Recorded evaluation configuration: {"Input Strategy":"Qwen3-VL-8B Plan","Evaluation Judge":"Gemini-3-Pro"}. This record does not assert independent reproduction.

### FLUX.1-Kontext-dev

Instruction Following: 2.32. Result ID: 6a06ef6547f34ce924ef22f1.

[Reporting paper](<https://www.sota2.com/research/paper/6a06a8cbdbc836b87d82a390.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa79a81e209131cea7eb79.md>)

Recorded evaluation configuration: {"Input Strategy":"High-Level Instruction","Evaluation Judge":"Gemini-3-Pro"}. This record does not assert independent reproduction.

### GenArtist

Instruction Following: 1.252. Result ID: 6a06ef6547f34ce924ef22e1.

[Reporting paper](<https://www.sota2.com/research/paper/6a06a8cbdbc836b87d82a390.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa756f1e209131cea67e3f.md>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.
