# Semiosis Quality Evaluation on HGI SemiosisArt

Source: [Semiosis Quality Evaluation on HGI SemiosisArt](<https://www.sota2.com/research/sota/semiosis-quality-evaluation-on-hgi-semiosisart>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-04-13T07:49:41.236Z

Evaluation of model performance on the HGI SemiosisArt dataset using correlation metrics (KRCC, SRCC, CCC) to assess Semiosis Quality.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Semiosis Quality Evaluation |
| Dataset (including any stated split) | HGI SemiosisArt |
| Metric | KRCC |
| Unit | SCALAR |
| Direction | Higher is better |
| Results loaded | 18 of 18 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [KRCC](<https://www.sota2.com/research/sota/semiosis-quality-evaluation-on-hgi-semiosisart.md?metric=KRCC>): Higher is better; unit: SCALAR.

- [SRCC](<https://www.sota2.com/research/sota/semiosis-quality-evaluation-on-hgi-semiosisart.md?metric=SRCC>): Higher is better; unit: SCALAR.

- [CCC](<https://www.sota2.com/research/sota/semiosis-quality-evaluation-on-hgi-semiosisart.md?metric=CCC>): Higher is better; unit: SCALAR.

## Recorded results

| Method | KRCC | Result date | Report paper |
| --- | --- | --- | --- |
| Human (Non-expert) | 0.79 | Not recorded | 69dc788d7d0172140ec78ad6 |
| SemJudge (Gemini-Flash) | 0.746 | Not recorded | 69dc788d7d0172140ec78ad6 |
| SemJudge (Qwen-35B-A3B) | 0.674 | Not recorded | 69dc788d7d0172140ec78ad6 |
| SemJudge (Qwen-9B) | 0.533 | Not recorded | 69dc788d7d0172140ec78ad6 |
| ArtCoT | 0.294 | Not recorded | 69dc788d7d0172140ec78ad6 |
| LMM4LMM | 0.274 | Not recorded | 69dc788d7d0172140ec78ad6 |
| VIEScore | 0.241 | Not recorded | 69dc788d7d0172140ec78ad6 |
| PickScore | 0.202 | Not recorded | 69dc788d7d0172140ec78ad6 |
| DSG | 0.153 | Not recorded | 69dc788d7d0172140ec78ad6 |
| CLIP-IQA | 0.08 | Not recorded | 69dc788d7d0172140ec78ad6 |
| ArtiMuse | 0.075 | Not recorded | 69dc788d7d0172140ec78ad6 |
| CLIPScore | 0.041 | Not recorded | 69dc788d7d0172140ec78ad6 |
| Aesthetic Predictor | 0.03 | Not recorded | 69dc788d7d0172140ec78ad6 |
| HPSv2 | 0.03 | Not recorded | 69dc788d7d0172140ec78ad6 |
| DeQA-Score | 0.023 | Not recorded | 69dc788d7d0172140ec78ad6 |
| ImageReward | -0.002 | Not recorded | 69dc788d7d0172140ec78ad6 |
| Random Guess | -0.023 | Not recorded | 69dc788d7d0172140ec78ad6 |
| GalleryGPT | -0.034 | Not recorded | 69dc788d7d0172140ec78ad6 |

### Human (Non-expert)

KRCC: 0.79. Result ID: 69dc9fdf47f34ce924c35982.

[Reporting paper](<https://www.sota2.com/research/paper/69dc788d7d0172140ec78ad6.md>)

Recorded evaluation configuration: {"Group":"Human Reference","Backbone":"-"}. This record does not assert independent reproduction.

### SemJudge (Gemini-Flash)

KRCC: 0.746. Result ID: 69dc9fdf47f34ce924c35985.

[Reporting paper](<https://www.sota2.com/research/paper/69dc788d7d0172140ec78ad6.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69dc788d7d0172140ec78ad6.md>)

[Implementation](<https://github.com/songrise/SemJudge>)

Recorded evaluation configuration: {"Group":"Semiosis Quality","Backbone":"Zero-shot MLLM"}. This record does not assert independent reproduction.

### SemJudge (Qwen-35B-A3B)

KRCC: 0.674. Result ID: 69dc9fdf47f34ce924c35984.

[Reporting paper](<https://www.sota2.com/research/paper/69dc788d7d0172140ec78ad6.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69dc788d7d0172140ec78ad6.md>)

[Implementation](<https://github.com/songrise/SemJudge>)

Recorded evaluation configuration: {"Group":"Semiosis Quality","Backbone":"Zero-shot MLLM"}. This record does not assert independent reproduction.

### SemJudge (Qwen-9B)

KRCC: 0.533. Result ID: 69dc9fdf47f34ce924c35983.

[Reporting paper](<https://www.sota2.com/research/paper/69dc788d7d0172140ec78ad6.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69dc788d7d0172140ec78ad6.md>)

[Implementation](<https://github.com/songrise/SemJudge>)

Recorded evaluation configuration: {"Group":"Semiosis Quality","Backbone":"Zero-shot MLLM"}. This record does not assert independent reproduction.

### ArtCoT

KRCC: 0.294. Result ID: 69dc9fdf47f34ce924c3597f.

[Reporting paper](<https://www.sota2.com/research/paper/69dc788d7d0172140ec78ad6.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7af61e209131cea85fca.md>)

Recorded evaluation configuration: {"Group":"Formal Analysis","Backbone":"Zero-shot MLLM"}. This record does not assert independent reproduction.

### LMM4LMM

KRCC: 0.274. Result ID: 69dc9fdf47f34ce924c3597c.

[Reporting paper](<https://www.sota2.com/research/paper/69dc788d7d0172140ec78ad6.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa77f61e209131cea763fb.md>)

Recorded evaluation configuration: {"Group":"Rationale&Scoring","Backbone":"Tuned MLLM"}. This record does not assert independent reproduction.

### VIEScore

KRCC: 0.241. Result ID: 69dc9fdf47f34ce924c3597d.

[Reporting paper](<https://www.sota2.com/research/paper/69dc788d7d0172140ec78ad6.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73651e209131cea5c307.md>)

Recorded evaluation configuration: {"Group":"Structured Rationale","Backbone":"Zero-shot MLLM"}. This record does not assert independent reproduction.

### PickScore

KRCC: 0.202. Result ID: 69dc9fdf47f34ce924c35979.

[Reporting paper](<https://www.sota2.com/research/paper/69dc788d7d0172140ec78ad6.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa713e1e209131cea500c1.md>)

[Implementation](<https://github.com/yuvalkirstain/PickScore>)

Recorded evaluation configuration: {"Group":"Preference Scoring","Backbone":"CLIP"}. This record does not assert independent reproduction.

### DSG

KRCC: 0.153. Result ID: 69dc9fdf47f34ce924c3597e.

[Reporting paper](<https://www.sota2.com/research/paper/69dc788d7d0172140ec78ad6.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa72851e209131cea56fa8.md>)

Recorded evaluation configuration: {"Group":"Structured Rationale","Backbone":"Zero-shot MLLM"}. This record does not assert independent reproduction.

### CLIP-IQA

KRCC: 0.08. Result ID: 69dc9fdf47f34ce924c35976.

[Reporting paper](<https://www.sota2.com/research/paper/69dc788d7d0172140ec78ad6.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6dfd1e209131cea3d39b.md>)

[Implementation](<https://github.com/IceClear/CLIP-IQA>)

Recorded evaluation configuration: {"Group":"Quality Scoring","Backbone":"CLIP"}. This record does not assert independent reproduction.

### ArtiMuse

KRCC: 0.075. Result ID: 69dc9fdf47f34ce924c35980.

[Reporting paper](<https://www.sota2.com/research/paper/69dc788d7d0172140ec78ad6.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7a7d1e209131cea836d3.md>)

Recorded evaluation configuration: {"Group":"Art Interpretation","Backbone":"Tuned MLLM"}. This record does not assert independent reproduction.

### CLIPScore

KRCC: 0.041. Result ID: 69dc9fdf47f34ce924c35975.

[Reporting paper](<https://www.sota2.com/research/paper/69dc788d7d0172140ec78ad6.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa69581e209131cea21ad0.md>)

[Implementation](<https://github.com/OpenAI/CLIP>)

Recorded evaluation configuration: {"Group":"Alignment Scoring","Backbone":"CLIP"}. This record does not assert independent reproduction.

### Aesthetic Predictor

KRCC: 0.03. Result ID: 69dc9fdf47f34ce924c35978.

[Reporting paper](<https://www.sota2.com/research/paper/69dc788d7d0172140ec78ad6.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6dad1e209131cea3b6a9.md>)

[Implementation](<https://github.com/rvencu/crawlingathome-gpu-hcloud>)

Recorded evaluation configuration: {"Group":"Preference Scoring","Backbone":"CLIP"}. This record does not assert independent reproduction.

### HPSv2

KRCC: 0.03. Result ID: 69dc9fdf47f34ce924c3597a.

[Reporting paper](<https://www.sota2.com/research/paper/69dc788d7d0172140ec78ad6.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa70621e209131cea4acbd.md>)

Recorded evaluation configuration: {"Group":"Preference Scoring","Backbone":"CLIP"}. This record does not assert independent reproduction.

### DeQA-Score

KRCC: 0.023. Result ID: 69dc9fdf47f34ce924c35977.

[Reporting paper](<https://www.sota2.com/research/paper/69dc788d7d0172140ec78ad6.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa77991e209131cea74442.md>)

Recorded evaluation configuration: {"Group":"Quality Scoring","Backbone":"Tuned MLLM"}. This record does not assert independent reproduction.

### ImageReward

KRCC: -0.002. Result ID: 69dc9fdf47f34ce924c3597b.

[Reporting paper](<https://www.sota2.com/research/paper/69dc788d7d0172140ec78ad6.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa719f1e209131cea52222.md>)

[Implementation](<https://github.com/THUDM/ImageReward>)

Recorded evaluation configuration: {"Group":"Preference Scoring","Backbone":"BLIP"}. This record does not assert independent reproduction.

### Random Guess

KRCC: -0.023. Result ID: 69dc9fdf47f34ce924c35974.

[Reporting paper](<https://www.sota2.com/research/paper/69dc788d7d0172140ec78ad6.md>)

Recorded evaluation configuration: {"Group":"Conventional Scorers","Backbone":"-"}. This record does not assert independent reproduction.

### GalleryGPT

KRCC: -0.034. Result ID: 69dc9fdf47f34ce924c35981.

[Reporting paper](<https://www.sota2.com/research/paper/69dc788d7d0172140ec78ad6.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa744f1e209131cea60f40.md>)

Recorded evaluation configuration: {"Group":"Art Interpretation","Backbone":"Tuned MLLM"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.