# Image Captioning Metric Evaluation on UCM-preference (test)

Source: [Image Captioning Metric Evaluation on UCM-preference (test)](<https://www.sota2.com/research/sota/image-captioning-metric-evaluation-on-ucm-preference-test>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-04-28T07:10:11.311Z

Evaluation of image captioning metrics on the test split of the UCM-preference dataset, assessed using Kendall's Tau correlation coefficients (τ\_b and τ\_c) to measure agreement.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Image Captioning Metric Evaluation |
| Dataset (including any stated split) | UCM-preference (test) |
| Metric | Kendall's Tau\_b |
| Unit | SCALAR |
| Direction | Higher is better |
| Results loaded | 10 of 10 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Kendall's Tau\_b](<https://www.sota2.com/research/sota/image-captioning-metric-evaluation-on-ucm-preference-test.md?metric=Kendall%27s%20Tau_b>): Higher is better; unit: SCALAR.

- [Kendall's Tau\_c](<https://www.sota2.com/research/sota/image-captioning-metric-evaluation-on-ucm-preference-test.md?metric=Kendall%27s%20Tau_c>): Higher is better; unit: SCALAR.

## Recorded results

| Method | Kendall's Tau\_b | Result date | Report paper |
| --- | --- | --- | --- |
| ReconScore | 28.75 | Not recorded | 69f03f0c55a3c32af89f9fe1 |
| CLIPScore | 25.75 | Not recorded | 69f03f0c55a3c32af89f9fe1 |
| SPICE | -12.35 | Not recorded | 69f03f0c55a3c32af89f9fe1 |
| BLEU-4 | -16.25 | Not recorded | 69f03f0c55a3c32af89f9fe1 |
| BLEU-3 | -16.36 | Not recorded | 69f03f0c55a3c32af89f9fe1 |
| METEOR | -21.7 | Not recorded | 69f03f0c55a3c32af89f9fe1 |
| BLEU-2 | -27.42 | Not recorded | 69f03f0c55a3c32af89f9fe1 |
| ROUGE | -31.27 | Not recorded | 69f03f0c55a3c32af89f9fe1 |
| BLEU-1 | -32.2 | Not recorded | 69f03f0c55a3c32af89f9fe1 |
| CIDEr | -43.59 | Not recorded | 69f03f0c55a3c32af89f9fe1 |

### ReconScore

Kendall's Tau\_b: 28.75. Result ID: 69f05c3b47f34ce924d5b801.

[Reporting paper](<https://www.sota2.com/research/paper/69f03f0c55a3c32af89f9fe1.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69f03f0c55a3c32af89f9fe1.md>)

Recorded evaluation configuration: {"Publication":"-","Reference-type":"Reference-free"}. This record does not assert independent reproduction.

### CLIPScore

Kendall's Tau\_b: 25.75. Result ID: 69f05c3b47f34ce924d5b800.

[Reporting paper](<https://www.sota2.com/research/paper/69f03f0c55a3c32af89f9fe1.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6c0f1e209131cea31cc9.md>)

Recorded evaluation configuration: {"Publication":"EMNLP21","Reference-type":"Reference-free"}. This record does not assert independent reproduction.

### SPICE

Kendall's Tau\_b: -12.35. Result ID: 69f05c3b47f34ce924d5b7ff.

[Reporting paper](<https://www.sota2.com/research/paper/69f03f0c55a3c32af89f9fe1.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa62a41e209131ce9fa82e.md>)

Recorded evaluation configuration: {"Publication":"ECCV16","Reference-type":"Reference-based"}. This record does not assert independent reproduction.

### BLEU-4

Kendall's Tau\_b: -16.25. Result ID: 69f05c3b47f34ce924d5b7fb.

[Reporting paper](<https://www.sota2.com/research/paper/69f03f0c55a3c32af89f9fe1.md>)

Recorded evaluation configuration: {"Publication":"ACL02","Reference-type":"Reference-based"}. This record does not assert independent reproduction.

### BLEU-3

Kendall's Tau\_b: -16.36. Result ID: 69f05c3b47f34ce924d5b7fa.

[Reporting paper](<https://www.sota2.com/research/paper/69f03f0c55a3c32af89f9fe1.md>)

Recorded evaluation configuration: {"Publication":"ACL02","Reference-type":"Reference-based"}. This record does not assert independent reproduction.

### METEOR

Kendall's Tau\_b: -21.7. Result ID: 69f05c3b47f34ce924d5b7fc.

[Reporting paper](<https://www.sota2.com/research/paper/69f03f0c55a3c32af89f9fe1.md>)

Recorded evaluation configuration: {"Publication":"ACL05","Reference-type":"Reference-based"}. This record does not assert independent reproduction.

### BLEU-2

Kendall's Tau\_b: -27.42. Result ID: 69f05c3b47f34ce924d5b7f9.

[Reporting paper](<https://www.sota2.com/research/paper/69f03f0c55a3c32af89f9fe1.md>)

Recorded evaluation configuration: {"Publication":"ACL02","Reference-type":"Reference-based"}. This record does not assert independent reproduction.

### ROUGE

Kendall's Tau\_b: -31.27. Result ID: 69f05c3b47f34ce924d5b7fd.

[Reporting paper](<https://www.sota2.com/research/paper/69f03f0c55a3c32af89f9fe1.md>)

Recorded evaluation configuration: {"Publication":"ACL04","Reference-type":"Reference-based"}. This record does not assert independent reproduction.

### BLEU-1

Kendall's Tau\_b: -32.2. Result ID: 69f05c3b47f34ce924d5b7f8.

[Reporting paper](<https://www.sota2.com/research/paper/69f03f0c55a3c32af89f9fe1.md>)

Recorded evaluation configuration: {"Publication":"ACL02","Reference-type":"Reference-based"}. This record does not assert independent reproduction.

### CIDEr

Kendall's Tau\_b: -43.59. Result ID: 69f05c3b47f34ce924d5b7fe.

[Reporting paper](<https://www.sota2.com/research/paper/69f03f0c55a3c32af89f9fe1.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa61e81e209131ce9f6a37.md>)

Recorded evaluation configuration: {"Publication":"CVPR15","Reference-type":"Reference-based"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.
