System ranking correlation research benchmarks

Benchmarks

Dataset NameSOTA methodMetricTrendResultsLast Updated

CRSArena-Eval (All)

FACE

0.918Pearson Correlation (r)

Apr 7, 2026

CRSArena-Eval Dial-level

FACE

0.913Pearson Correlation (r)

Apr 7, 2026

CRSArena-Eval Turn-level

FACE

0.93Pearson Correlation (r)

Apr 7, 2026

TREC-DL 23

Rank1-14B

90.6MAP@100

Feb 26, 2026

TREC-DL 22

Rank1-32B

92.5MAP@100

Feb 26, 2026

TREC-DL 21

UMBRELA

0.921MAP@100

Feb 26, 2026

TREC-DL 20

UMBRELA

0.93MAP@100

Feb 26, 2026

TREC-DL 2019 1 (test)

UMBRELA

0.946MAP@100

Feb 26, 2026

TREC-DL 2023 2 (test)

Rank1

0.909Kendall's tau (MAP@100)

Feb 26, 2026

TREC-DL 2022 2 (test)

Rank1

0.937Kendall's tau (MAP@100)

Feb 26, 2026

TREC-DL 2021 2 (test)

UMBRELA

0.921Kendall's tau (MAP@100)

Feb 26, 2026

TREC-DL 2020 1 (test)

UMBRELA

0.93Kendall's tau (MAP@100)

Feb 26, 2026