Annotation Agreement Estimation on iTAG vs real-world data within-n centered
0.901Pearson CorrelationDeepSeek-R1
Evaluation Results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| DeepSeek-R1Agreement Metric=SHD2026.04 | 0.901 | 10 | 0.879 | 0.811 | 0.575 | |
| DeepSeek-R1Agreement Metric=SID2026.04 | 0.893 | 10 | 0.865 | 0.797 | 0.672 | |
| DeepSeek-R1Agreement Metric=Average2026.04 | 0.885 | — | 0.862 | 0.784 | — | |
| DeepSeek-R1Agreement Metric=F1G2026.04 | 0.862 | 10 | 0.842 | 0.743 | 0.552 |