# Dialogue Annotation on Eedi (test) Source: [Dialogue Annotation on Eedi (test)]() Collection index: [llms.txt]() Data updated: 2026-04-06T08:57:23.424Z Evaluation of dialogue annotation performance on the test split of the Eedi dataset, measured using Cohen's Kappa and Accuracy. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Dialogue Annotation | | Dataset (including any stated split) | Eedi (test) | | Metric | Cohen's Kappa | | Unit | SCALAR | | Direction | Higher is better | | Results loaded | 12 of 12 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Cohen's Kappa](): Higher is better; unit: SCALAR. - [Accuracy](): Higher is better; unit: PERCENT. ## Recorded results | Method | Cohen's Kappa | Result date | Report paper | | --- | --- | --- | --- | | Sonnet 4.6 (RAG\_FINETUNED\_UTT) | 0.743 | Not recorded | 69d33df57d0172140ec73651 | | Qwen3-32b (RAG\_FINETUNED\_UTT) | 0.694 | Not recorded | 69d33df57d0172140ec73651 | | GPT-5.2 (RAG\_FINETUNED\_UTT) | 0.659 | Not recorded | 69d33df57d0172140ec73651 | | Sonnet 4.6 (RAG\_NO\_FINETUNE) | 0.632 | Not recorded | 69d33df57d0172140ec73651 | | Sonnet 4.6 (RAG\_FINETUNED\_CHUNK) | 0.632 | Not recorded | 69d33df57d0172140ec73651 | | Qwen3-32b (RAG\_NO\_FINETUNE) | 0.59 | Not recorded | 69d33df57d0172140ec73651 | | Qwen3-32b (RAG\_FINETUNED\_CHUNK) | 0.545 | Not recorded | 69d33df57d0172140ec73651 | | GPT-5.2 (RAG\_FINETUNED\_CHUNK) | 0.53 | Not recorded | 69d33df57d0172140ec73651 | | GPT-5.2 (RAG\_NO\_FINETUNE) | 0.505 | Not recorded | 69d33df57d0172140ec73651 | | Sonnet 4.6 (NO\_RAG) | 0.41 | Not recorded | 69d33df57d0172140ec73651 | | GPT-5.2 (NO\_RAG) | 0.351 | Not recorded | 69d33df57d0172140ec73651 | | Qwen3-32b (NO\_RAG) | 0.16 | Not recorded | 69d33df57d0172140ec73651 | ### Sonnet 4.6 (RAG\_FINETUNED\_UTT) Cohen's Kappa: 0.743. Result ID: 69d3757147f34ce924bb70bb. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Sonnet 4.6","Condition":"RAG\_FINETUNED\_UTT","Retrieval depth (k)":"3"}. This record does not assert independent reproduction. ### Qwen3-32b (RAG\_FINETUNED\_UTT) Cohen's Kappa: 0.694. Result ID: 69d3757147f34ce924bb70bf. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen3-32b","Condition":"RAG\_FINETUNED\_UTT","Retrieval depth (k)":"3"}. This record does not assert independent reproduction. ### GPT-5.2 (RAG\_FINETUNED\_UTT) Cohen's Kappa: 0.659. Result ID: 69d3757147f34ce924bb70b7. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"GPT-5.2","Condition":"RAG\_FINETUNED\_UTT","Retrieval depth (k)":"3"}. This record does not assert independent reproduction. ### Sonnet 4.6 (RAG\_NO\_FINETUNE) Cohen's Kappa: 0.632. Result ID: 69d3757147f34ce924bb70b9. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Sonnet 4.6","Condition":"RAG\_NO\_FINETUNE","Retrieval depth (k)":"3"}. This record does not assert independent reproduction. ### Sonnet 4.6 (RAG\_FINETUNED\_CHUNK) Cohen's Kappa: 0.632. Result ID: 69d3757147f34ce924bb70ba. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Sonnet 4.6","Condition":"RAG\_FINETUNED\_CHUNK","Retrieval depth (k)":"3"}. This record does not assert independent reproduction. ### Qwen3-32b (RAG\_NO\_FINETUNE) Cohen's Kappa: 0.59. Result ID: 69d3757147f34ce924bb70bd. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen3-32b","Condition":"RAG\_NO\_FINETUNE","Retrieval depth (k)":"3"}. This record does not assert independent reproduction. ### Qwen3-32b (RAG\_FINETUNED\_CHUNK) Cohen's Kappa: 0.545. Result ID: 69d3757147f34ce924bb70be. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen3-32b","Condition":"RAG\_FINETUNED\_CHUNK","Retrieval depth (k)":"3"}. This record does not assert independent reproduction. ### GPT-5.2 (RAG\_FINETUNED\_CHUNK) Cohen's Kappa: 0.53. Result ID: 69d3757147f34ce924bb70b6. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"GPT-5.2","Condition":"RAG\_FINETUNED\_CHUNK","Retrieval depth (k)":"3"}. This record does not assert independent reproduction. ### GPT-5.2 (RAG\_NO\_FINETUNE) Cohen's Kappa: 0.505. Result ID: 69d3757147f34ce924bb70b5. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"GPT-5.2","Condition":"RAG\_NO\_FINETUNE","Retrieval depth (k)":"3"}. This record does not assert independent reproduction. ### Sonnet 4.6 (NO\_RAG) Cohen's Kappa: 0.41. Result ID: 69d3757147f34ce924bb70b8. [Reporting paper]() Recorded evaluation configuration: {"Model":"Sonnet 4.6","Condition":"NO\_RAG","Retrieval depth (k)":"3"}. This record does not assert independent reproduction. ### GPT-5.2 (NO\_RAG) Cohen's Kappa: 0.351. Result ID: 69d3757147f34ce924bb70b4. [Reporting paper]() Recorded evaluation configuration: {"Model":"GPT-5.2","Condition":"NO\_RAG","Retrieval depth (k)":"3"}. This record does not assert independent reproduction. ### Qwen3-32b (NO\_RAG) Cohen's Kappa: 0.16. Result ID: 69d3757147f34ce924bb70bc. [Reporting paper]() Recorded evaluation configuration: {"Model":"Qwen3-32b","Condition":"NO\_RAG","Retrieval depth (k)":"3"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.