# Dialogue Annotation on TalkMoves (test) Source: [Dialogue Annotation on TalkMoves (test)]() Collection index: [llms.txt]() Data updated: 2026-04-06T08:57:23.399Z Evaluation of dialogue annotation performance on the test split of the TalkMoves dataset, measured using Cohen's Kappa and Accuracy. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Dialogue Annotation | | Dataset (including any stated split) | TalkMoves (test) | | Metric | Cohen's Kappa | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 12 of 12 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Cohen's Kappa](): Higher is better; unit: PERCENT. - [Accuracy](): Higher is better; unit: PERCENT. ## Recorded results | Method | Cohen's Kappa | Result date | Report paper | | --- | --- | --- | --- | | Sonnet 4.6 (RAG\_FINETUNED\_UTT) | 58 | Not recorded | 69d33df57d0172140ec73651 | | GPT-5.2 (RAG\_FINETUNED\_UTT) | 53.9 | Not recorded | 69d33df57d0172140ec73651 | | Qwen3-32b (RAG\_FINETUNED\_UTT) | 52.6 | Not recorded | 69d33df57d0172140ec73651 | | Sonnet 4.6 (RAG\_FINETUNED\_CHUNK) | 50.1 | Not recorded | 69d33df57d0172140ec73651 | | Sonnet 4.6 (RAG\_NO\_FINETUNE) | 49.8 | Not recorded | 69d33df57d0172140ec73651 | | GPT-5.2 (RAG\_NO\_FINETUNE) | 47.9 | Not recorded | 69d33df57d0172140ec73651 | | GPT-5.2 (RAG\_FINETUNED\_CHUNK) | 45.4 | Not recorded | 69d33df57d0172140ec73651 | | Qwen3-32b (RAG\_NO\_FINETUNE) | 44.2 | Not recorded | 69d33df57d0172140ec73651 | | Qwen3-32b (RAG\_FINETUNED\_CHUNK) | 42.8 | Not recorded | 69d33df57d0172140ec73651 | | Sonnet 4.6 (NO\_RAG) | 41.3 | Not recorded | 69d33df57d0172140ec73651 | | GPT-5.2 (NO\_RAG) | 31.5 | Not recorded | 69d33df57d0172140ec73651 | | Qwen3-32b (NO\_RAG) | 27.5 | Not recorded | 69d33df57d0172140ec73651 | ### Sonnet 4.6 (RAG\_FINETUNED\_UTT) Cohen's Kappa: 58. Result ID: 69d3757147f34ce924bb70af. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Sonnet 4.6","Condition":"RAG\_FINETUNED\_UTT","Retrieval depth (k)":"3"}. This record does not assert independent reproduction. ### GPT-5.2 (RAG\_FINETUNED\_UTT) Cohen's Kappa: 53.9. Result ID: 69d3757147f34ce924bb70ab. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"GPT-5.2","Condition":"RAG\_FINETUNED\_UTT","Retrieval depth (k)":"3"}. This record does not assert independent reproduction. ### Qwen3-32b (RAG\_FINETUNED\_UTT) Cohen's Kappa: 52.6. Result ID: 69d3757147f34ce924bb70b3. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen3-32b","Condition":"RAG\_FINETUNED\_UTT","Retrieval depth (k)":"3"}. This record does not assert independent reproduction. ### Sonnet 4.6 (RAG\_FINETUNED\_CHUNK) Cohen's Kappa: 50.1. Result ID: 69d3757147f34ce924bb70ae. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Sonnet 4.6","Condition":"RAG\_FINETUNED\_CHUNK","Retrieval depth (k)":"3"}. This record does not assert independent reproduction. ### Sonnet 4.6 (RAG\_NO\_FINETUNE) Cohen's Kappa: 49.8. Result ID: 69d3757147f34ce924bb70ad. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Sonnet 4.6","Condition":"RAG\_NO\_FINETUNE","Retrieval depth (k)":"3"}. This record does not assert independent reproduction. ### GPT-5.2 (RAG\_NO\_FINETUNE) Cohen's Kappa: 47.9. Result ID: 69d3757147f34ce924bb70a9. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"GPT-5.2","Condition":"RAG\_NO\_FINETUNE","Retrieval depth (k)":"3"}. This record does not assert independent reproduction. ### GPT-5.2 (RAG\_FINETUNED\_CHUNK) Cohen's Kappa: 45.4. Result ID: 69d3757147f34ce924bb70aa. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"GPT-5.2","Condition":"RAG\_FINETUNED\_CHUNK","Retrieval depth (k)":"3"}. This record does not assert independent reproduction. ### Qwen3-32b (RAG\_NO\_FINETUNE) Cohen's Kappa: 44.2. Result ID: 69d3757147f34ce924bb70b1. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen3-32b","Condition":"RAG\_NO\_FINETUNE","Retrieval depth (k)":"3"}. This record does not assert independent reproduction. ### Qwen3-32b (RAG\_FINETUNED\_CHUNK) Cohen's Kappa: 42.8. Result ID: 69d3757147f34ce924bb70b2. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen3-32b","Condition":"RAG\_FINETUNED\_CHUNK","Retrieval depth (k)":"3"}. This record does not assert independent reproduction. ### Sonnet 4.6 (NO\_RAG) Cohen's Kappa: 41.3. Result ID: 69d3757147f34ce924bb70ac. [Reporting paper]() Recorded evaluation configuration: {"Model":"Sonnet 4.6","Condition":"NO\_RAG","Retrieval depth (k)":"3"}. This record does not assert independent reproduction. ### GPT-5.2 (NO\_RAG) Cohen's Kappa: 31.5. Result ID: 69d3757147f34ce924bb70a8. [Reporting paper]() Recorded evaluation configuration: {"Model":"GPT-5.2","Condition":"NO\_RAG","Retrieval depth (k)":"3"}. This record does not assert independent reproduction. ### Qwen3-32b (NO\_RAG) Cohen's Kappa: 27.5. Result ID: 69d3757147f34ce924bb70b0. [Reporting paper]() Recorded evaluation configuration: {"Model":"Qwen3-32b","Condition":"NO\_RAG","Retrieval depth (k)":"3"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.