Dialogue Annotation on TalkMoves (test)
58Cohen's KappaSonnet 4.6 (RAG_FINETUNED_UTT)
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Sonnet 4.6 (RAG_FINETUNED_UTT)Model=Sonnet 4.6, Condition=RAG_FINETUNED_UTT, Retrieval depth (k)=32026.04 | 58 | 70.8 | |
| GPT-5.2 (RAG_FINETUNED_UTT)Model=GPT-5.2, Condition=RAG_FINETUNED_UTT, Retrieval depth (k)=32026.04 | 53.9 | 68.8 | |
| Qwen3-32b (RAG_FINETUNED_UTT)Model=Qwen3-32b, Condition=RAG_FINETUNED_UTT, Retrieval depth (k)=32026.04 | 52.6 | 66.7 | |
| Sonnet 4.6 (RAG_FINETUNED_CHUNK)Model=Sonnet 4.6, Condition=RAG_FINETUNED_CHUNK, Retrieval depth (k)=32026.04 | 50.1 | 64.7 | |
| Sonnet 4.6 (RAG_NO_FINETUNE)Model=Sonnet 4.6, Condition=RAG_NO_FINETUNE, Retrieval depth (k)=32026.04 | 49.8 | 65.3 | |
| GPT-5.2 (RAG_NO_FINETUNE)Model=GPT-5.2, Condition=RAG_NO_FINETUNE, Retrieval depth (k)=32026.04 | 47.9 | 63.9 | |
| GPT-5.2 (RAG_FINETUNED_CHUNK)Model=GPT-5.2, Condition=RAG_FINETUNED_CHUNK, Retrieval depth (k)=32026.04 | 45.4 | 62 | |
| Qwen3-32b (RAG_NO_FINETUNE)Model=Qwen3-32b, Condition=RAG_NO_FINETUNE, Retrieval depth (k)=32026.04 | 44.2 | 61.7 | |
| Qwen3-32b (RAG_FINETUNED_CHUNK)Model=Qwen3-32b, Condition=RAG_FINETUNED_CHUNK, Retrieval depth (k)=32026.04 | 42.8 | 58.5 | |
| Sonnet 4.6 (NO_RAG)Model=Sonnet 4.6, Condition=NO_RAG, Retrieval depth (k)=32026.04 | 41.3 | 57.8 | |
| GPT-5.2 (NO_RAG)Model=GPT-5.2, Condition=NO_RAG, Retrieval depth (k)=32026.04 | 31.5 | 50 | |
| Qwen3-32b (NO_RAG)Model=Qwen3-32b, Condition=NO_RAG, Retrieval depth (k)=32026.04 | 27.5 | 40.4 |