Dialogue Annotation on Eedi (test)
0.743Cohen's KappaSonnet 4.6 (RAG_FINETUNED_UTT)
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Sonnet 4.6 (RAG_FINETUNED_UTT)Model=Sonnet 4.6, Condition=RAG_FINETUNED_UTT, Retrieval depth (k)=32026.04 | 0.743 | 84.7 | |
| Qwen3-32b (RAG_FINETUNED_UTT)Model=Qwen3-32b, Condition=RAG_FINETUNED_UTT, Retrieval depth (k)=32026.04 | 0.694 | 81.8 | |
| GPT-5.2 (RAG_FINETUNED_UTT)Model=GPT-5.2, Condition=RAG_FINETUNED_UTT, Retrieval depth (k)=32026.04 | 0.659 | 81 | |
| Sonnet 4.6 (RAG_NO_FINETUNE)Model=Sonnet 4.6, Condition=RAG_NO_FINETUNE, Retrieval depth (k)=32026.04 | 0.632 | 79.7 | |
| Sonnet 4.6 (RAG_FINETUNED_CHUNK)Model=Sonnet 4.6, Condition=RAG_FINETUNED_CHUNK, Retrieval depth (k)=32026.04 | 0.632 | 78 | |
| Qwen3-32b (RAG_NO_FINETUNE)Model=Qwen3-32b, Condition=RAG_NO_FINETUNE, Retrieval depth (k)=32026.04 | 0.59 | 76.9 | |
| Qwen3-32b (RAG_FINETUNED_CHUNK)Model=Qwen3-32b, Condition=RAG_FINETUNED_CHUNK, Retrieval depth (k)=32026.04 | 0.545 | 72.5 | |
| GPT-5.2 (RAG_FINETUNED_CHUNK)Model=GPT-5.2, Condition=RAG_FINETUNED_CHUNK, Retrieval depth (k)=32026.04 | 0.53 | 74 | |
| GPT-5.2 (RAG_NO_FINETUNE)Model=GPT-5.2, Condition=RAG_NO_FINETUNE, Retrieval depth (k)=32026.04 | 0.505 | 70.9 | |
| Sonnet 4.6 (NO_RAG)Model=Sonnet 4.6, Condition=NO_RAG, Retrieval depth (k)=32026.04 | 0.41 | 71.4 | |
| GPT-5.2 (NO_RAG)Model=GPT-5.2, Condition=NO_RAG, Retrieval depth (k)=32026.04 | 0.351 | 57.5 | |
| Qwen3-32b (NO_RAG)Model=Qwen3-32b, Condition=NO_RAG, Retrieval depth (k)=32026.04 | 0.16 | 35.4 |