Multiclass Classification on Therapist Q&A (test)
0.446Kappa (κF1)MentalRoBERTa (RLP Labels)
Evaluation results
| Method | Links | |
|---|---|---|
| MentalRoBERTa (RLP Labels)Backbone Model=MentalRoBERTa, Labeling Strategy=RLP Labels, Annotator Model=GPT-4, Confidence Threshold=0.72025.11 | 0.446 | |
| MentalRoBERTa (RLP Labels)Backbone Model=MentalRoBERTa, Labeling Strategy=RLP Labels, Annotator Model=GPT-4, Confidence Threshold=0.52025.11 | 0.438 | |
| MentalRoBERTa (RLP Labels)Backbone Model=MentalRoBERTa, Labeling Strategy=RLP Labels, Annotator Model=GPT-4o, Confidence Threshold=0.52025.11 | 0.351 | |
| MentalRoBERTa (RLP Labels)Backbone Model=MentalRoBERTa, Labeling Strategy=RLP Labels, Annotator Model=GPT-4o, Confidence Threshold=0.72025.11 | 0.309 | |
| MentalRoBERTa (Golden Labels)Backbone Model=MentalRoBERTa, Labeling Strategy=Golden Labels, Annotator Model=GPT-4, Confidence Threshold=0.52025.11 | 0.257 | |
| MentalRoBERTa (Golden Labels)Backbone Model=MentalRoBERTa, Labeling Strategy=Golden Labels, Annotator Model=GPT-4o, Confidence Threshold=0.52025.11 | 0.254 | |
| MentalRoBERTa (Golden Labels)Backbone Model=MentalRoBERTa, Labeling Strategy=Golden Labels, Annotator Model=GPT-4o, Confidence Threshold=0.72025.11 | 0.25 | |
| MentalRoBERTa (Golden Labels)Backbone Model=MentalRoBERTa, Labeling Strategy=Golden Labels, Annotator Model=GPT-4, Confidence Threshold=0.72025.11 | 0.241 |