Multilabel Classification on Therapist Q&A (test)
0.348Kappa (κF1)MentalRoBERTa (RLP Labels)
Evaluation Results
| Method | Links | |
|---|---|---|
| MentalRoBERTa (RLP Labels)Backbone Model=MentalRoBERTa, Labeling Strategy=RLP Labels, Annotator Model=GPT-4, Confidence Threshold=0.52025.11 | 0.348 | |
| MentalRoBERTa (RLP Labels)Backbone Model=MentalRoBERTa, Labeling Strategy=RLP Labels, Annotator Model=GPT-4, Confidence Threshold=0.72025.11 | 0.331 | |
| MentalRoBERTa (MLP Labels)Backbone Model=MentalRoBERTa, Labeling Strategy=MLP Labels, Annotator Model=GPT-4, Confidence Threshold=0.72025.11 | 0.324 | |
| MentalRoBERTa (MLP Labels)Backbone Model=MentalRoBERTa, Labeling Strategy=MLP Labels, Annotator Model=GPT-4o, Confidence Threshold=0.52025.11 | 0.317 | |
| MentalRoBERTa (MLP Labels)Backbone Model=MentalRoBERTa, Labeling Strategy=MLP Labels, Annotator Model=GPT-4o, Confidence Threshold=0.72025.11 | 0.31 | |
| MentalRoBERTa (MLP Labels)Backbone Model=MentalRoBERTa, Labeling Strategy=MLP Labels, Annotator Model=GPT-4, Confidence Threshold=0.52025.11 | 0.283 | |
| MentalRoBERTa (RLP Labels)Backbone Model=MentalRoBERTa, Labeling Strategy=RLP Labels, Annotator Model=GPT-4o, Confidence Threshold=0.72025.11 | 0.279 | |
| MentalRoBERTa (RLP Labels)Backbone Model=MentalRoBERTa, Labeling Strategy=RLP Labels, Annotator Model=GPT-4o, Confidence Threshold=0.52025.11 | 0.229 | |
| MentalRoBERTa (Golden Labels)Backbone Model=MentalRoBERTa, Labeling Strategy=Golden Labels, Annotator Model=GPT-4o, Confidence Threshold=0.72025.11 | 0.169 | |
| MentalRoBERTa (Golden Labels)Backbone Model=MentalRoBERTa, Labeling Strategy=Golden Labels, Annotator Model=GPT-4, Confidence Threshold=0.72025.11 | 0.163 | |
| MentalRoBERTa (Golden Labels)Backbone Model=MentalRoBERTa, Labeling Strategy=Golden Labels, Annotator Model=GPT-4, Confidence Threshold=0.52025.11 | 0.138 | |
| MentalRoBERTa (Golden Labels)Backbone Model=MentalRoBERTa, Labeling Strategy=Golden Labels, Annotator Model=GPT-4o, Confidence Threshold=0.52025.11 | 0.11 |