Binary Classification on Therapist Q&A (test)
0.598Kappa (κF1)MentalRoBERTa (MLP Labels)
Evaluation Results
| Method | Links | |
|---|---|---|
| MentalRoBERTa (MLP Labels)Backbone Model=MentalRoBERTa, Labeling Strategy=MLP Labels, Annotator Model=GPT-4, Confidence Threshold=0.72025.11 | 0.598 | |
| MentalRoBERTa (MLP Labels)Backbone Model=MentalRoBERTa, Labeling Strategy=MLP Labels, Annotator Model=GPT-4, Confidence Threshold=0.52025.11 | 0.574 | |
| MentalRoBERTa (RLP Labels)Backbone Model=MentalRoBERTa, Labeling Strategy=RLP Labels, Annotator Model=GPT-4, Confidence Threshold=0.72025.11 | 0.572 | |
| MentalRoBERTa (RLP Labels)Backbone Model=MentalRoBERTa, Labeling Strategy=RLP Labels, Annotator Model=GPT-4, Confidence Threshold=0.52025.11 | 0.571 | |
| MentalRoBERTa (RLP Labels)Backbone Model=MentalRoBERTa, Labeling Strategy=RLP Labels, Annotator Model=GPT-4o, Confidence Threshold=0.52025.11 | 0.556 | |
| MentalRoBERTa (RLP Labels)Backbone Model=MentalRoBERTa, Labeling Strategy=RLP Labels, Annotator Model=GPT-4o, Confidence Threshold=0.72025.11 | 0.551 | |
| MentalRoBERTa (MLP Labels)Backbone Model=MentalRoBERTa, Labeling Strategy=MLP Labels, Annotator Model=GPT-4o, Confidence Threshold=0.72025.11 | 0.531 | |
| MentalRoBERTa (Golden Labels)Backbone Model=MentalRoBERTa, Labeling Strategy=Golden Labels, Annotator Model=GPT-4, Confidence Threshold=0.52025.11 | 0.515 | |
| MentalRoBERTa (Golden Labels)Backbone Model=MentalRoBERTa, Labeling Strategy=Golden Labels, Annotator Model=GPT-4o, Confidence Threshold=0.52025.11 | 0.503 | |
| MentalRoBERTa (Golden Labels)Backbone Model=MentalRoBERTa, Labeling Strategy=Golden Labels, Annotator Model=GPT-4, Confidence Threshold=0.72025.11 | 0.481 | |
| MentalRoBERTa (Golden Labels)Backbone Model=MentalRoBERTa, Labeling Strategy=Golden Labels, Annotator Model=GPT-4o, Confidence Threshold=0.72025.11 | 0.475 | |
| MentalRoBERTa (MLP Labels)Backbone Model=MentalRoBERTa, Labeling Strategy=MLP Labels, Annotator Model=GPT-4o, Confidence Threshold=0.52025.11 | 0.469 |