Psychiatric symptom identification on PRIMATE (5-fold cross-val)
71.2Macro RecallSynSym w/o Real
Evaluation results
| Method | Links | ||
|---|---|---|---|
| SynSym w/o RealBackbone=MentalBERT, Training Data=Synthetic solely, Note=Individual sentence evaluation aggregated2026.03 | 71.2 | 55.7 | |
| SynSym with RealBackbone=MentalBERT, Training Data=Synthetic + Real (fine-tuned)2026.03 | 64.5 | 65 | |
| MentalBERTTraining Data=Real, Evaluation Protocol=Fine-tuned2026.03 | 62.9 | 64.3 | |
| BERTTraining Data=Real, Evaluation Protocol=Fine-tuned2026.03 | 62.5 | 64.6 | |
| DeBERTaTraining Data=Real, Evaluation Protocol=Fine-tuned2026.03 | 60.4 | 61.4 | |
| GPT4o-ZSLPrompting Strategy=Direct multi-label classification, Evaluation Protocol=Zero-shot2026.03 | 56.9 | 56.7 | |
| GPT4o-COTPrompting Strategy=Chain-of-Thought, Evaluation Protocol=Zero-shot2026.03 | 56.7 | 56.8 | |
| GPT4o-PSPrompting Strategy=Plan-and-solve, Evaluation Protocol=Zero-shot2026.03 | 52.9 | 54.3 |