Social factor prediction on SocialDial Synthetic to Human
23.22Location Macro F1IMO-CY
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| IMO-CYPre-trained Model=CHATYUAN2024.04 | 23.22 | 46.04 | 42.71 | 37.32 | |
| IMO-CY lastPre-trained Model=CHATYUAN, Masking Position=last layer2024.04 | 21.47 | 44.73 | 39.89 | 35.36 | |
| CHATGPTModel version=gpt-3.5-turbo, Evaluation Protocol=Few-shot In-context Learning2024.04 | 21.44 | 38.46 | 35.12 | 31.67 | |
| CHATGLM-6BEvaluation Protocol=Few-shot In-context Learning2024.04 | 20.57 | 20.53 | 11.55 | 17.55 | |
| CHATYUANEvaluation Protocol=Fine-tuning2024.04 | 18.23 | 34.94 | 33.92 | 29.03 | |
| BERT-zhEvaluation Protocol=Fine-tuning2024.04 | 18.11 | 35.05 | 32.39 | 28.51 | |
| BERT-UDAData Augmentation=UDA, Evaluation Protocol=Fine-tuning2024.04 | 15.2 | 33.59 | 27.03 | 25.27 | |
| IMO-CY B2TPre-trained Model=CHATYUAN, Search Strategy=bottom-up layer-wise search2024.04 | 14.31 | 30.29 | 32.45 | 25.68 | |
| BERT-EDAData Augmentation=EDA, Evaluation Protocol=Fine-tuning2024.04 | 13.98 | 35.71 | 26.38 | 25.36 | |
| IMO-CY w/o sqPre-trained Model=CHATYUAN, Search Strategy=simultaneous search2024.04 | 13.37 | 29.81 | 29.05 | 24.07 |