# Social factor prediction on SocialDial Synthetic to Human

Source: [Social factor prediction on SocialDial Synthetic to Human](<https://www.sota2.com/research/sota/social-factor-prediction-on-socialdial-synthetic-to-human>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-02-26T05:48:40.464Z

Evaluation of model performance on predicting various social factors (Location, Social Dynamics, Social Relations) using the SocialDial Synthetic to Human dataset, measured by Macro F1 scores for each factor and overall average F1.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Social factor prediction |
| Dataset (including any stated split) | SocialDial Synthetic to Human |
| Metric | Location Macro F1 |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 10 of 10 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Location Macro F1](<https://www.sota2.com/research/sota/social-factor-prediction-on-socialdial-synthetic-to-human.md?metric=Location%20Macro%20F1>): Higher is better; unit: PERCENT.

- [Social Dynamics Macro F1](<https://www.sota2.com/research/sota/social-factor-prediction-on-socialdial-synthetic-to-human.md?metric=Social%20Dynamics%20Macro%20F1>): Higher is better; unit: PERCENT.

- [Social Relations Macro F1](<https://www.sota2.com/research/sota/social-factor-prediction-on-socialdial-synthetic-to-human.md?metric=Social%20Relations%20Macro%20F1>): Higher is better; unit: PERCENT.

- [Average Macro F1](<https://www.sota2.com/research/sota/social-factor-prediction-on-socialdial-synthetic-to-human.md?metric=Average%20Macro%20F1>): Higher is better; unit: PERCENT.

## Recorded results

| Method | Location Macro F1 | Result date | Report paper |
| --- | --- | --- | --- |
| IMO-CY | 23.22 | Not recorded | 68fa72f91e209131cea59b4c |
| IMO-CY last | 21.47 | Not recorded | 68fa72f91e209131cea59b4c |
| CHATGPT | 21.44 | Not recorded | 68fa72f91e209131cea59b4c |
| CHATGLM-6B | 20.57 | Not recorded | 68fa72f91e209131cea59b4c |
| CHATYUAN | 18.23 | Not recorded | 68fa72f91e209131cea59b4c |
| BERT-zh | 18.11 | Not recorded | 68fa72f91e209131cea59b4c |
| BERT-UDA | 15.2 | Not recorded | 68fa72f91e209131cea59b4c |
| IMO-CY B2T | 14.31 | Not recorded | 68fa72f91e209131cea59b4c |
| BERT-EDA | 13.98 | Not recorded | 68fa72f91e209131cea59b4c |
| IMO-CY w/o sq | 13.37 | Not recorded | 68fa72f91e209131cea59b4c |

### IMO-CY

Location Macro F1: 23.22. Result ID: 6992a6413a6c31484910655e.

[Reporting paper](<https://www.sota2.com/research/paper/68fa72f91e209131cea59b4c.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa72f91e209131cea59b4c.md>)

[Implementation](<https://github.com/WilliamsToTo/IMO>)

Recorded evaluation configuration: {"Pre-trained Model":"CHATYUAN"}. This record does not assert independent reproduction.

### IMO-CY last

Location Macro F1: 21.47. Result ID: 6992a6413a6c314849106561.

[Reporting paper](<https://www.sota2.com/research/paper/68fa72f91e209131cea59b4c.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa72f91e209131cea59b4c.md>)

[Implementation](<https://github.com/WilliamsToTo/IMO>)

Recorded evaluation configuration: {"Pre-trained Model":"CHATYUAN","Masking Position":"last layer"}. This record does not assert independent reproduction.

### CHATGPT

Location Macro F1: 21.44. Result ID: 6992a6413a6c31484910655c.

[Reporting paper](<https://www.sota2.com/research/paper/68fa72f91e209131cea59b4c.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa727a1e209131cea56a1e.md>)

[Implementation](<https://github.com/openai/evals>)

Recorded evaluation configuration: {"Model version":"gpt-3.5-turbo","Evaluation Protocol":"Few-shot In-context Learning"}. This record does not assert independent reproduction.

### CHATGLM-6B

Location Macro F1: 20.57. Result ID: 6992a6413a6c31484910655d.

[Reporting paper](<https://www.sota2.com/research/paper/68fa72f91e209131cea59b4c.md>)

Recorded evaluation configuration: {"Evaluation Protocol":"Few-shot In-context Learning"}. This record does not assert independent reproduction.

### CHATYUAN

Location Macro F1: 18.23. Result ID: 6992a6413a6c314849106559.

[Reporting paper](<https://www.sota2.com/research/paper/68fa72f91e209131cea59b4c.md>)

Recorded evaluation configuration: {"Evaluation Protocol":"Fine-tuning"}. This record does not assert independent reproduction.

### BERT-zh

Location Macro F1: 18.11. Result ID: 6992a6413a6c314849106558.

[Reporting paper](<https://www.sota2.com/research/paper/68fa72f91e209131cea59b4c.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa65c31e209131cea0c4ed.md>)

[Implementation](<https://github.com/google-research/bert>)

Recorded evaluation configuration: {"Evaluation Protocol":"Fine-tuning"}. This record does not assert independent reproduction.

### BERT-UDA

Location Macro F1: 15.2. Result ID: 6992a6413a6c31484910655b.

[Reporting paper](<https://www.sota2.com/research/paper/68fa72f91e209131cea59b4c.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa68a61e209131cea1d69a.md>)

[Implementation](<https://github.com/google-research/uda>)

Recorded evaluation configuration: {"Data Augmentation":"UDA","Evaluation Protocol":"Fine-tuning"}. This record does not assert independent reproduction.

### IMO-CY B2T

Location Macro F1: 14.31. Result ID: 6992a6413a6c31484910655f.

[Reporting paper](<https://www.sota2.com/research/paper/68fa72f91e209131cea59b4c.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa72f91e209131cea59b4c.md>)

[Implementation](<https://github.com/WilliamsToTo/IMO>)

Recorded evaluation configuration: {"Pre-trained Model":"CHATYUAN","Search Strategy":"bottom-up layer-wise search"}. This record does not assert independent reproduction.

### BERT-EDA

Location Macro F1: 13.98. Result ID: 6992a6413a6c31484910655a.

[Reporting paper](<https://www.sota2.com/research/paper/68fa72f91e209131cea59b4c.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa66421e209131cea0f41a.md>)

Recorded evaluation configuration: {"Data Augmentation":"EDA","Evaluation Protocol":"Fine-tuning"}. This record does not assert independent reproduction.

### IMO-CY w/o sq

Location Macro F1: 13.37. Result ID: 6992a6413a6c314849106560.

[Reporting paper](<https://www.sota2.com/research/paper/68fa72f91e209131cea59b4c.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa72f91e209131cea59b4c.md>)

[Implementation](<https://github.com/WilliamsToTo/IMO>)

Recorded evaluation configuration: {"Pre-trained Model":"CHATYUAN","Search Strategy":"simultaneous search"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.
