Human Response Prediction on Synthetic Persona Dataset
0.681Pearson RGemini 2.0 Flash
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Gemini 2.0 FlashCase=Case_6c, Description=AI (Given Demography + Traits)2026.01 | 0.681 | 21.32 | -0.025 | -19.097 | |
| GPT-4oCase=Case_4, Description=Full Information2026.01 | 0.667 | 39.67 | -0.008 | -6.401 | |
| Claude-3.5-SonnetCase=Case_4, Description=Full Information2026.01 | 0.663 | 39.98 | 0.024 | 17.909 | |
| Claude-3.5-SonnetCase=Case_7c, Description=Traits+Identity2026.01 | 0.662 | 40.29 | -0.053 | -40.485 | |
| GPT-4oCase=Case_7c, Description=Traits+Identity2026.01 | 0.658 | 39.68 | -0.074 | -56.354 | |
| Claude-3.5-SonnetCase=Case_2, Description=Demographics + Narratives2026.01 | 0.656 | 38.56 | 0.012 | 8.791 | |
| GPT-4oCase=Case_3, Description=Demographics + Traits2026.01 | 0.656 | 38.6 | 0.1 | 75.499 | |
| Claude-3.5-SonnetCase=Case_3, Description=Demographics + Traits2026.01 | 0.65 | 39.6 | 0.055 | 41.287 | |
| GPT-4oCase=Case_5c, Description=AI Summary (1000–1500w)2026.01 | 0.649 | 38.07 | -0.051 | -38.42 | |
| Gemini 2.0 FlashCase=Case_6b, Description=AI (Given Demography + Narratives)2026.01 | 0.649 | 20.01 | -0.041 | -30.71 | |
| Claude-3.5-SonnetCase=Case_7a, Description=Identity Only2026.01 | 0.649 | 38.26 | -0.072 | -54.323 | |
| Claude-3.5-SonnetCase=Case_5c, Description=AI Summary (1000–1500w)2026.01 | 0.648 | 38.33 | -0.055 | -41.981 | |
| GPT-4oCase=Case_5d, Description=AI Summary (No Limit)2026.01 | 0.646 | 38.26 | -0.041 | -31.116 | |
| GPT-4oCase=Case_5a, Description=AI Summary (300–500w)2026.01 | 0.645 | 37.9 | -0.04 | -30.373 | |
| GPT-4oCase=Case_5b, Description=AI Summary (500–1000w)2026.01 | 0.645 | 38.12 | -0.039 | -29.8 | |
| GPT-4oCase=Case_2, Description=Demographics + Narratives2026.01 | 0.642 | 37.74 | 0.011 | 8.298 | |
| Claude-3.5-SonnetCase=Case_5a, Description=AI Summary (300–500w)2026.01 | 0.642 | 37.95 | -0.036 | -27.367 | |
| Gemini 2.0 FlashCase=Case_6a, Description=AI (Given Demography only)2026.01 | 0.642 | 19.78 | -0.008 | -5.976 | |
| Claude-3.5-SonnetCase=Case_5b, Description=AI Summary (500–1000w)2026.01 | 0.639 | 37.74 | -0.058 | -43.601 | |
| Claude-3.5-SonnetCase=Case_7b, Description=Traits Only2026.01 | 0.639 | 38.92 | -0.088 | -66.353 | |
| Claude-3.5-SonnetCase=Case_5d, Description=AI Summary (No Limit)2026.01 | 0.638 | 37.92 | -0.056 | -42.387 | |
| Gemini-2.5-Pro-ThinkingCase=Case_7c, Description=Traits+Identity2026.01 | 0.636 | 33.89 | -0.101 | -76.371 | |
| GPT-4oCase=AVERAGE, Description=—2026.01 | 0.636 | 35.53 | — | — | |
| Gemini-2.5-Pro-ThinkingCase=Case_4, Description=Full Information2026.01 | 0.633 | 25.63 | -0.016 | -11.74 | |
| GPT-4oCase=Case_7a, Description=Identity Only2026.01 | 0.633 | 37.44 | -0.086 | -65.16 | |
| Claude-3.5-SonnetCase=Case_1, Description=Demographics Only2026.01 | 0.627 | 36 | 0.153 | 115.666 | |
| DeepSeek R1Case=Case_4, Scenario=Full Information2026.01 | 0.625 | 37.2 | -0.039 | -36.3 | |
| GPT-4oCase=Case_1, Description=Demographics Only2026.01 | 0.624 | 35.07 | 0.134 | 101.23 | |
| GPT-4oCase=Case_7b, Description=Traits Only2026.01 | 0.624 | 37.95 | -0.094 | -71.092 | |
| GPT 5.1Case=Case_4, Scenario=Full Information2026.01 | 0.621 | 32.4 | 0 | 0.3 | |
| Gemini-2.5-Pro-ThinkingCase=Case_5c, Description=AI Summary (1000–1500w)2026.01 | 0.62 | 32.13 | -0.12 | -90.75 | |
| GPT-4oCase=Case_6b, Description=AI (Given Demography + Narratives)2026.01 | 0.619 | 25.55 | -0.034 | -25.81 | |
| Gemini-2.5-Pro-ThinkingCase=Case_5b, Description=AI Summary (500–1000w)2026.01 | 0.618 | 31.74 | -0.114 | -86.013 | |
| Gemini-2.5-Pro-ThinkingCase=Case_2, Description=Demographics + Narratives2026.01 | 0.616 | 27.58 | -0.016 | -12.245 | |
| GPT 5.1Case=Case_2, Scenario=Demographics + Narrative2026.01 | 0.615 | 31.8 | 0.035 | 32.2 | |
| Gemini-2.5-Pro-ThinkingCase=Case_3, Description=Demographics + Traits2026.01 | 0.614 | 22.56 | -0.042 | -31.64 | |
| Gemini 2.0 FlashCase=Case_4, Description=Full Information2026.01 | 0.614 | 35.51 | -0.064 | -48.671 | |
| Gemini-2.5-Pro-ThinkingCase=Case_5d, Description=AI Summary (No Limit)2026.01 | 0.614 | 32.13 | -0.111 | -84.019 | |
| Claude-3.5-SonnetCase=Case_6f, Description=AI (Given Identity + Traits)2026.01 | 0.614 | 43.15 | -0.038 | -33.91 | |
| Gemini 2.0 FlashCase=AVERAGE, Description=—2026.01 | 0.613 | 31.68 | — | — | |
| Gemini 2.0 FlashCase=Case_7c, Description=Traits+Identity2026.01 | 0.612 | 37.86 | -0.099 | -74.563 | |
| GPT 5.1Case=Case_7c, Scenario=Traits+Identity2026.01 | 0.612 | 33.07 | -0.074 | -5,574.9 | |
| Gemini 2.0 FlashCase=Case_2, Description=Demographics + Narratives2026.01 | 0.609 | 33.58 | -0.101 | -76.79 | |
| DeepSeek R1Case=Case_2, Scenario=Demographics + Narrative2026.01 | 0.609 | 35.4 | -0.022 | -20.7 | |
| Gemini-2.5-Pro-ThinkingCase=Case_5a, Description=AI Summary (300–500w)2026.01 | 0.608 | 30.34 | -0.103 | -78.201 | |
| GPT-4oCase=Case_6f, Description=AI (Given Identity + Traits)2026.01 | 0.608 | 43.42 | -0.046 | -40.4 | |
| DeepSeek R1Case=Case_7c, Scenario=Traits+Identity2026.01 | 0.608 | 36.7 | -0.09 | -6,813.8 | |
| GPT-4oCase=Case_6c, Description=AI (Given Demography + Traits)2026.01 | 0.606 | 27.86 | 0.016 | 12.249 | |
| Claude-3.5-SonnetCase=AVERAGE, Description=—2026.01 | 0.605 | 36.5 | — | — | |
| Gemini-2.5-Pro-ThinkingCase=Case_7a, Description=Identity Only2026.01 | 0.604 | 30.22 | -0.123 | -93.316 | |
| Gemini 2.0 FlashCase=Case_5a, Description=AI Summary (300–500w)2026.01 | 0.602 | 34.44 | -0.077 | -58.009 | |
| DeepSeek R1Case=Case_3, Scenario=Demographics + Traits2026.01 | 0.602 | 35.1 | -0.011 | -10.7 | |
| GPT 5.1Case=Case_5c, Scenario=AI Summary (1000–1500w)2026.01 | 0.602 | 32.5 | -0.048 | -45.1 | |
| Gemini 2.0 FlashCase=Case_7a, Description=Identity Only2026.01 | 0.601 | 35.75 | -0.101 | -76.547 | |
| Gemini 2.0 FlashCase=Case_3, Description=Demographics + Traits2026.01 | 0.6 | 36.71 | 0.012 | 9.087 | |
| GPT 5.1Case=Case_5b, Scenario=AI Summary (500–1000w)2026.01 | 0.599 | 32.7 | -0.015 | -14.4 | |
| GPT 5.1Case=Case_5d, Scenario=AI Summary (No Limit)2026.01 | 0.597 | 31.7 | -0.048 | -44.2 | |
| Gemini 2.0 FlashCase=Case_1, Description=Demographics Only2026.01 | 0.596 | 35.32 | 0.071 | 54.001 | |
| GPT 5.1Case=Case_5a, Scenario=AI Summary (300–500w)2026.01 | 0.596 | 33.3 | -0.017 | -16 | |
| Gemini 2.0 FlashCase=Case_7b, Description=Traits Only2026.01 | 0.595 | 36.28 | -0.121 | -91.436 | |
| Gemini-2.5-Pro-ThinkingCase=Case_7b, Description=Traits Only2026.01 | 0.595 | 34.27 | -0.115 | -87.329 | |
| Gemini 2.0 FlashCase=Case_5d, Description=AI Summary (No Limit)2026.01 | 0.594 | 32.17 | -0.096 | -72.49 | |
| DeepSeek R1Case=Case_5b, Scenario=AI Summary (500–1000w)2026.01 | 0.594 | 36 | -0.06 | -56.3 | |
| GPT 5.1Case=Case_1, Scenario=Demographics Only2026.01 | 0.593 | 29.3 | 0.116 | 108.2 | |
| Claude-3.5-SonnetCase=Case_6e, Description=AI (Given Traits alone)2026.01 | 0.592 | 41.45 | -0.071 | -63.02 | |
| Gemini 2.0 FlashCase=Case_5b, Description=AI Summary (500–1000w)2026.01 | 0.591 | 32.98 | -0.103 | -78.321 | |
| Gemini 2.0 FlashCase=Case_5c, Description=AI Summary (1000–1500w)2026.01 | 0.591 | 31.81 | -0.117 | -88.529 | |
| GPT 5.1Case=Case_3, Scenario=Demographics + Traits2026.01 | 0.591 | 29.1 | 0.05 | 46.9 | |
| DeepSeek R1Case=Case_5c, Scenario=AI Summary (1000–1500w)2026.01 | 0.591 | 36 | -0.053 | -49 | |
| DeepSeek R1Case=Case_5d, Scenario=AI Summary (No Limit)2026.01 | 0.589 | 34.7 | -0.071 | -65.8 | |
| GPT 5.1Case=Case_7a, Scenario=Narrative Only2026.01 | 0.589 | 31.2 | -0.086 | -6,510.3 | |
| Gemini-2.5-Pro-ThinkingCase=Case_1, Description=Demographics Only2026.01 | 0.588 | 29.78 | 0.074 | 55.646 | |
| Claude-3.5-SonnetCase=Case_6d, Description=AI (Given Identity alone)2026.01 | 0.588 | 41.36 | -0.054 | -48.15 | |
| GPT-4oCase=Case_6e, Description=AI (Given Traits alone)2026.01 | 0.588 | 41.65 | -0.08 | -71.43 | |
| Gemini-2.5-Pro-ThinkingCase=AVERAGE, Description=—2026.01 | 0.588 | 29.12 | — | — | |
| Gemini-2.5-Pro-ThinkingCase=Case_6f, Description=AI (Given Identity + Traits)2026.01 | 0.586 | 42.43 | -0.082 | -72.79 | |
| Qwen3 480bCase=Case_4, Scenario=Full Information2026.01 | 0.586 | 21.7 | -0.02 | -18.8 | |
| GPT-4oCase=Case_6a, Description=AI (Given Demography only)2026.01 | 0.584 | 25.53 | 0.079 | 59.915 | |
| DeepSeek R1Case=Case_5a, Scenario=AI Summary (300–500w)2026.01 | 0.584 | 34.9 | -0.062 | -57.5 | |
| DeepSeek R1Case=Case_7a, Scenario=Narrative Only2026.01 | 0.584 | 34.62 | -0.105 | -7,957 | |
| DeepSeek R1Case=Case_1, Scenario=Demographics Only2026.01 | 0.583 | 31.4 | 0.005 | 4.6 | |
| GPT 5.1Case=Case_7b, Scenario=Traits Only2026.01 | 0.581 | 31.63 | -0.094 | -7,114.7 | |
| Gemini-2.5-Pro-ThinkingCase=Case_6d, Description=AI (Given Identity alone)2026.01 | 0.578 | 40.61 | -0.094 | -83.38 | |
| Gemini-2.5-Pro-ThinkingCase=Case_6e, Description=AI (Given Traits alone)2026.01 | 0.577 | 40.37 | -0.099 | -87.9 | |
| GPT 5.1Case=Case_6b, Scenario=AI (Given Demography + Identity)2026.01 | 0.576 | 21.29 | -0.025 | -1,891.5 | |
| DeepSeek R1Case=Case_7b, Scenario=Traits Only2026.01 | 0.576 | 35.1 | -0.115 | -8,695.8 | |
| GPT-4oCase=Case_6d, Description=AI (Given Identity alone)2026.01 | 0.572 | 39.72 | -0.062 | -54.69 | |
| Qwen3 480bCase=Case_2, Scenario=Demographics + Narrative2026.01 | 0.572 | 20.9 | -0.02 | -19 | |
| DeepSeek R1Case=Case_6b, Scenario=AI (Given Demography + Identity)2026.01 | 0.572 | 23.63 | -0.03 | -2,311.8 | |
| GPT 5.1Case=Case_6f, Scenario=AI (Given Identity + Traits)2026.01 | 0.566 | 36.19 | -0.055 | -4,889.7 | |
| Qwen3 480bCase=Case_7c, Scenario=Traits+Identity2026.01 | 0.566 | 22.49 | -0.086 | -6,504 | |
| GPT 5.1Case=Case_6c, Scenario=AI (Given Demography + Traits)2026.01 | 0.564 | 23.22 | -0.015 | -1,130 | |
| Gemini 2.0 FlashCase=Case_6f, Description=AI (Given Identity + Traits)2026.01 | 0.561 | 40.76 | -0.079 | -70.22 | |
| DeepSeek R1Case=Case_6f, Scenario=AI (Given Identity + Traits)2026.01 | 0.561 | 40.15 | -0.067 | -5,976.3 | |
| DeepSeek R1Case=Case_6c, Scenario=AI (Given Demography + Traits)2026.01 | 0.56 | 25.76 | -0.018 | -1,381.1 | |
| Qwen3 480bCase=Case_5c, Scenario=AI Summary (1000–1500w)2026.01 | 0.556 | 22.6 | -0.024 | -22.1 | |
| Qwen3 480bCase=Case_3, Scenario=Demographics + Traits2026.01 | 0.553 | 19.1 | -0.033 | -30.4 | |
| Qwen3 480bCase=Case_5d, Scenario=AI Summary (No Limit)2026.01 | 0.552 | 22 | -0.057 | -53.3 | |
| GPT 5.1Case=Case_6e, Scenario=AI (Given Traits alone)2026.01 | 0.547 | 34.71 | -0.079 | -7,028.8 | |
| Qwen3 480bCase=Case_5a, Scenario=AI Summary (300–500w)2026.01 | 0.546 | 22.2 | -0.046 | -42.7 |