Per-country Preference Alignment on 20-country human AMCE (test)
47.1MIS (ARG)Tc scale
Evaluation results
| Method | Links | ||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Tc scaleBackbone=Phi-4, Model Size=14B, Supervision level=Oracle population-level supervision2026.05 | 47.1 | 53.5 | 35.4 | 52.8 | 50.1 | 58 | 65.5 | 57.6 | 54.2 | 38.2 | 32.7 | 53.8 | 50.9 | 52.3 | 49.7 | 56.4 | 56.3 | 49.4 | 56.4 | 56.1 | 51.3 | 95 | |
| Margin mcBackbone=Phi-4, Model Size=14B, Supervision level=Oracle population-level supervision2026.05 | 45.3 | 53.5 | 29.8 | 52.8 | 46.9 | 58 | 65.5 | 57.6 | 54.2 | 35.3 | 32.7 | 53.8 | 50.6 | 52.3 | 49.7 | 56.4 | 56.3 | 49.4 | 56.4 | 56.1 | 50.6 | 90 | |
| MC-DropoutBackbone=Phi-4, Model Size=14B, Rate=0.10, T=8, Supervision level=Country-agnostic2026.05 | 41.2 | 37.9 | 30.6 | 36.7 | 38.9 | 37.6 | 52.6 | 40.7 | 45.2 | 53.8 | 37.1 | 36.1 | 44.5 | 38.2 | 36.3 | 39.8 | 41 | 34.2 | 36.9 | 47.3 | 40.3 | 90 | |
| DISCABackbone=Phi-4, Model Size=14B, Supervision level=Within-country persona disagreement2026.05 | 38.9 | 36.8 | 29.9 | 21.4 | 43.8 | 25.5 | 48.5 | 31.9 | 27.4 | 53 | 19.5 | 39.3 | 42 | 35.7 | 32.8 | 38.5 | 38.5 | 31.3 | 24.3 | 33.6 | 34.6 | — |