Cultural Alignment on CulturalBench Hard
63.54AccuracyCAReDiO
Evaluation results
| Method | Links | ||
|---|---|---|---|
| CAReDiOFamily=GPT-4.12025.04 | 63.54 | 11 | |
| Role-PlayFamily=GPT-4.12025.04 | 63.47 | 11 | |
| CultureBankFamily=GPT-4.12025.04 | 60 | 9 | |
| GPT-5 + Role-PlayFamily=Proprietaty LLMs2025.04 | 59.99 | 11 | |
| GPT-5Family=Proprietaty LLMs2025.04 | 59.54 | — | |
| Raw ModelFamily=GPT-4.12025.04 | 59.45 | — | |
| CAReDiOFamily=Gemma-3-27B-IT2025.04 | 48.88 | 10 | |
| Role-PlayFamily=Gemma-3-27B-IT2025.04 | 48.28 | 10 | |
| CultureSPAFamily=Gemma-3-27B-IT2025.04 | 48 | 10 | |
| Raw ModelFamily=Gemma-3-27B-IT2025.04 | 46.59 | — | |
| CultureParkFamily=Gemma-3-27B-IT2025.04 | 46.34 | 8 | |
| CultureLLMFamily=Gemma-3-27B-IT2025.04 | 46.31 | 9 | |
| CultureDataFamily=Gemma-3-27B-IT2025.04 | 44.28 | 8 | |
| CultureBankFamily=Gemma-3-27B-IT2025.04 | 41.88 | 6 | |
| CAReDiOFamily=Qwen2.5-7B-Instruct2025.04 | 40.2 | 11 | |
| CultureDataFamily=Qwen2.5-7B-Instruct2025.04 | 40.11 | 10 | |
| Raw ModelFamily=Qwen2.5-7B-Instruct2025.04 | 38.9 | — | |
| Role-PlayFamily=Qwen2.5-7B-Instruct2025.04 | 36.73 | 8 | |
| CultureSPAFamily=Qwen2.5-7B-Instruct2025.04 | 36.25 | 6 | |
| CultureLLMFamily=Qwen2.5-7B-Instruct2025.04 | 34.79 | 6 | |
| CultureParkFamily=Qwen2.5-7B-Instruct2025.04 | 34.41 | 6 | |
| CultureInstructionFamily=Qwen2.5-7B-Instruct2025.04 | 27.75 | 2 | |
| CultureBankFamily=Qwen2.5-7B-Instruct2025.04 | 27.34 | 2 | |
| CultureInstructionFamily=Gemma-3-27B-IT2025.04 | 18.19 | 0 |