Average evaluation across 7 tasks on Thai Culture (test)
74.1BERTScoreTyphoon
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Typhoon2024.11 | 74.1 | 2.32 | 17.21 | 12.7 | 20.7 | 11.8 | 20 | 20 | — | |
| Seed-Free Synthetic Data Generation FrameworkFluency (F)=+, Culture (C)=+, Diversity (D)=+2024.11 | 69.5 | 0.1 | 9.47 | 6.7 | 7.7 | 3.3 | 7.5 | 7.6 | — | |
| WangchanX2024.11 | 68.8 | 2.24 | 17.28 | 11.3 | 13.4 | 5.8 | 12.6 | 12.7 | — | |
| OpenThai2024.11 | 64.5 | 0.95 | 14.54 | 8.2 | 12.2 | 5.6 | 11.7 | 11.6 | — | |
| Seed-Free Synthetic Data Generation FrameworkFluency (F)=+, Culture (C)=-, Diversity (D)=-2024.11 | 57.3 | 0.01 | 5.18 | 3.7 | 3.6 | 1 | 3.6 | 3.5 | — | |
| Seed-Free Synthetic Data Generation FrameworkFluency (F)=-, Culture (C)=+, Diversity (D)=-2024.11 | 49.6 | 0 | 2.9 | 1.7 | 1.8 | 0.2 | 1.8 | 1.8 | — | |
| Seed-Free Synthetic Data Generation FrameworkFluency (F)=-, Culture (C)=-, Diversity (D)=+2024.11 | 48.2 | 0 | 2.74 | 1.7 | 1.9 | 0.3 | 1.9 | 1.9 | — | |
| Seed-Free Synthetic Data Generation FrameworkFluency (F)=-, Culture (C)=-, Diversity (D)=-2024.11 | 45.9 | 0.02 | 4.38 | 2.2 | 1.3 | 0.2 | 1.2 | 1.2 | — |