Dialectal Bias Evaluation on Bengali Dialectal Question Sets
2.94Barishal Dialect Performancemistral_7b
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| mistral_7bSize=7B2026.03 | 2.94 | 1.39 | 2.29 | 2.15 | 1.94 | 1.77 | 2.78 | 1.84 | 3.28 | 2.26 | |
| llama3.2_3bSize=3B2026.03 | 3.83 | 1.92 | 3.69 | 4.74 | 3.78 | 2.13 | 4.08 | 2.79 | 4.79 | 3.53 | |
| deepseek-r1_8bSize=8B2026.03 | 5.16 | 3.45 | 4.48 | 5.03 | 4.52 | 4.02 | 5.98 | 4.72 | 5.91 | 4.81 | |
| llama3.1_8bSize=8B2026.03 | 5.6 | 3.25 | 4.52 | 5.84 | 4.76 | 4.1 | 5.14 | 4.17 | 5.76 | 4.79 | |
| deepseek-r1_32bSize=32B2026.03 | 5.83 | 1.2 | 4.39 | 7.02 | 5.99 | 3.14 | 4.4 | 3.01 | 5.43 | 4.49 | |
| phi4_14bSize=14B2026.03 | 6.72 | 5.46 | 6.54 | 7.53 | 6.22 | 5.96 | 7.94 | 6.64 | 7.87 | 6.77 | |
| ministral-3_8bSize=8B2026.03 | 7 | 6.83 | 7.97 | 8.45 | 7.6 | 7.73 | 8.4 | 7.18 | 8.23 | 7.71 | |
| qwen3_8bSize=8B2026.03 | 7.01 | 6.19 | 8.24 | 8.16 | 7.23 | 7.26 | 9.02 | 7.56 | 8.5 | 7.69 | |
| qwen3_4bSize=4B2026.03 | 7.17 | 4.7 | 7.72 | 8.24 | 6.76 | 7.09 | 8.58 | 7.14 | 8.3 | 7.3 | |
| gemma-3n-e2b-itSize=2B, Instruction-Tuned=true2026.03 | 7.32 | 6.13 | 7.94 | 7.63 | 7.34 | 7.63 | 8.1 | 7.41 | 8.23 | 7.52 | |
| gemma-3-12b-itSize=12B, Instruction-Tuned=true2026.03 | 7.36 | 7.22 | 9.16 | 8.49 | 7.81 | 8.41 | 8.41 | 7.97 | 8.33 | 8.13 | |
| ministral-3_14bSize=14B2026.03 | 7.43 | 7.61 | 8.7 | 8.8 | 8.12 | 8.15 | 9.22 | 8.54 | 9.09 | 8.41 | |
| gemma-3n-e4b-itSize=4B, Instruction-Tuned=true2026.03 | 7.56 | 5.67 | 8.82 | 7.2 | 7.77 | 7.94 | 8.53 | 8.33 | 8.14 | 7.77 | |
| gpt-oss-120bSize=120B2026.03 | 8.02 | 5.12 | 8.75 | 9.22 | 8.24 | 8.65 | 8.85 | 8.39 | 8.59 | 8.2 | |
| gemma-3-27b-itSize=27B, Instruction-Tuned=true2026.03 | 8.08 | 7.8 | 9.3 | 9.16 | 8.38 | 9.03 | 8.55 | 8.85 | 9.22 | 8.71 | |
| gpt-oss_20bSize=20B2026.03 | 8.13 | 8.32 | 9.14 | 9.19 | 8.11 | 8.6 | 9.14 | 8.72 | 8.99 | 8.7 | |
| qwen-3-235bSize=235B2026.03 | 8.2 | 5.6 | 9.17 | 8.9 | 8.2 | 8.4 | 8.92 | 7.92 | 8.89 | 8.25 | |
| llama-3.3-70bSize=70B2026.03 | 8.3 | 7.79 | 8.68 | 9.06 | 8.36 | 8 | 9.24 | 8.5 | 9 | 8.55 | |
| qwen3_32bSize=32B2026.03 | 8.51 | 7.74 | 9.01 | 9.03 | 8.24 | 8.47 | 9.42 | 8.21 | 9.35 | 8.67 |