Mathematical Reasoning on GSM8K FLORES-200 (10 low-resourced languages) (test)
67.93Kazakh (Cyrl) AccuracyDIP
Evaluation results
| Method | Links | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| DIPModel=GPT-4o, English Pivoting (EP)=true, Chain-of-thought (CT)=true2024.11 | 67.93 | 46.17 | 80.36 | 67.1 | 53.3 | 43.29 | 68.61 | 60.5 | 63.68 | 68.31 | 61.92 | |
| English Pivot ThoughtModel=GPT-4o, English Pivoting (EP)=true, Chain-of-thought (CT)=true2024.11 | 61.11 | 36.77 | 60.35 | 57.16 | 49.13 | 22.67 | 60.96 | 20.85 | 44.28 | 35.48 | 44.88 | |
| English PivotingModel=GPT-4o, English Pivoting (EP)=true, Chain-of-thought (CT)=false2024.11 | 23.65 | 14.03 | 19.94 | 19.79 | 19.26 | 10.31 | 21.91 | 9.55 | 17.82 | 17.44 | 17.37 | |
| DIP w/ EP w/o CTModel=GPT-4o, English Pivoting (EP)=true, Chain-of-thought (CT)=false2024.11 | 23.43 | 16.3 | 24.94 | 21.23 | 18.5 | 14.86 | 23.58 | 19.18 | 22.14 | 22.37 | 20.65 | |
| DIP w/o EP w/o CTModel=GPT-4o, English Pivoting (EP)=false, Chain-of-thought (CT)=false2024.11 | 20.39 | 12.05 | 22.06 | 15.47 | 13.57 | 12.43 | 19.03 | 13.95 | 16.38 | 18.2 | 16.35 | |
| Standard PromptingModel=GPT-4o, English Pivoting (EP)=false, Chain-of-thought (CT)=false2024.11 | 17.89 | 9.1 | 15.62 | 11.52 | 11.3 | 5.91 | 14.25 | 5.84 | 9.55 | 9.4 | 11.04 | |
| Non-insertion PromptingModel=GPT-4o2024.11 | 14.94 | 6.29 | 13.87 | 10.08 | 10.24 | 6.37 | 12.89 | 7.05 | 10.24 | 8.19 | 10.02 |