Math Word Problem Solving on SVAMP 10 low-resourced languages FLORES-200 (test)
44Kazakh (Cyrillic) AccuracyDIP
Evaluation results
| Method | Links | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| DIPBackbone=Mixtral, English Pivoting=true, Chain-of-thought=true2024.11 | 44 | 29 | 50.33 | 48 | 32 | 39.67 | 39 | 55.67 | 48.33 | 51.67 | 43.27 | |
| English PivotingBackbone=Mixtral2024.11 | 35 | 10.33 | 51.33 | 12 | 7.67 | 10.33 | 38 | 18 | 30.67 | 24 | 22.43 | |
| Standard PromptingBackbone=Mixtral2024.11 | 28 | 14 | 36 | 16.33 | 8.33 | 12.33 | 28.33 | 17 | 27.67 | 20.67 | 18.97 | |
| DIP w/ EP w/o CTBackbone=Mixtral, English Pivoting=true, Chain-of-thought=false2024.11 | 27.33 | 17.67 | 31 | 36 | 18.33 | 29.67 | 17 | 38.33 | 31.33 | 40.67 | 28 | |
| Non-insertion PromptingBackbone=Mixtral2024.11 | 25 | 23 | 30.67 | 27 | 23.33 | 22.67 | 21 | 29 | 25 | 24 | 22.67 | |
| English Pivot ThoughtBackbone=Mixtral2024.11 | 22.67 | 5.67 | 27.33 | 10.33 | 6.33 | 9 | 14 | 18 | 20.33 | 15.33 | 12.97 | |
| DIP w/o EP w/o CTBackbone=Mixtral, English Pivoting=false, Chain-of-thought=false2024.11 | 22.67 | 17 | 32.67 | 29 | 21.33 | 25.33 | 18 | 39.67 | 18 | 24.33 | 22.8 |