Math Reasoning on SVAMP (Multilingual Performance Breakdown)
78.33Kazakh (Cyrl) AccuracyDIP
Evaluation results
| Method | Links | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| DIPBackbone=GPT-4o, EP=true, CT=true2024.11 | 78.33 | 57 | 89.67 | 76.67 | 65.33 | 65.67 | 77.67 | 71 | 78.67 | 75 | 73.5 | |
| DIP w/ EP w/o CTBackbone=GPT-4o, EP=true, CT=false2024.11 | 66.67 | 55.67 | 75.67 | 73.67 | 57 | 55.67 | 70 | 62 | 67.33 | 66 | 64.97 | |
| DIP w/o EP w/o CTBackbone=GPT-4o, EP=false, CT=false2024.11 | 66 | 43 | 73 | 65.67 | 49 | 45.33 | 62.67 | 48.67 | 62.33 | 61.67 | 57.73 | |
| English Pivot ThoughtBackbone=GPT-4o2024.11 | 61.33 | 61.33 | 58.67 | 71 | 60.67 | 41.67 | 71 | 36.67 | 69.33 | 55.33 | 58.7 | |
| English PivotingBackbone=GPT-4o2024.11 | 54.67 | 51.33 | 51.67 | 63 | 53.67 | 40.33 | 63.67 | 35 | 57.33 | 51.67 | 52.23 | |
| Standard PromptingBackbone=GPT-4o2024.11 | 53.33 | 42 | 46.33 | 48.33 | 36.67 | 28.33 | 53 | 25.33 | 37.67 | 40.67 | 41.17 | |
| Non-insertion PromptingBackbone=GPT-4o2024.11 | 52.67 | 34.33 | 45 | 46 | 38.33 | 29.67 | 53 | 28.67 | 45.67 | 41.33 | 41.47 |