Date Understanding on FLORES-200 10-languages
74.8Performance (kaz_Cyrl)English Pivot Thought
Evaluation results
| Method | Links | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| English Pivot ThoughtModel=GPT-4o, EP=true, CT=true2024.11 | 74.8 | 58 | 63.6 | 64 | 73.2 | 52.4 | 69.6 | 40.4 | 56 | 63.6 | 61.56 | |
| DIPModel=GPT-4o, EP=true, CT=true2024.11 | 72.4 | 66.8 | 73.2 | 71.6 | 76 | 66.4 | 77.6 | 70 | 75.6 | 75.6 | 72.52 | |
| English PivotingModel=GPT-4o, EP=true, CT=false2024.11 | 49.6 | 42.8 | 45.2 | 44.8 | 49.6 | 48 | 47.2 | 43.2 | 42.8 | 43.2 | 45.64 | |
| DIP w/ EP w/o CTModel=GPT-4o, EP=true, CT=false2024.11 | 49.6 | 44.8 | 46.8 | 48 | 50.4 | 48.8 | 45.6 | 45.2 | 44 | 46.8 | 47 | |
| DIPBackbone=Mixtral2024.11 | 48.8 | 41.6 | 51.2 | 52 | 51.2 | 48.4 | 45.6 | 49.6 | 45.2 | 57.2 | 49.08 | |
| DIP w/o EP w/o CTBackbone=Mixtral, English pivoting (EP)=false, chain-of-thought (CT)=false2024.11 | 48 | 41.6 | 43.6 | 47.6 | 48.4 | 43.2 | 44 | 51.6 | 39.2 | 41.6 | 40.64 | |
| Non-insertion PromptingModel=GPT-4o, EP=false, CT=false2024.11 | 48 | 39.2 | 44.4 | 43.6 | 43.6 | 42.4 | 43.2 | 46 | 41.6 | 43.6 | 43.56 | |
| DIP w/o EP w/o CTModel=GPT-4o, EP=false, CT=false2024.11 | 48 | 38 | 44.4 | 45.6 | 47.6 | 40.4 | 41.6 | 38.4 | 39.2 | 44.8 | 42.8 | |
| Standard PromptingModel=GPT-4o, EP=false, CT=false2024.11 | 47.2 | 47.2 | 53.6 | 54.4 | 61.6 | 47.2 | 55.6 | 45.2 | 46.4 | 46 | 50.44 | |
| DIP w/ EP w/o CTBackbone=Mixtral, English pivoting (EP)=true, chain-of-thought (CT)=false2024.11 | 39.6 | 34.4 | 36.8 | 37.6 | 42.4 | 41.6 | 34 | 40 | 32.4 | 36 | 35.78 | |
| Standard PromptingBackbone=Mixtral2024.11 | 22.4 | 31.6 | 24.8 | 29.6 | 34.8 | 32.4 | 18.8 | 28.8 | 25.6 | 21.6 | 30.92 | |
| English Pivot ThoughtBackbone=Mixtral, English pivoting (EP)=true, chain-of-thought (CT)=true2024.11 | 21.6 | 24 | 29.6 | 25.2 | 25.6 | 26 | 18.8 | 29.6 | 24 | 21.6 | 25.01 | |
| Non-insertion PromptingBackbone=Mixtral2024.11 | 14.4 | 17.2 | 18 | 19.2 | 20.8 | 26 | 13.2 | 20.8 | 11.2 | 14.4 | 18.52 | |
| English PivotingBackbone=Mixtral, English pivoting (EP)=true2024.11 | 14.4 | 18.8 | 15.6 | 19.6 | 16.8 | 18 | 9.2 | 23.6 | 12 | 13.6 | 14.68 |