Multilingual Ability Evaluation on MLQA, XLSUM, XQuAD, GMMLU
60.9MLQA F1LLAMA-3-8B
Evaluation Results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| LLAMA-3-8BPhase 1 (P1) Dataset=Instruct, Protocol=Two-phase Continual Fine-tuning, Evaluation Phase=P12024.10 | 60.9 | 4.8 | 71.2 | 25 | 40.5 | |
| LLAMA-3-8BPhase 1 (P1) Dataset=ALPACA, Phase 2 (P2) Dataset=ALPACA, Protocol=Two-phase Continual Fine-tuning, Evaluation Phase=P22024.10 | 59.7 | 3.4 | 73.7 | 34 | 42.7 | |
| LLAMA-3-8BFine-tuning Protocol=Dataset Mixture, Dataset Mixture=ALPACA MULTIALPACA2024.10 | 48 | 4 | 13.9 | 50 | 28.9 | |
| LLAMA-3-8BPhase 1 (P1) Dataset=ALPACA, Phase 2 (P2) Dataset=ALPACA, Protocol=Two-phase Continual Fine-tuning, Evaluation Phase=P12024.10 | 43.8 | 3.3 | 58.6 | 53 | 39.7 | |
| MISTRAL-7BFine-tuning Protocol=Dataset Mixture, Dataset Mixture=ALPACA MULTIALPACA2024.10 | 40.6 | 7.9 | 21.7 | 41 | 27.8 | |
| LLAMA-3-8BPhase 1 (P1) Dataset=Instruct, Protocol=Two-phase Continual Fine-tuning, Evaluation Phase=P22024.10 | 32.1 | 2.7 | 41.7 | 44 | 30.1 | |
| MISTRAL-7BPhase 1 (P1) Dataset=Instruct, Phase 2 (P2) Dataset=MULTI, Protocol=Two-phase Continual Fine-tuning, Evaluation Phase=P22024.10 | 30.7 | 3.3 | 43.6 | 43 | 30.2 | |
| MISTRAL-7BPhase 1 (P1) Dataset=ALPACA, Protocol=Two-phase Continual Fine-tuning, Evaluation Phase=P22024.10 | 28.8 | 6 | 60.2 | 40 | 33.8 | |
| MISTRAL-7BPhase 1 (P1) Dataset=Instruct, Phase 2 (P2) Dataset=MULTI, Protocol=Two-phase Continual Fine-tuning, Evaluation Phase=P12024.10 | 24.6 | 1.2 | 35.1 | 44 | 26.2 | |
| MISTRAL-7BPhase 1 (P1) Dataset=ALPACA, Protocol=Two-phase Continual Fine-tuning, Evaluation Phase=P12024.10 | 22.9 | 1.2 | 29 | 42 | 23.8 |