Japanese Commonsense Reasoning on JSQuAD, JCQA, XWinograd, JAQKET
81.9JSQuADSwap&Insert
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Swap&InsertVocabulary Size=100k, Training Mode=continual training (insertion strategy)2024.06 | 81.9 | 80.2 | 69.2 | 61.2 | 73.1 | |
| Fujii et al. (2024)Vocabulary Size=100k, Training Mode=continual training (embedding initialization via average)2024.06 | 81.6 | 77.6 | 69.1 | 61.1 | 72.4 | |
| Llama2Vocabulary Size=32k, Training Mode=continual training (original vocab)2024.06 | 80.7 | 79.4 | 72.6 | 47.7 | 70.1 | |
| SwapVocabulary Size=100k, Training Mode=continual training (new parameters, no insertion)2024.06 | 79.2 | 80.2 | 67.5 | 56.3 | 70.8 | |
| From scratchVocabulary Size=100k, Training Mode=Full training from scratch2024.06 | 71.8 | 76 | 63.6 | 54.2 | 66.4 | |
| Llama2Vocabulary Size=32k, Training Mode=without training2024.06 | 71.2 | 60.8 | 62.4 | 15.3 | 52.4 |