Factoid Question Answering on LAMA, EntityQA, and WebQA (test)
90.5LAMA AccuracyREMIX
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| REMIXModel=Mistral-7B-v0.3, Stage 2 Training Dataset=PopQA, Mixing Strategy=Random / K-Pile2024.11 | 90.5 | 92.3 | 89 | 90.6 | |
| REMIXModel=Mistral-7B-v0.3, Stage 2 Training Dataset=TriviaQA, Mixing Strategy=Random / K-Pile2024.11 | 77 | 81.5 | 83.1 | 80.5 | |
| No MixingModel=Mistral-7B-v0.3, Stage 2 Training Dataset=TriviaQA, Mixing Strategy=None2024.11 | 71.6 | 86.4 | 91.5 | 83.2 | |
| No MixingModel=Mistral-7B-v0.3, Stage 2 Training Dataset=PopQA, Mixing Strategy=None2024.11 | 66.9 | 92.3 | 89.6 | 82.9 | |
| REMIXModel=Mistral-7B-v0.3, Stage 2 Training Dataset=KVR, Mixing Strategy=Random / K-Pile2024.11 | 47.5 | 44.1 | 39 | 43.5 | |
| No MixingModel=Mistral-7B-v0.3, Stage 2 Training Dataset=KVR, Mixing Strategy=None2024.11 | 0.1 | 15.4 | 29.6 | 15 |