General Reasoning and Coding on PopQA Non-Factoid
51.4GSM8K AccuracyREMIX (Random / -)
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| REMIX (Random / -)Backbone=Llama-3-8B, Mixing Strategy (DM1 / DM2)=Random / -2024.11 | 51.4 | 89.3 | 82.7 | 81.8 | 66 | 72.2 | |
| REMIX (K-Pile / -)Backbone=Llama-3-8B, Mixing Strategy (DM1 / DM2)=K-Pile / -2024.11 | 46.4 | 92.7 | 94 | 87.2 | 90.9 | 82.2 | |
| REMIX (Random / K-Pile)Backbone=Llama-3-8B, Mixing Strategy (DM1 / DM2)=Random / K-Pile2024.11 | 38.5 | 88.7 | 88.3 | 79.2 | 74.4 | 73.8 | |
| REMIX (- / K-Pile)Backbone=Llama-3-8B, Mixing Strategy (DM1 / DM2)=- / K-Pile2024.11 | 25.9 | 94 | 92.4 | 73.9 | 74.7 | 72.2 | |
| No MixingBackbone=Llama-3-8B, Mixing Strategy (DM1 / DM2)=No Mixing2024.11 | 19 | 92.4 | 77 | 55.1 | 48.5 | 58.4 | |
| REMIX (- / Random)Backbone=Llama-3-8B, Mixing Strategy (DM1 / DM2)=- / Random2024.11 | 16.8 | 93.5 | 87.5 | 59.3 | 70.7 | 65.6 |