Multiple-choice Question Answering on MMLU T5
50.8Accuracy (MMLU T5)Full Retraining
Evaluation results
| Method | Links | |
|---|---|---|
| Full RetrainingBackbone=LLaMA-3.1-Instruct-8B, Number of random seeds=32026.04 | 50.8 | |
| COMPASS-ECDABackbone=LLaMA-3.1-Instruct-8B, Number of random seeds=32026.04 | 50.6 | |
| Random RehearsalBackbone=LLaMA-3.1-Instruct-8B, Number of random seeds=32026.04 | 48.5 | |
| EWCBackbone=LLaMA-3.1-Instruct-8B, Number of random seeds=32026.04 | 48.4 | |
| Naive Fine-tuningBackbone=LLaMA-3.1-Instruct-8B, Number of random seeds=32026.04 | 39.1 |