General Performance on Aggregated LLM Evaluation Suite
47.9Average ScoreBTX
Evaluation results
| Method | Links | |
|---|---|---|
| BTXActive Experts (Top-k)=22024.03 | 47.9 | |
| BTXActive Experts (Top-k)=1, Sampling Strategy=Sample2024.03 | 47.3 | |
| Sparse upcyclingTraining Context=Data-Matching (DM), Active Experts (Top-k)=22024.03 | 46.3 | |
| LLAMA-2Model Size=13B2024.03 | 45.4 | |
| DenseTraining Context=Data-Matching (DM)2024.03 | 44.5 | |
| BTMActive Experts (Top-k)=22024.03 | 43.4 | |
| BTMActive Experts (Top-k)=12024.03 | 43.1 | |
| LLAMA-2Model Size=7B2024.03 | 40.7 | |
| CODELLAMAModel Size=7B2024.03 | 37.9 | |
| LLEMMAModel Size=7B2024.03 | 32.1 |