Downstream Task on BoolQ
61.93AccuracyLLaMA-MoE 2/8
Evaluation Results
| Method | Links | |
|---|---|---|
| LLaMA-MoE 2/8FFN Size=25% × 2, Shots=02025.06 | 61.93 | |
| FLAPFFN Size=50%, Shots=02025.06 | 61.86 | |
| LLaMA-MoE 1/8FFN Size=50% × 1, Shots=02025.06 | 61.71 | |
| DIVE 2/8FFN Size=25% × 2, Shots=02025.06 | 59.88 | |
| LLM-PrunerFFN Size=50%, Shots=02025.06 | 59.02 | |
| DIVE 1/8FFN Size=50% × 1, Shots=02025.06 | 58.81 | |
| TinyLlama-1.1BFFN Size=100%, Shots=02025.06 | 55.99 |