Downstream Task on SciQ
89.3AccuracyTinyLlama-1.1B
Evaluation results
| Method | Links | |
|---|---|---|
| TinyLlama-1.1BFFN Size=100%, Shots=02025.06 | 89.3 | |
| DIVE 1/8FFN Size=50% × 1, Shots=02025.06 | 83 | |
| DIVE 2/8FFN Size=25% × 2, Shots=02025.06 | 81.1 | |
| FLAPFFN Size=50%, Shots=02025.06 | 80.5 | |
| LLM-PrunerFFN Size=50%, Shots=02025.06 | 79.2 | |
| LLaMA-MoE 2/8FFN Size=25% × 2, Shots=02025.06 | 77.1 | |
| LLaMA-MoE 1/8FFN Size=50% × 1, Shots=02025.06 | 76.3 |