Throughput Measurement on LLaMA-2 13B
19.4Throughput (tokens/s)Token Filtering
Evaluation results
| Method | Links | |
|---|---|---|
| Token FilteringPruning Ratio=50%, Batch size=82025.12 | 19.4 | |
| FLAPPruning Ratio=50%, Batch size=82025.12 | 19.4 | |
| ProbePruning Ratio=50%, Batch size=82025.12 | 19.2 | |
| FLAPPruning Ratio=50%, Batch size=162025.12 | 19.1 | |
| Token FilteringPruning Ratio=50%, Batch size=162025.12 | 17.8 | |
| ProbePruning Ratio=50%, Batch size=162025.12 | 17.2 | |
| DensePruning Ratio=0%, Batch size=82025.12 | 16.8 | |
| DensePruning Ratio=0%, Batch size=162025.12 | 16.1 | |
| FLAPPruning Ratio=50%, Batch size=322025.12 | 15.4 | |
| Token FilteringPruning Ratio=50%, Batch size=322025.12 | 14.9 | |
| ProbePruning Ratio=50%, Batch size=322025.12 | 14.6 | |
| DensePruning Ratio=0%, Batch size=322025.12 | 11.8 | |
| FLAPPruning Ratio=50%, Batch size=642025.12 | 9.6 | |
| Token FilteringPruning Ratio=50%, Batch size=642025.12 | 8.8 | |
| ProbePruning Ratio=50%, Batch size=642025.12 | 7.5 | |
| FLAPPruning Ratio=50%, Batch size=1282025.12 | 5.8 | |
| Token FilteringPruning Ratio=50%, Batch size=1282025.12 | 5.2 | |
| DensePruning Ratio=0%, Batch size=642025.12 | 4.7 | |
| ProbePruning Ratio=50%, Batch size=1282025.12 | 4.4 | |
| DensePruning Ratio=0%, Batch size=1282025.12 | 2.7 |