Inference Throughput on Qwen3-8B
26.74Throughput (tokens/s)ROCKET
Evaluation Results
| Method | Links | |
|---|---|---|
| ROCKETCompression Ratio=20%, Batch Size=1, Context Length=2562026.02 | 26.74 | |
| ROCKETCompression Ratio=30%, Batch Size=1, Context Length=2562026.02 | 26.6 | |
| ROCKETCompression Ratio=40%, Batch Size=1, Context Length=2562026.02 | 26.36 | |
| CoSpaDiCompression Ratio=30%, Batch Size=1, Context Length=2562026.02 | 25.76 | |
| CoSpaDiCompression Ratio=40%, Batch Size=1, Context Length=2562026.02 | 25.62 | |
| CoSpaDiCompression Ratio=20%, Batch Size=1, Context Length=2562026.02 | 25.45 | |
| SVD-LLMCompression Ratio=40%, Batch Size=1, Context Length=2562026.02 | 24.72 | |
| SVD-LLMCompression Ratio=20%, Batch Size=1, Context Length=2562026.02 | 24.36 | |
| SVD-LLMCompression Ratio=30%, Batch Size=1, Context Length=2562026.02 | 24.31 |