Latency Speedup on SQuAD 2.0 (Question Answering)
12.94Throughput (TPS)CreditDecoding
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| CreditDecodingBackbone=LLaDA-8B-Instruct, Gen Length=256, Block Size=642025.10 | 12.94 | — | 706 | 22.4 | |
| Fast-dLLMBackbone=LLaDA-8B-Instruct, Gen Length=256, Block Size=642025.10 | 10.57 | — | — | — | |
| BaselineBackbone=LLaDA-8B-Instruct, Gen Length=256, Block Size=642025.10 | 1.61 | — | — | — | |
| Fast Post-Training Pruning FrameworkBatch size=32, Backbone=BERT_BASE, Hardware=NVIDIA V100 GPU, Framework=PyTorch, Accuracy degradation constraint=at most 1%2022.03 | — | 1.37 | — | — | |
| Fast Post-Training Pruning FrameworkBatch size=256, Backbone=BERT_BASE, Hardware=NVIDIA V100 GPU, Framework=PyTorch, Accuracy degradation constraint=at most 1%2022.03 | — | 1.4 | — | — |