Large Language Model Inference Efficiency on Aggregate of 16 datasets (test)
126.2Throughput (tokens/s)Hybrid Verified
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Hybrid VerifiedTarget Model=Llama3.1-8B2026.05 | 126.2 | 200.1 | 4 | 18.6 | 81.4 | |
| Frequency HybridTarget Model=Llama3.1-8B2026.05 | 98.3 | 234.9 | 3.26 | 9.4 | 90.6 | |
| SuffixTarget Model=Llama3.1-8B2026.05 | 78.2 | 264 | 2.79 | 93.9 | 0 | |
| EAGLE3Target Model=Llama3.1-8B2026.05 | 47.7 | 571 | 0.75 | 0 | 100 | |
| GreedyTarget Model=Llama3.1-8B2026.05 | 40.7 | 1,000 | — | — | — |