Multi-round conversation on MT-Bench
274.32Tokens Per SecondPRISM
Evaluation results
| Method | Links | ||
|---|---|---|---|
| PRISMModel=LLaMA-2, Temp=02026.02 | 274.32 | 4.66 | |
| HASSModel=LLaMA-2, Temp=02026.02 | 265.08 | 4.48 | |
| PRISMModel=LLaMA-2, Temp=T = 02026.02 | 254.18 | 4.69 | |
| PRISMModel=LLaMA-2, Temp=T = 12026.02 | 253.61 | 4.65 | |
| PRISMModel=LLaMA-2, Temp=12026.02 | 246.64 | 4.3 | |
| EAGLE-2Model=LLaMA-2, Temp=02026.02 | 246.18 | 4.16 | |
| HASSModel=LLaMA-2, Temp=T = 12026.02 | 241.22 | 4.43 | |
| HASSModel=LLaMA-2, Temp=T = 02026.02 | 238.65 | 4.42 | |
| HASSModel=LLaMA-2, Temp=12026.02 | 238.13 | 4.12 | |
| EAGLE-2Model=LLaMA-2, Temp=12026.02 | 223.69 | 3.87 | |
| EAGLE-2Model=LLaMA-2, Temp=T = 02026.02 | 223.32 | 4.12 | |
| EAGLE-2Model=LLaMA-2, Temp=T = 12026.02 | 223.28 | 4.12 | |
| PRISMModel=LLaMA-3, Temp=02026.02 | 201.51 | 4.29 | |
| PRISMModel=LLaMA-3, Temp=12026.02 | 201.47 | 4.29 | |
| PRISMModel=LLaMA-3, Temp=T = 12026.02 | 198.65 | 4.3 | |
| PRISMModel=LLaMA-3, Temp=T = 02026.02 | 198.21 | 4.27 | |
| HASSModel=LLaMA-3, Temp=T = 02026.02 | 184.12 | 4.01 | |
| HASSModel=LLaMA-3, Temp=T = 12026.02 | 183.37 | 4.01 | |
| HASSModel=LLaMA-3, Temp=12026.02 | 181.14 | 3.93 | |
| HASSModel=LLaMA-3, Temp=02026.02 | 180.5 | 3.93 | |
| EAGLE-2Model=LLaMA-3, Temp=12026.02 | 168.64 | 3.67 | |
| EAGLE-2Model=LLaMA-3, Temp=02026.02 | 168.26 | 3.67 | |
| EAGLE-2Model=LLaMA-3, Temp=T = 12026.02 | 168.1 | 3.77 | |
| EAGLE-2Model=LLaMA-3, Temp=T = 02026.02 | 166.75 | 3.76 | |
| Standard Speculative DecodingModel=LLaMA-3, Drafter Model=LLaMA-3.1-1B, Temp=02026.02 | 164.13 | 5.06 | |
| StandardModel=LLaMA-3, Temp=T = 12026.02 | 150.61 | 5.08 | |
| StandardModel=LLaMA-3, Temp=T = 02026.02 | 149.47 | 5.08 | |
| Standard Speculative DecodingModel=LLaMA-3, Drafter Model=LLaMA-3.1-1B, Temp=12026.02 | 142.69 | 4.42 | |
| Standard Speculative DecodingModel=LLaMA-2, Drafter Model=LLaMA-160M, Temp=12026.02 | 142.37 | 2.68 | |
| Standard Speculative DecodingModel=LLaMA-2, Drafter Model=LLaMA-160M, Temp=02026.02 | 142.35 | 2.68 | |
| StandardModel=LLaMA-2, Temp=T = 12026.02 | 136.56 | 2.78 | |
| StandardModel=LLaMA-2, Temp=T = 02026.02 | 134.85 | 2.77 | |
| VanillaModel=LLaMA-2, Temp=T = 12026.02 | 102.27 | — | |
| VanillaModel=LLaMA-2, Temp=T = 02026.02 | 101.3 | — | |
| VanillaModel=LLaMA-2, Temp=1, Description=Standard auto-regressive decoding2026.02 | 95.7 | — | |
| VanillaModel=LLaMA-2, Temp=0, Description=Standard auto-regressive decoding2026.02 | 95.66 | — | |
| VanillaModel=LLaMA-3, Temp=T = 12026.02 | 91.72 | — | |
| VanillaModel=LLaMA-3, Temp=T = 02026.02 | 91.6 | — | |
| VanillaModel=LLaMA-3, Temp=0, Description=Standard auto-regressive decoding2026.02 | 85.63 | — | |
| VanillaModel=LLaMA-3, Temp=1, Description=Standard auto-regressive decoding2026.02 | 85.39 | — |