Watermark Decoding on Llama2-7B Tokenizer 400k tokens (test)
1.44Decoding Latency (µs/token)HeRo¹ (3)
Evaluation Results
| Method | Links | |
|---|---|---|
| HeRo¹ (3)Batch Size=642026.07 | 1.44 | |
| HeRo² (2,2)Batch Size=642026.07 | 1.48 | |
| HeRo¹ (3)Batch Size=322026.07 | 1.57 | |
| HeRo² (2,2)Batch Size=322026.07 | 1.59 | |
| HeRo¹ (3)Batch Size=162026.07 | 1.66 | |
| HeRo² (2,2)Batch Size=162026.07 | 1.85 | |
| HeRo¹ (3)Batch Size=42026.07 | 2.43 | |
| HeRo² (2,2)Batch Size=42026.07 | 3.08 | |
| HeRo¹ (3)Batch Size=12026.07 | 7.75 | |
| HeRo² (2,2)Batch Size=12026.07 | 10.34 | |
| BiMark (20)Batch Size=42026.07 | 130.22 | |
| BiMark (20)Batch Size=12026.07 | 130.43 | |
| BiMark (20)Batch Size=162026.07 | 130.68 | |
| BiMark (20)Batch Size=322026.07 | 131.63 | |
| BiMark (20)Batch Size=642026.07 | 133.23 | |
| MPAC (2, 2.0)Batch Size=42026.07 | 479.3 | |
| MPAC (2, 2.0)Batch Size=642026.07 | 480.43 | |
| MPAC (2, 2.0)Batch Size=12026.07 | 488.13 | |
| MPAC (2, 2.0)Batch Size=322026.07 | 489.15 | |
| MPAC (2, 2.0)Batch Size=162026.07 | 491.43 |