Adaptive Token Selection on LLM Inference T=512
94Jaccard SimilarityCascade ADA
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| Cascade ADAModel=GPT-J, L=28, d=4096, r¯ind=205, tau=0.30, T=5122026.05 | 94 | 52 | 3 | |
| Cascade ADAModel=GPT-2, L=12, d=768, r¯ind=278, tau=0.30, T=5122026.05 | 86.3 | 22 | 13 | |
| Cascade ADAModel=OPT, L=32, d=4096, r¯ind=142, tau=0.30, T=5122026.05 | 82.5 | 63 | 2 |