List your company
ResearchFormerly WizWand

LLM Inference Acceleration on MT-bench

5.5818Mean Accepted Length (tau)CATS w/ EAGLE
1.6296962.6557233.681754.707777May 11, 2026
Updated 6mo ago

Evaluation results

MethodLinks
CATS w/ EAGLEModel=Llama2-7b, Decoding Strategy=Relaxed (temperature = 0.7), Decoding Structure=tree base2026.055.58185.58
CATS w/ EAGLEModel=Llama2-13b, Decoding Strategy=Relaxed (temperature = 0.7), Decoding Structure=tree base2026.055.34995.28
EAGLE-self, treeModel=Llama2-7b, Decoding Strategy=Relaxed (temperature = 0.7), Decoding Structure=tree base2026.055.24094.52
CATSModel=Llama2-7b, Decoding Strategy=Relaxed (temperature = 0.7), Decoding Structure=chain base2026.055.20045.2
EAGLE-self, treeModel=Llama2-13b, Decoding Strategy=Relaxed (temperature = 0.7), Decoding Structure=tree base2026.055.10515.05
CATS w/ EAGLEModel=Llama2-7b, Decoding Strategy=Greedy, Decoding Structure=tree base2026.055.06345.06
EAGLE-selfModel=Llama2-7b, Decoding Strategy=Relaxed (temperature = 0.7), Decoding Structure=chain base2026.054.97564.29
KangarooModel=Llama2-7b, Decoding Strategy=Relaxed (temperature = 0.7), Decoding Structure=chain base2026.054.96094.28
CATSModel=Llama2-13b, Decoding Strategy=Relaxed (temperature = 0.7), Decoding Structure=chain base2026.054.91334.87
CATS w/ EAGLEModel=Llama2-13b, Decoding Strategy=Greedy, Decoding Structure=tree base2026.054.81914.75
EAGLE-self, treeModel=Llama2-7b, Decoding Strategy=Greedy, Decoding Structure=tree base2026.054.71994.07
EAGLE-selfModel=Llama2-13b, Decoding Strategy=Relaxed (temperature = 0.7), Decoding Structure=chain base2026.054.70324.68
KangarooModel=Llama2-13b, Decoding Strategy=Relaxed (temperature = 0.7), Decoding Structure=chain base2026.054.69914.68
CATSModel=Llama2-7b, Decoding Strategy=Greedy, Decoding Structure=chain base2026.054.61814.62
EAGLE-self, treeModel=Llama2-13b, Decoding Strategy=Greedy, Decoding Structure=tree base2026.054.46994.42
CATSModel=Llama2-13b, Decoding Strategy=Greedy, Decoding Structure=chain base2026.054.27524.24
CATS w/ EAGLEModel=Vicuna-13b, Decoding Strategy=Relaxed (temperature = 0.7), Decoding Structure=tree base2026.054.2364.18
KangarooModel=Llama2-7b, Decoding Strategy=Greedy, Decoding Structure=chain base2026.054.1933.61
EAGLE-selfModel=Llama2-7b, Decoding Strategy=Greedy, Decoding Structure=chain base2026.054.18133.6
EAGLE-selfModel=Llama2-13b, Decoding Strategy=Greedy, Decoding Structure=chain base2026.053.97543.96
KangarooModel=Llama2-13b, Decoding Strategy=Greedy, Decoding Structure=chain base2026.053.93693.92
CATS w/ EAGLEModel=Vicuna-13b, Decoding Strategy=Greedy, Decoding Structure=tree base2026.053.68973.64
EAGLE-self, treeModel=Vicuna-13b, Decoding Strategy=Relaxed (temperature = 0.7), Decoding Structure=tree base2026.053.52323.49
CATSModel=Vicuna-13b, Decoding Strategy=Relaxed (temperature = 0.7), Decoding Structure=chain base2026.053.24213.22
EAGLE-self, treeModel=Vicuna-13b, Decoding Strategy=Greedy, Decoding Structure=tree base2026.053.04233.01
EAGLE-selfModel=Vicuna-13b, Decoding Strategy=Relaxed (temperature = 0.7), Decoding Structure=chain base2026.052.84442.83
CATSModel=Vicuna-13b, Decoding Strategy=Greedy, Decoding Structure=chain base2026.052.79462.77
MedusaModel=Vicuna-13b, Decoding Strategy=Relaxed (temperature = 0.7), Decoding Structure=chain base2026.052.79052.37
KangarooModel=Vicuna-13b, Decoding Strategy=Relaxed (temperature = 0.7), Decoding Structure=chain base2026.052.78942.78
MedusaModel=Vicuna-13b, Decoding Strategy=Greedy, Decoding Structure=chain base2026.052.58332.19
KangarooModel=Vicuna-13b, Decoding Strategy=Greedy, Decoding Structure=chain base2026.052.41842.41
EAGLE-selfModel=Vicuna-13b, Decoding Strategy=Greedy, Decoding Structure=chain base2026.052.38312.37
MedusaModel=Llama2-13b, Decoding Strategy=Relaxed (temperature = 0.7), Decoding Structure=chain base2026.052.06461.75
MedusaModel=Llama2-13b, Decoding Strategy=Greedy, Decoding Structure=chain base2026.051.97371.68
MedusaModel=Llama2-7b, Decoding Strategy=Relaxed (temperature = 0.7), Decoding Structure=chain base2026.051.86671.87
MedusaModel=Llama2-7b, Decoding Strategy=Greedy, Decoding Structure=chain base2026.051.78171.78