List your company
ResearchFormerly WizWand

Inference Efficiency on HumanEval

5.33Speedup FactorCATS w/ EAGLE
0.78521.96513.1454.3249Nov 16, 2024Feb 14, 2025May 15, 2025Aug 13, 2025Nov 11, 2025Feb 9, 2026May 11, 2026
Updated 5mo ago

Evaluation results

MethodLinks
CATS w/ EAGLEModel=Llama2-13b, Decoding Strategy=Relaxed (temperature = 0.7), Decoding Structure=tree base2026.055.33——5.3969
CATS w/ EAGLEModel=Llama2-7b, Decoding Strategy=Greedy, Decoding Structure=tree base2026.055.17——5.1741
FLyModel=Llama-3.1-Instruct-405B, Training-based=false, Temperature=02025.115.15——15.83
EAGLE-self, treeModel=Llama2-13b, Decoding Strategy=Relaxed (temperature = 0.7), Decoding Structure=tree base2026.055.12——5.1758
CATSModel=Llama2-13b, Decoding Strategy=Relaxed (temperature = 0.7), Decoding Structure=chain base2026.055.03——5.0678
FLyModel=Llama-3.1-Instruct-405B, Training-based=false, Temperature=12025.115——16.29
CATS w/ EAGLEModel=Llama2-7b, Decoding Strategy=Relaxed (temperature = 0.7), Decoding Structure=tree base2026.054.99——4.9882
CATS w/ EAGLEModel=Llama2-13b, Decoding Strategy=Greedy, Decoding Structure=tree base2026.054.95——5.0178
CATSModel=Llama2-7b, Decoding Strategy=Greedy, Decoding Structure=chain base2026.054.72——4.724
EAGLE-selfModel=Llama2-13b, Decoding Strategy=Relaxed (temperature = 0.7), Decoding Structure=chain base2026.054.69——4.7135
KangarooModel=Llama2-13b, Decoding Strategy=Relaxed (temperature = 0.7), Decoding Structure=chain base2026.054.68——4.695
CATS w/ EAGLEModel=Vicuna-13b, Decoding Strategy=Relaxed (temperature = 0.7), Decoding Structure=tree base2026.054.67——4.7269
CATSModel=Llama2-13b, Decoding Strategy=Greedy, Decoding Structure=chain base2026.054.66——4.6959
EAGLE-self, treeModel=Llama2-13b, Decoding Strategy=Greedy, Decoding Structure=tree base2026.054.65——4.6993
CATSModel=Llama2-7b, Decoding Strategy=Relaxed (temperature = 0.7), Decoding Structure=chain base2026.054.44——4.4388
CATS w/ EAGLEModel=Vicuna-13b, Decoding Strategy=Greedy, Decoding Structure=tree base2026.054.19——4.2428
EAGLE-self, treeModel=Llama2-7b, Decoding Strategy=Relaxed (temperature = 0.7), Decoding Structure=tree base2026.054.11——4.7703
KangarooModel=Llama2-13b, Decoding Strategy=Greedy, Decoding Structure=chain base2026.054.1——4.1133
EAGLE-selfModel=Llama2-13b, Decoding Strategy=Greedy, Decoding Structure=chain base2026.054.09——4.1082
EAGLE-3Model=Meta-Llama-3.3-Instruct-70B, Training-based=true, Temperature=02025.113.97——5.73
EAGLE-self, treeModel=Llama2-7b, Decoding Strategy=Greedy, Decoding Structure=tree base2026.053.95——4.5866
EAGLE-self, treeModel=Vicuna-13b, Decoding Strategy=Relaxed (temperature = 0.7), Decoding Structure=tree base2026.053.94——3.9784
EAGLE-3Model=Meta-Llama-3.3-Instruct-70B, Training-based=true, Temperature=12025.113.74——5.4
KangarooModel=Llama2-7b, Decoding Strategy=Greedy, Decoding Structure=chain base2026.053.7——4.2946
EAGLE-selfModel=Llama2-7b, Decoding Strategy=Relaxed (temperature = 0.7), Decoding Structure=chain base2026.053.68——4.2687
EAGLE-self, treeModel=Vicuna-13b, Decoding Strategy=Greedy, Decoding Structure=tree base2026.053.58——3.6168
EAGLE-selfModel=Llama2-7b, Decoding Strategy=Greedy, Decoding Structure=chain base2026.053.5——4.0588
KangarooModel=Llama2-7b, Decoding Strategy=Relaxed (temperature = 0.7), Decoding Structure=chain base2026.053.5——4.0587
CATSModel=Vicuna-13b, Decoding Strategy=Relaxed (temperature = 0.7), Decoding Structure=chain base2026.053.45——3.4792
SAM-Decoding[E2]Backbone=Vicuna-7B-v1.3, Auxiliary Method=EAGLE-2, Data Type=float16, Batch size=12024.113.354.95130.28—
KangarooModel=Vicuna-13b, Decoding Strategy=Relaxed (temperature = 0.7), Decoding Structure=chain base2026.053.27——3.2753
DARTModel=Qwen4B, Temperature=02026.013.25———
EAGLE-2Backbone=Vicuna-7B-v1.3, Data Type=float16, Batch size=12024.113.245.13125.77—
CATSModel=Vicuna-13b, Decoding Strategy=Greedy, Decoding Structure=chain base2026.053.2——3.2218
EAGLE-selfModel=Vicuna-13b, Decoding Strategy=Relaxed (temperature = 0.7), Decoding Structure=chain base2026.053.2——3.2212
DARTModel=L2 7B, Temperature=02026.012.98———
KangarooModel=Vicuna-13b, Decoding Strategy=Greedy, Decoding Structure=chain base2026.052.91——2.9183
EAGLE-selfModel=Vicuna-13b, Decoding Strategy=Greedy, Decoding Structure=chain base2026.052.9——2.9163
FLyModel=Llama-3.1-Instruct-70B, Training-based=false, Temperature=02025.112.86——12.61
SpSModel=Llama-3.1-Instruct-405B, Training-based=false, Temperature=02025.112.84——11.62
DARTModel=Qwen8B, Temperature=02026.012.8———
DARTModel=Qwen1.7B, Temperature=02026.012.79———
DARTModel=Qwen14B, Temperature=02026.012.79———
FLyModel=Meta-Llama-3.3-Instruct-70B, Training-based=false, Temperature=02025.112.75——12.2
FLyModel=Meta-Llama-3.3-Instruct-70B, Training-based=false, Temperature=12025.112.75——12.55
HydraModel=L2 7B, Temperature=02026.012.6———
FLyModel=Meta-Llama-3-Instruct-70B, Training-based=false, Temperature=02025.112.6——12.74
EAGLE-2Model=Meta-Llama-3-Instruct-70B, Training-based=true, Temperature=02025.112.56——4.24
FLyModel=Meta-Llama-3-Instruct-70B, Training-based=false, Temperature=12025.112.53——12.67
FLyModel=Llama-3.1-Instruct-70B, Training-based=false, Temperature=12025.112.52——11.96
MedusaModel=Vicuna-13b, Decoding Strategy=Relaxed (temperature = 0.7), Decoding Structure=chain base2026.052.51——2.9535
DARTModel=Qwen14B, Temperature=12026.012.48———
SAM-Decoding[T]Backbone=Vicuna-7B-v1.3, Auxiliary Method=Token Recycling, Data Type=float16, Batch size=12024.112.452.9495.08—
SpSModel=Llama-3.1-Instruct-405B, Training-based=false, Temperature=12025.112.39——9.78
MedusaModel=Vicuna-13b, Decoding Strategy=Greedy, Decoding Structure=chain base2026.052.39——2.8035
EAGLE-2Model=Meta-Llama-3-Instruct-70B, Training-based=true, Temperature=12025.112.38——3.95
EAGLE3Model=Qwen8B, Temperature=02026.012.37———
DARTModel=Qwen32B, Temperature=02026.012.37———
SAM-DecodingBackbone=Vicuna-7B-v1.3, Data Type=float16, Batch size=12024.112.292.6488.91—
MedusaModel=Llama2-7b, Decoding Strategy=Relaxed (temperature = 0.7), Decoding Structure=chain base2026.052.28——2.2848
EAGLE3Model=Qwen14B, Temperature=02026.012.27———
MedusaModel=L2 7B, Temperature=02026.012.22———
EAGLE3Model=Qwen4B, Temperature=02026.012.2———
EAGLE3Model=Qwen32B, Temperature=02026.012.19———
DARTModel=Qwen32B, Temperature=12026.012.19———
EAGLE3Model=Qwen14B, Temperature=12026.012.17———
TokenRecyclingModel=Llama-3.1-Instruct-70B, Training-based=false, Temperature=12025.112.15——3.06
RESTModel=Llama-3.1-Instruct-405B, Training-based=false, Temperature=02025.112.14——2.41
TokenRecyclingModel=Llama-3.1-Instruct-70B, Training-based=false, Temperature=02025.112.12——3.07
EAGLE3Model=Qwen32B, Temperature=12026.012.08———
MedusaModel=Llama2-7b, Decoding Strategy=Greedy, Decoding Structure=chain base2026.051.97——1.9657
MedusaModel=Llama2-13b, Decoding Strategy=Relaxed (temperature = 0.7), Decoding Structure=chain base2026.051.95——2.2848
Token RecyclingBackbone=Vicuna-7B-v1.3, Data Type=float16, Batch size=12024.111.942.7875.44—
EAGLE3Model=Qwen1.7B, Temperature=02026.011.94———
RESTModel=Llama-3.1-Instruct-70B, Training-based=false, Temperature=02025.111.89——2.22
PLDModel=L2 7B, Temperature=02026.011.84———
RESTModel=Llama-3.1-Instruct-405B, Training-based=false, Temperature=12025.111.83——2.07
RESTModel=Llama-3.1-Instruct-70B, Training-based=false, Temperature=12025.111.81——2.13
REST*Backbone=Vicuna-7B-v1.3, Data Type=float16, Batch size=12024.111.741.8534.6—
PIABackbone=Vicuna-7B-v1.3, Data Type=float16, Batch size=12024.111.682.6265.49—
MedusaModel=Llama2-13b, Decoding Strategy=Greedy, Decoding Structure=chain base2026.051.65——1.9425
LookaheadModel=L2 7B, Temperature=02026.011.64———
SpSModel=Llama-3.1-Instruct-70B, Training-based=false, Temperature=02025.111.64——9.89
TokenRecyclingModel=Llama-3.1-Instruct-405B, Training-based=false, Temperature=12025.111.56——2.91
Lookahead*Backbone=Vicuna-7B-v1.3, Data Type=float16, Batch size=12024.111.541.7630.81—
TokenRecyclingModel=Llama-3.1-Instruct-405B, Training-based=false, Temperature=02025.111.54——2.88
PLDBackbone=Vicuna-7B-v1.3, Data Type=float16, Batch size=12024.111.521.6559.04—
SpSModel=Llama-3.1-Instruct-70B, Training-based=false, Temperature=12025.111.07——6.29
SPSModel=Qwen32B, Temperature=02026.011.06———
SPSModel=Qwen14B, Temperature=02026.010.96———