# Mathematical Reasoning on GSM8K (AVGLEN & SPEEDUP)

Source: [Mathematical Reasoning on GSM8K (AVGLEN & SPEEDUP)](<https://www.sota2.com/research/sota/mathematical-reasoning-on-gsm8k-avglen-and-speedup>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-05-29T08:35:09.030Z

Evaluation of model performance on grade-school level mathematical word problems (GSM8K), focusing on average output length (AVGLEN) and inference speedup (SPEEDUP).

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Mathematical Reasoning |
| Dataset (including any stated split) | GSM8K |
| Metric | Average Length |
| Unit | SCALAR |
| Direction | Higher is better |
| Results loaded | 61 of 61 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Average Length](<https://www.sota2.com/research/sota/mathematical-reasoning-on-gsm8k-avglen-and-speedup.md?metric=Average%20Length>): Higher is better; unit: SCALAR.

- [Speedup](<https://www.sota2.com/research/sota/mathematical-reasoning-on-gsm8k-avglen-and-speedup.md?metric=Speedup>): Higher is better; unit: SCALAR.

## Recorded results

| Method | Average Length | Result date | Report paper |
| --- | --- | --- | --- |
| Draft-OPD | 6.6 | Not recorded | 6a191dafdbd7150506f82956 |
| Draft-OPD | 6.57 | Not recorded | 6a191dafdbd7150506f82956 |
| Draft-OPD | 6.13 | Not recorded | 6a191dafdbd7150506f82956 |
| DFlash | 6.04 | Not recorded | 6a191dafdbd7150506f82956 |
| DFlash | 6.04 | Not recorded | 6a191dafdbd7150506f82956 |
| Draft-OPD | 6.02 | Not recorded | 6a191dafdbd7150506f82956 |
| EAGLE-3 | 5.99 | Not recorded | 6a191dafdbd7150506f82956 |
| Draft-OPD | 5.96 | Not recorded | 6a191dafdbd7150506f82956 |
| EAGLE-3 | 5.84 | Not recorded | 6a191dafdbd7150506f82956 |
| Draft-OPD | 5.73 | Not recorded | 6a191dafdbd7150506f82956 |
| DFlash | 5.73 | Not recorded | 6a191dafdbd7150506f82956 |
| EAGLE-3 | 5.69 | Not recorded | 6a191dafdbd7150506f82956 |
| EAGLE-3 | 5.66 | Not recorded | 6a191dafdbd7150506f82956 |
| EAGLE-3 | 5.64 | Not recorded | 6a191dafdbd7150506f82956 |
| DFlash | 5.62 | Not recorded | 6a191dafdbd7150506f82956 |
| DFlash | 5.51 | Not recorded | 6a191dafdbd7150506f82956 |
| EAGLE-3 | 5.33 | Not recorded | 6a191dafdbd7150506f82956 |
| EAGLE-3 | 5.33 | Not recorded | 6a191dafdbd7150506f82956 |
| DFlash | 5.19 | Not recorded | 6a191dafdbd7150506f82956 |
| Draft-OPD | 5.13 | Not recorded | 6a191dafdbd7150506f82956 |
| Draft-OPD | 5.05 | Not recorded | 6a191dafdbd7150506f82956 |
| EAGLE-3 | 5.03 | Not recorded | 6a191dafdbd7150506f82956 |
| DFlash | 4.77 | Not recorded | 6a191dafdbd7150506f82956 |
| DFlash | 4.66 | Not recorded | 6a191dafdbd7150506f82956 |
| CATS w/ EAGLE | 4.5345 | Not recorded | 6a040579dbc836b87d827444 |
| CATS | 3.9652 | Not recorded | 6a040579dbc836b87d827444 |
| EAGLE-self | 3.7629 | Not recorded | 6a040579dbc836b87d827444 |
| CATS w/ EAGLE | 3.6869 | Not recorded | 6a040579dbc836b87d827444 |
| Kangaroo | 3.0513 | Not recorded | 6a040579dbc836b87d827444 |
| CATS | 3.0198 | Not recorded | 6a040579dbc836b87d827444 |
| EAGLE-self | 3.0059 | Not recorded | 6a040579dbc836b87d827444 |
| EAGLE-self | 2.9515 | Not recorded | 6a040579dbc836b87d827444 |
| Opt-Hydra | 2.94 | Not recorded | 69b78e5b0636e5bf4afbda1d |
| Lookahead | 2.701 | Not recorded | 6a040579dbc836b87d827444 |
| Opt-Hydra | 2.69 | Not recorded | 69b78e5b0636e5bf4afbda1d |
| Medusa | 2.6837 | Not recorded | 6a040579dbc836b87d827444 |
| OSD-Hydra | 2.56 | Not recorded | 69b78e5b0636e5bf4afbda1d |
| OSD-Hydra | 2.55 | Not recorded | 69b78e5b0636e5bf4afbda1d |
| Medusa | 2.4966 | Not recorded | 6a040579dbc836b87d827444 |
| Kangaroo | 2.4856 | Not recorded | 6a040579dbc836b87d827444 |
| Ens-EAGLE-3 | 2.33 | Not recorded | 69b78e5b0636e5bf4afbda1d |
| EAGLE-self | 2.3213 | Not recorded | 6a040579dbc836b87d827444 |
| OSD-EAGLE-3 | 2.25 | Not recorded | 69b78e5b0636e5bf4afbda1d |
| Ens-EAGLE-3 | 2.16 | Not recorded | 69b78e5b0636e5bf4afbda1d |
| Hydra | 2.14 | Not recorded | 69b78e5b0636e5bf4afbda1d |
| OSD-EAGLE-3 | 2.07 | Not recorded | 69b78e5b0636e5bf4afbda1d |
| Ens-EAGLE | 2.01 | Not recorded | 69b78e5b0636e5bf4afbda1d |
| OSD-EAGLE | 1.92 | Not recorded | 69b78e5b0636e5bf4afbda1d |
| EAGLE-3 | 1.82 | Not recorded | 69b78e5b0636e5bf4afbda1d |
| EAGLE-3 | 1.78 | Not recorded | 69b78e5b0636e5bf4afbda1d |
| Ens-EAGLE | 1.58 | Not recorded | 69b78e5b0636e5bf4afbda1d |
| OSD | 1.52 | Not recorded | 69b78e5b0636e5bf4afbda1d |
| Hydra | 1.52 | Not recorded | 69b78e5b0636e5bf4afbda1d |
| REST | 1.5192 | Not recorded | 6a040579dbc836b87d827444 |
| REST | 1.5045 | Not recorded | 6a040579dbc836b87d827444 |
| EAGLE | 1.48 | Not recorded | 69b78e5b0636e5bf4afbda1d |
| OSD-EAGLE | 1.45 | Not recorded | 69b78e5b0636e5bf4afbda1d |
| OSD | 1.4 | Not recorded | 69b78e5b0636e5bf4afbda1d |
| Vanilla SD | 1.25 | Not recorded | 69b78e5b0636e5bf4afbda1d |
| Vanilla SD | 1.25 | Not recorded | 69b78e5b0636e5bf4afbda1d |
| EAGLE | 1.19 | Not recorded | 69b78e5b0636e5bf4afbda1d |

### Draft-OPD

Average Length: 6.6. Result ID: 6a194facfbc373592c13b0b2.

[Reporting paper](<https://www.sota2.com/research/paper/6a191dafdbd7150506f82956.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a191dafdbd7150506f82956.md>)

[Implementation](<https://github.com/bingyang-lei/Draft-OPD>)

Recorded evaluation configuration: {"Model":"Q3-4B","Thinking Mode":"Disabled","Temperature":"0"}. This record does not assert independent reproduction.

### Draft-OPD

Average Length: 6.57. Result ID: 6a194facfbc373592c13b0b5.

[Reporting paper](<https://www.sota2.com/research/paper/6a191dafdbd7150506f82956.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a191dafdbd7150506f82956.md>)

[Implementation](<https://github.com/bingyang-lei/Draft-OPD>)

Recorded evaluation configuration: {"Model":"Q3-8B","Thinking Mode":"Disabled","Temperature":"0"}. This record does not assert independent reproduction.

### Draft-OPD

Average Length: 6.13. Result ID: 6a194facfbc373592c13b0b8.

[Reporting paper](<https://www.sota2.com/research/paper/6a191dafdbd7150506f82956.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a191dafdbd7150506f82956.md>)

[Implementation](<https://github.com/bingyang-lei/Draft-OPD>)

Recorded evaluation configuration: {"Model":"Q3-4B","Thinking Mode":"Disabled","Temperature":"0.6"}. This record does not assert independent reproduction.

### DFlash

Average Length: 6.04. Result ID: 6a194facfbc373592c13b0b1.

[Reporting paper](<https://www.sota2.com/research/paper/6a191dafdbd7150506f82956.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa738a1e209131cea5d174.md>)

Recorded evaluation configuration: {"Model":"Q3-4B","Thinking Mode":"Disabled","Temperature":"0"}. This record does not assert independent reproduction.

### DFlash

Average Length: 6.04. Result ID: 6a194facfbc373592c13b0b4.

[Reporting paper](<https://www.sota2.com/research/paper/6a191dafdbd7150506f82956.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa738a1e209131cea5d174.md>)

Recorded evaluation configuration: {"Model":"Q3-8B","Thinking Mode":"Disabled","Temperature":"0"}. This record does not assert independent reproduction.

### Draft-OPD

Average Length: 6.02. Result ID: 6a194facfbc373592c13b0bb.

[Reporting paper](<https://www.sota2.com/research/paper/6a191dafdbd7150506f82956.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a191dafdbd7150506f82956.md>)

[Implementation](<https://github.com/bingyang-lei/Draft-OPD>)

Recorded evaluation configuration: {"Model":"Q3-8B","Thinking Mode":"Disabled","Temperature":"0.6"}. This record does not assert independent reproduction.

### EAGLE-3

Average Length: 5.99. Result ID: 6a194facfbc373592c13b0b3.

[Reporting paper](<https://www.sota2.com/research/paper/6a191dafdbd7150506f82956.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa783c1e209131cea7758e.md>)

Recorded evaluation configuration: {"Model":"Q3-8B","Thinking Mode":"Disabled","Temperature":"0"}. This record does not assert independent reproduction.

### Draft-OPD

Average Length: 5.96. Result ID: 6a194facfbc373592c13b0a6.

[Reporting paper](<https://www.sota2.com/research/paper/6a191dafdbd7150506f82956.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a191dafdbd7150506f82956.md>)

[Implementation](<https://github.com/bingyang-lei/Draft-OPD>)

Recorded evaluation configuration: {"Model":"Q3-4B","Thinking Mode":"Enabled","Temperature":"0"}. This record does not assert independent reproduction.

### EAGLE-3

Average Length: 5.84. Result ID: 6a194facfbc373592c13b0b0.

[Reporting paper](<https://www.sota2.com/research/paper/6a191dafdbd7150506f82956.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa783c1e209131cea7758e.md>)

Recorded evaluation configuration: {"Model":"Q3-4B","Thinking Mode":"Disabled","Temperature":"0"}. This record does not assert independent reproduction.

### Draft-OPD

Average Length: 5.73. Result ID: 6a194facfbc373592c13b0a9.

[Reporting paper](<https://www.sota2.com/research/paper/6a191dafdbd7150506f82956.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a191dafdbd7150506f82956.md>)

[Implementation](<https://github.com/bingyang-lei/Draft-OPD>)

Recorded evaluation configuration: {"Model":"Q3-8B","Thinking Mode":"Enabled","Temperature":"0"}. This record does not assert independent reproduction.

### DFlash

Average Length: 5.73. Result ID: 6a194facfbc373592c13b0b7.

[Reporting paper](<https://www.sota2.com/research/paper/6a191dafdbd7150506f82956.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa738a1e209131cea5d174.md>)

Recorded evaluation configuration: {"Model":"Q3-4B","Thinking Mode":"Disabled","Temperature":"0.6"}. This record does not assert independent reproduction.

### EAGLE-3

Average Length: 5.69. Result ID: 6a194facfbc373592c13b0b9.

[Reporting paper](<https://www.sota2.com/research/paper/6a191dafdbd7150506f82956.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa783c1e209131cea7758e.md>)

Recorded evaluation configuration: {"Model":"Q3-8B","Thinking Mode":"Disabled","Temperature":"0.6"}. This record does not assert independent reproduction.

### EAGLE-3

Average Length: 5.66. Result ID: 6a194facfbc373592c13b0b6.

[Reporting paper](<https://www.sota2.com/research/paper/6a191dafdbd7150506f82956.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa783c1e209131cea7758e.md>)

Recorded evaluation configuration: {"Model":"Q3-4B","Thinking Mode":"Disabled","Temperature":"0.6"}. This record does not assert independent reproduction.

### EAGLE-3

Average Length: 5.64. Result ID: 6a194facfbc373592c13b0a7.

[Reporting paper](<https://www.sota2.com/research/paper/6a191dafdbd7150506f82956.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa783c1e209131cea7758e.md>)

Recorded evaluation configuration: {"Model":"Q3-8B","Thinking Mode":"Enabled","Temperature":"0"}. This record does not assert independent reproduction.

### DFlash

Average Length: 5.62. Result ID: 6a194facfbc373592c13b0ba.

[Reporting paper](<https://www.sota2.com/research/paper/6a191dafdbd7150506f82956.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa738a1e209131cea5d174.md>)

Recorded evaluation configuration: {"Model":"Q3-8B","Thinking Mode":"Disabled","Temperature":"0.6"}. This record does not assert independent reproduction.

### DFlash

Average Length: 5.51. Result ID: 6a194facfbc373592c13b0a5.

[Reporting paper](<https://www.sota2.com/research/paper/6a191dafdbd7150506f82956.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa738a1e209131cea5d174.md>)

Recorded evaluation configuration: {"Model":"Q3-4B","Thinking Mode":"Enabled","Temperature":"0"}. This record does not assert independent reproduction.

### EAGLE-3

Average Length: 5.33. Result ID: 6a194facfbc373592c13b0a4.

[Reporting paper](<https://www.sota2.com/research/paper/6a191dafdbd7150506f82956.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa783c1e209131cea7758e.md>)

Recorded evaluation configuration: {"Model":"Q3-4B","Thinking Mode":"Enabled","Temperature":"0"}. This record does not assert independent reproduction.

### EAGLE-3

Average Length: 5.33. Result ID: 6a194facfbc373592c13b0ad.

[Reporting paper](<https://www.sota2.com/research/paper/6a191dafdbd7150506f82956.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa783c1e209131cea7758e.md>)

Recorded evaluation configuration: {"Model":"Q3-8B","Thinking Mode":"Enabled","Temperature":"0.6"}. This record does not assert independent reproduction.

### DFlash

Average Length: 5.19. Result ID: 6a194facfbc373592c13b0a8.

[Reporting paper](<https://www.sota2.com/research/paper/6a191dafdbd7150506f82956.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa738a1e209131cea5d174.md>)

Recorded evaluation configuration: {"Model":"Q3-8B","Thinking Mode":"Enabled","Temperature":"0"}. This record does not assert independent reproduction.

### Draft-OPD

Average Length: 5.13. Result ID: 6a194facfbc373592c13b0ac.

[Reporting paper](<https://www.sota2.com/research/paper/6a191dafdbd7150506f82956.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a191dafdbd7150506f82956.md>)

[Implementation](<https://github.com/bingyang-lei/Draft-OPD>)

Recorded evaluation configuration: {"Model":"Q3-4B","Thinking Mode":"Enabled","Temperature":"0.6"}. This record does not assert independent reproduction.

### Draft-OPD

Average Length: 5.05. Result ID: 6a194facfbc373592c13b0af.

[Reporting paper](<https://www.sota2.com/research/paper/6a191dafdbd7150506f82956.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a191dafdbd7150506f82956.md>)

[Implementation](<https://github.com/bingyang-lei/Draft-OPD>)

Recorded evaluation configuration: {"Model":"Q3-8B","Thinking Mode":"Enabled","Temperature":"0.6"}. This record does not assert independent reproduction.

### EAGLE-3

Average Length: 5.03. Result ID: 6a194facfbc373592c13b0aa.

[Reporting paper](<https://www.sota2.com/research/paper/6a191dafdbd7150506f82956.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa783c1e209131cea7758e.md>)

Recorded evaluation configuration: {"Model":"Q3-4B","Thinking Mode":"Enabled","Temperature":"0.6"}. This record does not assert independent reproduction.

### DFlash

Average Length: 4.77. Result ID: 6a194facfbc373592c13b0ab.

[Reporting paper](<https://www.sota2.com/research/paper/6a191dafdbd7150506f82956.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa738a1e209131cea5d174.md>)

Recorded evaluation configuration: {"Model":"Q3-4B","Thinking Mode":"Enabled","Temperature":"0.6"}. This record does not assert independent reproduction.

### DFlash

Average Length: 4.66. Result ID: 6a194facfbc373592c13b0ae.

[Reporting paper](<https://www.sota2.com/research/paper/6a191dafdbd7150506f82956.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa738a1e209131cea5d174.md>)

Recorded evaluation configuration: {"Model":"Q3-8B","Thinking Mode":"Enabled","Temperature":"0.6"}. This record does not assert independent reproduction.

### CATS w/ EAGLE

Average Length: 4.5345. Result ID: 6a0427b447f34ce924e9a8a7.

[Reporting paper](<https://www.sota2.com/research/paper/6a040579dbc836b87d827444.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a040579dbc836b87d827444.md>)

Recorded evaluation configuration: {"Decoding Strategy":"Relaxed (temperature = 0.7)","Decoding Structure":"tree base","Backbone":"Vicuna-7B","Drafting steps":"5","top-K":"10"}. This record does not assert independent reproduction.

### CATS

Average Length: 3.9652. Result ID: 6a0427b447f34ce924e9a8a5.

[Reporting paper](<https://www.sota2.com/research/paper/6a040579dbc836b87d827444.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a040579dbc836b87d827444.md>)

Recorded evaluation configuration: {"Decoding Strategy":"Relaxed (temperature = 0.7)","Decoding Structure":"chain base","Backbone":"Vicuna-7B","Drafting steps":"5"}. This record does not assert independent reproduction.

### EAGLE-self

Average Length: 3.7629. Result ID: 6a0427b447f34ce924e9a8a6.

[Reporting paper](<https://www.sota2.com/research/paper/6a040579dbc836b87d827444.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa77391e209131cea71c7d.md>)

Recorded evaluation configuration: {"Decoding Strategy":"Relaxed (temperature = 0.7)","Decoding Structure":"tree base","Backbone":"Vicuna-7B","Drafting steps":"5","top-K":"10"}. This record does not assert independent reproduction.

### CATS w/ EAGLE

Average Length: 3.6869. Result ID: 6a0427b447f34ce924e9a8a0.

[Reporting paper](<https://www.sota2.com/research/paper/6a040579dbc836b87d827444.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a040579dbc836b87d827444.md>)

Recorded evaluation configuration: {"Decoding Strategy":"Greedy","Decoding Structure":"tree base","Backbone":"Vicuna-7B","Drafting steps":"5","top-K":"10"}. This record does not assert independent reproduction.

### Kangaroo

Average Length: 3.0513. Result ID: 6a0427b447f34ce924e9a8a2.

[Reporting paper](<https://www.sota2.com/research/paper/6a040579dbc836b87d827444.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73081e209131cea59efc.md>)

Recorded evaluation configuration: {"Decoding Strategy":"Relaxed (temperature = 0.7)","Decoding Structure":"chain base","Backbone":"Vicuna-7B","Drafting steps":"5"}. This record does not assert independent reproduction.

### CATS

Average Length: 3.0198. Result ID: 6a0427b447f34ce924e9a89e.

[Reporting paper](<https://www.sota2.com/research/paper/6a040579dbc836b87d827444.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a040579dbc836b87d827444.md>)

Recorded evaluation configuration: {"Decoding Strategy":"Greedy","Decoding Structure":"chain base","Backbone":"Vicuna-7B","Drafting steps":"5"}. This record does not assert independent reproduction.

### EAGLE-self

Average Length: 3.0059. Result ID: 6a0427b447f34ce924e9a89f.

[Reporting paper](<https://www.sota2.com/research/paper/6a040579dbc836b87d827444.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa77391e209131cea71c7d.md>)

Recorded evaluation configuration: {"Decoding Strategy":"Greedy","Decoding Structure":"tree base","Backbone":"Vicuna-7B","Drafting steps":"5","top-K":"10"}. This record does not assert independent reproduction.

### EAGLE-self

Average Length: 2.9515. Result ID: 6a0427b447f34ce924e9a8a4.

[Reporting paper](<https://www.sota2.com/research/paper/6a040579dbc836b87d827444.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa77391e209131cea71c7d.md>)

Recorded evaluation configuration: {"Decoding Strategy":"Relaxed (temperature = 0.7)","Decoding Structure":"chain base","Backbone":"Vicuna-7B","Drafting steps":"5"}. This record does not assert independent reproduction.

### Opt-Hydra

Average Length: 2.94. Result ID: 69b7c5bc47f34ce9249f8e54.

[Reporting paper](<https://www.sota2.com/research/paper/69b78e5b0636e5bf4afbda1d.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69b78e5b0636e5bf4afbda1d.md>)

Recorded evaluation configuration: {"Target model":"meta-llama / Llama-2-7B-Chat"}. This record does not assert independent reproduction.

### Lookahead

Average Length: 2.701. Result ID: 6a0427b447f34ce924e9a89a.

[Reporting paper](<https://www.sota2.com/research/paper/6a040579dbc836b87d827444.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa721d1e209131cea54995.md>)

Recorded evaluation configuration: {"Decoding Strategy":"Greedy","Decoding Structure":"chain base","Backbone":"Vicuna-7B"}. This record does not assert independent reproduction.

### Opt-Hydra

Average Length: 2.69. Result ID: 69b7c5bc47f34ce9249f8e49.

[Reporting paper](<https://www.sota2.com/research/paper/69b78e5b0636e5bf4afbda1d.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69b78e5b0636e5bf4afbda1d.md>)

Recorded evaluation configuration: {"Target model":"lmsys / Vicuna-7B-v1.3"}. This record does not assert independent reproduction.

### Medusa

Average Length: 2.6837. Result ID: 6a0427b447f34ce924e9a8a3.

[Reporting paper](<https://www.sota2.com/research/paper/6a040579dbc836b87d827444.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73b11e209131cea5df0f.md>)

Recorded evaluation configuration: {"Decoding Strategy":"Relaxed (temperature = 0.7)","Decoding Structure":"chain base","Backbone":"Vicuna-7B","Number of heads":"2"}. This record does not assert independent reproduction.

### OSD-Hydra

Average Length: 2.56. Result ID: 69b7c5bc47f34ce9249f8e48.

[Reporting paper](<https://www.sota2.com/research/paper/69b78e5b0636e5bf4afbda1d.md>)

Recorded evaluation configuration: {"Target model":"lmsys / Vicuna-7B-v1.3"}. This record does not assert independent reproduction.

### OSD-Hydra

Average Length: 2.55. Result ID: 69b7c5bc47f34ce9249f8e53.

[Reporting paper](<https://www.sota2.com/research/paper/69b78e5b0636e5bf4afbda1d.md>)

Recorded evaluation configuration: {"Target model":"meta-llama / Llama-2-7B-Chat"}. This record does not assert independent reproduction.

### Medusa

Average Length: 2.4966. Result ID: 6a0427b447f34ce924e9a89c.

[Reporting paper](<https://www.sota2.com/research/paper/6a040579dbc836b87d827444.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73b11e209131cea5df0f.md>)

Recorded evaluation configuration: {"Decoding Strategy":"Greedy","Decoding Structure":"chain base","Backbone":"Vicuna-7B","Number of heads":"2"}. This record does not assert independent reproduction.

### Kangaroo

Average Length: 2.4856. Result ID: 6a0427b447f34ce924e9a89b.

[Reporting paper](<https://www.sota2.com/research/paper/6a040579dbc836b87d827444.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73081e209131cea59efc.md>)

Recorded evaluation configuration: {"Decoding Strategy":"Greedy","Decoding Structure":"chain base","Backbone":"Vicuna-7B","Drafting steps":"5"}. This record does not assert independent reproduction.

### Ens-EAGLE-3

Average Length: 2.33. Result ID: 69b7c5bc47f34ce9249f8e5a.

[Reporting paper](<https://www.sota2.com/research/paper/69b78e5b0636e5bf4afbda1d.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69b78e5b0636e5bf4afbda1d.md>)

Recorded evaluation configuration: {"Target model":"meta-llama / Llama-2-7B-Chat"}. This record does not assert independent reproduction.

### EAGLE-self

Average Length: 2.3213. Result ID: 6a0427b447f34ce924e9a89d.

[Reporting paper](<https://www.sota2.com/research/paper/6a040579dbc836b87d827444.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa77391e209131cea71c7d.md>)

Recorded evaluation configuration: {"Decoding Strategy":"Greedy","Decoding Structure":"chain base","Backbone":"Vicuna-7B","Drafting steps":"5"}. This record does not assert independent reproduction.

### OSD-EAGLE-3

Average Length: 2.25. Result ID: 69b7c5bc47f34ce9249f8e59.

[Reporting paper](<https://www.sota2.com/research/paper/69b78e5b0636e5bf4afbda1d.md>)

Recorded evaluation configuration: {"Target model":"meta-llama / Llama-2-7B-Chat"}. This record does not assert independent reproduction.

### Ens-EAGLE-3

Average Length: 2.16. Result ID: 69b7c5bc47f34ce9249f8e4f.

[Reporting paper](<https://www.sota2.com/research/paper/69b78e5b0636e5bf4afbda1d.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69b78e5b0636e5bf4afbda1d.md>)

Recorded evaluation configuration: {"Target model":"lmsys / Vicuna-7B-v1.3"}. This record does not assert independent reproduction.

### Hydra

Average Length: 2.14. Result ID: 69b7c5bc47f34ce9249f8e47.

[Reporting paper](<https://www.sota2.com/research/paper/69b78e5b0636e5bf4afbda1d.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa75111e209131cea6579f.md>)

Recorded evaluation configuration: {"Target model":"lmsys / Vicuna-7B-v1.3"}. This record does not assert independent reproduction.

### OSD-EAGLE-3

Average Length: 2.07. Result ID: 69b7c5bc47f34ce9249f8e4e.

[Reporting paper](<https://www.sota2.com/research/paper/69b78e5b0636e5bf4afbda1d.md>)

Recorded evaluation configuration: {"Target model":"lmsys / Vicuna-7B-v1.3"}. This record does not assert independent reproduction.

### Ens-EAGLE

Average Length: 2.01. Result ID: 69b7c5bc47f34ce9249f8e4c.

[Reporting paper](<https://www.sota2.com/research/paper/69b78e5b0636e5bf4afbda1d.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69b78e5b0636e5bf4afbda1d.md>)

Recorded evaluation configuration: {"Target model":"lmsys / Vicuna-7B-v1.3"}. This record does not assert independent reproduction.

### OSD-EAGLE

Average Length: 1.92. Result ID: 69b7c5bc47f34ce9249f8e4b.

[Reporting paper](<https://www.sota2.com/research/paper/69b78e5b0636e5bf4afbda1d.md>)

Recorded evaluation configuration: {"Target model":"lmsys / Vicuna-7B-v1.3"}. This record does not assert independent reproduction.

### EAGLE-3

Average Length: 1.82. Result ID: 69b7c5bc47f34ce9249f8e58.

[Reporting paper](<https://www.sota2.com/research/paper/69b78e5b0636e5bf4afbda1d.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa783c1e209131cea7758e.md>)

Recorded evaluation configuration: {"Target model":"meta-llama / Llama-2-7B-Chat"}. This record does not assert independent reproduction.

### EAGLE-3

Average Length: 1.78. Result ID: 69b7c5bc47f34ce9249f8e4d.

[Reporting paper](<https://www.sota2.com/research/paper/69b78e5b0636e5bf4afbda1d.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa783c1e209131cea7758e.md>)

Recorded evaluation configuration: {"Target model":"lmsys / Vicuna-7B-v1.3"}. This record does not assert independent reproduction.

### Ens-EAGLE

Average Length: 1.58. Result ID: 69b7c5bc47f34ce9249f8e57.

[Reporting paper](<https://www.sota2.com/research/paper/69b78e5b0636e5bf4afbda1d.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69b78e5b0636e5bf4afbda1d.md>)

Recorded evaluation configuration: {"Target model":"meta-llama / Llama-2-7B-Chat"}. This record does not assert independent reproduction.

### OSD

Average Length: 1.52. Result ID: 69b7c5bc47f34ce9249f8e46.

[Reporting paper](<https://www.sota2.com/research/paper/69b78e5b0636e5bf4afbda1d.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73851e209131cea5ce69.md>)

Recorded evaluation configuration: {"Target model":"lmsys / Vicuna-7B-v1.3"}. This record does not assert independent reproduction.

### Hydra

Average Length: 1.52. Result ID: 69b7c5bc47f34ce9249f8e52.

[Reporting paper](<https://www.sota2.com/research/paper/69b78e5b0636e5bf4afbda1d.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa75111e209131cea6579f.md>)

Recorded evaluation configuration: {"Target model":"meta-llama / Llama-2-7B-Chat"}. This record does not assert independent reproduction.

### REST

Average Length: 1.5192. Result ID: 6a0427b447f34ce924e9a8a1.

[Reporting paper](<https://www.sota2.com/research/paper/6a040579dbc836b87d827444.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa72cd1e209131cea588ee.md>)

Recorded evaluation configuration: {"Decoding Strategy":"Relaxed (temperature = 0.7)","Decoding Structure":"chain base","Backbone":"Vicuna-7B"}. This record does not assert independent reproduction.

### REST

Average Length: 1.5045. Result ID: 6a0427b447f34ce924e9a899.

[Reporting paper](<https://www.sota2.com/research/paper/6a040579dbc836b87d827444.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa72cd1e209131cea588ee.md>)

Recorded evaluation configuration: {"Decoding Strategy":"Greedy","Decoding Structure":"chain base","Backbone":"Vicuna-7B"}. This record does not assert independent reproduction.

### EAGLE

Average Length: 1.48. Result ID: 69b7c5bc47f34ce9249f8e4a.

[Reporting paper](<https://www.sota2.com/research/paper/69b78e5b0636e5bf4afbda1d.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa77391e209131cea71c7d.md>)

Recorded evaluation configuration: {"Target model":"lmsys / Vicuna-7B-v1.3"}. This record does not assert independent reproduction.

### OSD-EAGLE

Average Length: 1.45. Result ID: 69b7c5bc47f34ce9249f8e56.

[Reporting paper](<https://www.sota2.com/research/paper/69b78e5b0636e5bf4afbda1d.md>)

Recorded evaluation configuration: {"Target model":"meta-llama / Llama-2-7B-Chat"}. This record does not assert independent reproduction.

### OSD

Average Length: 1.4. Result ID: 69b7c5bc47f34ce9249f8e51.

[Reporting paper](<https://www.sota2.com/research/paper/69b78e5b0636e5bf4afbda1d.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73851e209131cea5ce69.md>)

Recorded evaluation configuration: {"Target model":"meta-llama / Llama-2-7B-Chat"}. This record does not assert independent reproduction.

### Vanilla SD

Average Length: 1.25. Result ID: 69b7c5bc47f34ce9249f8e45.

[Reporting paper](<https://www.sota2.com/research/paper/69b78e5b0636e5bf4afbda1d.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6f6f1e209131cea458d8.md>)

Recorded evaluation configuration: {"Target model":"lmsys / Vicuna-7B-v1.3"}. This record does not assert independent reproduction.

### Vanilla SD

Average Length: 1.25. Result ID: 69b7c5bc47f34ce9249f8e50.

[Reporting paper](<https://www.sota2.com/research/paper/69b78e5b0636e5bf4afbda1d.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6f6f1e209131cea458d8.md>)

Recorded evaluation configuration: {"Target model":"meta-llama / Llama-2-7B-Chat"}. This record does not assert independent reproduction.

### EAGLE

Average Length: 1.19. Result ID: 69b7c5bc47f34ce9249f8e55.

[Reporting paper](<https://www.sota2.com/research/paper/69b78e5b0636e5bf4afbda1d.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa77391e209131cea71c7d.md>)

Recorded evaluation configuration: {"Target model":"meta-llama / Llama-2-7B-Chat"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.
