Mathematical Reasoning on GSM8K (Acceptance Rate & Speedup Ratio)
79.3Acceptance RateGSD
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| GSDTarget Model=OPT-13b, Draft Model=OPT-350m, k (maximum out-degree)=4, tau (redundant node threshold)=1, theta_prob=0.4, theta_sib=0.1, Decoding Setting=Deterministic2024.07 | 79.3 | 1.22 | |
| GSDTarget Model=LLaMA-2-7b, Draft Model=LLaMA-160m, k=4, tau=1, theta_prob=0.4, theta_sib=0.12024.07 | 79.3 | 1.16 | |
| TSDTarget Model=LLaMA-2-7b, Draft Model=LLaMA-160m, k=4, tau=1, theta_prob=0.4, theta_sib=0.12024.07 | 77.5 | 0.6 | |
| TSDTarget Model=OPT-13b, Draft Model=OPT-350m, k (maximum out-degree)=4, tau (redundant node threshold)=1, theta_prob=0.4, theta_sib=0.1, Decoding Setting=Deterministic2024.07 | 77 | 0.62 | |
| GSDTarget Model=BLOOM-7b1, Draft Model=BLOOM-560m, k (maximum out-degree)=4, tau (threshold for redundant node)=1, theta_prob=0.4, theta_sib=0.12024.07 | 75.5 | 1.26 | |
| TSDTarget Model=BLOOM-7b1, Draft Model=BLOOM-560m, k (maximum out-degree)=4, tau (threshold for redundant node)=1, theta_prob=0.4, theta_sib=0.12024.07 | 74.1 | 0.32 | |
| SSDTarget Model=BLOOM-7b1, Draft Model=BLOOM-560m, k (maximum out-degree)=4, tau (threshold for redundant node)=1, theta_prob=0.4, theta_sib=0.12024.07 | 65.3 | 1.18 | |
| SSDTarget Model=OPT-13b, Draft Model=OPT-350m, k (maximum out-degree)=4, tau (redundant node threshold)=1, theta_prob=0.4, theta_sib=0.1, Decoding Setting=Deterministic2024.07 | 60.2 | 1.08 | |
| SSDTarget Model=LLaMA-2-7b, Draft Model=LLaMA-160m, k=4, tau=1, theta_prob=0.4, theta_sib=0.12024.07 | 60.1 | 1.04 |