Mathematical Reasoning on GSM8K (test) (Accuracy, Tokens/sec)
65.4Throughput (Tokens/sec)SelfJudge-F
Evaluation results
| Method | Links | ||
|---|---|---|---|
| SelfJudge-FModel (Target/Draft)=Llama-3.1-70B/8B, Hardware=4 A100 GPUs, Tensor Parallelism=4-way, Draft length (gamma)=202025.09 | 65.4 | 94 | |
| SelfJudge-RModel (Target/Draft)=Llama-3.1-70B/8B, Hardware=4 A100 GPUs, Tensor Parallelism=4-way, Draft length (gamma)=202025.09 | 59.57 | 93.8 | |
| SDModel (Target/Draft)=Llama-3.1-70B/8B, Hardware=4 A100 GPUs, Tensor Parallelism=4-way, Draft length (gamma)=202025.09 | 53.45 | 93 |