General Language Understanding on MMLU (test)
48.09Throughput (Tokens/sec)SelfJudge-F
Evaluation results
| Method | Links | ||
|---|---|---|---|
| SelfJudge-FModel (Target/Draft)=Llama-3.1-70B/8B, Hardware=4 A100 GPUs, Tensor Parallelism=4-way, Draft length (gamma)=152025.09 | 48.09 | 79.8 | |
| SelfJudge-RModel (Target/Draft)=Llama-3.1-70B/8B, Hardware=4 A100 GPUs, Tensor Parallelism=4-way, Draft length (gamma)=152025.09 | 43.3 | 81.4 | |
| SDModel (Target/Draft)=Llama-3.1-70B/8B, Hardware=4 A100 GPUs, Tensor Parallelism=4-way, Draft length (gamma)=152025.09 | 39.19 | 80.6 |