Python Programming on HumanEval (test)
10.45BE ScoreSpecTr-GBV
Evaluation results
| Method | Links | ||
|---|---|---|---|
| SpecTr-GBVTarget Model=DeepSeek-33B, Draft Model=DeepSeek-1.3B, Draft length (L)=12, Temperature (T)=0.4, Draft number (K)=32026.04 | 10.45 | 2.5 | |
| SpecTrTarget Model=DeepSeek-33B, Draft Model=DeepSeek-1.3B, Draft length (L)=12, Temperature (T)=0.4, Draft number (K)=32026.04 | 10.25 | 2.21 | |
| GBVTarget Model=DeepSeek-33B, Draft Model=DeepSeek-1.3B, Draft length (L)=12, Temperature (T)=0.4, Draft number (K)=32026.04 | 9.6 | 2.03 | |
| SDTarget Model=DeepSeek-33B, Draft Model=DeepSeek-1.3B, Draft length (L)=12, Temperature (T)=0.4, Draft number (K)=32026.04 | 9.53 | 2.02 | |
| SpecTr-GBVTarget Model=DeepSeek-6.7B, Draft Model=DeepSeek-1.3B, Draft length (L)=8, Temperature (T)=0.4, Draft number (K)=32026.04 | 7.7 | 1.28 | |
| SpecTrTarget Model=DeepSeek-6.7B, Draft Model=DeepSeek-1.3B, Draft length (L)=8, Temperature (T)=0.4, Draft number (K)=32026.04 | 7.53 | 1.15 | |
| SDTarget Model=DeepSeek-6.7B, Draft Model=DeepSeek-1.3B, Draft length (L)=8, Temperature (T)=0.4, Draft number (K)=32026.04 | 7.06 | 1.19 | |
| GBVTarget Model=DeepSeek-6.7B, Draft Model=DeepSeek-1.3B, Draft length (L)=8, Temperature (T)=0.4, Draft number (K)=32026.04 | 7.06 | 1.19 | |
| ARTarget Model=DeepSeek-33B, Draft Model=DeepSeek-1.3B, Draft length (L)=12, Temperature (T)=0.4, Draft number (K)=32026.04 | 1 | 1 | |
| ARTarget Model=DeepSeek-6.7B, Draft Model=DeepSeek-1.3B, Draft length (L)=8, Temperature (T)=0.4, Draft number (K)=32026.04 | 1 | 1 |