Conversation on MT-Bench (Speedup and Tau Metrics)
3.87SpeedupDFLARE
Evaluation results
| Method | Links | ||
|---|---|---|---|
| DFLAREModel=GPT-20B, Temperature=02026.06 | 3.87 | 5.32 | |
| DFlashModel=GPT-20B, Temperature=02026.06 | 3.74 | 5.2 | |
| DFLAREModel=Q3-4B, Temperature=02026.06 | 3.33 | 5.3 | |
| DFLAREModel=Q3-8B, Temperature=02026.06 | 3.23 | 5.09 | |
| DFlashModel=Q3-4B, Temperature=02026.06 | 3.13 | 4.4 | |
| DFLAREModel=Q3-4B, Temperature=12026.06 | 3 | 4.77 | |
| DFlashModel=Q3-8B, Temperature=02026.06 | 2.85 | 4.3 | |
| DFLAREModel=Q3-8B, Temperature=12026.06 | 2.81 | 4.37 | |
| DFlashModel=Q3-4B, Temperature=12026.06 | 2.69 | 4.03 | |
| DFlashModel=Q3-8B, Temperature=12026.06 | 2.57 | 3.79 | |
| EAGLE3Model=Q3-4B, Temperature=02026.06 | 2.39 | 3.57 | |
| EAGLE3Model=Q3-4B, Temperature=12026.06 | 2.22 | 3.49 | |
| EAGLE3Model=Q3-8B, Temperature=02026.06 | 2.2 | 3.25 | |
| EAGLE3Model=Q3-8B, Temperature=12026.06 | 1.99 | 3.06 | |
| DFLAREModel=GPT-20B, Temperature=12026.06 | 1.81 | 2.89 | |
| DFlashModel=GPT-20B, Temperature=12026.06 | 1.76 | 2.76 |