Command Generation on CmdGen-NVIDIA (test)
89.37Cmd SimVelocitune
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| VelocituneBackbone=Llama-3 8B2024.11 | 89.37 | 75.59 | 76.59 | 51.21 | 39.02 | 61.95 | |
| BaselineBackbone=Llama-3 8B2024.11 | 87.53 | 72.15 | 69.27 | 46.34 | 37.07 | 57.07 | |
| Llama-3Backbone=Llama-3 8B, Configuration=Base2024.11 | 86.41 | 69.09 | 64.39 | 41.95 | 32.68 | 50.73 | |
| DBLBackbone=Llama-3 8B2024.11 | 83.83 | 64.87 | 63.9 | 38.54 | 27.8 | 45.85 | |
| BaselineBackbone=Mistral-7B2024.11 | 83.36 | 65.26 | 66.34 | 23.9 | 21.46 | 32.2 | |
| VelocituneBackbone=Mistral-7B2024.11 | 82.85 | 64.3 | 68.29 | 27.32 | 21.95 | 36.59 | |
| MistralBackbone=Mistral-7B, Configuration=Base2024.11 | 80.57 | 58.67 | 61.95 | 24.88 | 19.02 | 30.73 | |
| DBLBackbone=Mistral-7B2024.11 | 79.83 | 65.97 | 59.02 | 24.39 | 19.02 | 32.2 |