Command Generation on CmdGen-AMD (test)
89.23Cmd SimVelocitune
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| VelocituneBackbone=Llama-3 8B2024.11 | 89.23 | 72.97 | 74.36 | 49.74 | 31.28 | 54.87 | |
| DBLBackbone=Llama-3 8B2024.11 | 88.8 | 68 | 69.23 | 39.49 | 27.69 | 45.64 | |
| BaselineBackbone=Llama-3 8B2024.11 | 88.69 | 69.6 | 70.26 | 46.15 | 29.23 | 51.79 | |
| Llama-3Backbone=Llama-3 8B, Configuration=Base2024.11 | 88.08 | 71.22 | 67.18 | 41.54 | 27.69 | 47.18 | |
| BaselineBackbone=Mistral-7B2024.11 | 87.58 | 65.72 | 70.77 | 25.13 | 18.46 | 30.77 | |
| VelocituneBackbone=Mistral-7B2024.11 | 86.84 | 62.99 | 69.23 | 27.18 | 18.46 | 33.33 | |
| DBLBackbone=Mistral-7B2024.11 | 84.61 | 67.37 | 65.13 | 26.15 | 17.95 | 32.82 | |
| MistralBackbone=Mistral-7B, Configuration=Base2024.11 | 84.25 | 59.49 | 61.54 | 25.64 | 15.9 | 29.23 |