Natural Language to Bash Commands Translation on NLC2CMD Challenge (Leaderboard)
53.2AccuracyMagnum
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| MagnumModel=Transformer, Data Augment=No2023.02 | 53.2 | 682.3 | 0.709 | |
| HubrisModel=GPT-2, Data Augment=No2023.02 | 51.3 | 809.6 | 14.87 | |
| JbModel=Classifier+Transformer, Data Augment=Yes2023.02 | 49.9 | 828.9 | 3.142 | |
| AICoreModel=Two-stage Transformer, Data Augment=No2023.02 | 48.9 | 596.9 | 0.423 | |
| TellinaModel=BRNN (GRU), Data Augment=No2023.02 | 13.8 | 916.1 | 3.242 |