Next token prediction on py150
67.2MRRTravTrans
Evaluation Results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| TravTransApplication=Function parameter name2020.03 | 67.2 | 7.7 | — | — | — | — | — | |
| TravTransApplication=Name (variable, module)2020.03 | 66.6 | 15.5 | — | — | — | — | — | |
| TravTransApplication=Numeric constant2020.03 | 63.5 | 10.3 | — | — | — | — | — | |
| TravTransApplication=Attribute access2020.03 | 60.5 | 19.3 | — | — | — | — | — | |
| PointerMixtureApplication=Function parameter name2020.03 | 59.7 | 7.7 | — | — | — | — | — | |
| TravTransArchitecture=Attention, Input Representation=AST2020.03 | 58 | — | — | — | — | — | — | |
| TravTransApplication=All leaf nodes2020.03 | 58 | 21.1 | — | — | — | — | — | |
| PathTransArchitecture=Attention, Input Representation=AST2020.03 | 55.1 | — | — | — | — | — | — | |
| SeqTransArchitecture=Transformer, Input Representation=Sequence2020.03 | 54.9 | — | — | — | — | — | — | |
| PointerMixtureApplication=Attribute access2020.03 | 54.2 | 19.3 | — | — | — | — | — | |
| PointerMixtureApplication=All leaf nodes2020.03 | 51.9 | 21.1 | — | — | — | — | — | |
| PointerMixtureApplication=Numeric constant2020.03 | 50.6 | 10.3 | — | — | — | — | — | |
| PointerMixtureApplication=Name (variable, module)2020.03 | 49.5 | 15.5 | — | — | — | — | — | |
| Deep3Architecture=Decision tree, Input Representation=AST2020.03 | 43.9 | — | — | — | — | — | — | |
| Code2SeqArchitecture=Attention, Input Representation=AST2020.03 | 43.7 | — | — | — | — | — | — | |
| SeqRNNArchitecture=RNN, Input Representation=Sequence2020.03 | 36.6 | — | — | — | — | — | — | |
| Code2Seq2020.03 | — | — | 39.3 | 49.5 | 45.8 | 56.8 | 43.7 | |
| Deep32020.03 | — | — | 45.3 | 53.2 | 48.9 | 58.1 | 43.9 | |
| PathTrans2020.03 | — | — | 57.2 | 59.1 | 63.5 | 65.8 | 55.1 | |
| TravTrans2020.03 | — | — | 60.5 | 63.5 | 66.6 | 67.2 | 58 |