Type Inference on DeepTyper TypeScript (test)
54.01Acc@1DeepTyper BiLSTM
Evaluation results
| Method | Links | ||
|---|---|---|---|
| DeepTyper BiLSTMPre-training Objective=ContraCode2020.07 | 54.01 | 85.55 | |
| DeepTyper BiLSTMPre-training Objective=None2020.07 | 51.73 | 82.71 | |
| DeepTyper BiLSTMPre-training Objective=ROBERTa MLM2020.07 | 50.24 | 82.85 | |
| TransformerPre-training Objective=ContraCode + MLM (hybrid)2020.07 | 47.16 | 81.44 | |
| TransformerPre-training Objective=ContraCode2020.07 | 46.86 | 81.85 | |
| TransformerPre-training Objective=None2020.07 | 45.66 | 80.08 | |
| TypeScript CheckJSType=Static Analysis2020.07 | 45.11 | — | |
| TransformerPre-training Objective=ROBERTa MLM2020.07 | 40.85 | 75.76 | |
| GPT-3 CodexPrompting Setup=few-shot, Model Scale=175B2020.07 | 30.55 | — | |
| DeepTyperInput Constraints=variable name only2020.07 | 28.94 | 70.07 | |
| GPT-3 CodexPrompting Setup=zero-shot, Model Scale=175B2020.07 | 26.62 | — |