Code Generation on Django (test)
83AccuracyCOARSE2FINE
Evaluation results
| Method | Links | ||
|---|---|---|---|
| COARSE2FINEoracle_sketch=true2018.05 | 83 | — | |
| COARSE2FINEdecoding_mode=coarse-to-fine2018.05 | 74.1 | — | |
| TRANXparent feeding=with2018.10 | 73.7 | — | |
| TRANXparent feeding=without2018.10 | 72.7 | — | |
| COARSE2FINEsketch_encoder=removed2018.05 | 72.1 | — | |
| SNM+COPY2018.05 | 71.6 | — | |
| YN172018.10 | 71.6 | — | |
| ONESTAGEdecoding_mode=one stage without sketches2018.05 | 69.5 | — | |
| Latent Predictor Networkcomponents=full system2016.03 | 62.3 | 77.6 | |
| LPN+COPY2018.05 | 62.3 | — | |
| LPN2018.10 | 62.3 | — | |
| Latent Predictor Networkablation=model compression removed2016.03 | 61.3 | 76.3 | |
| Latent Predictor Networkablation=C2W removed2016.03 | 60.9 | 75.9 | |
| SEQ2SEQ+UNK replacement2018.05 | 45.1 | — | |
| NMT2018.10 | 45.1 | — | |
| Latent Predictor Networkablation=pointer networks (LPN) removed2016.03 | 40.8 | 63.3 | |
| SEQ2TREE+UNK replacement2018.05 | 39.4 | — | |
| SEQ2TREE2018.10 | 39.4 | — | |
| Attention (Baseline)2016.03 | 38.8 | 58.9 | |
| Latent Predictor Networkablation=attention component removed2016.03 | 34.5 | 48.8 | |
| Sequence2016.03 | 33.2 | 44.1 | |
| Phrase2016.03 | 31.5 | 47.6 | |
| Phrasal SMT2018.05 | 31.5 | — | |
| Phrasal Statistical MT2018.10 | 31.5 | — | |
| Retrieval2016.03 | 14.7 | 18.6 | |
| Retrieval System2018.05 | 14.7 | — | |
| Hierarchical2016.03 | 9.5 | 35.9 | |
| Hierarchical SMT2018.05 | 9.5 | — |