Code Generation on Python dataset (test)
16.66BLEUMAT
Evaluation results
| Method | Links | ||
|---|---|---|---|
| MATρ=0.1, Na=4, d=256, dh=1024, number of parameters=37M2020.06 | 16.66 | 79.9 | |
| MATρ=0, Na=4, d=256, dh=1024, number of parameters=37M2020.06 | 16.35 | 65.5 | |
| MATρ=0.2, Na=4, d=256, dh=1024, number of parameters=37M2020.06 | 15.63 | 84 | |
| Transformerd=512, dh=1024, number of parameters=55M2020.06 | 15.49 | 73.3 | |
| MATρ=0.3, Na=4, d=256, dh=1024, number of parameters=37M2020.06 | 12.85 | 75.3 | |
| Dual2020.06 | 12.09 | 51.9 |