Program Synthesis on VizDoom (test)
62.22Exact MatchEVAPS
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| EVAPSTop-k=Top-202023.12 | 62.22 | 79.2 | 76.84 | |
| TransformerTop-k=Top-202023.12 | 51.53 | 64.69 | 63.09 | |
| SEDTop-k=Top-202023.12 | 51.23 | 71.42 | 67.06 | |
| EVAPSTop-k=Top-52023.12 | 50.62 | 69.82 | 67.82 | |
| Inferred TraceTop-k=Top-202023.12 | 37.75 | 66.69 | 63.96 | |
| EVAPSTop-k=Top-12023.12 | 36.4 | 50.29 | 48.9 | |
| TransformerTop-k=Top-52023.12 | 34.8 | 47.96 | 46.76 | |
| Inferred TraceTop-k=Top-52023.12 | 27.49 | 55.13 | 52.8 | |
| SEDTop-k=Top-52023.12 | 25.48 | 66.76 | 62.25 | |
| TransformerTop-k=Top-12023.12 | 15.53 | 26.51 | 25.93 | |
| Inferred TraceTop-k=Top-12023.12 | 14.73 | 36.87 | 35.09 | |
| SEDTop-k=Top-12023.12 | 12.69 | 43.07 | 38.63 | |
| LGRLTop-k=Top-202023.12 | 9.16 | 66.22 | 63.49 | |
| Latent ExecutionTop-k=Top-202023.12 | 9.16 | 59.93 | 57.53 | |
| Latent ExecutionTop-k=Top-52023.12 | 4.91 | 50.76 | 48.62 | |
| LGRLTop-k=Top-52023.12 | 4.58 | 54.25 | 51.56 | |
| Latent ExecutionTop-k=Top-12023.12 | 3.53 | 44.4 | 42.25 | |
| LGRLTop-k=Top-12023.12 | 2.18 | 33.42 | 31.49 |