# Program Synthesis on VizDoom (test) Source: [Program Synthesis on VizDoom (test)]() Collection index: [llms.txt]() Data updated: 2026-02-26T05:42:27.152Z Evaluation of models performing Program Synthesis on the test split of the VizDoom dataset, measured using Exact Match, Semantic Match, and Generalization metrics. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Program Synthesis | | Dataset (including any stated split) | VizDoom (test) | | Metric | Exact Match | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 18 of 18 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Exact Match](): Higher is better; unit: PERCENT. - [Semantic Match](): Higher is better; unit: PERCENT. - [Generalization](): Higher is better; unit: PERCENT. ## Recorded results | Method | Exact Match | Result date | Report paper | | --- | --- | --- | --- | | EVAPS | 62.22 | Not recorded | 68fa71801e209131cea51aa6 | | Transformer | 51.53 | Not recorded | 68fa71801e209131cea51aa6 | | SED | 51.23 | Not recorded | 68fa71801e209131cea51aa6 | | EVAPS | 50.62 | Not recorded | 68fa71801e209131cea51aa6 | | Inferred Trace | 37.75 | Not recorded | 68fa71801e209131cea51aa6 | | EVAPS | 36.4 | Not recorded | 68fa71801e209131cea51aa6 | | Transformer | 34.8 | Not recorded | 68fa71801e209131cea51aa6 | | Inferred Trace | 27.49 | Not recorded | 68fa71801e209131cea51aa6 | | SED | 25.48 | Not recorded | 68fa71801e209131cea51aa6 | | Transformer | 15.53 | Not recorded | 68fa71801e209131cea51aa6 | | Inferred Trace | 14.73 | Not recorded | 68fa71801e209131cea51aa6 | | SED | 12.69 | Not recorded | 68fa71801e209131cea51aa6 | | LGRL | 9.16 | Not recorded | 68fa71801e209131cea51aa6 | | Latent Execution | 9.16 | Not recorded | 68fa71801e209131cea51aa6 | | Latent Execution | 4.91 | Not recorded | 68fa71801e209131cea51aa6 | | LGRL | 4.58 | Not recorded | 68fa71801e209131cea51aa6 | | Latent Execution | 3.53 | Not recorded | 68fa71801e209131cea51aa6 | | LGRL | 2.18 | Not recorded | 68fa71801e209131cea51aa6 | ### EVAPS Exact Match: 62.22. Result ID: 6991ec2a3a6c3148490b772b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Top-k":"Top-20"}. This record does not assert independent reproduction. ### Transformer Exact Match: 51.53. Result ID: 6991ec2a3a6c3148490b772a. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Top-k":"Top-20"}. This record does not assert independent reproduction. ### SED Exact Match: 51.23. Result ID: 6991ec2a3a6c3148490b7727. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Top-k":"Top-20"}. This record does not assert independent reproduction. ### EVAPS Exact Match: 50.62. Result ID: 6991ec2a3a6c3148490b7725. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Top-k":"Top-5"}. This record does not assert independent reproduction. ### Inferred Trace Exact Match: 37.75. Result ID: 6991ec2a3a6c3148490b7728. [Reporting paper]() Recorded evaluation configuration: {"Top-k":"Top-20"}. This record does not assert independent reproduction. ### EVAPS Exact Match: 36.4. Result ID: 6991ec2a3a6c3148490b771f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Top-k":"Top-1"}. This record does not assert independent reproduction. ### Transformer Exact Match: 34.8. Result ID: 6991ec2a3a6c3148490b7724. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Top-k":"Top-5"}. This record does not assert independent reproduction. ### Inferred Trace Exact Match: 27.49. Result ID: 6991ec2a3a6c3148490b7722. [Reporting paper]() Recorded evaluation configuration: {"Top-k":"Top-5"}. This record does not assert independent reproduction. ### SED Exact Match: 25.48. Result ID: 6991ec2a3a6c3148490b7721. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Top-k":"Top-5"}. This record does not assert independent reproduction. ### Transformer Exact Match: 15.53. Result ID: 6991ec2a3a6c3148490b771e. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Top-k":"Top-1"}. This record does not assert independent reproduction. ### Inferred Trace Exact Match: 14.73. Result ID: 6991ec2a3a6c3148490b771c. [Reporting paper]() Recorded evaluation configuration: {"Top-k":"Top-1"}. This record does not assert independent reproduction. ### SED Exact Match: 12.69. Result ID: 6991ec2a3a6c3148490b771b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Top-k":"Top-1"}. This record does not assert independent reproduction. ### LGRL Exact Match: 9.16. Result ID: 6991ec2a3a6c3148490b7726. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Top-k":"Top-20"}. This record does not assert independent reproduction. ### Latent Execution Exact Match: 9.16. Result ID: 6991ec2a3a6c3148490b7729. [Reporting paper]() Recorded evaluation configuration: {"Top-k":"Top-20"}. This record does not assert independent reproduction. ### Latent Execution Exact Match: 4.91. Result ID: 6991ec2a3a6c3148490b7723. [Reporting paper]() Recorded evaluation configuration: {"Top-k":"Top-5"}. This record does not assert independent reproduction. ### LGRL Exact Match: 4.58. Result ID: 6991ec2a3a6c3148490b7720. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Top-k":"Top-5"}. This record does not assert independent reproduction. ### Latent Execution Exact Match: 3.53. Result ID: 6991ec2a3a6c3148490b771d. [Reporting paper]() Recorded evaluation configuration: {"Top-k":"Top-1"}. This record does not assert independent reproduction. ### LGRL Exact Match: 2.18. Result ID: 6991ec2a3a6c3148490b771a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Top-k":"Top-1"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.