# DiffusionHarmonizer: Bridging Neural Reconstruction and Photorealistic Simulation with Online Diffusion Enhancer Source: [DiffusionHarmonizer: Bridging Neural Reconstruction and Photorealistic Simulation with Online Diffusion Enhancer]() Collection index: [llms.txt]() Data updated: 2026-03-07T05:00:04.447Z Authors: Yuxuan Zhang, Katar\\'ina T\\'othov\\'a, Zian Wang, Kangxue Yin, Haithem Turki, Riccardo de Lutio, Yen-Yu Chang, Or Litany, Sanja Fidler, Zan Gojcic Published: 2026-02-27T15:35:30.000Z ## Abstract Simulation is essential to the development and evaluation of autonomous robots such as self-driving vehicles. Neural reconstruction is emerging as a promising solution as it enables simulating a wide variety of scenarios from real-world data alone in an automated and scalable way. However, while methods such as NeRF and 3D Gaussian Splatting can produce visually compelling results, they often exhibit artifacts particularly when rendering novel views, and fail to realistically integrate inserted dynamic objects, especially when they were captured from different scenes. To overcome these limitations, we introduce DiffusionHarmonizer, an online generative enhancement framework that transforms renderings from such imperfect scenes into temporally consistent outputs while improving their realism. At its core is a single-step temporally-conditioned enhancer that is converted from a pretrained multi-step image diffusion model, capable of running in online simulators on a single GPU. The key to training it effectively is a custom data curation pipeline that constructs synthetic-real pairs emphasizing appearance harmonization, artifact correction, and lighting realism. The result is a scalable system that significantly elevates simulation fidelity in both research and production environments. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [ISP Modification on ISP Modification Data Videos (Holdout Set)](): ISP Modification; ISP Modification Data Videos (Holdout Set); PSNR: 28.1. - [Novel Trajectory Simulation on Novel Trajectory Simulation In-domain (test)](): Novel Trajectory Simulation; Novel Trajectory Simulation In-domain (test); FID: 120.23. - [Object Insertion Simulation on Object Insertion Simulation Out-of-domain (test)](): Object Insertion Simulation; Object Insertion Simulation Out-of-domain (test); FID: 101.27. - [Shadow Simulation on PBR Shadow Data Videos (Holdout Set)](): Shadow Simulation; PBR Shadow Data Videos (Holdout Set); PSNR: 26.31. - [Generative Video Enhancement on Novel Trajectory and Object Insertion Simulation (test)](): Generative Video Enhancement; Novel Trajectory and Object Insertion Simulation (test); Human Eval (%): 90.11. - [Harmonization on ISP Modification Data (Holdout Set)](): Harmonization; ISP Modification Data (Holdout Set); PSNR: 28.58. - [Relighting on Relighting Data Images (Holdout Set)](): Relighting; Relighting Data Images (Holdout Set); PSNR: 23.93. ## Original sources and implementation [Original paper]() [PDF]()