# Reference-based Controllable Scene Stylization with Gaussian Splatting

Source: [Reference-based Controllable Scene Stylization with Gaussian Splatting](<https://www.sota2.com/research/paper/68fa74111e209131cea5f994>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-01-02T06:43:59.118Z

Authors: Yiqun Mei, Jiacong Xu, Vishal M. Patel

Published: 2024-07-09T20:30:29.000Z

## Abstract

Referenced-based scene stylization that edits the appearance based on a content-aligned reference image is an emerging research area. Starting with a pretrained neural radiance field (NeRF), existing methods typically learn a novel appearance that matches the given style. Despite their effectiveness, they inherently suffer from time-consuming volume rendering, and thus are impractical for many real-time applications. In this work, we propose ReGS, which adapts 3D Gaussian Splatting (3DGS) for reference-based stylization to enable real-time stylized view synthesis. Editing the appearance of a pretrained 3DGS is challenging as it uses discrete Gaussians as 3D representation, which tightly bind appearance with geometry. Simply optimizing the appearance as prior methods do is often insufficient for modeling continuous textures in the given reference image. To address this challenge, we propose a novel texture-guided control mechanism that adaptively adjusts local responsible Gaussians to a new geometric arrangement, serving for desired texture details. The proposed process is guided by texture clues for effective appearance editing, and regularized by scene depth for preserving original geometric structure. With these novel designs, we show ReGs can produce state-of-the-art stylization results that respect the reference texture while embracing real-time rendering speed for free-view navigation.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Scene Stylization on Synthetic, LLFF, and Tanks & Temples (test)](<https://www.sota2.com/research/sota/scene-stylization-on-synthetic-llff-and-tanks-and-temples-test.md>): Scene Stylization; Synthetic, LLFF, and Tanks & Temples (test); Ref-LPIPS: 0.202.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2407.07220>)

[PDF](<https://arxiv.org/pdf/2407.07220.pdf>)