https://arxiv.org/api/hEOoPWb5wtOKIEWFCo7UDaCnBes 2026-09-27T19:43:13Z 40 14 0 http://arxiv.org/abs/2606.27604v1 2026-06-25T23:30:57Z

In this paper we present a spectral optical inversion for skin for path tracing-based rendering of subsurface scattering. Skin is a complex multilayered medium, with appearance determined by the mixture of biophysical chromophores. However, current methods rely on medium homogeneization, with optical parameters obtained via albedo inversion from a reflectance texture and hand-tuned scattering distance and anisotropy. This results into significant art-skilled manual labor for authoring, and an inaccurate scattering profile for skin. To solve these problems, we generalize existing albedo inversion techniques, and propose a framework that predicts full-spectral skin scattering parameters from a single RGB diffuse albedo. Our method builds upon a new mixture-of-media representation, that approximates the aggregated multilayered appearance of skin by mixing the aggregated scattering of three uncorrelated media. We train a chained neural decoder that maps RGB diffuse albedo to the optical properties of the mixture of media, including anisotropy, scattering radius and scattering albedo. Then, we show this mixture can be used in a random-walk-based path tracer with minimal modifications, by simply randomly selecting the medium to traverse.

2026-06-25T23:30:57Z 14 pages, 9 figures Carlos Aliaga Adrian Jarabo http://arxiv.org/abs/2604.25178v1 2026-04-28T03:37:24Z

Achieving a desirable balance between rendering quality and real-time performance is a long-standing challenge in modern game and rendering engines, particularly on resource-constrained mobile devices such as laptops, tablets, and smartphones. Existing approaches to automatic rendering parameter optimization either depend on exhaustive per-scene pre-computation that spans several days, suffer from the prohibitive inference overhead of neural networks that prevents per-frame adaptation, or lack generalizability across heterogeneous hardware and diverse scenes. In this paper, we propose \textbf{LUT-Opt}, a lightweight, general-purpose framework for adaptive per-frame rendering parameter optimization. Our method decomposes the joint optimization of rendering time and image quality into a tractable two-stage pipeline. In the offline stage, we train a pair of XGBoost regressors to predict rendering time and image quality from rendering parameters, hardware state, and scene complexity descriptors. The trained ensemble models are then distilled into compact lookup tables (LUTs) through systematic discretization and a two-phase linear search that first constrains rendering time and subsequently maximizes structural similarity (SSIM). During runtime, the pre-computed LUT is queried every frame in sub-millisecond time, enabling truly adaptive parameter selection with negligible computational overhead. We validate LUT-Opt on two representative rendering techniques -- subsurface scattering (SSS) and hybrid-pipeline ambient occlusion (AO) -- implemented within Unreal Engine 5. Extensive experiments across multiple scenes and GPU configurations demonstrate that LUT-Opt reduces subsurface scattering rendering time by approximately 40\% and ambient occlusion rendering time by roughly 70\%, while incurring only about 2\% increase in image quality error, with per-frame inference latency below 0.1\ ms.

2026-04-28T03:37:24Z Baijun Tan Francesco Moretti http://arxiv.org/abs/2601.12020v1 2026-01-17T12:02:14Z

Subsurface scattering (SSS) gives translucent materials -- such as wax, jade, marble, and skin -- their characteristic soft shadows, color bleeding, and diffuse glow. Modeling these effects in neural rendering remains challenging due to complex light transport and the need for densely captured multi-view, multi-light datasets (often more than 100 views and 112 OLATs). We present DIAMOND-SSS, a data-efficient framework for high-fidelity translucent reconstruction from extremely sparse supervision -- even as few as ten images. We fine-tune diffusion models for novel-view synthesis and relighting, conditioned on estimated geometry and trained on less than 7 percent of the dataset, producing photorealistic augmentations that can replace up to 95 percent of missing captures. To stabilize reconstruction under sparse or synthetic supervision, we introduce illumination-independent geometric priors: a multi-view silhouette consistency loss and a multi-view depth consistency loss. Across all sparsity regimes, DIAMOND-SSS achieves state-of-the-art quality in relightable Gaussian rendering, reducing real capture requirements by up to 90 percent compared to SSS-3DGS.

2026-01-17T12:02:14Z Guillermo Figueroa-Araneda Iris Diana Jimenez Florian Hofherr Manny Ko Hector Andrade-Loarca Daniel Cremers http://arxiv.org/abs/2510.15846v1 2025-10-17T17:37:42Z

Rendering novel, relit views of a human head, given a monocular portrait image as input, is an inherently underconstrained problem. The traditional graphics solution is to explicitly decompose the input image into geometry, material and lighting via differentiable rendering; but this is constrained by the multiple assumptions and approximations of the underlying models and parameterizations of these scene components. We propose 3DPR, an image-based relighting model that leverages generative priors learnt from multi-view One-Light-at-A-Time (OLAT) images captured in a light stage. We introduce a new diverse and large-scale multi-view 4K OLAT dataset of 139 subjects to learn a high-quality prior over the distribution of high-frequency face reflectance. We leverage the latent space of a pre-trained generative head model that provides a rich prior over face geometry learnt from in-the-wild image datasets. The input portrait is first embedded in the latent manifold of such a model through an encoder-based inversion process. Then a novel triplane-based reflectance network trained on our lightstage data is used to synthesize high-fidelity OLAT images to enable image-based relighting. Our reflectance network operates in the latent space of the generative head model, crucially enabling a relatively small number of lightstage images to train the reflectance model. Combining the generated OLATs according to a given HDRI environment maps yields physically accurate environmental relighting results. Through quantitative and qualitative evaluations, we demonstrate that 3DPR outperforms previous methods, particularly in preserving identity and in capturing lighting effects such as specularities, self-shadows, and subsurface scattering. Project Page: https://vcai.mpi-inf.mpg.de/projects/3dpr/

2025-10-17T17:37:42Z Accepted at ACM SIGGRAPH ASIA 2025 Conference Proceedings Pramod Rao Abhimitra Meka Xilong Zhou Gereon Fox Mallikarjun B R Fangneng Zhan Tim Weyrich Bernd Bickel Hanspeter Pfister Wojciech Matusik Thabo Beeler Mohamed Elgharib Marc Habermann Christian Theobalt http://arxiv.org/abs/2507.18385v1 2025-07-24T12:59:42Z

Full-body Human inverse rendering based on physically-based rendering aims to acquire high-quality materials, which helps achieve photo-realistic rendering under arbitrary illuminations. This task requires estimating multiple material maps and usually relies on the constraint of rendering result. The absence of constraints on the material maps makes inverse rendering an ill-posed task. Previous works alleviated this problem by building material dataset for training, but their simplified material data and rendering equation lead to rendering results with limited realism, especially that of skin. To further alleviate this problem, we construct a higher-quality dataset (OpenHumanBRDF) based on scanned real data and statistical material data. In addition to the normal, diffuse albedo, roughness, specular albedo, we produce displacement and subsurface scattering to enhance the realism of rendering results, especially for the skin. With the increase in prediction tasks for more materials, using an end-to-end model as in the previous work struggles to balance the importance among various material maps, and leads to model underfitting. Therefore, we design a model (HumanMaterial) with progressive training strategy to make full use of the supervision information of the material maps and improve the performance of material estimation. HumanMaterial first obtain the initial material results via three prior models, and then refine the results by a finetuning model. Prior models estimate different material maps, and each map has different significance for rendering results. Thus, we design a Controlled PBR Rendering (CPR) loss, which enhances the importance of the materials to be optimized during the training of prior models. Extensive experiments on OpenHumanBRDF dataset and real data demonstrate that our method achieves state-of-the-art performance.

2025-07-24T12:59:42Z 14 Yu Jiang Jiahao Xia Jiongming Qin Yusen Wang Tuo Cao Chunxia Xiao http://arxiv.org/abs/2501.01980v1 2024-12-29T07:57:25Z

Acquisition and modeling of polarized light reflection and scattering help reveal the shape, structure, and physical characteristics of an object, which is increasingly important in computer graphics. However, current polarimetric acquisition systems are limited to static and opaque objects. Human faces, on the other hand, present a particularly difficult challenge, given their complex structure and reflectance properties, the strong presence of spatially-varying subsurface scattering, and their dynamic nature. We present a new polarimetric acquisition method for dynamic human faces, which focuses on capturing spatially varying appearance and precise geometry, across a wide spectrum of skin tones and facial expressions. It includes both single and heterogeneous subsurface scattering, index of refraction, and specular roughness and intensity, among other parameters, while revealing biophysically-based components such as inner- and outer-layer hemoglobin, eumelanin and pheomelanin. Our method leverages such components' unique multispectral absorption profiles to quantify their concentrations, which in turn inform our model about the complex interactions occurring within the skin layers. To our knowledge, our work is the first to simultaneously acquire polarimetric and spectral reflectance information alongside biophysically-based skin parameters and geometry of dynamic human faces. Moreover, our polarimetric skin model integrates seamlessly into various rendering pipelines.

2024-12-29T07:57:25Z ACM Transactions on Graphics 43, 6, Article 275 (December 2024) Hyunho Ha Inseung Hwang Nestor Monzon Jaemin Cho Donggun Kim Seung-Hwan Baek Adolfo Muñoz Diego Gutierrez Min H. Kim 10.1145/3687767 http://arxiv.org/abs/2408.12282v2 2024-10-31T12:30:46Z

3D reconstruction and relighting of objects made from scattering materials present a significant challenge due to the complex light transport beneath the surface. 3D Gaussian Splatting introduced high-quality novel view synthesis at real-time speeds. While 3D Gaussians efficiently approximate an object's surface, they fail to capture the volumetric properties of subsurface scattering. We propose a framework for optimizing an object's shape together with the radiance transfer field given multi-view OLAT (one light at a time) data. Our method decomposes the scene into an explicit surface represented as 3D Gaussians, with a spatially varying BRDF, and an implicit volumetric representation of the scattering component. A learned incident light field accounts for shadowing. We optimize all parameters jointly via ray-traced differentiable rendering. Our approach enables material editing, relighting and novel view synthesis at interactive rates. We show successful application on synthetic data and introduce a newly acquired multi-view multi-light dataset of objects in a light-stage setup. Compared to previous work we achieve comparable or better results at a fraction of optimization and rendering time while enabling detailed control over material attributes. Project page https://sss.jdihlmann.com/

2024-08-22T10:34:01Z Project page: https://sss.jdihlmann.com/ Jan-Niklas Dihlmann Arjun Majumdar Andreas Engelhardt Raphael Braun Hendrik P. A. Lensch http://arxiv.org/abs/2406.13227v1 2024-06-19T05:28:52Z

Face retouching aims to remove facial blemishes, such as pigmentation and acne, and still retain fine-grain texture details. Nevertheless, existing methods just remove the blemishes but focus little on realism of the intermediate process, limiting their use more to beautifying facial images on social media rather than being effective tools for simulating changes in facial pigmentation and ance. Motivated by this limitation, we propose our Controllable and Gradual Face Retouching (CGFR). Our CGFR is based on physical modelling, adopting Sum-of-Gaussians to approximate skin subsurface scattering in a decomposed melanin and haemoglobin color space. Our CGFR offers a user-friendly control over the facial blemishes, achieving realistic and gradual blemishes retouching. Experimental results based on actual clinical data shows that CGFR can realistically simulate the blemishes' gradual recovering process.

2024-06-19T05:28:52Z 7 pages, 6 figures. The paper has been accepted by the IEEE Conference on Multimedia Expo 2024 Chenhao Shuai Rizhao Cai Bandara Dissanayake Amanda Newman Dayan Guan Dennis Sng Ling Li Alex Kot http://arxiv.org/abs/2312.06085v2 2025-03-15T07:51:12Z

In this study, we introduce a novel two-stage technique for decomposing and reconstructing facial features from sparse-view images, a task made challenging by the unique geometry and complex skin reflectance of each individual. To synthesize 3D facial models more realistically, we endeavor to decouple key facial attributes from the RGB color, including geometry, diffuse reflectance, and specular reflectance. Specifically, we design a Sparse-view Face Decomposition Model (SFDM): 1) In the first stage, we create a general facial template from a wide array of individual faces, encapsulating essential geometric and reflectance characteristics. 2) Guided by this template, we refine a specific facial model for each individual in the second stage, considering the interaction between geometry and reflectance, as well as the effects of subsurface scattering on the skin. With these advances, our method can reconstruct high-quality facial representations from as few as three images. The comprehensive evaluation and comparison reveal that our approach outperforms existing methods by effectively disentangling geometric and reflectance components, significantly enhancing the quality of synthesized novel views, and paving the way for applications in facial relighting and reflectance editing.

2023-12-11T03:14:58Z CVPR25 Daisheng Jin Jiangbei Hu Baixin Xu Yuxin Dai Chen Qian Ying He http://arxiv.org/abs/2311.15354v1 2023-11-26T17:09:37Z

In this work, we present a 2.5D pipeline approach to creating dynamic paintings that can be re-rendered interactively in real-time on the Web. Using this 2.5D approach, any existing simple painting such as portraits can be turned into an interactive dynamic web-based artwork. Our interactive system provides most global illumination effects such as reflection, refraction, shadow, and subsurface scattering by processing images. In our system, the scene is defined only by a set of images. These include (1) a shape image, (2) two diffuse images, (3) a background image, (4) one foreground image, and (5) one transparency image. A shape image is either a normal map or a height. Two diffuse images are usually hand-painted. They are interpolated using illumination information. The transparency image is used to define the transparent and reflective regions that can reflect the foreground image and refract the background image, both of which are also hand-drawn. This framework, which mainly uses hand-drawn images, provides qualitatively convincing painterly global illumination effects such as reflection and refraction. We also include parameters to provide additional artistic controls. For instance, using our piecewise linear Fresnel function, it is possible to control the ratio of reflection and refraction. This system is the result of a long line of research contributions. On the other hand, the art-directed Fresnel function that provides physically plausible compositing of reflection and refraction with artistic control is completely new. Art-directed warping equations that provide qualitatively convincing refraction and reflection effects with linearized artistic control are also new. You can try our web-based system for interactive dynamic real-time paintings at http://mock3d.tamu.edu/.

2023-11-26T17:09:37Z 22 pages ISEA 1 (2022) 1-12 Ergun Akleman Youyou wang Yinan Xiong Anusha Shanker Fermi Perumal Ozgur Gonen Motahareh Fard http://arxiv.org/abs/2304.05472v1 2023-04-11T19:54:50Z

Physically-based rendering (PBR) is key for immersive rendering effects used widely in the industry to showcase detailed realistic scenes from computer graphics assets. A well-known caveat is that producing the same is computationally heavy and relies on complex capture devices. Inspired by the success in quality and efficiency of recent volumetric neural rendering, we want to develop a physically-based neural shader to eliminate device dependency and significantly boost performance. However, no existing lighting and material models in the current neural rendering approaches can accurately represent the comprehensive lighting models and BRDFs properties required by the PBR process. Thus, this paper proposes a novel lighting representation that models direct and indirect light locally through a light sampling strategy in a learned light sampling field. We also propose BRDF models to separately represent surface/subsurface scattering details to enable complex objects such as translucent material (i.e., skin, jade). We then implement our proposed representations with an end-to-end physically-based neural face skin shader, which takes a standard face asset (i.e., geometry, albedo map, and normal map) and an HDRI for illumination as inputs and generates a photo-realistic rendering as output. Extensive experiments showcase the quality and efficiency of our PBR face skin shader, indicating the effectiveness of our proposed lighting and material representations.

2023-04-11T19:54:50Z ICLR 2023 Poster Jing Yang Hanyuan Xiao Wenbin Teng Yunxuan Cai Yajie Zhao http://arxiv.org/abs/2301.01719v1 2023-01-04T17:36:31Z

Presenting real-time rendering of 3D surfaces using radiance textures for fast synthesis of complex incidence-variable effects and environment interactions. This includes iridescence, parallax occlusion and interior mapping, (specular, regular, diffuse, total-internal) reflections with many bounces, refraction, subsurface scattering, transparency, and possibly more. This method divides textures into a matrix of radiance buckets, where each bucket represent some data at various incidence angles. Data can show final pixel color, or deferred rendering ambient occlusion, reflections, shadow map, etc. Resolution of the final synthesized output is the radiance bucket matrix size. Technique can be implemented with a simple fragment shader. The computational footprint of this technique is of simple diffuse-only graphics, but with visual fidelity of complex (off-line) ray-traced render at the cost of storage memory footprint. Balance between computational footprint and storage memory footprint can be easily achieved with variable compression ratio of repetitive radiance scene textures.

2023-01-04T17:36:31Z 4 pages, 3 figures, WIP Jakub Maksymilian Fober http://arxiv.org/abs/2212.06125v1 2022-12-12T18:55:03Z

Creating realistic virtual assets is a time-consuming process: it usually involves an artist designing the object, then spending a lot of effort on tweaking its appearance. Intricate details and certain effects, such as subsurface scattering, elude representation using real-time BRDFs, making it impossible to fully capture the appearance of certain objects. Inspired by the recent progress of neural rendering, we propose an approach for capturing real-world objects in everyday environments faithfully and fast. We use a novel neural representation to reconstruct volumetric effects, such as translucent object parts, and preserve photorealistic object appearance. To support real-time rendering without compromising rendering quality, our model uses a grid of features and a small MLP decoder that is transpiled into efficient shader code with interactive framerates. This leads to a seamless integration of the proposed neural assets with existing mesh environments and objects. Thanks to the use of standard shader code rendering is portable across many existing hardware and software systems.

2022-12-12T18:55:03Z Aljaž Božič Denis Gladkov Luke Doukakis Christoph Lassner http://arxiv.org/abs/2112.05957v1 2021-12-11T11:36:30Z

Over the last years, many face analysis tasks have accomplished astounding performance, with applications including face generation and 3D face reconstruction from a single "in-the-wild" image. Nevertheless, to the best of our knowledge, there is no method which can produce render-ready high-resolution 3D faces from "in-the-wild" images and this can be attributed to the: (a) scarcity of available data for training, and (b) lack of robust methodologies that can successfully be applied on very high-resolution data. In this work, we introduce the first method that is able to reconstruct photorealistic render-ready 3D facial geometry and BRDF from a single "in-the-wild" image. We capture a large dataset of facial shape and reflectance, which we have made public. We define a fast facial photorealistic differentiable rendering methodology with accurate facial skin diffuse and specular reflection, self-occlusion and subsurface scattering approximation. With this, we train a network that disentangles the facial diffuse and specular BRDF components from a shape and texture with baked illumination, reconstructed with a state-of-the-art 3DMM fitting method. Our method outperforms the existing arts by a significant margin and reconstructs high-resolution 3D faces from a single low-resolution image, that can be rendered in various applications, and bridge the uncanny valley.

2021-12-11T11:36:30Z Project and Dataset page: ( https://github.com/lattas/AvatarMe ). 20 pages, including supplemental materials. Accepted for publishing at IEEE Transactions on Pattern Analysis and Machine Intelligence on 13 November 2021. Copyright 2021 IEEE. Personal use of this material is permitted Alexandros Lattas Stylianos Moschoglou Stylianos Ploumpis Baris Gecer Abhijeet Ghosh Stefanos Zafeiriou 10.1109/TPAMI.2021.3125598