-
NnD: Diffusion-based Generation of Physically-Nonnegative Objects
Authors:
Nadav Torem,
Tamar Sde-Chen,
Yoav Y. Schechner
Abstract:
…and analysis (inference). NnD trains on high-quality physically simulated objects. Once trained, it can be used for generation and inference. We demonstrate generation of 3D volumetric clouds, comprising inherently nonnegative microphysical fields. Our generated clouds are consistent with cloud physics trends. They are…
▽ More
Most natural objects have inherent complexity and variability. While some simple objects can be modeled from first principles, many real-world phenomena, such as cloud formation, require computationally expensive simulations that limit scalability. This work focuses on a class of physically meaningful, nonnegative objects that are computationally tractable but costly to simulate. To dramatically reduce computational costs, we propose nonnegative diffusion (NnD). This is a learned generative model using score based diffusion. It adapts annealed Langevin dynamics to enforce, by design, non-negativity throughout iterative scene generation and analysis (inference). NnD trains on high-quality physically simulated objects. Once trained, it can be used for generation and inference. We demonstrate generation of 3D volumetric clouds, comprising inherently nonnegative microphysical fields. Our generated clouds are consistent with cloud physics trends. They are effectively not distinguished as non-physical by expert perception.
△ Less
Submitted 11 June, 2025;
originally announced June 2025.
-
Machine Learning-Driven Volumetric Cloud Rendering: Procedural Shader Optimization and Dynamic Lighting in Unreal Engine for Realistic Atmospheric Simulation
Authors:
Shruti Singh,
Shantanu Kumar
Abstract:
This study advances real-time volumetric cloud rendering in Computer Graphics (CG) by developing a specialized shader in Unreal Engine (UE), focusing on realistic cloud modeling and lighting. By leveraging ray-casting-based lighting algorithms, this work demonstrates the practical application of a dual-layered procedur…
▽ More
This study advances real-time volumetric cloud rendering in Computer Graphics (CG) by developing a specialized shader in Unreal Engine (UE), focusing on realistic cloud modeling and lighting. By leveraging ray-casting-based lighting algorithms, this work demonstrates the practical application of a dual-layered procedural noise model, eliminating the need for conventional two-dimensional (2D) weather textures. The shader allows for procedurally configured skies with a defined parameter set, offering flexibility for both artistic expression and realistic simulation. Empirical results reveal that the shader achieves an average rendering time of 35ms per frame while maintaining high visual accuracy and scene realism. Visual fidelity assessments indicate a 15% improvement in cloud realism over traditional 2D techniques, particularly in dynamic lighting scenarios. This research contributes to CG by bridging technical and aesthetic elements, enhancing real-time visual storytelling and immersion within gigital media environments.
△ Less
Submitted 11 February, 2025;
originally announced February 2025.
-
Light Transport-aware Diffusion Posterior Sampling for Single-View Reconstruction of 3D Volumes
Authors:
Ludwic Leonard,
Nils Thuerey,
Ruediger Westermann
Abstract:
…contrast to classic NeRF approaches and makes the reconstructions better aligned with observed data. Through various experiments, we demonstrate single-view reconstruction of volumetric clouds at a previously unattainable quality.
▽ More
We introduce a single-view reconstruction technique of volumetric fields in which multiple light scattering effects are omnipresent, such as in clouds. We model the unknown distribution of volumetric fields using an unconditional diffusion model trained on a novel benchmark dataset comprising 1,000 synthetically simulated volumetric density fields. The neural diffusion model is trained on the latent codes of a novel, diffusion-friendly, monoplanar representation. The generative model is used to incorporate a tailored parametric diffusion posterior sampling technique into different reconstruction tasks. A physically-based differentiable volume renderer is employed to provide gradients with respect to light transport in the latent space. This stands in contrast to classic NeRF approaches and makes the reconstructions better aligned with observed data. Through various experiments, we demonstrate single-view reconstruction of volumetric clouds at a previously unattainable quality.
△ Less
Submitted 28 March, 2025; v1 submitted 9 January, 2025;
originally announced January 2025.
-
Volumetric Cloud Field Reconstruction
Authors:
Jacob Lin,
Miguel Farinha,
Edward Gryspeerdt,
Ronald Clark
Abstract:
Volumetric phenomena, such as clouds and fog, present a significant challenge for 3D reconstruction systems due to their translucent nature and their complex interactions with light. Conventional techniques for reconstructing scattering volumes rely on controlled setups, limiting practical applications. This paper introduces an approach to reconstructing volumes from a few input stereo pairs. We p…
▽ More
Volumetric phenomena, such as clouds and fog, present a significant challenge for 3D reconstruction systems due to their translucent nature and their complex interactions with light. Conventional techniques for reconstructing scattering volumes rely on controlled setups, limiting practical applications. This paper introduces an approach to reconstructing volumes from a few input stereo pairs. We propose a novel deep learning framework that integrates a deep stereo model with a 3D Convolutional Neural Network (3D CNN) and an advection module, capable of capturing the shape and dynamics of volumes. The stereo depths are used to carve empty space around volumes, providing the 3D CNN with a prior for coping with the lack of input views. Refining our output, the advection module leverages the temporal evolution of the medium, providing a mechanism to infer motion and improve temporal consistency. The efficacy of our system is demonstrated through its ability to estimate density and velocity fields of large-scale volumes, in this case, clouds, from a sparse set of stereo image pairs.
△ Less
Submitted 29 November, 2023;
originally announced November 2023.
-
Model-Centric Volumetric Point Cloud Attributes
Authors:
Ricardo L. de Queiroz,
Camilo Dorea,
Davi R. Freitas,
Maja Krivokuca,
Gustavo P. Sandri
Abstract:
…from which it may calculate the image viewed by a given camera. We want to be able to describe transparent or translucid objects, mirrors, fishbowls, fog and smoke. Volumetric clouds may allow us to describe the air, however ``empty'', and introduce air particles, in a manner independent of the viewer position…
▽ More
Point clouds have recently gained interest, especially for real-time applications and for 3D-scanned material, such as is used in autonomous driving, architecture, and engineering, to model real estate for renovation or display. Point clouds are associated with geometry information and attributes such as color. Be the color unique or direction-dependent (in the case of plenoptic point clouds), it reflects the colors observed by cameras displaced around the object. Hence, not only are the viewing references assumed, but the illumination spectrum and illumination geometry is also implicit. We propose a model-centric description of the 3D object, that is independent of the illumination and of the position of the cameras. We want to be able to describe the objects themselves such that, at a later stage, the rendering of the model may decide where to place illumination, from which it may calculate the image viewed by a given camera. We want to be able to describe transparent or translucid objects, mirrors, fishbowls, fog and smoke. Volumetric clouds may allow us to describe the air, however ``empty'', and introduce air particles, in a manner independent of the viewer position. For that, we rely on some eletromagnetic properties to arrive at seven attributes per voxel that would describe the material and its color or transparency. Three attributes are for the transmissivity of each color, three are for the attenuation of each color, and another attribute is for diffuseness. These attributes give information about the object to the renderer, with whom lies the decision on how to render and depict each object.
△ Less
Submitted 29 June, 2021;
originally announced June 2021.
