Showing 1–25 of 55 results for all: cloud rendering volumetric

CT2Yarn: Yarn-Level Reconstruction of Crochet from Computed Tomography

Authors: Chang Luo, Nobuyuki Umetani

Abstract: …underlying yarn path from external observations is challenging because of severe self-occlusion. While micro-CT reveals the full internal structure of a crochet object, the volumetric scan alone does not explicitly encode how the yarn traverses the object. This challenge stems from the hierarchical structure of yarn: a yarn consists of multiple twisted plies… ▽ More We introduce CT2Yarn, a human-in-the-loop framework for recovering a single continuous yarn path from micro-computed tomography (micro-CT) scans of real crochet objects. Crochet is a craft that creates complex three-dimensional shapes by interlocking loops formed from a single yarn. Recovering the underlying yarn path from external observations is challenging because of severe self-occlusion. While micro-CT reveals the full internal structure of a crochet object, the volumetric scan alone does not explicitly encode how the yarn traverses the object. This challenge stems from the hierarchical structure of yarn: a yarn consists of multiple twisted plies, and each ply itself consists of twisted fibers. Consequently, local fiber orientations observed in micro-CT scans are not aligned with the overall yarn direction. To recover yarn-level orientations from ply-level fiber orientations, we first estimate local fiber directions using Gabor filtering and convert the volume into an oriented point cloud. We then introduce an anisotropic mean-shift procedure that aggregates local fiber orientations within a neighborhood of the yarn radius into yarn-level orientation estimates. Combined with an automatic topology skeletonization strategy, our method extracts yarn-path fragments. Subsequent fragment linking, junction cleaning, and loop detection merge these trees into a small number of long curves. Where the automatic reconstruction remains ambiguous, a sketch-based user interface enables users to interactively complete the single continuous yarn path. The recovered yarn path enables downstream applications including physically based simulation, ply-level rendering, and stitch-pattern extraction. △ Less

Submitted 6 September, 2026; originally announced September 2026.

Comments: Accepted to the Conference Track of Pacific Graphics 2026

Journal ref: Pacific Graphics 2026 - Conference Papers and Posters

SE(2) Navigation Mesh

Authors: Shuyang Shi, Kaixian Qu, Changan Chen, Ines Kast, Yuntao Ma, Marco Hutter

Abstract: …environments requires representations that accurately capture traversable regions while enabling efficient path planning. Current approaches present key limitations: Point clouds and… ▽ More Global navigation for ground robots in complex multi-level environments requires representations that accurately capture traversable regions while enabling efficient path planning. Current approaches present key limitations: Point clouds and volumetric occupancy maps lack explicit surface structure for traversability estimation, whereas direct pathfinding on dense triangle meshes is computationally prohibitive. Navigation meshes mitigate these challenges through polygonal abstraction of the underlying mesh, but assume yaw-invariant traversability, rendering them unsuitable for non-circular robots in constrained spaces. We propose SE(2) Navigation Mesh (SE(2) NavMesh), a polygonal representation of traversable regions that encodes yaw-dependent traversability. Our method evaluates traversability using footprint masks and constructs a graph over yaw-specific layers with explicit translational and rotational connectivity. Grounded in this representation, we develop an A*-String Pulling-A* (ASA) pathfinding strategy that hierarchically optimizes robot position and heading. We also present an online method that incrementally updates the SE(2) NavMesh from streaming point clouds during concurrent geometry reconstruction. In simulation, the SE(2) NavMesh captures over 50% more traversable area than classical NavMeshes, and the SE(2) NavMesh + ASA pipeline consistently outperforms sampling-based baselines in constrained environments. Extensive real-world experiments on a physical robot validate real-time online generation and successful navigation across multiple environments. △ Less

Submitted 1 July, 2026; originally announced July 2026.

Comments: Project page: https://se2-navmesh.github.io/

AugSplat: Radiance Field-Informed Gaussian Splatting for Sparse-View Settings

Authors: Lorenzo Lazzaroni, Riccardo Bollati, Daniel Barath, Michael Niemeyer, Keisuke Tateno

Abstract: …challenge in 3D vision, particularly in sparse-view scenarios. Neural radiance fields have demonstrated robust reconstruction from limited observations, but their reliance on volumetric… ▽ More Generating high-quality novel views at real-time frame rates remains a central challenge in 3D vision, particularly in sparse-view scenarios. Neural radiance fields have demonstrated robust reconstruction from limited observations, but their reliance on volumetric rendering leads to high computational cost and slow inference. In contrast, Gaussian Splatting methods achieve real-time rendering through rasterization, but their optimization is highly sensitive to the quality of the initial geometry. This sensitivity becomes especially problematic in sparse-view settings, where limited observations often lead to incomplete or noisy point-cloud reconstructions. In this work, we present AugSplat, a simple framework for improving Gaussian Splatting in sparse-view regimes using radiance-field-based view augmentation. We first train a radiance field on the sparse input views and use it to synthesize additional images from nearby novel viewpoints, increasing the effective view-space coverage available for supervision. These synthetic views are then used as auxiliary supervision during Gaussian Splatting optimization. We study two variants: Staged AugSplat, which uses synthetic views for an initial optimization phase before switching to real images, and Dual AugSplat, which jointly trains on real and synthetic views with a decaying synthetic loss weight. Experiments on sparse-view mip-NeRF 360 scenes show that AugSplat improves reconstruction quality over standard Gaussian Splatting. Staged AugSplat achieves the strongest average performance, while Dual AugSplat provides a closely performing formulation that keeps real-image supervision active throughout training, and both variants preserve real-time rendering at inference. △ Less

Submitted 30 June, 2026; originally announced June 2026.

Comments: 9 pages, 5 figures

Recent Advances and Trends in Learning-based 3D Representations

Authors: Adrien Schockaert, Hamid Laga, Hazem Wannous, Vincent Magnier, Guillaume Dufaye, Jean-françois Witz

Abstract: …decision that dictates the efficiency, quality, and capabilities of modern computer vision and graphics pipelines for tasks such as 3D reconstruction, novel-view synthesis and rendering, shape and motion analysis, recognition, and generation. While traditional representations (\eg meshes, point… ▽ More The selection of an appropriate 3D representation is a fundamental design decision that dictates the efficiency, quality, and capabilities of modern computer vision and graphics pipelines for tasks such as 3D reconstruction, novel-view synthesis and rendering, shape and motion analysis, recognition, and generation. While traditional representations (\eg meshes, point clouds, and volumetric grids) remain standard outputs of 3D sensors (\eg LiDAR and 3D scanners) and are widely used in downstream applications (\eg editing and simulation), recent neural and primitive-based representations (\eg 3D Gaussian Splatting) offer compact and differentiable alternatives opening a wide range of opportunities in applications such as games, AR/VR, autonomous driving, robot navigation, and medical imaging, to name a few. The goal of this paper is to survey the main families of 3D representations from discrete explicit formats to continuous implicit fields based either on neural rendering or primitive splatting. For each type of representation, we present the general formulation and its variants, discuss its benefits and limitations, and highlight key applications. We conclude the paper by outlining the open challenges and potential directions for future research. Distinct from recent surveys that broadly cover 3D object and scene reconstruction, this paper provides a focused analysis on the evolution of 3D representations themselves. We specifically emphasize the paradigm shift toward implicit representations, offering a novel perspective on how these emerging formats fundamentally alter 3D/4D workflows. △ Less

Submitted 3 June, 2026; originally announced June 2026.

Full Dynamic Range Sky-Modelling For Image Based Lighting

Authors: Ian J. Maquignaz

Abstract: …visual arts, immersive virtual reality and a wide range of scientific and engineering applications. To alleviate the burden of physical-capture, physically-simulation and volumetric… ▽ More Accurate environment maps are a key component to modelling real-world outdoor scenes. They enable captivating visual arts, immersive virtual reality and a wide range of scientific and engineering applications. To alleviate the burden of physical-capture, physically-simulation and volumetric rendering, sky-models have been proposed as fast, flexible, and cost-saving alternatives. In recent years, sky-models have been extended through deep learning to be more comprehensive and inclusive of cloud formations, but recent work has demonstrated these models fall short in faithfully recreating accurate and photorealistic natural skies. Particularly at higher resolutions, DNN sky-models struggle to accurately model the 14EV+ class-imbalanced solar region, resulting in poor visual quality and scenes illuminated with skewed light transmission, shadows and tones. In this work, we propose Icarus, an all-weather sky-model capable of learning the exposure range of Full Dynamic Range (FDR) physically captured outdoor imagery. Our model allows conditional generation of environment maps with intuitive user-positioning of solar and cloud formations, and extends on current state-of-the-art to enable user-controlled texturing of atmospheric formations. Through our evaluation, we demonstrate Icarus is interchangeable with FDR physically captured outdoor imagery or parametric sky-models, and illuminates scenes with unprecedented accuracy, photorealism, lighting directionality (shadows), and tones in Image Based Lightning (IBL). △ Less

Submitted 5 March, 2026; originally announced March 2026.

SceMoS: Scene-Aware 3D Human Motion Synthesis by Planning with Geometry-Grounded Tokens

Authors: Anindita Ghosh, Vladislav Golyanik, Taku Komura, Philipp Slusallek, Christian Theobalt, Rishabh Dabral

Abstract: …use generative frameworks that simultaneously learn high-level planning and low-level contact reasoning, and rely on computationally expensive 3D scene data such as point clouds or voxel occupancy grids. We propose SceMoS, a scene-aware motion synthesis framework that shows that structured 2D scene representations can serve as a powerful alternative to full… ▽ More Synthesizing text-driven 3D human motion within realistic scenes requires learning both semantic intent ("walk to the couch") and physical feasibility (e.g., avoiding collisions). Current methods use generative frameworks that simultaneously learn high-level planning and low-level contact reasoning, and rely on computationally expensive 3D scene data such as point clouds or voxel occupancy grids. We propose SceMoS, a scene-aware motion synthesis framework that shows that structured 2D scene representations can serve as a powerful alternative to full 3D supervision in physically grounded motion synthesis. SceMoS disentangles global planning from local execution using lightweight 2D cues and relying on (1) a text-conditioned autoregressive global motion planner that operates on a bird's-eye-view (BEV) image rendered from an elevated corner of the scene, encoded with DINOv2 features, as the scene representation, and (2) a geometry-grounded motion tokenizer trained via a conditional VQ-VAE, that uses 2D local scene heightmap, thus embedding surface physics directly into a discrete vocabulary. This 2D factorization reaches an efficiency-fidelity trade-off: BEV semantics capture spatial layout and affordance for global reasoning, while local heightmaps enforce fine-grained physical adherence without full 3D volumetric reasoning. SceMoS achieves state-of-the-art motion realism and contact accuracy on the TRUMANS benchmark, reducing the number of trainable parameters for scene encoding by over 50%, showing that 2D scene cues can effectively ground 3D human-scene interaction. △ Less

Submitted 23 February, 2026; originally announced February 2026.

Comments: 13 pages, 6 figures, 4 tables

Gabor Fields: Orientation-Selective Level-of-Detail for Volume Rendering

Authors: Jorge Condor, Nicolai Hermann, Mehmet Ata Yurtsever, Piotr Didyk

Abstract: Gaussian-based representations have enabled efficient physically-based volume rendering at a fraction of the memory cost of regular, discrete, voxel-based distributions. However, several remaining issues hamper their widespread use. One of the advantages of classic voxel grids is the ease of constructing hierarchical representations by either storing… ▽ More Gaussian-based representations have enabled efficient physically-based volume rendering at a fraction of the memory cost of regular, discrete, voxel-based distributions. However, several remaining issues hamper their widespread use. One of the advantages of classic voxel grids is the ease of constructing hierarchical representations by either storing volumetric mipmaps or selectively pruning branches of an already hierarchical voxel grid. Such strategies reduce rendering time and eliminate aliasing when lower levels of detail are required. Constructing similar strategies for Gaussian-based volumes is not trivial. Straightforward solutions, such as prefiltering or computing mipmap-style representations, lead to increased memory requirements or expensive re-fitting of each level separately. Additionally, such solutions do not guarantee a smooth transition between different hierarchy levels. To address these limitations, we propose Gabor Fields, an orientation-selective mixture of Gabor kernels that enables continuous frequency filtering at no cost. The frequency content of the asset is reduced by selectively pruning primitives, directly benefiting rendering performance. Beyond filtering, we demonstrate that stochastically sampling from different frequencies and orientations at each ray recursion enables masking substantial portions of the volume, accelerating ray traversal time in single- and multiple-scattering settings. Furthermore, inspired by procedural volumes, we present an application for efficient design and rendering of procedural clouds as Gabor-noise-modulated Gaussians. △ Less

Submitted 16 July, 2026; v1 submitted 4 February, 2026; originally announced February 2026.

Comments: Accepted to ACM Transactions on Graphics (SIGGRAPH 2026)

ACM Class: I.3.0

Journal ref: ACM Transactions on Graphics 2026, Vol. 45, No. 4, Article 62

Immersive Volumetric Video Playback: Near-RT Resource Allocation and O-RAN-based Implementation

Authors: Yao Wen, Luping Xiang, Kun Yang

Abstract: Immersive volumetric video streaming in extended reality (XR) demands ultra-low motion-to-photon (MTP) latency, which conventional edge-centric architectures struggle to meet due to per-frame computationally intensive… ▽ More Immersive volumetric video streaming in extended reality (XR) demands ultra-low motion-to-photon (MTP) latency, which conventional edge-centric architectures struggle to meet due to per-frame computationally intensive rendering tightly coupled with user motion. To address this challenge, we propose an Open Radio Access Network (O-RAN)-integrated playback framework that jointly orchestrates radio, compute, and content resources in near real time (Near-RT) control loop. The system formulates the rendered-pixel ratio as a continuous control variable and jointly optimizes it over the Open Cloud (O-Cloud) compute, gNB transmit power, and bandwidth under a Weber-Fechner quality of experience (QoE) model, explicitly balancing resolution, computation, and latency. A Soft Actor-Critic (SAC) agent with structured action decomposition and QoE-aware reward shaping resolves the resulting high-dimensional control problem. Experiments on a 5G O-RAN testbed and system simulations show that SAC reduces median MTP latency by above $11\%$ and improves both mean QoE and fairness, demonstrating the feasibility of RIC-driven joint radio-compute-content control for scalable, latency-aware immersive streaming. △ Less

Submitted 21 March, 2026; v1 submitted 28 January, 2026; originally announced January 2026.

Comments: Accepted to TMC, IEEE Transactions on Mobile Computing. https://doi.org/10.1109/TMC.2026.3674975

Journal ref: IEEE Transactions on Mobile Computing (2026)

VSRD++: Autolabeling for 3D Object Detection via Instance-Aware Volumetric Silhouette Rendering

Authors: Zihua Liu, Hiroki Sakuma, Masatoshi Okutomi

Abstract: …challenging task in 3D scene understanding. Existing approaches heavily depend on supervised learning with extensive 3D annotations, which are often acquired from LiDAR point clouds through labor-intensive labeling processes. To tackle this problem, we propose VSRD++, a novel weakly supervised framework for monocular 3D object detection that eliminates the r… ▽ More Monocular 3D object detection is a fundamental yet challenging task in 3D scene understanding. Existing approaches heavily depend on supervised learning with extensive 3D annotations, which are often acquired from LiDAR point clouds through labor-intensive labeling processes. To tackle this problem, we propose VSRD++, a novel weakly supervised framework for monocular 3D object detection that eliminates the reliance on 3D annotations and leverages neural-field-based volumetric rendering with weak 2D supervision. VSRD++ consists of a two-stage pipeline: multi-view 3D autolabeling and subsequent monocular 3D detector training. In the multi-view autolabeling stage, object surfaces are represented as signed distance fields (SDFs) and rendered as instance masks via the proposed instance-aware volumetric silhouette rendering. To optimize 3D bounding boxes, we decompose each instance's SDF into a cuboid SDF and a residual distance field (RDF) that captures deviations from the cuboid. To address the geometry inconsistency commonly observed in volume rendering methods applied to dynamic objects, we model the dynamic objects by including velocity into bounding box attributes as well as assigning confidence to each pseudo-label. Moreover, we also employ a 3D attribute initialization module to initialize the dynamic bounding box parameters. In the monocular 3D object detection phase, the optimized 3D bounding boxes serve as pseudo labels for training monocular 3D object detectors. Extensive experiments on the KITTI-360 dataset demonstrate that VSRD++ significantly outperforms existing weakly supervised approaches for monocular 3D object detection on both static and dynamic scenes. Code is available at https://github.com/Magicboomliu/VSRD_plus_plus △ Less

Submitted 30 November, 2025; originally announced December 2025.

Comments: arXiv admin note: text overlap with arXiv:2404.00149

CPSL: Representing Volumetric Video via Content-Promoted Scene Layers

Authors: Kaiyuan Hu, Yili Jin, Junhua Liu, Xize Duan, Hong Kang, Xue Liu

Abstract: Volumetric video enables immersive and interactive visual experiences by supporting free viewpoint exploration and realistic motion parallax. However, existing… ▽ More Volumetric video enables immersive and interactive visual experiences by supporting free viewpoint exploration and realistic motion parallax. However, existing volumetric representations from explicit point clouds to implicit neural fields, remain costly in capture, computation, and rendering, which limits their scalability for on-demand video and reduces their feasibility for real-time communication. To bridge this gap, we propose Content-Promoted Scene Layers (CPSL), a compact 2.5D video representation that brings the perceptual benefits of volumetric video to conventional 2D content. Guided by per-frame depth and content saliency, CPSL decomposes each frame into a small set of geometry-consistent layers equipped with soft alpha bands and an edge-depth cache that jointly preserve occlusion ordering and boundary continuity. These lightweight, 2D-encodable assets enable parallax-corrected novel-view synthesis via depth-weighted warping and front-to-back alpha compositing, bypassing expensive 3D reconstruction. Temporally, CPSL maintains inter-frame coherence using motion-guided propagation and per-layer encoding, supporting real-time playback with standard video codecs. Across multiple benchmarks, CPSL achieves superior perceptual quality and boundary fidelity compared with layer-based and neural-field baselines while reducing storage and rendering cost by several folds. Our approach offer a practical path from 2D video to scalable 2.5D immersive media. △ Less

Submitted 18 November, 2025; originally announced November 2025.

3D Gaussian Modeling and Ray Marching of OpenVDB datasets for Scientific Visualization

Authors: Isha Sharma, Dieter Schmalstieg

Abstract: …formats are dense-grid data structures that store every grid cell, irrespective of its contribution. OpenVDB library and data format were introduced for representing sparse volumetric data specifically for visual effects use cases such as… ▽ More 3D Gaussians are currently being heavily investigated for their scene modeling and compression abilities. In 3D volumes, their use is being explored for representing dense volumes as sparsely as possible. However, most of these methods begin with a memory inefficient data format. Specially in Scientific Visualization(SciVis), where most popular formats are dense-grid data structures that store every grid cell, irrespective of its contribution. OpenVDB library and data format were introduced for representing sparse volumetric data specifically for visual effects use cases such as clouds, fire, fluids etc. It avoids storing empty cells by masking them during storage. It presents an opportunity for use in SciVis, specifically as a modeling framework for conversion to 3D Gaussian particles for further compression and for a unified modeling approach for different scientific volume types. This compression head-start is non-trivial and this paper would like to present this with a rendering algorithm based on line integration implemented in OptiX8.1 for calculating 3D Gaussians contribution along a ray for optical-depth accumulation. For comparing the rendering results of our ray marching Gaussians renderer, we also implement a SciVis style primary-ray only NanoVDB HDDA based ray marcher for OpenVDB voxel grids. Finally, this paper also explores application of this Gaussian model to formats of volumes other than regular grids, such as AMR volumes and point clouds, using internal representation of OpenVDB grid class types for data hierarchy and subdivision structure. △ Less

Submitted 14 September, 2025; originally announced September 2025.

TensoIS: A Step Towards Feed-Forward Tensorial Inverse Subsurface Scattering for Perlin Distributed Heterogeneous Media

Authors: Ashish Tiwari, Satyam Bhardwaj, Yash Bachwana, Parag Sarvoday Sahu, T. M. Feroz Ali, Bhargava Chintalapati, Shanmuganathan Raman

Abstract: …problem. Most of the existing approaches model BSSRDF either through an analysis-by-synthesis approach, approximating complex path integrals, or using differentiable volume rendering techniques to account for heterogeneity. However, only a few studies have applied learning-based methods to estimate subsurface scattering parameters, but they assume homogeneou… ▽ More Estimating scattering parameters of heterogeneous media from images is a severely under-constrained and challenging problem. Most of the existing approaches model BSSRDF either through an analysis-by-synthesis approach, approximating complex path integrals, or using differentiable volume rendering techniques to account for heterogeneity. However, only a few studies have applied learning-based methods to estimate subsurface scattering parameters, but they assume homogeneous media. Interestingly, no specific distribution is known to us that can explicitly model the heterogeneous scattering parameters in the real world. Notably, procedural noise models such as Perlin and Fractal Perlin noise have been effective in representing intricate heterogeneities of natural, organic, and inorganic surfaces. Leveraging this, we first create HeteroSynth, a synthetic dataset comprising photorealistic images of heterogeneous media whose scattering parameters are modeled using Fractal Perlin noise. Furthermore, we propose Tensorial Inverse Scattering (TensoIS), a learning-based feed-forward framework to estimate these Perlin-distributed heterogeneous scattering parameters from sparse multi-view image observations. Instead of directly predicting the 3D scattering parameter volume, TensoIS uses learnable low-rank tensor components to represent the scattering volume. We evaluate TensoIS on unseen heterogeneous variations over shapes from the HeteroSynth test set, smoke and cloud geometries obtained from open-source realistic volumetric simulations, and some real-world samples to establish its effectiveness for inverse scattering. Overall, this study is an attempt to explore Perlin noise distribution, given the lack of any such well-defined distribution in literature, to potentially model real-world heterogeneous scattering in a feed-forward manner. △ Less

Submitted 4 September, 2025; originally announced September 2025.

Comments: To appear in Pacific Graphics 2025 (CGF Journal Track), Project page: https://yashbachwana.github.io/TensoIS/

Efficient Geometry Compression and Communication for 3D Gaussian Splatting Point Clouds

Authors: Liang Xie, Yanting Li, Luyang Tang, Wei Gao

Abstract: …data volume causes excessive storage space occupancy. To address this issue, we propose adopting the AVS PCRM reference software for efficient compression of Gaussian point cloud geometry data. The strategy deeply integrates the advanced encoding capabilities of AVS PCRM into the i3DV platform, forming technical complementarity with the original rate-distort… ▽ More Storage and transmission challenges in dynamic 3D scene representation based on the i3DV platform, With increasing scene complexity, the explosive growth of 3D Gaussian data volume causes excessive storage space occupancy. To address this issue, we propose adopting the AVS PCRM reference software for efficient compression of Gaussian point cloud geometry data. The strategy deeply integrates the advanced encoding capabilities of AVS PCRM into the i3DV platform, forming technical complementarity with the original rate-distortion optimization mechanism based on binary hash tables. On one hand, the hash table efficiently caches inter-frame Gaussian point transformation relationships, which allows for high-fidelity transmission within a 40 Mbps bandwidth constraint. On the other hand, AVS PCRM performs precise compression on geometry data. Experimental results demonstrate that the joint framework maintains the advantages of fast rendering and high-quality synthesis in 3D Gaussian technology while achieving significant 10\%-25\% bitrate savings on universal test sets. It provides a superior rate-distortion tradeoff solution for the storage, transmission, and interaction of 3D volumetric video. △ Less

Submitted 2 September, 2025; originally announced September 2025.

Comments: 8 pages,5 figures

Journal ref: ACM MOBICOM 2025

Holo-Artisan: A Personalized Multi-User Holographic Experience for Virtual Museums on the Edge Intelligence

Authors: Nan-Hong Kuo, Hojjat Baghban

Abstract: …process real-time user data -- including pose, facial expression, and voice -- for multiple visitors concurrently. Generative AI models then drive digital artworks (e.g., a volumetric Mona Lisa) to respond uniquely to each viewer. For instance, the Mona Lisa can return a smile to one visitor while engaging in a spoken Q\&A with another, all in real time.… ▽ More We present Holo-Artisan, a novel system architecture enabling immersive multi-user experiences in virtual museums through true holographic displays and personalized edge intelligence. In our design, local edge computing nodes process real-time user data -- including pose, facial expression, and voice -- for multiple visitors concurrently. Generative AI models then drive digital artworks (e.g., a volumetric Mona Lisa) to respond uniquely to each viewer. For instance, the Mona Lisa can return a smile to one visitor while engaging in a spoken Q\&A with another, all in real time. A cloud-assisted collaboration platform composes these interactions in a shared scene using a universal scene description, and employs ray tracing to render high-fidelity, personalized views with a direct pipeline to glasses-free holographic displays. To preserve user privacy and continuously improve personalization, we integrate federated learning (FL) -- edge devices locally fine-tune AI models and share only model updates for aggregation. This edge-centric approach minimizes latency and bandwidth usage, ensuring a synchronized shared experience with individual customization. Through Holo-Artisan, static museum exhibits are transformed into dynamic, living artworks that engage each visitor in a personal dialogue, heralding a new paradigm of cultural heritage interaction. △ Less

Submitted 20 August, 2025; originally announced August 2025.

PC-JND: Subjective Study and Dataset on Just Noticeable Difference for Point Clouds in 6DoF Virtual Reality

Authors: Chunling Fan, Yun Zhang, Dietmar Saupe, Raouf Hamzaoui, Weisi Lin

Abstract: …a difference between a pristine stimulus and its distorted version. The JND concept has been widely applied in visual signal processing tasks, including coding, transmission, rendering, and quality assessment, to optimize human-centric media experiences. A point… ▽ More The Just Noticeable Difference (JND) accounts for the minimum distortion at which humans can perceive a difference between a pristine stimulus and its distorted version. The JND concept has been widely applied in visual signal processing tasks, including coding, transmission, rendering, and quality assessment, to optimize human-centric media experiences. A point cloud is a mainstream volumetric data representation consisting of both geometry information and attributes (e.g. color). Point clouds are used for advanced immersive 3D media such as Virtual Reality (VR). However, the JND characteristics of viewing point clouds in VR have not been explored before. In this paper, we study the point cloud-wise JND (PCJND) characteristics in a Six Degrees of Freedom (6DoF) VR environment using a head-mounted display. Our findings reveal that the texture PCJND of human eyes is smaller than the geometry PCJND for most point clouds. Furthermore, we identify a correlation between colorfulness and texture PCJND. However, there is no significant correlation between colorfulness and the geometry PCJND, nor between the number of points and neither the texture or geometry PCJND. To support future research in JND prediction and perception-driven signal processing, we introduce PC-JND, a novel point cloud-based JND dataset. This dataset will be made publicly available to facilitate advancements in perceptual optimization for immersive media. △ Less

Submitted 29 July, 2025; originally announced July 2025.

Comments: 13 pages, 10 figures, Journal

L3GS: Layered 3D Gaussian Splats for Efficient 3D Scene Delivery

Authors: Yi-Zhen Tsai, Xuechen Zhang, Zheng Li, Jiasi Chen

Abstract: Traditional 3D content representations include dense point clouds that consume large amounts of data and hence network bandwidth, while newer representations such as neural radiance fields suffer from poor frame rates due to their non-standard… ▽ More Traditional 3D content representations include dense point clouds that consume large amounts of data and hence network bandwidth, while newer representations such as neural radiance fields suffer from poor frame rates due to their non-standard volumetric rendering pipeline. 3D Gaussian splats (3DGS) can be seen as a generalization of point clouds that meet the best of both worlds, with high visual quality and efficient rendering for real-time frame rates. However, delivering 3DGS scenes from a hosting server to client devices is still challenging due to high network data consumption (e.g., 1.5 GB for a single scene). The goal of this work is to create an efficient 3D content delivery framework that allows users to view high quality 3D scenes with 3DGS as the underlying data representation. The main contributions of the paper are: (1) Creating new layered 3DGS scenes for efficient delivery, (2) Scheduling algorithms to choose what splats to download at what time, and (3) Trace-driven experiments from users wearing virtual reality headsets to evaluate the visual quality and latency. Our system for Layered 3D Gaussian Splats delivery L3GS demonstrates high visual quality, achieving 16.9% higher average SSIM compared to baselines, and also works with other compressed 3DGS representations. △ Less

Submitted 7 April, 2025; originally announced April 2025.

DreamPrinting: Volumetric Printing Primitives for High-Fidelity 3D Printing

Authors: Youjia Wang, Ruixiang Cao, Teng Xu, Yifei Liu, Dong Zhang, Yiwen Wu, Jingyi Yu

Abstract: Translating the rich visual fidelity of volumetric… ▽ More Translating the rich visual fidelity of volumetric rendering techniques into physically realizable 3D prints remains an open challenge. We introduce DreamPrinting, a novel pipeline that transforms radiance-based volumetric representations into explicit, material-centric Volumetric Printing Primitives (VPPs). While volumetric rendering primitives (e.g., NeRF) excel at capturing intricate geometry and appearance, they lack the physical constraints necessary for real-world fabrication, such as pigment compatibility and material density. DreamPrinting addresses these challenges by integrating the Kubelka-Munk model with a spectrophotometric calibration process to characterize and mix pigments for accurate reproduction of color and translucency. The result is a continuous-to-discrete mapping that determines optimal pigment concentrations for each voxel, ensuring fidelity to both geometry and optical properties. A 3D stochastic halftoning procedure then converts these concentrations into printable labels, enabling fine-grained control over opacity, texture, and color gradients. Our evaluations show that DreamPrinting achieves exceptional detail in reproducing semi-transparent structures-such as fur, leaves, and clouds-while outperforming traditional surface-based methods in managing translucency and internal consistency. Furthermore, by seamlessly integrating VPPs with cutting-edge 3D generation techniques, DreamPrinting expands the potential for complex, high-quality volumetric prints, providing a robust framework for printing objects that closely mirror their digital origins. △ Less

Submitted 2 March, 2025; originally announced March 2025.

Machine Learning-Driven Volumetric Cloud Rendering: Procedural Shader Optimization and Dynamic Lighting in Unreal Engine for Realistic Atmospheric Simulation

Authors: Shruti Singh, Shantanu Kumar

Abstract: This study advances real-time volumetric… ▽ More This study advances real-time volumetric cloud rendering in Computer Graphics (CG) by developing a specialized shader in Unreal Engine (UE), focusing on realistic cloud modeling and lighting. By leveraging ray-casting-based lighting algorithms, this work demonstrates the practical application of a dual-layered procedural noise model, eliminating the need for conventional two-dimensional (2D) weather textures. The shader allows for procedurally configured skies with a defined parameter set, offering flexibility for both artistic expression and realistic simulation. Empirical results reveal that the shader achieves an average rendering time of 35ms per frame while maintaining high visual accuracy and scene realism. Visual fidelity assessments indicate a 15% improvement in cloud realism over traditional 2D techniques, particularly in dynamic lighting scenarios. This research contributes to CG by bridging technical and aesthetic elements, enhancing real-time visual storytelling and immersion within gigital media environments. △ Less

Submitted 11 February, 2025; originally announced February 2025.

Light Transport-aware Diffusion Posterior Sampling for Single-View Reconstruction of 3D Volumes

Authors: Ludwic Leonard, Nils Thuerey, Ruediger Westermann

Abstract: We introduce a single-view reconstruction technique of volumetric fields in which multiple light scattering effects are omnipresent, such as in… ▽ More We introduce a single-view reconstruction technique of volumetric fields in which multiple light scattering effects are omnipresent, such as in clouds. We model the unknown distribution of volumetric fields using an unconditional diffusion model trained on a novel benchmark dataset comprising 1,000 synthetically simulated volumetric density fields. The neural diffusion model is trained on the latent codes of a novel, diffusion-friendly, monoplanar representation. The generative model is used to incorporate a tailored parametric diffusion posterior sampling technique into different reconstruction tasks. A physically-based differentiable volume renderer is employed to provide gradients with respect to light transport in the latent space. This stands in contrast to classic NeRF approaches and makes the reconstructions better aligned with observed data. Through various experiments, we demonstrate single-view reconstruction of volumetric clouds at a previously unattainable quality. △ Less

Submitted 28 March, 2025; v1 submitted 9 January, 2025; originally announced January 2025.

Comments: CVPR 2025

ACM Class: I.2.10; I.4.5

AI-Driven Innovations in Volumetric Video Streaming: A Review

Authors: Erfan Entezami, Hui Guan

Abstract: Recent efforts to enhance immersive and interactive user experiences have driven the development of volumetric video, a form of 3D content that enables 6 DoF. Unlike traditional 2D content,… ▽ More Recent efforts to enhance immersive and interactive user experiences have driven the development of volumetric video, a form of 3D content that enables 6 DoF. Unlike traditional 2D content, volumetric content can be represented in various ways, such as point clouds, meshes, or neural representations. However, due to its complex structure and large amounts of data size, deploying this new form of 3D data presents significant challenges in transmission and rendering. These challenges have hindered the widespread adoption of volumetric video in daily applications. In recent years, researchers have proposed various AI-driven techniques to address these challenges and improve the efficiency and quality of volumetric content streaming. This paper provides a comprehensive overview of recent advances in AI-driven approaches to facilitate volumetric content streaming. Through this review, we aim to offer insights into the current state-of-the-art and suggest potential future directions for advancing the deployment of volumetric video streaming in real-world applications. △ Less

Submitted 15 December, 2024; originally announced December 2024.

Representing Long Volumetric Video with Temporal Gaussian Hierarchy

Authors: Zhen Xu, Yinghao Xu, Zhiyuan Yu, Sida Peng, Jiaming Sun, Hujun Bao, Xiaowei Zhou

Abstract: This paper aims to address the challenge of reconstructing long volumetric videos from multi-view RGB videos. Recent dynamic view synthesis methods leverage powerful 4D representations, like feature grids or point… ▽ More This paper aims to address the challenge of reconstructing long volumetric videos from multi-view RGB videos. Recent dynamic view synthesis methods leverage powerful 4D representations, like feature grids or point cloud sequences, to achieve high-quality rendering results. However, they are typically limited to short (1~2s) video clips and often suffer from large memory footprints when dealing with longer videos. To solve this issue, we propose a novel 4D representation, named Temporal Gaussian Hierarchy, to compactly model long volumetric videos. Our key observation is that there are generally various degrees of temporal redundancy in dynamic scenes, which consist of areas changing at different speeds. Motivated by this, our approach builds a multi-level hierarchy of 4D Gaussian primitives, where each level separately describes scene regions with different degrees of content change, and adaptively shares Gaussian primitives to represent unchanged scene content over different temporal segments, thus effectively reducing the number of Gaussian primitives. In addition, the tree-like structure of the Gaussian hierarchy allows us to efficiently represent the scene at a particular moment with a subset of Gaussian primitives, leading to nearly constant GPU memory usage during the training or rendering regardless of the video length. Extensive experimental results demonstrate the superiority of our method over alternative methods in terms of training cost, rendering speed, and storage usage. To our knowledge, this work is the first approach capable of efficiently handling minutes of volumetric video data while maintaining state-of-the-art rendering quality. Our project page is available at: https://zju3dv.github.io/longvolcap. △ Less

Submitted 12 December, 2024; originally announced December 2024.

Comments: SIGGRAPH Asia 2024 (TOG). Project page: https://zju3dv.github.io/longvolcap

GSurf: Learning Signed Distance Fields from Splatting Opaque Gaussians for High-quality 3D Reconstruction

Authors: Baixin Xu, Jiangbei Hu, Jiaze Li, Ying He

Abstract: …is a core problem in 3D computer vision. While neural implicit surfaces like SDFs offer smooth geometry, they are often bottlenecked by the computational intensity of volume rendering. Conversely, 3D Gaussian Splatting (3DGS) provides rapid training but lacks geometry continuity, often leading to fragmented surfaces. This paper presents a novel framework tha… ▽ More High-fidelity surface reconstruction from multi-view images is a core problem in 3D computer vision. While neural implicit surfaces like SDFs offer smooth geometry, they are often bottlenecked by the computational intensity of volume rendering. Conversely, 3D Gaussian Splatting (3DGS) provides rapid training but lacks geometry continuity, often leading to fragmented surfaces. This paper presents a novel framework that integrates Signed Distance Fields directly into the splatting pipeline. By leveraging the continuous nature of SDFs to regularize Gaussian primitives, our method effectively fills geometric holes and suppresses noise inherent in sparse point clouds. Unlike hybrid approaches that rely on heavy volumetric sampling, our approach utilizes the efficiency of splatting to achieve faster convergence. Extensive evaluations demonstrate that our method produces high-quality surfaces with significantly fewer primitives, offering a more compact and efficient representation for both indoor and outdoor environments. △ Less

Submitted 9 July, 2026; v1 submitted 24 November, 2024; originally announced November 2024.

Comments: see https://github.com/xubaixinxbx/Gsurf

VAIR: Visuo-Acoustic Implicit Representations for Low-Cost, Multi-Modal Transparent Surface Reconstruction in Indoor Scenes

Authors: Advaith V. Sethuraman, Onur Bagoren, Harikrishnan Seetharaman, Dalton Richardson, Joseph Taylor, Katherine A. Skinner

Abstract: …optimization to learn an implicit representation of indoor scenes consisting of transparent surfaces. We demonstrate that we can query the implicit representation to enable volumetric rendering in image space or 3D geometry reconstruction (point clouds or mesh) with transparent s… ▽ More Mobile robots operating indoors must be prepared to navigate challenging scenes that contain transparent surfaces. This paper proposes a novel method for the fusion of acoustic and visual sensing modalities through implicit neural representations to enable dense reconstruction of transparent surfaces in indoor scenes. We propose a novel model that leverages generative latent optimization to learn an implicit representation of indoor scenes consisting of transparent surfaces. We demonstrate that we can query the implicit representation to enable volumetric rendering in image space or 3D geometry reconstruction (point clouds or mesh) with transparent surface prediction. We evaluate our method's effectiveness qualitatively and quantitatively on a new dataset collected using a custom, low-cost sensing platform featuring RGB-D cameras and ultrasonic sensors. Our method exhibits significant improvement over state-of-the-art for transparent surface reconstruction. △ Less

Submitted 7 November, 2024; originally announced November 2024.

Comments: https://umfieldrobotics.github.io/VAIR_site/

DeformStream: Deformation-based Adaptive Volumetric Video Streaming

Authors: Boyan Li, Yongting Chen, Dayou Zhang, Fangxin Wang

Abstract: Volumetric video streaming offers immersive 3D experiences but faces significant challenges due to high bandwidth requirements and latency issues in transmitting detailed content in real time. Traditional methods like point… ▽ More Volumetric video streaming offers immersive 3D experiences but faces significant challenges due to high bandwidth requirements and latency issues in transmitting detailed content in real time. Traditional methods like point cloud streaming compromise visual quality when zoomed in, and neural rendering techniques are too computationally intensive for real-time use. Though mesh-based streaming stands out by preserving surface detail and connectivity, offering a more refined representation for 3D content, traditional mesh streaming methods typically transmit data on a per-frame basis, failing to take full advantage of temporal redundancies across frames. This results in inefficient bandwidth usage and poor adaptability to fluctuating network conditions. We introduce Deformation-based Adaptive Volumetric Video Streaming, a novel framework that enhances volumetric video streaming performance by leveraging the inherent deformability of mesh-based representations. DeformStream uses embedded deformation to reconstruct subsequent frames from inter-frame motion, significantly reducing bandwidth usage while ensuring visual coherence between frames. To address frame reconstruction overhead and network adaptability, we formulate a new QoE model that accounts for client-side deformation latency and design a dynamic programming algorithm to optimize the trade-off between visual quality and bandwidth consumption under varying network conditions. Our evaluation demonstrates that Deformation-based Adaptive Volumetric Video Streaming outperforms existing mesh-based streaming systems in both bandwidth efficiency and visual quality, offering a robust solution for real-time volumetric video applications. △ Less

Submitted 25 September, 2024; originally announced September 2024.

Towards Real-Time Gaussian Splatting: Accelerating 3DGS through Photometric SLAM

Authors: Yan Song Hu, Dayou Mao, Yuhao Chen, John Zelek

Abstract: Initial applications of 3D Gaussian Splatting (3DGS) in Visual Simultaneous Localization and Mapping (VSLAM) demonstrate the generation of high-quality volumetric reconstructions from monocular video streams. However, despite these promising advancements, current 3DGS integrations have reduced tracking performance and lower operating speeds compared to tradi… ▽ More Initial applications of 3D Gaussian Splatting (3DGS) in Visual Simultaneous Localization and Mapping (VSLAM) demonstrate the generation of high-quality volumetric reconstructions from monocular video streams. However, despite these promising advancements, current 3DGS integrations have reduced tracking performance and lower operating speeds compared to traditional VSLAM. To address these issues, we propose integrating 3DGS with Direct Sparse Odometry, a monocular photometric SLAM system. We have done preliminary experiments showing that using Direct Sparse Odometry point cloud outputs, as opposed to standard structure-from-motion methods, significantly shortens the training time needed to achieve high-quality renders. Reducing 3DGS training time enables the development of 3DGS-integrated SLAM systems that operate in real-time on mobile hardware. These promising initial findings suggest further exploration is warranted in combining traditional VSLAM systems with 3DGS. △ Less

Submitted 7 August, 2024; originally announced August 2024.

Comments: This extended abstract has been submitted to be presented at an IEEE conference. It will be made available online by IEEE but will not be published in IEEE Xplore