Showing 1–20 of 20 results for all: snow rendering

The Domain Is a Residue: Adapting Self-Supervised Features, Not Generators

Authors: Thomas Deixelberger, Markus Steinberger

Abstract: Clearing fog, rain or snow from footage, or turning… ▽ More Clearing fog, rain or snow from footage, or turning renders into photographs, must remove the source domain and keep the scene. Unpaired translators carry it through because their generator sees the source appearance (pixels, a near-invertible latent or a control map) and keeps it. A DINO feature map fixes what is in the scene and carries weather, lighting and rendering style as a residue of 13 to 14% of the feature norm. We propose the Representation Feature Adapter (RFA), a 2.9M-parameter network that moves this residue. We train only the adapter and its discriminators; the encoder and a feature-conditioned decoder, trained once for all conditions, stay frozen. Against CycleGAN-Turbo it is ahead on both metrics on fog and on KID on night, and level within noise on snow, rain and haze. On sim-to-real it leads REGEN and HyPER-GAN on both metrics. Only the RFA removes the rain while keeping the scene. The removal costs scene structure: CycleGAN-Turbo keeps more on every condition but fog. On VAE latents the identical adapter collapses to the identity, and decoders from other groups that never saw it render its output. The RFA has about 160 times fewer trainable parameters than CycleGAN-Turbo and under a fifth of its per-condition training time. △ Less

Submitted 29 September, 2026; originally announced September 2026.

Comments: 9 pages main text, 28 pages including appendix. 12 figures, 13 tables

ACM Class: I.2.10; I.4.3; I.4.8

VersaGauss: A Versatile Framework for Generating Multiphase Dynamics with 3D Gaussians

Authors: Ruijie Su, Lingxiao Yang, Xiaohua Xie, Jianhuang Lai

Abstract: …dynamic generation of solid objects and only handle single-phase collision interactions. We introduce VersaGauss, a unified framework for generation, simulation, and rendering that supports versatile physics-based dynamic generation, particularly for multiphase interactions. Our system takes a few images as input and produces a realistic, physics-driven 3D d… ▽ More Recent progress has been made in 3D Gaussian representation for reconstruction, generation, and physical simulation. However, current approaches mainly concentrate on physics-based dynamic generation of solid objects and only handle single-phase collision interactions. We introduce VersaGauss, a unified framework for generation, simulation, and rendering that supports versatile physics-based dynamic generation, particularly for multiphase interactions. Our system takes a few images as input and produces a realistic, physics-driven 3D dynamic scene with multiple objects. To optimize the Gaussian kernel distribution, we develop a particle pruning algorithm. We also propose the Coupled Multiphase Point Method (CMPM) to effectively model and generate multiphase interactions. Additionally, harmonic interpolation within CMPM and a Gaussian evolution strategy are introduced to achieve realistic fluid rendering. Extensive experiments demonstrate that our framework can simulate interactions among various materials such as fluid, rubber, sand, snow, and others. Code is available at https://github.com/Elowen-surj/VersaGauss. △ Less

Submitted 28 August, 2026; originally announced August 2026.

Beyond Clear Skies: Synthetic Seasonal and Weather Variations for Real-World Drone Detection

Authors: Tamara R. Lenhard, Andreas Weinmann, Tobias Koch

Abstract: …conditions. Conversely, synthetic data offers a scalable alternative: environmental variation becomes controllable, while modern game-engine-based pipelines provide realistic rendering and automatic annotations. Leveraging this potential, we introduce SynDroneVision-Weather (SDV-W), an systematic extension of SynDroneVision (SDV) targeting adverse-weather an… ▽ More Reliable drone detection under real-world deployment conditions requires training data that spans the full operational design domain, including adverse weather and seasonal appearance variation. However, acquiring and annotating such data at scale remains highly resource-intensive, as adverse-weather conditions are inherently difficult to control, reproduce, and sample systematically. Existing datasets therefore typically provide only limited coverage of such conditions. Conversely, synthetic data offers a scalable alternative: environmental variation becomes controllable, while modern game-engine-based pipelines provide realistic rendering and automatic annotations. Leveraging this potential, we introduce SynDroneVision-Weather (SDV-W), an systematic extension of SynDroneVision (SDV) targeting adverse-weather and seasonal domain shifts in urban drone detection. SDV-W comprises 55,187 annotated high-resolution images from three urban environments, rendered across three seasonal configurations and diverse weather conditions, including rain, snow, and fog at multiple severity levels. By preserving SDV's scene and trajectory configuration, SDV-W enables matched clean-adverse comparisons and quantification of condition-specific detector degradation. Across representative YOLO models and real-world datasets, we show that SDV-W improves detector reliability under adverse appearance shifts, reduces missed detections and false alarms, and is most effective as a complement to general-purpose synthetic drone-detection data. SDV-W will be publicly released upon paper acceptance. △ Less

Submitted 17 August, 2026; originally announced August 2026.

Drone-Assisted UAV-UGV Collaboration for Autonomous Navigation in Snow-Covered Terrain

Authors: Shreyam Gupta, P. Agrawal, Priyam Gupta, R. Gautam

Abstract: This paper presents a collaborative UAV-UGV navigation framework for high-altitude, snow-covered terrain, where reduced visibility and unstable ground… ▽ More This paper presents a collaborative UAV-UGV navigation framework for high-altitude, snow-covered terrain, where reduced visibility and unstable ground render conventional methods ineffective. We introduce a custom efficient U-Net architecture that falls under the computational constraints for real-time road segmentation, utilizing a novel synthetic snow data augmentation technique to achieve 96.5% segmentation accuracy. For UAV localization, we implement an Extended Kalman Filter (EKF) fusing onboard GPS and IMU data, achieving a maximum observed positional error of +-0.5 meters. The UGV position is determined via a visual tracking pipeline using YOLOv5 and depth data from the UAV's RGB-D camera. A dynamic path planning algorithm utilizes this segmentation to adjust for snow drifts, enabling successful navigation in obscured test environment with minimal deviation. △ Less

Submitted 7 August, 2026; originally announced August 2026.

Fewer, Better Frames: A Compute-Normalized Proof of Concept for Coherence-First World-Model Rendering with Model-Guided FSR4 Frame Generation

Authors: Paweł Katarzyński

Abstract: …that generates about 30 FPS natively under the same seed, route, control script, target presentation duration, and local time-scaling regime. Across forest, sword, desert, and snow scenes, the coherence-first branch preserves path geometry, object identity, large silhouettes, and depth layering longer, while the baseline degrades earlier into brightness drif… ▽ More World models are often evaluated by native frame cadence, but higher nominal frame rate can trade away long-horizon scene stability. This article reports an independent proof of concept implemented using Overworld's Waypoint-1.5 family and WorldEngine runtime on a Windows fallback stack with ONNX Runtime + DirectML and an FSR4 DX12 bridge. The tested coherence-first branch generates higher-context anchor frames at a 15 FPS presentation-timeline cadence and reconstructs presentation to 30 FPS using latent-delta motion guidance and synthesized depth. It is compared against a lower-context cadence-first baseline that generates about 30 FPS natively under the same seed, route, control script, target presentation duration, and local time-scaling regime. Across forest, sword, desert, and snow scenes, the coherence-first branch preserves path geometry, object identity, large silhouettes, and depth layering longer, while the baseline degrades earlier into brightness drift and geometric distortion. Lightweight temporal metrics and paired videos support the visual comparison, with LPIPS favoring the coherence-first branch across all tested scenes. Here compute-normalized means approximately matched same-GPU, same-timescale operating points, not exact FLOP parity or measured realtime throughput. A separate heavier sword-scene probe suggests local non-monotonicity: more context and denoising did not automatically improve quality. These results support coherence-first allocation as a practical proof-of-concept strategy under limited inference budget, not as a finished realtime renderer. △ Less

Submitted 11 May, 2026; originally announced June 2026.

Comments: 19 pages, 8 figures, independent systems proof of concept

Improving Model Performance by Adapting the KGE Metric to Account for System Non-Stationarity

Authors: M Jawad, HV Gupta, YH Wang, MA Farmani, A Behrangi, GY Niu

Abstract: …seasonal and climatic variability in hydrometeorological drivers, and from natural and anthropogenic changes to land use and cover. As has been pointed out, such variability renders "the assumption of statistical stationarity obsolete in water management", and requires us to "account for, rather than ignore, non-stationary trends" in the data… ▽ More Geoscientific systems tend to be characterized by pronounced temporal non-stationarity, arising from seasonal and climatic variability in hydrometeorological drivers, and from natural and anthropogenic changes to land use and cover. As has been pointed out, such variability renders "the assumption of statistical stationarity obsolete in water management", and requires us to "account for, rather than ignore, non-stationary trends" in the data. However, metrics used for model development are typically based on the implicit and unjustifiable assumption that the data generating process is time-stationary. Here, we introduce the JKGE_ss metric (adapted from KGE_ss) that detects and accounts for dynamical non-stationarity in the statistical properties of the data and thereby improves information extraction and model performance. Unlike NSE and KGE_ss, which use the long-term mean as a benchmark against which to evaluate model efficiency, JKGE_ss emphasizes reproduction of temporal variations in system storage. We tested the robustness of the new metric by training physical-conceptual and data-based catchment-scale models of varying complexity across a wide range of hydroclimatic conditions, from recent-precipitation-dominated to snow-dominated to strongly arid. In all cases, the result was improved reproduction of system temporal dynamics at all time scales, across wet to dry years, and over the full range of flow levels (especially recession periods). Since traditional metrics fail to adequately account for temporal shifts in system dynamics, potentially resulting in misleading assessments of model performance under changing conditions, we recommend the adoption of JKGE_ss for geoscientific model development. △ Less

Submitted 9 April, 2026; v1 submitted 4 April, 2026; originally announced April 2026.

3D Roadway Scene Object Detection with LIDARs in Snowfall Conditions

Authors: Ghazal Farhani, Taufiq Rahman, Syed Mostaquim Ali, Andrew Liu, Mohamed Zaki, Dominique Charlebois, Benoit Anctil

Abstract: …in clean and clear weather conditions, their performance significantly deteriorates in adverse weather conditions such as those involving atmospheric precipitation. This may render perception capabilities of autonomous systems that use LiDAR data in learning based models to perform object detection and ranging ineffective. While efforts have been made to enh… ▽ More Because 3D structure of a roadway environment can be characterized directly by a Light Detection and Ranging (LiDAR) sensors, they can be used to obtain exceptional situational awareness for assitive and autonomous driving systems. Although LiDARs demonstrate good performance in clean and clear weather conditions, their performance significantly deteriorates in adverse weather conditions such as those involving atmospheric precipitation. This may render perception capabilities of autonomous systems that use LiDAR data in learning based models to perform object detection and ranging ineffective. While efforts have been made to enhance the accuracy of these models, the extent of signal degradation under various weather conditions remains largely not quantified. In this study, we focus on the performance of an automotive grade LiDAR in snowy conditions in order to develop a physics-based model that examines failure modes of a LiDAR sensor. Specifically, we investigated how the LiDAR signal attenuates with different snowfall rates and how snow particles near the source serve as small but efficient reflectors. Utilizing our model, we transform data from clear conditions to simulate snowy scenarios, enabling a comparison of our synthetic data with actual snowy conditions. Furthermore, we employ this synthetic data, representative of different snowfall rates, to explore the impact on a pre-trained object detection model, assessing its performance under varying levels of snowfall △ Less

Submitted 25 October, 2025; originally announced October 2025.

Comments: 2024 IEEE 27th International Conference on Intelligent Transportation Systems (ITSC), pp. 1441--1448, Sept. 2024

Stronger, Steadier & Superior: Geometric Consistency in Depth VFM Forges Domain Generalized Semantic Segmentation

Authors: Siyu Chen, Ting Han, Changshe Zhang, Xin Luo, Meiliu Wu, Guorong Cai, Jinhe Su

Abstract: …Generalized Semantic Segmentation (DGSS). However, recent methods often overlook the fact that visual cues are susceptible, whereas the underlying geometry remains stable, rendering depth information more robust. In this paper, we investigate the potential of integrating depth information with features from VFMs, to improve the geometric consistency within a… ▽ More Vision Foundation Models (VFMs) have delivered remarkable performance in Domain Generalized Semantic Segmentation (DGSS). However, recent methods often overlook the fact that visual cues are susceptible, whereas the underlying geometry remains stable, rendering depth information more robust. In this paper, we investigate the potential of integrating depth information with features from VFMs, to improve the geometric consistency within an image and boost the generalization performance of VFMs. We propose a novel fine-tuning DGSS framework, named DepthForge, which integrates the visual cues from frozen DINOv2 or EVA02 and depth cues from frozen Depth Anything V2. In each layer of the VFMs, we incorporate depth-aware learnable tokens to continuously decouple domain-invariant visual and spatial information, thereby enhancing depth awareness and attention of the VFMs. Finally, we develop a depth refinement decoder and integrate it into the model architecture to adaptively refine multi-layer VFM features and depth-aware learnable tokens. Extensive experiments are conducted based on various DGSS settings and five different datsets as unseen target domains. The qualitative and quantitative results demonstrate that our method significantly outperforms alternative approaches with stronger performance, steadier visual-spatial attention, and superior generalization ability. In particular, DepthForge exhibits outstanding performance under extreme conditions (e.g., night and snow). Code is available at https://github.com/anonymouse-xzrptkvyqc/DepthForge. △ Less

Submitted 15 July, 2025; v1 submitted 17 April, 2025; originally announced April 2025.

Comments: Accepted by ICCV 2025

StyledStreets: Multi-style Street Simulator with Spatial and Temporal Consistency

Authors: Yuyin Chen, Yida Wang, Xueyang Zhang, Kun Zhan, Peng Jia, Yifei Zhan, Xianpeng Lang

Abstract: …disentangles persistent scene geometry from transient style attributes, allowing realistic environmental edits while preserving structural integrity. Second, uncertainty-aware rendering mitigates supervision noise from diffusion priors, enabling robust training across extreme style variations. Third, a unified parametric model prevents geometric drift throug… ▽ More Urban scene reconstruction requires modeling both static infrastructure and dynamic elements while supporting diverse environmental conditions. We present \textbf{StyledStreets}, a multi-style street simulator that achieves instruction-driven scene editing with guaranteed spatial and temporal consistency. Building on a state-of-the-art Gaussian Splatting framework for street scenarios enhanced by our proposed pose optimization and multi-view training, our method enables photorealistic style transfers across seasons, weather conditions, and camera setups through three key innovations: First, a hybrid embedding scheme disentangles persistent scene geometry from transient style attributes, allowing realistic environmental edits while preserving structural integrity. Second, uncertainty-aware rendering mitigates supervision noise from diffusion priors, enabling robust training across extreme style variations. Third, a unified parametric model prevents geometric drift through regularized updates, maintaining multi-view consistency across seven vehicle-mounted cameras. Our framework preserves the original scene's motion patterns and geometric relationships. Qualitative results demonstrate plausible transitions between diverse conditions (snow, sandstorm, night), while quantitative evaluations show state-of-the-art geometric accuracy under style transfers. The approach establishes new capabilities for urban simulation, with applications in autonomous vehicle testing and augmented reality systems requiring reliable environmental consistency. Codes will be publicly available upon publication. △ Less

Submitted 26 March, 2025; originally announced March 2025.

Comments: 14 pages

Large-Area Emergency Lockdowns with Automated Driving Systems

Authors: Noah Goodall

Abstract: Region-wide restrictions on personal vehicle travel have a long history in the United States, from riot curfews in the late 1960s, to travel bans during snow events, to the 2013 shelter-in-place "lockdown" during the search for the perpetrator of the Boston Marathon bombing. Because lockdowns require tremendous resources to enforce, they are often li… ▽ More Region-wide restrictions on personal vehicle travel have a long history in the United States, from riot curfews in the late 1960s, to travel bans during snow events, to the 2013 shelter-in-place "lockdown" during the search for the perpetrator of the Boston Marathon bombing. Because lockdowns require tremendous resources to enforce, they are often limited in duration or scope. The introduction of automated driving systems may allow governments to quickly and cheaply effect large-area lockdowns by jamming wireless communications, spoofing road closures on digital maps, exploiting a vehicle's programming to obey all traffic control devices, or coordinating with vehicle developers. Future vehicles may lack conventional controls, rendering them undrivable by the public. As travel restrictions become easier to implement, governments may enforce them more frequently, over longer durations and wider areas. This article explores the practical, legal, and ethical implications of lockdowns when most driving is highly automated, and provides guidance for the development of lockdown policies. △ Less

Submitted 19 July, 2024; originally announced July 2024.

ScatterNeRF: Seeing Through Fog with Physically-Based Inverse Neural Rendering

Authors: Andrea Ramazzina, Mario Bijelic, Stefanie Walz, Alessandro Sanvito, Dominik Scheuble, Felix Heide

Abstract: Vision in adverse weather conditions, whether it be snow, rain, or fog is challenging. In these scenarios, scattering and attenuation severly degrades image quality. Handling such inclement weather conditions, however, is essential to operate autonomous vehicles, drones and robotic applications where human performance is impeded the most. A large body of wor… ▽ More Vision in adverse weather conditions, whether it be snow, rain, or fog is challenging. In these scenarios, scattering and attenuation severly degrades image quality. Handling such inclement weather conditions, however, is essential to operate autonomous vehicles, drones and robotic applications where human performance is impeded the most. A large body of work explores removing weather-induced image degradations with dehazing methods. Most methods rely on single images as input and struggle to generalize from synthetic fully-supervised training approaches or to generate high fidelity results from unpaired real-world datasets. With data as bottleneck and most of today's training data relying on good weather conditions with inclement weather as outlier, we rely on an inverse rendering approach to reconstruct the scene content. We introduce ScatterNeRF, a neural rendering method which adequately renders foggy scenes and decomposes the fog-free background from the participating media-exploiting the multiple views from a short automotive sequence without the need for a large training data corpus. Instead, the rendering approach is optimized on the multi-view scene itself, which can be typically captured by an autonomous vehicle, robot or drone during operation. Specifically, we propose a disentangled representation for the scattering volume and the scene objects, and learn the scene reconstruction with physics-inspired losses. We validate our method by capturing multi-view In-the-Wild data and controlled captures in a large-scale fog chamber. △ Less

Submitted 3 May, 2023; originally announced May 2023.

ClimateNeRF: Extreme Weather Synthesis in Neural Radiance Field

Authors: Yuan Li, Zhi-Hao Lin, David Forsyth, Jia-Bin Huang, Shenlong Wang

Abstract: …of physical phenomena in those scenes. Our application -- Climate NeRF -- allows people to visualize what climate change outcomes will do to them. ClimateNeRF allows us to render realistic weather effects, including smog, snow, and flood. Results can be controlled with physically meaningful variables like water level.… ▽ More Physical simulations produce excellent predictions of weather effects. Neural radiance fields produce SOTA scene models. We describe a novel NeRF-editing procedure that can fuse physical simulations with NeRF models of scenes, producing realistic movies of physical phenomena in those scenes. Our application -- Climate NeRF -- allows people to visualize what climate change outcomes will do to them. ClimateNeRF allows us to render realistic weather effects, including smog, snow, and flood. Results can be controlled with physically meaningful variables like water level. Qualitative and quantitative studies show that our simulated results are significantly more realistic than those from SOTA 2D image editing and SOTA 3D NeRF stylization. △ Less

Submitted 8 June, 2023; v1 submitted 23 November, 2022; originally announced November 2022.

Comments: project page: https://climatenerf.github.io/

Attacking Motion Estimation with Adversarial Snow

Authors: Jenny Schmalfuss, Lukas Mehl, Andrés Bruhn

Abstract: …per-pixel perturbations, which are unlikely to appear in the real world. In contrast, we exploit a real-world weather phenomenon for a novel attack with adversarially optimized snow. At the core of our attack is a differentiable… ▽ More Current adversarial attacks for motion estimation (optical flow) optimize small per-pixel perturbations, which are unlikely to appear in the real world. In contrast, we exploit a real-world weather phenomenon for a novel attack with adversarially optimized snow. At the core of our attack is a differentiable renderer that consistently integrates photorealistic snowflakes with realistic motion into the 3D scene. Through optimization we obtain adversarial snow that significantly impacts the optical flow while being indistinguishable from ordinary snow. Surprisingly, the impact of our novel attack is largest on methods that previously showed a high robustness to small L_p perturbations. △ Less

Submitted 20 October, 2022; originally announced October 2022.

A Systematic Review on Computer Vision-Based Parking Lot Management Applied on Public Datasets

Authors: Paulo Ricardo Lisboa de Almeida, Jeovane Honório Alves, Rafael Stubs Parpinelli, Jean Paul Barddal

Abstract: …In addition, we have noticed that several important factors such as the presence of the same cars across consecutive images, have been neglected in most studies, thereby rendering unrealistic assessment protocols. Furthermore, the analysis of the datasets also revealed that certain features that should be present when developing new benchmarks, such as the… ▽ More Computer vision-based parking lot management methods have been extensively researched upon owing to their flexibility and cost-effectiveness. To evaluate such methods authors often employ publicly available parking lot image datasets. In this study, we surveyed and compared robust publicly available image datasets specifically crafted to test computer vision-based methods for parking lot management approaches and consequently present a systematic and comprehensive review of existing works that employ such datasets. The literature review identified relevant gaps that require further research, such as the requirement of dataset-independent approaches and methods suitable for autonomous detection of position of parking spaces. In addition, we have noticed that several important factors such as the presence of the same cars across consecutive images, have been neglected in most studies, thereby rendering unrealistic assessment protocols. Furthermore, the analysis of the datasets also revealed that certain features that should be present when developing new benchmarks, such as the availability of video sequences and images taken in more diverse conditions, including nighttime and snow, have not been incorporated. △ Less

Submitted 12 March, 2022; originally announced March 2022.

Comments: Paper accepted at Expert Systems with Applications (ESWA)

Joint Characterization of the Cryospheric Spectral Feature Space

Authors: Christopher Small, Daniel Sousa

Abstract: …objective of this study is to compare and contrast two approaches for identifying feature space basis vectors via dimensionality reduction. These approaches can be combined to render a joint characterization that reveals spectral properties not apparent using either approach alone. We use a diverse collection of AVIRIS-NG reflectance spectra of the… ▽ More Hyperspectral feature spaces are useful for many remote sensing applications ranging from spectral mixture modeling to discrete thematic classification. In such cases, characterization of the feature space dimensionality, geometry and topology can provide guidance for effective model design. The objective of this study is to compare and contrast two approaches for identifying feature space basis vectors via dimensionality reduction. These approaches can be combined to render a joint characterization that reveals spectral properties not apparent using either approach alone. We use a diverse collection of AVIRIS-NG reflectance spectra of the snow-firn-ice continuum to illustrate the utility of joint characterization and identify physical properties inferred from the spectra. Spectral feature spaces combining principal components (PCs) and t-distributed Stochastic Neighbor Embeddings (t-SNEs) provide physically interpretable dimensions representing the global (PC) structure of cryospheric reflectance properties and local (t-SNE) manifold structures revealing clustering not resolved in the global continuum. Joint characterization reveals distinct continua for snow-firn gradients on different parts of the Greenland Ice Sheet and multiple clusters of ice reflectance properties common to both glacier and sea ice in different locations. Clustering revealed in t-SNE feature spaces, and extended to the joint characterization, distinguishes differences in spectral curvature specific to location within the snow accumulation zone, and BRDF effects related to view geometry. The ability of PC+t-SNE joint characterization to produce a physically interpretable spectral feature spaces revealing global topology while preserving local manifold structures suggests that this characterization might be extended to the much higher dimensional hyperspectral feature space of all terrestrial land cover. △ Less

Submitted 2 December, 2021; originally announced December 2021.

Comments: 30 pages, 11 figures

Kilometer-scale autonomous navigation in subarctic forests: challenges and lessons learned

Authors: Dominic Baril, Simon-Pierre Deschênes, Olivier Gamache, Maxime Vaidis, Damien LaRocque, Johann Laconte, Vladimír Kubelka, Philippe Giguère, François Pomerleau

Abstract: …systems. To this end, we present a field report analyzing teach-and-repeat navigation in a subarctic forest while subject to fluctuating weather, including light and heavy snow, rain and drizzle. First, we describe the system, which relies on point cloud registration to localize a mobile robot through a boreal forest, while simultaneously building a map. We… ▽ More Challenges inherent to autonomous wintertime navigation in forests include lack of reliable a Global Navigation Satellite System (GNSS) signal, low feature contrast, high illumination variations and changing environment. This type of off-road environment is an extreme case of situations autonomous cars could encounter in northern regions. Thus, it is important to understand the impact of this harsh environment on autonomous navigation systems. To this end, we present a field report analyzing teach-and-repeat navigation in a subarctic forest while subject to fluctuating weather, including light and heavy snow, rain and drizzle. First, we describe the system, which relies on point cloud registration to localize a mobile robot through a boreal forest, while simultaneously building a map. We experimentally evaluate this system in over 18.8 km of autonomous navigation in the teach-and-repeat mode. Over 14 repeat runs, only four manual interventions were required, three of which were due to localization failure and another one caused by battery power outage. We show that dense vegetation perturbs the GNSS signal, rendering it unsuitable for navigation in forest trails. Furthermore, we highlight the increased uncertainty related to localizing using point cloud registration in forest trails. We demonstrate that it is not snow precipitation, but snow accumulation, that affects our system's ability to localize within the environment. Finally, we expose some challenges and lessons learned from our field campaign to support better experimental work in winter conditions. Our dataset is available online. △ Less

Submitted 26 July, 2022; v1 submitted 27 November, 2021; originally announced November 2021.

Comments: Published in Field Robotics Volume 2. Paper number 50

New Encoder Learning for Captioning Heavy Rain Images via Semantic Visual Feature Matching

Authors: Chang-Hwan Son, Pung-Hwi Ye

Abstract: …text that describes scenes from input images. It has been developed for high quality images taken in clear weather. However, in bad weather conditions, such as heavy rain, snow, and dense fog, the poor visibility owing to rain streaks, rain accumulation, and snowflakes causes a serious degradation of image quality. This hinders the extraction of useful visua… ▽ More Image captioning generates text that describes scenes from input images. It has been developed for high quality images taken in clear weather. However, in bad weather conditions, such as heavy rain, snow, and dense fog, the poor visibility owing to rain streaks, rain accumulation, and snowflakes causes a serious degradation of image quality. This hinders the extraction of useful visual features and results in deteriorated image captioning performance. To address practical issues, this study introduces a new encoder for captioning heavy rain images. The central idea is to transform output features extracted from heavy rain input images into semantic visual features associated with words and sentence context. To achieve this, a target encoder is initially trained in an encoder-decoder framework to associate visual features with semantic words. Subsequently, the objects in a heavy rain image are rendered visible by using an initial reconstruction subnetwork (IRS) based on a heavy rain model. The IRS is then combined with another semantic visual feature matching subnetwork (SVFMS) to match the output features of the IRS with the semantic visual features of the pretrained target encoder. The proposed encoder is based on the joint learning of the IRS and SVFMS. It is is trained in an end-to-end manner, and then connected to the pretrained decoder for image captioning. It is experimentally demonstrated that the proposed encoder can generate semantic visual features associated with words even from heavy rain images, thereby increasing the accuracy of the generated captions. △ Less

Submitted 15 September, 2021; v1 submitted 28 May, 2021; originally announced May 2021.

Journal ref: Journal of Imaging Science and Technology, vol. 65, no. 5, pp. 050402-1~050402-12, Sept. 2021

A Featureless Infrared Transmission Spectrum for the Super-Puff Planet Kepler-79d

Authors: Yayaati Chachan, Daniel Jontof-Hutter, Heather A. Knutson, Danica Adams, Peter Gao, Björn Benneke, Zachory Berta-Thompson, Fei Dai, Drake Deming, Eric Ford, Eve J. Lee, Jessica E. Libby-Roberts, Nikku Madhusudhan, Hannah R. Wakeford, Ian Wong

Abstract: …), their large envelopes may have been accreted beyond the water snow line and many appear to be susceptible to catastrophic mass loss. Both the presence of water and the importance of mass loss can be explored using transmission spectroscopy. Here, we present new HST WFC3 spectroscopy and updated Kepler transit depth measurements for the super-puff Kepler-7… ▽ More Extremely low density planets ('super-puffs') are a small but intriguing subset of the transiting planet population. With masses in the super-Earth range ($1-10$ M$_{\oplus}$) and radii akin to those of giant planets ($>4$ R$_{\oplus}$), their large envelopes may have been accreted beyond the water snow line and many appear to be susceptible to catastrophic mass loss. Both the presence of water and the importance of mass loss can be explored using transmission spectroscopy. Here, we present new HST WFC3 spectroscopy and updated Kepler transit depth measurements for the super-puff Kepler-79d. We do not detect any molecular absorption features in the $1.1-1.7$ $μ$m WFC3 bandpass and the combination of Kepler and WFC3 data are consistent with a flat line model, indicating the presence of aerosols in the atmosphere. We compare the shape of Kepler-79d's transmission spectrum to predictions from a microphysical haze model that incorporates an outward particle flux due to ongoing mass loss. We find that photochemical hazes offer an attractive explanation for the observed properties of super-puffs like Kepler-79d, as they simultaneously render the near-infrared spectrum featureless and reduce the inferred envelope mass loss rate by moving the measured radius (optical depth unity surface during transit) to lower pressures. We revisit the broader question of mass loss rates for super-puffs and find that the age estimates and mass loss rates for the majority of super-puffs can be reconciled if hazes move the photosphere from the typically assumed pressure of $\sim 10$ mbar to $\sim 10 \; μ$bar. △ Less

Submitted 19 August, 2020; v1 submitted 12 August, 2020; originally announced August 2020.

Comments: Awaiting publication in AJ. Small updates in Table 6 and Fig 11. Table 3 and 6 will be provided in MRT format upon publication

Journal ref: AJ, 160, 201 (2020)

Image De-raining Using a Conditional Generative Adversarial Network

Authors: He Zhang, Vishwanath Sindagi, Vishal M. Patel

Abstract: Severe weather conditions such as rain and snow adversely affect the visual quality of images captured under such conditions thus… ▽ More Severe weather conditions such as rain and snow adversely affect the visual quality of images captured under such conditions thus rendering them useless for further usage and sharing. In addition, such degraded images drastically affect performance of vision systems. Hence, it is important to solve the problem of single image de-raining/de-snowing. However, this is a difficult problem to solve due to its inherent ill-posed nature. Existing approaches attempt to introduce prior information to convert it into a well-posed problem. In this paper, we investigate a new point of view in addressing the single image de-raining problem. Instead of focusing only on deciding what is a good prior or a good framework to achieve good quantitative and qualitative performance, we also ensure that the de-rained image itself does not degrade the performance of a given computer vision algorithm such as detection and classification. In other words, the de-rained result should be indistinguishable from its corresponding clear image to a given discriminator. This criterion can be directly incorporated into the optimization framework by using the recently introduced conditional generative adversarial networks (GANs). To minimize artifacts introduced by GANs and ensure better visual quality, a new refined loss function is introduced. Based on this, we propose a novel single image de-raining method called Image De-raining Conditional General Adversarial Network (ID-CGAN), which considers quantitative, visual and also discriminative performance into the objective function. Experiments evaluated on synthetic images and real images show that the proposed method outperforms many recent state-of-the-art single image de-raining methods in terms of quantitative and visual performance. △ Less

Submitted 2 June, 2019; v1 submitted 20 January, 2017; originally announced January 2017.

Mountain Peak Detection in Online Social Media

Authors: Roman Fedorov

Abstract: …that, given as input a geo-tagged photograph, estimates its FOV (Field Of View) and the direction of the camera using a matching algorithm on the photograph edge maps and a rendered view of the mountain silhouettes that should be seen from the observer's point of view. The extraction algorithm then identifies the mountain peaks present in the photograph… ▽ More We present a system for the classification of mountain panoramas from user-generated photographs followed by identification and extraction of mountain peaks from those panoramas. We have developed an automatic technique that, given as input a geo-tagged photograph, estimates its FOV (Field Of View) and the direction of the camera using a matching algorithm on the photograph edge maps and a rendered view of the mountain silhouettes that should be seen from the observer's point of view. The extraction algorithm then identifies the mountain peaks present in the photograph and their profiles. We discuss possible applications in social fields such as photograph peak tagging on social portals, augmented reality on mobile devices when viewing a mountain panorama, and generation of collective intelligence systems (such as environmental models) from massive social media collections (e.g. snow water availability maps based on mountain peak states extracted from photograph hosting services). △ Less

Submitted 12 August, 2015; originally announced August 2015.

Comments: M.Sc. Thesis, Politecnico di Milano, Italy, 2013