Showing 1–49 of 49 results for abstract: "cloth simulation"

A Simple Gripper Interface for Simulator-Agnostic Cloth Manipulation

Authors: Abhilash Nayak, Franco Coltraro, Maria Alberich-Carramiñana, Carme Torras

Abstract: …to discrete cloth positions and a mechanism for imposing target positions as constraints. We implement our grasping model in conjunction with a constraint-based inextensible cloth simulator, where grasping is implemented as moving positional equality constraints coupled with stretch, shear, collision, and table contact… ▽ More This paper presents a grasping model for cloth manipulation specifically tailored to ease the deployment of robotic control methods. The model is robust, fast and easy to implement avoiding at the same time contact and friction considerations between the gripper and the cloth in favor of simple positional constraints. The gripper is described by its pose, jaw state, and an attached grasping volume. Two kinds of grasping volumes are considered: an axis-aligned box to simulate a pinch grasping and a square pyramidal volume to simulate point grasping. When the gripper closes, the discrete cloth positions lying inside this volume are selected, stored in the local gripper frame, and then transported with the gripper motion. A simple squeezing step is also included to progressively move the selected cloth positions toward the center of the grasping region, avoiding an instantaneous displacement at closure. The model can be used in any simulator as it only requires access to discrete cloth positions and a mechanism for imposing target positions as constraints. We implement our grasping model in conjunction with a constraint-based inextensible cloth simulator, where grasping is implemented as moving positional equality constraints coupled with stretch, shear, collision, and table contact projection steps. The same gripper trajectory is applied on a robot arm to fold a real piece of cloth, serving as a simple bridge between simulation and physical cloth manipulation and showcasing the realism and practicality of our idealized grasping model. △ Less

Submitted 28 September, 2026; v1 submitted 24 September, 2026; originally announced September 2026.

Comments: Accepted for a special session on Hybrid Intelligence at the 23rd International Conference on Informatics in Control, Automation and Robotics, 26-28 October, 2026

Learnable Persistent Wrinkle Formation in Cloth Simulation

Authors: Deshan Gong, Ningtao Mao, Xiaoyuan Yang, Xinyu Lu, He Wang, Taku Komura

Abstract: …(elastic), recoverable (friction-driven), and unrecoverable (plastic)), from cyclic loading-unloading measurements. Building on this data, we propose a differentiable cloth simulator combining an elasto-plastic model with friction, designed to capture recoverable and unrecoverable wrinkle formation. Our simulator is di… ▽ More The mechanical memory of fabrics often leads to persistent wrinkles, which reflect key physical properties and habitual wear patterns. Simulating these wrinkles accurately is essential for visual plausibility in digital garments, yet no dedicated approach exists for inferring the parameters that govern their formation due to the lack of precise datasets and estimation methods. We introduce Fabric-101, an inclusive, accurate, and extendable fabric dataset comprising over 101 common fabrics following textile standards. Unlike existing datasets, it captures three physically distinct deformation components (i.e., self-recoverable (elastic), recoverable (friction-driven), and unrecoverable (plastic)), from cyclic loading-unloading measurements. Building on this data, we propose a differentiable cloth simulator combining an elasto-plastic model with friction, designed to capture recoverable and unrecoverable wrinkle formation. Our simulator is differentiable and uses adjoint method to learn fabric physical parameters from the measured hysteresis curves, learning fabric-specific wrinkle behaviors. Through extensive experiments, we demonstrate that our model reproduces persistent wrinkles that are visually and physically similar to real fabrics across diverse materials and motions. Dataset and code are available in https://github.com/GongDeshan/Fabric_101_for_Wrinkles. △ Less

Submitted 12 September, 2026; originally announced September 2026.

Comments: A paper accepted in the SIGRRAPH Asia 2026 conference

HyperBones: Realtime Bone-driven Neural Garment Simulation with Hypernetwork Conditioning

Authors: Astitva Srivastava, Hsiao-Yu Chen, Ryan Goldade, Philipp Herholz, Zhongshi Jiang, Gene Wei-Chin Lin, Lingchen Yang, Nikolaos Sarafianos, Tuur Stuyck, Avinash Sharma, Egor Larionov

Abstract: Recent advances in cloth simulation have led to accurate garment physics, but the methods are computationally expensive for real-time applications. In contrast, Linear Blend Skinning (LBS) is efficient, but cannot capture the complex dynamics of loose-fitting garments, leading to unrealistic motion and visual artifacts… ▽ More Recent advances in cloth simulation have led to accurate garment physics, but the methods are computationally expensive for real-time applications. In contrast, Linear Blend Skinning (LBS) is efficient, but cannot capture the complex dynamics of loose-fitting garments, leading to unrealistic motion and visual artifacts. Neural methods offer a promising alternative, yet they still struggle to animate loose clothing plausibly under strict runtime constraints. We present a fast and physically-informed framework for dynamic garment simulation, consisting of a reduced-space neural dynamics simulator with independent coarse and fine-level components. At the coarse level, the garment is driven by virtual bones integrated with a lightweight neural network for predicting corrections over LBS. Fine-scale wrinkle details are then recovered using a convolutional MLP defined in UV space. By decoupling identity-specific computation from shape conditioning via hypernetwork, our neural framework offers high performance, trained using an effective physics-based self-supervised training paradigm without relying on an offline simulator. Experiments show that our method produces physically plausible garment dynamics, generalizes across diverse motions and unseen body shapes, and delivers over 30x speedup compared to state-of-the-art autoregressive neural simulators, achieving interactive inference at ~1 ms per frame on a consumer GPU. △ Less

Submitted 4 September, 2026; v1 submitted 19 May, 2026; originally announced May 2026.

Comments: Project page is available at http://sarcastitva.me/publications/hyperbones

ClothTransformer: Unified Latent-Space Transformers for Scalable Cloth Simulation

Authors: Yu Zhang, Yidi Shao, Wenqi Ouyang, Yushi Lan, Zhexin Liang, Chengrui Wu, Xudong Xu, Xingang Pan

Abstract: …rendering processes, and motion in videos. In this work, we take a step further by investigating whether modern Transformer techniques can tackle the challenging task of cloth… ▽ More Unified and scalable Transformers have recently achieved remarkable success in modeling diverse phenomena traditionally associated with computer graphics, such as 3D visual effects, rendering processes, and motion in videos. In this work, we take a step further by investigating whether modern Transformer techniques can tackle the challenging task of cloth simulation. To this end, we present ClothTransformer, a framework that reformulates cloth simulation as autoregressive sequence modeling in a learned latent space. Existing neural cloth simulators are largely specialized to single scenarios, intrinsically coupled to the mesh discretization, and lack robust collision handling. Our approach addresses these limitations through three contributions: (1) a unified Transformer architecture that handles diverse scenarios -- body-driven garments, robotic manipulation, and free-fall collisions -- under a single model and achieves approximately $4$--$9{\times}$ lower error than prior state-of-the-art methods across all scenarios; (2) a scalable latent-space formulation that compresses arbitrary-resolution meshes into a fixed-size set of latent tokens, making temporal dynamics computation independent of mesh resolution; and (3) a diverse-scenario high-fidelity penetration-free dataset of ${\sim}$493.4k frames spanning all three settings, which enables a differentiable Continuous Collision Detection (CCD) module to suppress penetration artifacts. Project Page: https://yucrazing.github.io/clothtransformer/ △ Less

Submitted 22 July, 2026; v1 submitted 26 May, 2026; originally announced May 2026.

Wearing A Coat: Dual-Arm Robot-Assisted Dressing with Differentiable Clothing Simulation

Authors: Yiming Liu, Lijun Han, Hesheng Wang

Abstract: …the partial worn garments under contact constraints. To overcome this challenge, we propose a novel robotic dressing control algorithm that integrates realtime differentiable clothing simulation. The simulation algorithm employs explicit iterative scheme with intentionally introduced higher-order perturbations to enhan… ▽ More The development of assistive robots for dressing tasks serves to augment human convenience and improve the quality of life for individuals with physical impairments. However, due to the intricate contact interactions between garments and the human limbs during dressing, most robot-assisted dressing algorithms treat clothing as an assembly of discrete segments, thereby struggling to manage the partial worn garments under contact constraints. To overcome this challenge, we propose a novel robotic dressing control algorithm that integrates realtime differentiable clothing simulation. The simulation algorithm employs explicit iterative scheme with intentionally introduced higher-order perturbations to enhance computational efficiency while maintaining stability under large time-step conditions. Through simulation, we resolve the garment state under contact constraints, which then enables a multi-phase control strategy for successful coat dressing assistance. To further improve real-time performance, we introduce a constrained local model along with its corresponding optimization solver, permitting high-frequency local compensation for the differentiable simulation based global controller. Finally, we experimentally validate our approach through both simulated and physical dressing scenarios, conclusively demonstrating its feasibility and efficacy △ Less

Submitted 12 July, 2026; originally announced July 2026.

PatternGSL: A Structured Specification Language for Template-Free and Simulation-Ready 3D Garments

Authors: Zhenyang Li, Lutao Jiang, Yizhou Zhao, Ying-Cong Chen, Xin Wang, Weikai Chen, Yifan Peng

Abstract: …VLM training for structured garment reconstruction. Experiments demonstrate improved pattern accuracy over prior baselines, explicit sewing-structure recovery, reliable cloth simulation, and pattern-level editing through the same deterministic decoding pipeline. Code and data-processing scripts will be released at http… ▽ More Reconstructing realistic, physically plausible garments from a single image remains a fundamental challenge. Template-free methods capture surface geometry but lack explicit sewing structure for simulation; while programmatic systems are simulation-ready but constrained by predefined templates. This reveals a fundamental representation gap between geometric reconstruction and structured garment construction. We present PatternGSL, a structured garment representation in the form of a template-free and learnable specification language that encodes complete sewing patterns, including panel boundaries, parameterized seams, and explicit stitch topology, in a compact and standardized form. PatternGSL preserves the physical rigor of pattern-based models while removing template dependence, elevating sewing structure as a first-class target for generative modeling. We further propose a vision-language framework that predicts PatternGSL specifications directly from a single image and decodes them into garments using lightweight deterministic validity handling, without optimization-based refinement or manual cleanup. In addition, we introduce PatternGSLData, the first large-scale image-to-GSL paired dataset comprising 300K samples with complete sewing pattern annotations, enabling supervised VLM training for structured garment reconstruction. Experiments demonstrate improved pattern accuracy over prior baselines, explicit sewing-structure recovery, reliable cloth simulation, and pattern-level editing through the same deterministic decoding pipeline. Code and data-processing scripts will be released at https://lagrangeli.github.io/PatternGSL/. △ Less

Submitted 1 July, 2026; v1 submitted 23 June, 2026; originally announced June 2026.

Comments: 11 pages, 6 figures

EASE: Parametric garment design with explicit and local ease control

Authors: Kristijan Bartol, Frieda Hentschel, Nataliya Sadretdinova, Benjamin Russig, Melinos Averkiou, Yordan Kyosev, Stefan Gumhold

Abstract: …mitigated for target poses, and the ease distribution is accurately transferred to target shapes. The approach is implemented as a virtual try-on framework, with physics-based cloth simulation used for final garment visualization. We will publicly release our framework and detailed documentation. ▽ More Garment fit and comfort depend critically on ease, the local allowance of excess material relative to the body. In existing design pipelines, ease is typically a byproduct of geometry or simulation rather than an independent design variable, making it difficult to specify, edit, transfer, or redistribute without re-running simulation or optimization. We propose a garment representation that embeds meshes directly on the surface of a parametric human body model and represents ease explicitly as spatially varying, anisotropic per-triangle scales. These scales act as primary design variables, decoupling the specification of material allowance from its physical deformation. Given a design specified by parametric and user-defined surface cuts together with local scale fields, we optimize sewing patterns that enforce the prescribed ease distribution while satisfying geometric and seam constraints. The representation enables three capabilities that are unavailable without explicit ease control: (1) direct specification and editing of local material allowance on the body surface; (2) intent-preserving transfer to new body shapes that reproduces the specified ease distribution without re-running simulation; and (3) intent-modifying pose adaptation that redistributes ease to relieve strain in high-stretch regions. We verify each of these experimentally: ease is closely retained after optimization, excessive strain is significantly mitigated for target poses, and the ease distribution is accurately transferred to target shapes. The approach is implemented as a virtual try-on framework, with physics-based cloth simulation used for final garment visualization. We will publicly release our framework and detailed documentation. △ Less

Submitted 28 June, 2026; originally announced June 2026.

Comments: Special Section on SMI2026 (Shape Modeling International). Link to official publication: https://www.sciencedirect.com/science/article/pii/S0097849326001159

Report number: 104644

Journal ref: Computer & Graphics, Volume 138, 2026 (12 pages)

Locality-Aware Automatic Differentiation on the GPU for Mesh-Based Computations

Authors: Ahmed H. Mahmoud, Rahul Goel, Jonathan Ragan-Kelley, Justin Solomon

Abstract: …dynamic interaction-driven sparsity updates, and seamless integration with external GPU sparse linear solvers. We evaluate our system on applications including elastic and cloth simulation, surface parameterization, mesh smoothing, frame field design, ARAP deformation, and spherical manifold optimization. Across these… ▽ More We present a GPU-based system for automatic differentiation (AD) of functions defined on triangle meshes, designed to exploit the locality and sparsity in mesh-based computation. Our system evaluates derivatives using per-element forward-mode AD, confining all computation to registers and shared memory and assembling global gradients, sparse Jacobians, and sparse Hessians directly on the GPU. By avoiding global computation graphs, intermediate buffers, and device-host synchronization, our approach minimizes memory traffic and enables efficient differentiation under both static and dynamically changing sparsity. Our programming model lets users express energy terms over mesh neighborhoods, while our system automatically manages parallel execution, derivative propagation, sparse assembly, and matrix-free operations such as Hessian-vector products. Our system supports both scalar- and vector-valued objectives, dynamic interaction-driven sparsity updates, and seamless integration with external GPU sparse linear solvers. We evaluate our system on applications including elastic and cloth simulation, surface parameterization, mesh smoothing, frame field design, ARAP deformation, and spherical manifold optimization. Across these tasks, our system consistently outperforms state-of-the-art differentiation frameworks, including PyTorch, JAX, Warp, DrJIT, EnzymeAD, and Thallo. We demonstrate speedups across a range of solver types, from Newton and Gauss-Newton for nonlinear least squares to L-BFGS and gradient descent, and across different derivative usage modes, including Hessian-vector products as well as full sparse Hessian and Jacobian construction. Our system is available as open source at https://github.com/owensgroup/RXMesh. △ Less

Submitted 3 June, 2026; v1 submitted 30 August, 2025; originally announced September 2025.

Comments: SIGGRAPH 2026

Journal ref: ACM Transactions on Graphics, 45(4), July 2026

Dynamic robotic cloth folding with efficient Koopman operator-based model predictive control

Authors: Edoardo Caldarelli, Franco Coltraro, Adrià Colomé, Lorenzo Rosasco, Carme Torras

Abstract: …for nonlinear system identification, is used to obtain a linear model for the cloth being folded. Such a surrogate model, trained with data from a high-fidelity, physics-based cloth simulator, can then be employed within a suitable model predictive control algorithm, in place of the costly, nonlinear one, to efficientl… ▽ More Robotic cloth folding is a challenging task, particularly when considering dynamic folding tasks, which aim at folding cloth by fast motions that leverage its dynamics. When subject to such fast motions, the complexity of cloth dynamics hinders both system identification and planning of folding trajectories, resulting in a difficult simulation-to-reality transfer when using physical models of cloth. Compared to the dexterity that humans exhibit when performing folding tasks, robotic approaches usually employ small garments with quite rigid dynamics, and are either too slow, or fast but imprecise, requiring several attempts to achieve a reasonably good fold. In this paper, we tackle these challenges by generating fast folding trajectories with a novel model predictive controller, integrating physics-based simulation of cloth dynamics and efficient, kernel-based Koopman operator regression. Koopman operator regression, an increasingly popular machine learning technique for nonlinear system identification, is used to obtain a linear model for the cloth being folded. Such a surrogate model, trained with data from a high-fidelity, physics-based cloth simulator, can then be employed within a suitable model predictive control algorithm, in place of the costly, nonlinear one, to efficiently generate folding trajectories to be executed by a robotic manipulator. Both in simulated and real-robot experiments, we show how the linearization supplied by the Koopman operator-based model can be employed to efficiently generate fast folding trajectories to unseen poses, without sacrificing folding accuracy. △ Less

Submitted 18 May, 2026; originally announced May 2026.

Comments: Accepted for presentation at the 2026 IEEE International Conference on Robotics and Automation (ICRA)

Velox: Learning Representations of 4D Geometry and Appearance

Authors: Anagh Malik, Dorian Chan, Xiaoming Zhao, David B. Lindell, Oncel Tuzel, Jen-Hao Rick Chang

Abstract: …Gaussians, helping learn appearance. To demonstrate the utility of our representation, we evaluate it across three downstream tasks -- video-to-4D generation, 3D tracking, and cloth simulation via image-to-4D generation -- and observe strong performances in all settings. ▽ More We introduce a framework for learning latent representations of 4D objects which are descriptive, faithfully capturing object geometry and appearance; compressive, aiding in downstream efficiency; and accessible, requiring minimal input, i.e., an unstructured dynamic point cloud, to construct. Specifically, Velox trains an encoder to compress spatiotemporal color point clouds into a set of dynamic shape tokens. These tokens are supervised using two complementary decoders: a 4D surface decoder, which models the time-varying surface distribution capturing the geometry; and a Gaussian decoder, which maps the tokens to 3D Gaussians, helping learn appearance. To demonstrate the utility of our representation, we evaluate it across three downstream tasks -- video-to-4D generation, 3D tracking, and cloth simulation via image-to-4D generation -- and observe strong performances in all settings. △ Less

Submitted 6 May, 2026; originally announced May 2026.

Comments: CVPR 2026, Project page: https://apple.github.io/ml-velox

Efficient B-Spline Finite Elements for Cloth Simulation

Authors: Yuqi Meng, Yihao Shi, Kemeng Huang, Zixuan Lu, Ning Guo, Taku Komura, Yin Yang, Minchen Li

Abstract: We present an efficient B-spline finite element method (FEM) for cloth… ▽ More We present an efficient B-spline finite element method (FEM) for cloth simulation. While higher-order FEM has long promised higher accuracy, its adoption in cloth simulators has been limited by its larger computational costs while generating results with similar visual quality. Our contribution is a full algorithmic pipeline that makes cloth simulation using quadratic B-spline surfaces faster than standard linear FEM in practice while consistently improving accuracy and visual fidelity. Using quadratic B-spline basis functions, we obtain a globally $C^1$-continuous displacement field that supports consistent discretization of both membrane and bending energies, effectively reducing locking artifacts and mesh dependence common to linear elements. To close the performance gap, we introduce a reduced integration scheme that separately optimizes quadrature rules for membrane and bending energies, an accelerated Hessian assembly procedure tailored to the spline structure, and an optimized linear solver based on partial factorization. Together, these optimizations make high-order, smooth cloth simulation competitive at scale, yielding an average $2\times$ speedup over linear FEM in our tests. Extensive experiments demonstrate improved accuracy, wrinkle detail, and robustness, including contact-rich scenarios, relative to linear FEM and recent higher-order approaches. Our method enables realistic wrinkling dynamics across a wide range of material parameters and supports practical garment animation, providing a new promising spatial discretization for high-quality cloth simulation. △ Less

Submitted 1 May, 2026; v1 submitted 23 June, 2025; originally announced June 2025.

Comments: 25 pages, 28 figures

D-Garment: Physically Grounded Latent Diffusion for Dynamic Garment Deformations

Authors: Antoine Dumoulin, Adnane Boukhayma, Laurence Boissieux, Bharath Bhushan Damodaran, Pierre Hellier, Stefanie Wuhrer

Abstract: …metrics such as strain or curvature. Existing work studies pose-dependent garment modeling to generate garment deformations from example data, and possibly data-driven dynamic cloth simulation to generate realistic garments in motion. We propose D-Garment, a learning-based approach trained on new data generated with a… ▽ More We present a method to dynamically deform 3D garments, in the form of a 3D polygon mesh, based on body shape, motion, and physical cloth material properties. Considering physical cloth properties allows to learn a physically grounded model, with the advantage of being more accurate in terms of physically inspired metrics such as strain or curvature. Existing work studies pose-dependent garment modeling to generate garment deformations from example data, and possibly data-driven dynamic cloth simulation to generate realistic garments in motion. We propose D-Garment, a learning-based approach trained on new data generated with a physics-based simulator. Compared to prior work, our 3D generative model learns garment deformations conditioned by physical material properties, which allows to model loose cloth geometry, especially for large deformations and dynamic wrinkles driven by body motion. Furthermore, the model can be efficiently fitted to observations captured using vision sensors such as 3D point clouds. We leverage the capability of diffusion models to learn flexible and powerful generative priors by modeling the 3D garment in a 2D parameter space independently from the mesh resolution. This representation allows to learn a template-specific latent diffusion model. This allows to condition global and local geometry with body and cloth material information. We quantitatively and qualitatively evaluate D-Garment on both simulations and data captured with a multi-view acquisition platform. Compared to recent baselines, our method is more realistic and accurate in terms of shape similarity and physical validity metrics. Code and data are available for research purposes at https://dumoulina.github.io/d-garment/ △ Less

Submitted 8 April, 2026; v1 submitted 4 April, 2025; originally announced April 2025.

Comments: 18 pages, 11 figures

Journal ref: Transactions on Machine Learning Research (TMLR), 2026

DInf-Grid: A Neural Differential Equation Solver with Differentiable Feature Grids

Authors: Navami Kairanda, Shanthika Naik, Marc Habermann, Avinash Sharma, Christian Theobalt, Vladislav Golyanik

Abstract: …on a variety of tasks, including the Poisson equation for image reconstruction, the Helmholtz equation for wave fields, and the Kirchhoff-Love boundary value problem for cloth simulation. Our results demonstrate a 5-20x speed-up over coordinate-based MLP-based methods, solving differential equations in seconds or minut… ▽ More We present a novel differentiable grid-based representation for efficiently solving differential equations (DEs). Widely used architectures for neural solvers, such as sinusoidal neural networks, are coordinate-based MLPs that are both computationally intensive and slow to train. Although grid-based alternatives for implicit representations (e.g., Instant-NGP and K-Planes) train faster by exploiting signal structure, their reliance on linear interpolation restricts their ability to compute higher-order derivatives, rendering them unsuitable for solving DEs. Our approach overcomes these limitations by combining the efficiency of feature grids with radial basis function interpolation, which is infinitely differentiable. To effectively capture high-frequency solutions and enable stable and faster computation of global gradients, we introduce a multi-resolution decomposition with co-located grids. Our proposed representation, DInf-Grid, is trained implicitly using the differential equations as loss functions, enabling accurate modelling of physical fields. We validate DInf-Grid on a variety of tasks, including the Poisson equation for image reconstruction, the Helmholtz equation for wave fields, and the Kirchhoff-Love boundary value problem for cloth simulation. Our results demonstrate a 5-20x speed-up over coordinate-based MLP-based methods, solving differential equations in seconds or minutes while maintaining comparable accuracy and compactness. △ Less

Submitted 15 January, 2026; originally announced January 2026.

Comments: 25 pages; 16 figures; project page: https://4dqv.mpi-inf.mpg.de/DInf-Grid/

FNOPT: Resolution-Agnostic, Self-Supervised Cloth Simulation using Meta-Optimization with Fourier Neural Operators

Authors: Ruochen Chen, Thuy Tran, Shaifali Parashar

Abstract: We present FNOpt, a self-supervised cloth… ▽ More We present FNOpt, a self-supervised cloth simulation framework that formulates time integration as an optimization problem and trains a resolution-agnostic neural optimizer parameterized by a Fourier neural operator (FNO). Prior neural simulators often rely on extensive ground truth data or sacrifice fine-scale detail, and generalize poorly across resolutions and motion patterns. In contrast, FNOpt learns to simulate physically plausible cloth dynamics and achieves stable and accurate rollouts across diverse mesh resolutions and motion patterns without retraining. Trained only on a coarse grid with physics-based losses, FNOpt generalizes to finer resolutions, capturing fine-scale wrinkles and preserving rollout stability. Extensive evaluations on a benchmark cloth simulation dataset demonstrate that FNOpt outperforms prior learning-based approaches in out-of-distribution settings in both accuracy and robustness. These results position FNO-based meta-optimization as a compelling alternative to previous neural simulators for cloth, thus reducing the need for curated data and improving cross-resolution reliability. △ Less

Submitted 5 December, 2025; originally announced December 2025.

Comments: Accepted for WACV

MonoCloth: Reconstruction and Animation of Cloth-Decoupled Human Avatars from Monocular Videos

Authors: Daisheng Jin, Ying He

Abstract: …difficulty and deformation complexity across these components. Specifically, we focus on detailed geometry recovery for the face and hands. For clothing, we propose a dedicated cloth simulation module that captures garment deformation using temporal motion cues and geometric constraints. Experimental results demonstrat… ▽ More Reconstructing realistic 3D human avatars from monocular videos is a challenging task due to the limited geometric information and complex non-rigid motion involved. We present MonoCloth, a new method for reconstructing and animating clothed human avatars from monocular videos. To overcome the limitations of monocular input, we introduce a part-based decomposition strategy that separates the avatar into body, face, hands, and clothing. This design reflects the varying levels of reconstruction difficulty and deformation complexity across these components. Specifically, we focus on detailed geometry recovery for the face and hands. For clothing, we propose a dedicated cloth simulation module that captures garment deformation using temporal motion cues and geometric constraints. Experimental results demonstrate that MonoCloth improves both visual reconstruction quality and animation realism compared to existing methods. Furthermore, thanks to its part-based design, MonoCloth also supports additional tasks such as clothing transfer, underscoring its versatility and practical utility. △ Less

Submitted 15 November, 2025; v1 submitted 6 August, 2025; originally announced August 2025.

Real Garment Benchmark (RGBench): A Comprehensive Benchmark for Robotic Garment Manipulation featuring a High-Fidelity Scalable Simulator

Authors: Wenkang Hu, Xincheng Tang, Yanzhi E, Yitong Li, Zhengjie Shu, Wei Li, Huamin Wang, Ruigang Yang

Abstract: …protocol to evaluate garment simulation quality with carefully measured real garment dynamics. Our experiments demonstrate that our simulator outperforms currently available cloth simulators by a large margin, reducing simulation error by 20% while maintaining a speed of 3 times faster. We will publicly release RGBench… ▽ More While there has been significant progress to use simulated data to learn robotic manipulation of rigid objects, applying its success to deformable objects has been hindered by the lack of both deformable object models and realistic non-rigid body simulators. In this paper, we present Real Garment Benchmark (RGBench), a comprehensive benchmark for robotic manipulation of garments. It features a diverse set of over 6000 garment mesh models, a new high-performance simulator, and a comprehensive protocol to evaluate garment simulation quality with carefully measured real garment dynamics. Our experiments demonstrate that our simulator outperforms currently available cloth simulators by a large margin, reducing simulation error by 20% while maintaining a speed of 3 times faster. We will publicly release RGBench to accelerate future research in robotic garment manipulation. Website: https://rgbench.github.io/ △ Less

Submitted 12 November, 2025; v1 submitted 9 November, 2025; originally announced November 2025.

Comments: 2026 AAAI Accept

Real-Time Cloth Simulation Using WebGPU: Evaluating Limits of High-Resolution

Authors: Nak-Jun Sung, Jun Ma, TaeHeon Kim, Yoo-joo Choi, Min-Hyung Choi, Min Hong

Abstract: This study explores the capabilities of WebGPU, an emerging web graphics paradigm, for real-time cloth… ▽ More This study explores the capabilities of WebGPU, an emerging web graphics paradigm, for real-time cloth simulation. Traditional WebGL-based methods have been in handling complex physical simulations due to their emphasis on graphics rendering rather than general-purpose GPU (GPGPU) operations. WebGPU, designed to provide modern 3D graphics and computational capabilities, offers significant improvements through parallel processing and support for computational shaders. In this work, we implemented a cloth simulation system using the Mass-Spring Method within the WebGPU framework, integrating collision detection and response handling with the 3D surface model. First, comparative performance evaluations demonstrate that WebGPU substantially outperforms WebGL, particularly in high-resolution simulations, maintaining 60 frames per second (fps) even with up to 640K nodes. The second experiment aimed to determine the real-time limitations of WebGPU and confirmed that WebGPU can handle real-time collisions between 4K and 100k cloth node models and a 100K triangle surface model in real-time. These experiments also highlight the importance of balancing real-time performance with realistic rendering when handling collisions between cloth models and complex 3D objects. Our source code is available at https://github.com/nakjun/Cloth-Simulation-WebGPU △ Less

Submitted 15 July, 2025; originally announced July 2025.

Metamizer: a versatile neural optimizer for fast and accurate physics simulations

Authors: Nils Wandel, Stefan Schulz, Reinhard Klein

Abstract: …- sometimes approaching machine precision - across multiple PDEs after training on the Laplace, advection-diffusion and incompressible Navier-Stokes equation as well as on cloth simulations. Remarkably, the model also generalizes to PDEs that were not covered during training such as the Poisson, wave and Burgers equati… ▽ More Efficient physics simulations are essential for numerous applications, ranging from realistic cloth animations or smoke effects in video games, to analyzing pollutant dispersion in environmental sciences, to calculating vehicle drag coefficients in engineering applications. Unfortunately, analytical solutions to the underlying physical equations are rarely available, and numerical solutions require high computational resources. Latest developments in the field of physics-based Deep Learning have led to promising efficiency improvements but still suffer from limited generalization capabilities and low accuracy compared to numerical solvers. In this work, we introduce Metamizer, a novel neural optimizer that iteratively solves a wide range of physical systems with high accuracy by minimizing a physics-based loss function. To this end, our approach leverages a scale-invariant architecture that enhances gradient descent updates to accelerate convergence. Since the neural network itself acts as an optimizer, training this neural optimizer falls into the category of meta-optimization approaches. We demonstrate that Metamizer achieves unprecedented accuracy for deep learning based approaches - sometimes approaching machine precision - across multiple PDEs after training on the Laplace, advection-diffusion and incompressible Navier-Stokes equation as well as on cloth simulations. Remarkably, the model also generalizes to PDEs that were not covered during training such as the Poisson, wave and Burgers equation. Our results suggest that Metamizer could have a profound impact on future numerical solvers, paving the way for fast and accurate neural physics simulations without the need for retraining. △ Less

Submitted 20 April, 2025; v1 submitted 10 October, 2024; originally announced October 2024.

Comments: to be published at International Conference on Learning Representations (ICLR) 2025

TanDepth: Leveraging Global DEMs for Metric Monocular Depth Estimation in UAVs

Authors: Horatiu Florea, Sergiu Nedevschi

Abstract: …leverages sparse measurements from Global Digital Elevation Models (GDEM) by projecting them to the camera view using extrinsic and intrinsic information. An adaptation to the Cloth Simulation Filter is presented, which allows selecting ground points from the estimated depth map to then correlate with the projected ref… ▽ More Aerial scene understanding systems face stringent payload restrictions and must often rely on monocular depth estimation for modeling scene geometry, which is an inherently ill-posed problem. Moreover, obtaining accurate ground truth data required by learning-based methods raises significant additional challenges in the aerial domain. Self-supervised approaches can bypass this problem, at the cost of providing only up-to-scale results. Similarly, recent supervised solutions which make good progress towards zero-shot generalization also provide only relative depth values. This work presents TanDepth, a practical scale recovery method for obtaining metric depth results from relative estimations at inference-time, irrespective of the type of model generating them. Tailored for Unmanned Aerial Vehicle (UAV) applications, our method leverages sparse measurements from Global Digital Elevation Models (GDEM) by projecting them to the camera view using extrinsic and intrinsic information. An adaptation to the Cloth Simulation Filter is presented, which allows selecting ground points from the estimated depth map to then correlate with the projected reference points. We evaluate and compare our method against alternate scaling methods adapted for UAVs, on a variety of real-world scenes. Considering the limited availability of data for this domain, we construct and release a comprehensive, depth-focused extension to the popular UAVid dataset to further research. △ Less

Submitted 24 February, 2025; v1 submitted 8 September, 2024; originally announced September 2024.

Journal ref: IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 18, 2025, pp. 5445-5459

A Physics-embedded Deep Learning Framework for Cloth Simulation

Authors: Zhiwei Zhao

Abstract: Delicate cloth… ▽ More Delicate cloth simulations have long been desired in computer graphics. Various methods were proposed to improve engaged force interactions, collision handling, and numerical integrations. Deep learning has the potential to achieve fast and real-time simulation, but common neural network structures often demand many parameters to capture cloth dynamics. This paper proposes a physics-embedded learning framework that directly encodes physical features of cloth simulation. The convolutional neural network is used to represent spatial correlations of the mass-spring system, after which three branches are designed to learn linear, nonlinear, and time derivate features of cloth physics. The framework can also integrate with other external forces and collision handling through either traditional simulators or sub neural networks. The model is tested across different cloth animation cases, without training with new data. Agreement with baselines and predictive realism successfully validate its generalization ability. Inference efficiency of the proposed model also defeats traditional physics simulation. This framework is also designed to easily integrate with other visual refinement techniques like wrinkle carving, which leaves significant chances to incorporate prevailing macing learning techniques in 3D cloth amination. △ Less

Submitted 3 December, 2024; v1 submitted 19 March, 2024; originally announced March 2024.

Comments: updated version

ACM Class: I.2.0; I.3.7

Shallow Signed Distance Functions for Kinematic Collision Bodies

Authors: Osman Akar, Yushan Han, Yizhou Chen, Weixian Lan, Benn Gallagher, Ronald Fedkiw, Joseph Teran

Abstract: …shapes with modest memory requirements compared to traditional representations over dense grids. However, the computational expense of DeepSDFs prevents their use in real-time clothing simulation applications. We design a learning-based representation of SDFs for human avatars whoes bodies change shape kinematically du… ▽ More We present learning-based implicit shape representations designed for real-time avatar collision queries arising in the simulation of clothing. Signed distance functions (SDFs) have been used for such queries for many years due to their computational efficiency. Recently deep neural networks have been used for implicit shape representations (DeepSDFs) due to their ability to represent multiple shapes with modest memory requirements compared to traditional representations over dense grids. However, the computational expense of DeepSDFs prevents their use in real-time clothing simulation applications. We design a learning-based representation of SDFs for human avatars whoes bodies change shape kinematically due to joint-based skinning. Rather than using a single DeepSDF for the entire avatar, we use a collection of extremely computationally efficient (shallow) neural networks that represent localized deformations arising from changes in body shape induced by the variation of a single joint. This requires a stitching process to combine each shallow SDF in the collection together into one SDF representing the signed closest distance to the boundary of the entire body. To achieve this we augment each shallow SDF with an additional output that resolves whether or not the individual shallow SDF value is referring to a closest point on the boundary of the body, or to a point on the interior of the body (but on the boundary of the individual shallow SDF). Our model is extremely fast and accurate and we demonstrate its applicability with real-time simulation of garments driven by animated characters. △ Less

Submitted 11 November, 2024; originally announced November 2024.

Comments: Preprint

NeuralClothSim: Neural Deformation Fields Meet the Thin Shell Theory

Authors: Navami Kairanda, Marc Habermann, Christian Theobalt, Vladislav Golyanik

Abstract: Despite existing 3D cloth… ▽ More Despite existing 3D cloth simulators producing realistic results, they predominantly operate on discrete surface representations (e.g. points and meshes) with a fixed spatial resolution, which often leads to large memory consumption and resolution-dependent simulations. Moreover, back-propagating gradients through the existing solvers is difficult, and they cannot be easily integrated into modern neural architectures. In response, this paper re-thinks physically plausible cloth simulation: We propose NeuralClothSim, i.e., a new quasistatic cloth simulator using thin shells, in which surface deformation is encoded in neural network weights in the form of a neural field. Our memory-efficient solver operates on a new continuous coordinate-based surface representation called neural deformation fields (NDFs); it supervises NDF equilibria with the laws of the non-linear Kirchhoff-Love shell theory with a non-linear anisotropic material model. NDFs are adaptive: They 1) allocate their capacity to the deformation details and 2) allow surface state queries at arbitrary spatial resolutions without re-training. We show how to train NeuralClothSim while imposing hard boundary conditions and demonstrate multiple applications, such as material interpolation and simulation editing. The experimental results highlight the effectiveness of our continuous neural formulation. See our project page: https://4dqv.mpi-inf.mpg.de/NeuralClothSim/. △ Less

Submitted 7 November, 2024; v1 submitted 24 August, 2023; originally announced August 2023.

Comments: 33 pages, 23 figures and 3 tables; project page: https://4dqv.mpi-inf.mpg.de/NeuralClothSim/

Inverse Garment and Pattern Modeling with a Differentiable Simulator

Authors: Boyang Yu, Frederic Cordier, Hyewon Seo

Abstract: …on fields like shape capture in social VR, and virtual try-on in the fashion industry. To align with the garment modeling process standardized by the fashion industry as well as cloth… ▽ More The capability to generate simulation-ready garment models from 3D shapes of clothed humans will significantly enhance the interpretability of captured geometry of real garments, as well as their faithful reproduction in the virtual world. This will have notable impact on fields like shape capture in social VR, and virtual try-on in the fashion industry. To align with the garment modeling process standardized by the fashion industry as well as cloth simulation softwares, it is required to recover 2D patterns. This involves an inverse garment design problem, which is the focus of our work here: Starting with an arbitrary target garment geometry, our system estimates an animatable garment model by automatically adjusting its corresponding 2D template pattern, along with the material parameters of the physics-based simulation (PBS). Built upon a differentiable cloth simulator, the optimization process is directed towards minimizing the deviation of the simulated garment shape from the target geometry. Moreover, our produced patterns meet manufacturing requirements such as left-to-right-symmetry, making them suited for reverse garment fabrication. We validate our approach on examples of different garment types, and show that our method faithfully reproduces both the draped garment shape and the sewing pattern. △ Less

Submitted 21 October, 2024; v1 submitted 11 March, 2024; originally announced March 2024.

Efficient GPU Cloth Simulation with Non-distance Barriers and Subspace Reuse

Authors: Lei Lan, Zixuan Lu, Jingyi Long, Chun Yuan, Xuan Li, Xiaowei He, Huamin Wang, Chenfanfu Jiang, Yin Yang

Abstract: This paper pushes the performance of cloth… ▽ More This paper pushes the performance of cloth simulation, making the simulation interactive even for high-resolution garment models while keeping every triangle untangled. The penetration-free guarantee is inspired by the interior point method, which converts the inequality constraints to barrier potentials. We propose a major overhaul of this modality within the projective dynamics framework by leveraging an adaptive weighting mechanism inspired by barrier formulation. This approach does not depend on the distance between mesh primitives, but on the virtual life span of a collision event and thus keeps all the vertices within feasible region. Such a non-distance barrier model allows a new way to integrate collision resolution into the simulation pipeline. Another contributor to the performance boost comes from the subspace reuse strategy. This is based on the observation that low-frequency strain propagation is near orthogonal to the deformation induced by collisions or self-collisions, often of high frequency. Subspace reuse then takes care of low-frequency residuals, while high-frequency residuals can also be effectively smoothed by GPU-based iterative solvers. We show that our method outperforms existing fast cloth simulators by at least one order while producing high-quality animations of high-resolution models. △ Less

Submitted 27 September, 2024; v1 submitted 28 March, 2024; originally announced March 2024.

Implicit frictional dynamics with soft constraints

Authors: Egor Larionov, Andreas Longva, Uri M. Ascher, Jan Bender, Dinesh K. Pai

Abstract: Dynamics simulation with frictional contacts is important for a wide range of applications, from cloth simulation to object manipulation. Recent methods using smoothed lagged friction forces have enabled robust and differentiable simulation of elastodynamics with friction. However, the resulting frictional behavior can… ▽ More Dynamics simulation with frictional contacts is important for a wide range of applications, from cloth simulation to object manipulation. Recent methods using smoothed lagged friction forces have enabled robust and differentiable simulation of elastodynamics with friction. However, the resulting frictional behavior can be inaccurate and may not converge to analytic solutions. Here we evaluate the accuracy of lagged friction models in comparison with implicit frictional contact systems. We show that major inaccuracies near the stick-slip threshold in such systems are caused by lagging of friction forces rather than by smoothing the Coulomb friction curve. Furthermore, we demonstrate how systems involving implicit or lagged friction can be correctly used with higher-order time integration and highlight limitations in earlier attempts. We demonstrate how to exploit forward-mode automatic differentiation to simplify and, in some cases, improve the performance of the inexact Newton method. Finally, we show that other complex phenomena can also be simulated effectively while maintaining smoothness of the entire system. We extend our method to exhibit stick-slip frictional behavior and preserve volume on compressible and nearly-incompressible media using soft constraints. △ Less

Submitted 31 July, 2024; v1 submitted 19 November, 2022; originally announced November 2022.

SENC: Handling Self-collision in Neural Cloth Simulation

Authors: Zhouyingcheng Liao, Sinan Wang, Taku Komura

Abstract: We present SENC, a novel self-supervised neural cloth simulator that addresses the challenge of cloth self-collision. This problem has remained unresolved due to the gap in simulation setup between recent collision detection and response approaches and self-supervised neural simulators. The former requires collision-fr… ▽ More We present SENC, a novel self-supervised neural cloth simulator that addresses the challenge of cloth self-collision. This problem has remained unresolved due to the gap in simulation setup between recent collision detection and response approaches and self-supervised neural simulators. The former requires collision-free initial setups, while the latter necessitates random cloth instantiation during training. To tackle this issue, we propose a novel loss based on Global Intersection Analysis (GIA). This loss extracts the volume surrounded by the cloth region that forms the penetration. By constructing an energy based on this volume, our self-supervised neural simulator can effectively address cloth self-collisions. Moreover, we develop a self-collision-aware graph neural network capable of learning to handle self-collisions, even for parts that are topologically distant from one another. Additionally, we introduce an effective external force scheme that enables the simulation to learn the cloth's behavior in response to random external forces. We validate the efficacy of SENC through extensive quantitative and qualitative experiments, demonstrating that it effectively reduces cloth self-collision while maintaining high-quality animation results. △ Less

Submitted 17 July, 2024; originally announced July 2024.

Comments: Accepted at ECCV 2024

WordRobe: Text-Guided Generation of Textured 3D Garments

Authors: Astitva Srivastava, Pranav Manu, Amit Raj, Varun Jampani, Avinash Sharma

Abstract: …texture synthesis, supported by quantitative evaluation and qualitative user study. The unposed 3D garment meshes generated using WordRobe can be directly fed to standard cloth simulation & animation pipelines without any post-processing. ▽ More In this paper, we tackle a new and challenging problem of text-driven generation of 3D garments with high-quality textures. We propose "WordRobe", a novel framework for the generation of unposed & textured 3D garment meshes from user-friendly text prompts. We achieve this by first learning a latent representation of 3D garments using a novel coarse-to-fine training strategy and a loss for latent disentanglement, promoting better latent interpolation. Subsequently, we align the garment latent space to the CLIP embedding space in a weakly supervised manner, enabling text-driven 3D garment generation and editing. For appearance modeling, we leverage the zero-shot generation capability of ControlNet to synthesize view-consistent texture maps in a single feed-forward inference step, thereby drastically decreasing the generation time as compared to existing methods. We demonstrate superior performance over current SOTAs for learning 3D garment latent space, garment interpolation, and text-driven texture synthesis, supported by quantitative evaluation and qualitative user study. The unposed 3D garment meshes generated using WordRobe can be directly fed to standard cloth simulation & animation pipelines without any post-processing. △ Less

Submitted 14 July, 2024; v1 submitted 26 March, 2024; originally announced March 2024.

ContourCraft: Learning to Resolve Intersections in Neural Multi-Garment Simulations

Authors: Artur Grigorev, Giorgio Becherini, Michael J. Black, Otmar Hilliges, Bernhard Thomaszewski

Abstract: Learning-based approaches to cloth… ▽ More Learning-based approaches to cloth simulation have started to show their potential in recent years. However, handling collisions and intersections in neural simulations remains a largely unsolved problem. In this work, we present \moniker{}, a learning-based solution for handling intersections in neural cloth simulations. Unlike conventional approaches that critically rely on intersection-free inputs, \moniker{} robustly recovers from intersections introduced through missed collisions, self-penetrating bodies, or errors in manually designed multi-layer outfits. The technical core of \moniker{} is a novel intersection contour loss that penalizes interpenetrations and encourages rapid resolution thereof. We integrate our intersection loss with a collision-avoiding repulsion objective into a neural cloth simulation method based on graph neural networks (GNNs). We demonstrate our method's ability across a challenging set of diverse multi-layer outfits under dynamic human motions. Our extensive analysis indicates that \moniker{} significantly improves collision handling for learned simulation and produces visually compelling results. △ Less

Submitted 24 May, 2024; v1 submitted 15 May, 2024; originally announced May 2024.

Comments: Accepted for publication by SIGGRAPH 2024, conference track

4D-DRESS: A 4D Dataset of Real-world Human Clothing with Semantic Annotations

Authors: Wenbo Wang, Hsuan-I Ho, Chen Guo, Boxiang Rong, Artur Grigorev, Jie Song, Juan Jose Zarate, Otmar Hilliges

Abstract: …to accurately label 4D scans in diverse garments and body movements. Leveraging precise annotations and high-quality garment meshes, we establish several benchmarks for clothing simulation and reconstruction. 4D-DRESS offers realistic and challenging data that complements synthetic sources, paving the way for advanceme… ▽ More The studies of human clothing for digital avatars have predominantly relied on synthetic datasets. While easy to collect, synthetic data often fall short in realism and fail to capture authentic clothing dynamics. Addressing this gap, we introduce 4D-DRESS, the first real-world 4D dataset advancing human clothing research with its high-quality 4D textured scans and garment meshes. 4D-DRESS captures 64 outfits in 520 human motion sequences, amounting to 78k textured scans. Creating a real-world clothing dataset is challenging, particularly in annotating and segmenting the extensive and complex 4D human scans. To address this, we develop a semi-automatic 4D human parsing pipeline. We efficiently combine a human-in-the-loop process with automation to accurately label 4D scans in diverse garments and body movements. Leveraging precise annotations and high-quality garment meshes, we establish several benchmarks for clothing simulation and reconstruction. 4D-DRESS offers realistic and challenging data that complements synthetic sources, paving the way for advancements in research of lifelike human clothing. Website: https://ait.ethz.ch/4d-dress. △ Less

Submitted 29 April, 2024; originally announced April 2024.

Comments: CVPR 2024 paper, 21 figures, 9 tables

DiffAvatar: Simulation-Ready Garment Optimization with Differentiable Simulation

Authors: Yifei Li, Hsiao-yu Chen, Egor Larionov, Nikolaos Sarafianos, Wojciech Matusik, Tuur Stuyck

Abstract: …and customization. While physical simulations can produce realistic motions for clothed humans, they require high-quality garment assets with associated physical parameters for cloth simulations. However, manually creating these assets and calibrating their parameters is labor-intensive and requires specialized experti… ▽ More The realism of digital avatars is crucial in enabling telepresence applications with self-expression and customization. While physical simulations can produce realistic motions for clothed humans, they require high-quality garment assets with associated physical parameters for cloth simulations. However, manually creating these assets and calibrating their parameters is labor-intensive and requires specialized expertise. Current methods focus on reconstructing geometry, but don't generate complete assets for physics-based applications. To address this gap, we propose \papername,~a novel approach that performs body and garment co-optimization using differentiable simulation. By integrating physical simulation into the optimization loop and accounting for the complex nonlinear behavior of cloth and its intricate interaction with the body, our framework recovers body and garment geometry and extracts important material parameters in a physically plausible way. Our experiments demonstrate that our approach generates realistic clothing and body shape suitable for downstream applications. We provide additional insights and results on our webpage: https://people.csail.mit.edu/liyifei/publication/diffavatar/ △ Less

Submitted 29 March, 2024; v1 submitted 20 November, 2023; originally announced November 2023.

Comments: CVPR 2024; Project page: https://people.csail.mit.edu/liyifei/publication/diffavatar/

Digital 3D Smocking Design

Authors: Jing Ren, Aviv Segall, Olga Sorkine-Hornung

Abstract: …experimentation. This motivates us to seek a digital smocking design method. Straightforward attempts to compute smocked fabric geometry using surface deformation or cloth simulation methods fail to produce realistic results, likely due to the intricate structure of the designs, the large number of contacts and high-cu… ▽ More We develop an optimization-based method to model smocking, a surface embroidery technique that provides decorative geometric texturing while maintaining stretch properties of the fabric. During smocking, multiple pairs of points on the fabric are stitched together, creating non-manifold geometric features and visually pleasing textures. Designing smocking patterns is challenging, because the outcome of stitching is unpredictable: the final texture is often revealed only when the whole smocking process is completed, necessitating painstaking physical fabrication and time consuming trial-and-error experimentation. This motivates us to seek a digital smocking design method. Straightforward attempts to compute smocked fabric geometry using surface deformation or cloth simulation methods fail to produce realistic results, likely due to the intricate structure of the designs, the large number of contacts and high-curvature folds. We instead formulate smocking as a graph embedding and shape deformation problem. We extract a coarse graph representing the fabric and the stitching constraints, and then derive the graph structure of the smocked result. We solve for the 3D embedding of this graph, which in turn reliably guides the deformation of the high-resolution fabric mesh. Our optimization based method is simple, efficient, and flexible, which allows us to build an interactive system for smocking pattern exploration. To demonstrate the accuracy of our method, we compare our results to real fabrications on a large set of smocking patterns △ Less

Submitted 13 September, 2023; originally announced September 2023.

Comments: 17 pages, 35 figures

DiffMimic: Efficient Motion Mimicking with Differentiable Physics

Authors: Jiawei Ren, Cunjun Yu, Siwei Chen, Xiao Ma, Liang Pan, Ziwei Liu

Abstract: …require about a day of training to cycle Backflip. More importantly, we hope DiffMimic can benefit more differentiable animation systems with techniques like differentiable clothes simulation in future research. ▽ More Motion mimicking is a foundational task in physics-based character animation. However, most existing motion mimicking methods are built upon reinforcement learning (RL) and suffer from heavy reward engineering, high variance, and slow convergence with hard explorations. Specifically, they usually take tens of hours or even days of training to mimic a simple motion sequence, resulting in poor scalability. In this work, we leverage differentiable physics simulators (DPS) and propose an efficient motion mimicking method dubbed DiffMimic. Our key insight is that DPS casts a complex policy learning task to a much simpler state matching problem. In particular, DPS learns a stable policy by analytical gradients with ground-truth physical priors hence leading to significantly faster and stabler convergence than RL-based methods. Moreover, to escape from local optima, we utilize a Demonstration Replay mechanism to enable stable gradient backpropagation in a long horizon. Extensive experiments on standard benchmarks show that DiffMimic has a better sample efficiency and time efficiency than existing methods (e.g., DeepMimic). Notably, DiffMimic allows a physically simulated character to learn Backflip after 10 minutes of training and be able to cycle it after 3 hours of training, while the existing approach may require about a day of training to cycle Backflip. More importantly, we hope DiffMimic can benefit more differentiable animation systems with techniques like differentiable clothes simulation in future research. △ Less

Submitted 26 April, 2023; v1 submitted 6 April, 2023; originally announced April 2023.

Comments: ICLR 2023 Code is at https://github.com/jiawei-ren/diffmimic Project page is at https://diffmimic.github.io/

Self-supervised Cloth Reconstruction via Action-conditioned Cloth Tracking

Authors: Zixuan Huang, Xingyu Lin, David Held

Abstract: …full state of crumpled clothes by training a mesh reconstruction model in simulation. However, such models are prone to suffer from a sim-to-real gap due to differences between cloth simulation and the real world. In this work, we propose a self-supervised method to finetune a mesh reconstruction model in the real worl… ▽ More State estimation is one of the greatest challenges for cloth manipulation due to cloth's high dimensionality and self-occlusion. Prior works propose to identify the full state of crumpled clothes by training a mesh reconstruction model in simulation. However, such models are prone to suffer from a sim-to-real gap due to differences between cloth simulation and the real world. In this work, we propose a self-supervised method to finetune a mesh reconstruction model in the real world. Since the full mesh of crumpled cloth is difficult to obtain in the real world, we design a special data collection scheme and an action-conditioned model-based cloth tracking method to generate pseudo-labels for self-supervised learning. By finetuning the pretrained mesh reconstruction model on this pseudo-labeled dataset, we show that we can improve the quality of the reconstructed mesh without requiring human annotations, and improve the performance of downstream manipulation task. △ Less

Submitted 19 February, 2023; originally announced February 2023.

Journal ref: International Conference on Robotics and Automation 2023

Neural Cloth Simulation

Authors: Hugo Bertiche, Meysam Madadi, Sergio Escalera

Abstract: …do not handle cloth dynamics. Here, we propose the first methodology able to learn realistic cloth dynamics unsupervisedly, and henceforth, a general formulation for neural cloth… ▽ More We present a general framework for the garment animation problem through unsupervised deep learning inspired in physically based simulation. Existing trends in the literature already explore this possibility. Nonetheless, these approaches do not handle cloth dynamics. Here, we propose the first methodology able to learn realistic cloth dynamics unsupervisedly, and henceforth, a general formulation for neural cloth simulation. The key to achieve this is to adapt an existing optimization scheme for motion from simulation based methodologies to deep learning. Then, analyzing the nature of the problem, we devise an architecture able to automatically disentangle static and dynamic cloth subspaces by design. We will show how this improves model performance. Additionally, this opens the possibility of a novel motion augmentation technique that greatly improves generalization. Finally, we show it also allows to control the level of motion in the predictions. This is a useful, never seen before, tool for artists. We provide of detailed analysis of the problem to establish the bases of neural cloth simulation and guide future research into the specifics of this domain. △ Less

Submitted 13 December, 2022; originally announced December 2022.

Journal ref: Neural Cloth Simulation. ACM Trans. Graph. 41, 6, Article 220 (December 2022), 14 pages

Designing Personalized Garments with Body Movement

Authors: Katja Wolff, Philipp Herholz, Verena Ziegler, Frauke Link, Nico Brügel, Olga Sorkine-Hornung

Abstract: …an underlying sewing pattern, and on the adjustment of that garment's rest shape while interpolating and moving through the different input poses. We alternate between cloth simulation and rest shape adjustment based on stretch to achieve the final shape of the garment. At any step in the real-time process, we allo… ▽ More The standardized sizes used in the garment industry do not cover the range of individual differences in body shape for most people, leading to ill-fitting clothes, high return rates and overproduction. Recent research efforts in both industry and academia therefore focus on virtual try-on and on-demand fabrication of individually fitting garments. We propose an interactive design tool for creating custom-fit garments based on 3D body scans of the intended wearer. Our method explicitly incorporates transitions between various body poses to ensure a better fit and freedom of movement. The core of our method focuses on tools to create a 3D garment shape directly on an avatar without an underlying sewing pattern, and on the adjustment of that garment's rest shape while interpolating and moving through the different input poses. We alternate between cloth simulation and rest shape adjustment based on stretch to achieve the final shape of the garment. At any step in the real-time process, we allow for interactive changes to the garment. Once the garment shape is finalized for production, established techniques can be used to parameterize it into a 2D sewing pattern or transform it into a knitting pattern. △ Less

Submitted 19 September, 2022; v1 submitted 10 February, 2021; originally announced February 2021.

Comments: 13 pages, 14 figures

DiffCloth: Differentiable Cloth Simulation with Dry Frictional Contact

Authors: Yifei Li, Tao Du, Kui Wu, Jie Xu, Wojciech Matusik

Abstract: Cloth… ▽ More Cloth simulation has wide applications in computer animation, garment design, and robot-assisted dressing. This work presents a differentiable cloth simulator whose additional gradient information facilitates cloth-related applications. Our differentiable simulator extends a state-of-the-art cloth simulator based on Projective Dynamics (PD) and with dry frictional contact. We draw inspiration from previous work to propose a fast and novel method for deriving gradients in PD-based cloth simulation with dry frictional contact. Furthermore, we conduct a comprehensive analysis and evaluation of the usefulness of gradients in contact-rich cloth simulation. Finally, we demonstrate the efficacy of our simulator in a number of downstream applications, including system identification, trajectory optimization for assisted dressing, closed-loop control, inverse design, and real-to-sim transfer. We observe a substantial speedup obtained from using our gradient information in solving most of these applications. △ Less

Submitted 17 July, 2022; v1 submitted 9 June, 2021; originally announced June 2021.

Journal ref: ACM Transactions on Graphics (TOG), 2022

N-Cloth: Predicting 3D Cloth Deformation with Mesh-Based Networks

Authors: Yudi Li, Min Tang, Yun Yang, Zi Huang, Ruofeng Tong, Shuangcai Yang, Yao Li, Dinesh Manocha

Abstract: …up to 100K triangles and scenes with various objects corresponding to SMPL humans, non-SMPL humans or rigid bodies. In practice, our approach can be used to generate plausible cloth… ▽ More We present a novel mesh-based learning approach (N-Cloth) for plausible 3D cloth deformation prediction. Our approach is general and can handle cloth or obstacles represented by triangle meshes with arbitrary topologies. We use graph convolution to transform the cloth and object meshes into a latent space to reduce the non-linearity in the mesh space. Our network can predict the target 3D cloth mesh deformation based on the initial state of the cloth mesh template and the target obstacle mesh. Our approach can handle complex cloth meshes with up to 100K triangles and scenes with various objects corresponding to SMPL humans, non-SMPL humans or rigid bodies. In practice, our approach can be used to generate plausible cloth simulation at 30-45 fps on an NVIDIA GeForce RTX 3090 GPU. We highlight its benefits over prior learning-based methods and physically-based cloth simulators. △ Less

Submitted 27 May, 2022; v1 submitted 12 December, 2021; originally announced December 2021.

Comments: 12 pages

Neural Collision Detection for Deformable Objects

Authors: Ryan S. Zesch, Bethany R. Witemeyer, Ziyan Xiong, David I. W. Levin, Shinjiro Sueda

Abstract: …to use and implement, and it can readily be employed on the GPU. We demonstrate our approach with two concrete examples: a haptics application with a finite element mesh, and cloth simulation with a skinned character. ▽ More We propose a neural network-based approach for collision detection with deformable objects. Unlike previous approaches based on bounding volume hierarchies, our neural approach does not require an update of the spatial data structure when the object deforms. Our network is trained on the reduced degrees of freedom of the object, so that we can use the same network to query for collisions even when the object deforms. Our approach is simple to use and implement, and it can readily be employed on the GPU. We demonstrate our approach with two concrete examples: a haptics application with a finite element mesh, and cloth simulation with a skinned character. △ Less

Submitted 4 February, 2022; originally announced February 2022.

Active Learning of Neural Collision Handler for Complex 3D Mesh Deformations

Authors: Qingyang Tan, Zherong Pan, Breannan Smith, Takaaki Shiratori, Dinesh Manocha

Abstract: …handling of complex, 3D meshes coming from several datasets with different shapes and topologies, including datasets corresponding to dressed and undressed human poses, cloth simulations, and human hand poses acquired using multiview capture systems. Our approach outperforms supervised learning methods and achieves… ▽ More We present a robust learning algorithm to detect and handle collisions in 3D deforming meshes. Our collision detector is represented as a bilevel deep autoencoder with an attention mechanism that identifies colliding mesh sub-parts. We use a numerical optimization algorithm to resolve penetrations guided by the network. Our learned collision handler can resolve collisions for unseen, high-dimensional meshes with thousands of vertices. To obtain stable network performance in such large and unseen spaces, we progressively insert new collision data based on the errors in network inferences. We automatically label these data using an analytical collision detector and progressively fine-tune our detection networks. We evaluate our method for collision handling of complex, 3D meshes coming from several datasets with different shapes and topologies, including datasets corresponding to dressed and undressed human poses, cloth simulations, and human hand poses acquired using multiview capture systems. Our approach outperforms supervised learning methods and achieves $93.8-98.1\%$ accuracy compared to the groundtruth by analytic methods. Compared to prior learning methods, our approach results in a $5.16\%-25.50\%$ lower false negative rate in terms of collision checking and a $9.65\%-58.91\%$ higher success rate in collision handling. △ Less

Submitted 8 October, 2021; originally announced October 2021.

Multi-feature super-resolution network for cloth wrinkle synthesis

Authors: Lan Chen, Juntao Ye, Xiaopeng Zhang

Abstract: Existing physical cloth simulators suffer from expensive computation and difficulties in tuning mechanical parameters to get desired wrinkling behaviors. Data-driven methods provide an alternative solution. It typically synthesizes cloth animation at a much lower computational cost, and also creates wrinkling effects t… ▽ More Existing physical cloth simulators suffer from expensive computation and difficulties in tuning mechanical parameters to get desired wrinkling behaviors. Data-driven methods provide an alternative solution. It typically synthesizes cloth animation at a much lower computational cost, and also creates wrinkling effects that highly resemble the much controllable training data. In this paper we propose a deep learning based method for synthesizing cloth animation with high resolution meshes. To do this we first create a dataset for training: a pair of low and high resolution meshes are simulated and their motions are synchronized. As a result the two meshes exhibit similar large-scale deformation but different small wrinkles. Each simulated mesh pair are then converted into a pair of low and high resolution "images" (a 2D array of samples), with each sample can be interpreted as any of three features: the displacement, the normal and the velocity. With these image pairs, we design a multi-feature super-resolution (MFSR) network that jointly train an upsampling synthesizer for the three features. The MFSR architecture consists of two key components: a sharing module that takes multiple features as input to learn low-level representations from corresponding super-resolution tasks simultaneously; and task-specific modules focusing on various high-level semantics. Frame-to-frame consistency is well maintained thanks to the proposed kinematics-based loss function. Our method achieves realistic results at high frame rates: 12-14 times faster than traditional physical simulation. We demonstrate the performance of our method with various experimental scenes, including a dressed character with sophisticated collisions. △ Less

Submitted 15 June, 2021; v1 submitted 8 April, 2020; originally announced April 2020.

Journal ref: JOURNAL OF COMPUTER SCIENCE AND TECHNOLOGY 36(3): 478-493 May 2021

MulayCap: Multi-layer Human Performance Capture Using A Monocular Video Camera

Authors: Zhaoqi Su, Weilin Wan, Tao Yu, Lingjie Liu, Lu Fang, Wenping Wang, Yebin Liu

Abstract: …The key technique behind is a Garment-from-Video (GfV) method for optimizing the garment shape and reconstructing the dynamic cloth to fit the input video sequence, based on a cloth simulation model which is effectively solved with gradient descent. For texture rendering, we decompose each input image frame into a sha… ▽ More We introduce MulayCap, a novel human performance capture method using a monocular video camera without the need for pre-scanning. The method uses "multi-layer" representations for geometry reconstruction and texture rendering, respectively. For geometry reconstruction, we decompose the clothed human into multiple geometry layers, namely a body mesh layer and a garment piece layer. The key technique behind is a Garment-from-Video (GfV) method for optimizing the garment shape and reconstructing the dynamic cloth to fit the input video sequence, based on a cloth simulation model which is effectively solved with gradient descent. For texture rendering, we decompose each input image frame into a shading layer and an albedo layer, and propose a method for fusing a fixed albedo map and solving for detailed garment geometry using the shading layer. Compared with existing single view human performance capture systems, our "multi-layer" approach bypasses the tedious and time consuming scanning step for obtaining a human specific mesh template. Experimental results demonstrate that MulayCap produces realistic rendering of dynamically changing details that has not been achieved in any previous monocular video camera systems. Benefiting from its fully semantic modeling, MulayCap can be applied to various important editing applications, such as cloth editing, re-targeting, relighting, and AR applications. △ Less

Submitted 1 October, 2020; v1 submitted 13 April, 2020; originally announced April 2020.

P-Cloth: Interactive Complex Cloth Simulation on Multi-GPU Systems using Dynamic Matrix Assembly and Pipelined Implicit Integrators

Authors: Cheng Li, Min Tang, Ruofeng Tong, Ming Cai, Jieyi Zhao, Dinesh Manocha

Abstract: We present a novel parallel algorithm for cloth… ▽ More We present a novel parallel algorithm for cloth simulation that exploits multiple GPUs for fast computation and the handling of very high resolution meshes. To accelerate implicit integration, we describe new parallel algorithms for sparse matrix-vector multiplication (SpMV) and for dynamic matrix assembly on a multi-GPU workstation. Our algorithms use a novel work queue generation scheme for a fat-tree GPU interconnect topology. Furthermore, we present a novel collision handling scheme that uses spatial hashing for discrete and continuous collision detection along with a non-linear impact zone solver. Our parallel schemes can distribute the computation and storage overhead among multiple GPUs and enable us to perform almost interactive simulation on complex cloth meshes, which can hardly be handled on a single GPU due to memory limitations. We have evaluated the performance with two multi-GPU workstations (with 4 and 8 GPUs, respectively) on cloth meshes with 0.5-1.65M triangles. Our approach can reliably handle the collisions and generate vivid wrinkles and folds at 2-5 fps, which is significantly faster than prior cloth simulation systems. We observe almost linear speedups with respect to the number of GPUs. △ Less

Submitted 4 August, 2020; v1 submitted 2 August, 2020; originally announced August 2020.

Efficient Cloth Simulation using Miniature Cloth and Upscaling Deep Neural Networks

Authors: Tae Min Lee, Young Jin Oh, In-Kwon Lee

Abstract: Cloth… ▽ More Cloth simulation requires a fast and stable method for interactively and realistically visualizing fabric materials using computer graphics. We propose an efficient cloth simulation method using miniature cloth simulation and upscaling Deep Neural Networks (DNN). The upscaling DNNs generate the target cloth simulation from the results of physically-based simulations of a miniature cloth that has similar physical properties to those of the target cloth. We have verified the utility of the proposed method through experiments, and the results demonstrate that it is possible to generate fast and stable cloth simulations under various conditions. △ Less

Submitted 8 July, 2019; originally announced July 2019.

Comments: 11 pages, 15 figures, 8 tables

Ground Profile Recovery from Aerial 3D LiDAR-based Maps

Authors: Adelya Sabirova, Maksim Rassabin, Roman Fedorenko, Ilya Afanasyev

Abstract: The paper presents the study and implementation of the ground detection methodology with filtration and removal of forest points from LiDAR-based 3D point cloud using the Cloth Simulation Filtering (CSF) algorithm. The methodology allows to recover a terrestrial relief and create a landscape map of a forestry region. A… ▽ More The paper presents the study and implementation of the ground detection methodology with filtration and removal of forest points from LiDAR-based 3D point cloud using the Cloth Simulation Filtering (CSF) algorithm. The methodology allows to recover a terrestrial relief and create a landscape map of a forestry region. As the proof-of-concept, we provided the outdoor flight experiment, launching a hexacopter under a mixed forestry region with sharp ground changes nearby Innopolis city (Russia), which demonstrated the encouraging results for both ground detection and methodology robustness. △ Less

Submitted 26 March, 2019; originally announced March 2019.

Comments: 8 pages, FRUCT-2019 conference

SimulCap : Single-View Human Performance Capture with Cloth Simulation

Authors: Tao Yu, Zerong Zheng, Yuan Zhong, Jianhui Zhao, Qionghai Dai, Gerard Pons-Moll, Yebin Liu

Abstract: …using multi-layer surface representation, which includes the undressed body surface and separate clothing meshes. For performance capture, we perform skeleton tracking, cloth… ▽ More This paper proposes a new method for live free-viewpoint human performance capture with dynamic details (e.g., cloth wrinkles) using a single RGBD camera. Our main contributions are: (i) a multi-layer representation of garments and body, and (ii) a physics-based performance capture procedure. We first digitize the performer using multi-layer surface representation, which includes the undressed body surface and separate clothing meshes. For performance capture, we perform skeleton tracking, cloth simulation, and iterative depth fitting sequentially for the incoming frame. By incorporating cloth simulation into the performance capture pipeline, we can simulate plausible cloth dynamics and cloth-body interactions even in the occluded regions, which was not possible in previous capture methods. Moreover, by formulating depth fitting as a physical process, our system produces cloth tracking results consistent with the depth observation while still maintaining physical constraints. Results and evaluations show the effectiveness of our method. Our method also enables new types of applications such as cloth retargeting, free-viewpoint video rendering and animations. △ Less

Submitted 18 March, 2019; v1 submitted 14 March, 2019; originally announced March 2019.

Comments: CVPR 2019

CLoTH: a Simulator for HTLC Payment Networks

Authors: Marco Conoscenti, Antonio Vetrò, Juan Carlos De Martin, Federico Spini, Fabio Castaldo, Sebastiano Scròfina

Abstract: …might arise in the development of an HTLC payment network, and to estimate the effects of an optimisation before deploying it. In upcoming works we'll publish the results of CLoTH simulations. ▽ More The Lightning Network (LN) is one of the most promising off-chain scaling solutions for Bitcoin, as it enables off-chain payments which are not subject to the well-known blockchain scalability limit. In this work, we introduce CLoTH, a simulator for HTLC payment networks, of which LN is the best working example. It simulates input-defined payments on an input-defined HTLC network and produces performance measures in terms of payment-related statistics, such as time to complete payments and probability of payment failure. CLoTH helps to predict issues that might arise in the development of an HTLC payment network, and to estimate the effects of an optimisation before deploying it. In upcoming works we'll publish the results of CLoTH simulations. △ Less

Submitted 24 December, 2018; originally announced December 2018.

Landslide Monitoring based on Terrestrial Laser Scanning: A Novel Semi-automatic Workflow

Authors: Yue Pan

Abstract: …between multiple periods, Iterative Global Similarity Point (IGSP) algorithm is applied to accomplish multiple-phase registration, which outperforms ICP in experiments. Then the cloth simulation filtering (CSF) algorithm was used together with manual post-processing to remove vegetation on the slope. After that, the mo… ▽ More In this paper, we propose a workflow that uses Terrestrial Laser Scanning(TLS) to semi-automatically monitor landslide and then test it in practice. Firstly, several groups of TLS stations are set on different time to collect the raw point cloud of the object mountain. Next, Hierarchical Merging Based Multi-view (HMMR) registration algorithm is adapted to accomplish single-phase multi-view registration.In order to analyze deformation between multiple periods, Iterative Global Similarity Point (IGSP) algorithm is applied to accomplish multiple-phase registration, which outperforms ICP in experiments. Then the cloth simulation filtering (CSF) algorithm was used together with manual post-processing to remove vegetation on the slope. After that, the mountain slope's digital terrain model (DTM) is generated for each period, and the distance between adjacent DTMs are calculated as the landslide deformation mass. Furthermore, average deformation rate of the landslide surface is calculated and analyzed.To validate the effectiveness of proposed workflow, we uses the TLS data of five periods of the landslide in the Shanhou village of northern Changshan Island from 2013 to 2015. The results indicate that the method can obtain centimeter-level deformation monitoring accuracy which can effectively monitor and analyze long-term landslide morphology and trend as well as position the significant deformation area and determine the type of landslide. In addition, the process can be automated to provide end-to-end TLS based long-term landslide monitoring applications, providing reference for monitoring and early warning of potential landslides. △ Less

Submitted 1 September, 2018; originally announced September 2018.

Comments: 10 pages, 13 figures ,3 tables

Hierarchical Cloth Simulation using Deep Neural Networks

Authors: Young Jin Oh, Tae Min Lee, In-Kwon Lee

Abstract: …simulation techniques are essential for providing flexible visual effects for computer graphics content. In this paper, we propose a fast and reliable hierarchical cloth… ▽ More Fast and reliable physically-based simulation techniques are essential for providing flexible visual effects for computer graphics content. In this paper, we propose a fast and reliable hierarchical cloth simulation method, which combines conventional physically-based simulation with deep neural networks (DNN). Simulations of the coarsest level of the hierarchical model are calculated using conventional physically-based simulations, and more detailed levels are generated by inference using DNN models. We demonstrate that our method generates reliable and fast cloth simulation results through experiments under various conditions. △ Less

Submitted 9 February, 2018; originally announced February 2018.

Comments: 9 pages, 11 figures

Detailed Garment Recovery from a Single-View Image

Authors: Shan Yang, Tanya Ambert, Zherong Pan, Ke Wang, Licheng Yu, Tamara Berg, Ming C. Lin

Abstract: …from a single-view image by using statistical, geometric, and physical priors and a combina- tion of parameter estimation, semantic parsing, shape recovery, and physics- based cloth simulation. We demonstrate the effectiveness of our algorithm by re-purposing the reconstructed garments for virtual try-on and garment tr… ▽ More Most recent garment capturing techniques rely on acquiring multiple views of clothing, which may not always be readily available, especially in the case of pre-existing photographs from the web. As an alternative, we pro- pose a method that is able to compute a rich and realistic 3D model of a human body and its outfits from a single photograph with little human in- teraction. Our algorithm is not only able to capture the global shape and geometry of the clothing, it can also extract small but important details of cloth, such as occluded wrinkles and folds. Unlike previous methods using full 3D information (i.e. depth, multi-view images, or sampled 3D geom- etry), our approach achieves detailed garment recovery from a single-view image by using statistical, geometric, and physical priors and a combina- tion of parameter estimation, semantic parsing, shape recovery, and physics- based cloth simulation. We demonstrate the effectiveness of our algorithm by re-purposing the reconstructed garments for virtual try-on and garment transfer applications, as well as cloth animation for digital characters. △ Less

Submitted 12 September, 2016; v1 submitted 3 August, 2016; originally announced August 2016.

Comments: Comparison added. Algorithm added. Equations cleaned up