-
PINNOCHIO: Physics-Informed Neural Network for Coupled Hyperelastic Interface-Volume Simulation in Orthognathic Surgery
Authors:
Jungwook Lee,
Daeseung Kim,
Kevin Gu,
Zhangfeng Hu,
Tianshu Kuang,
Finn Hopeman,
Michael A. K. Liebschner,
Jaime Gateno,
Pingkun Yan
Abstract:
Predicting patient-specific facial soft-tissue deformation is critical for iterative orthognathic surgery planning. However, current computational methods face a strict accuracy-efficiency trade-off: high-fidelity Finite Element Methods (FEM) are computationally prohibitive, whereas pure deep learning models often produce biomechanically inconsistent results…
▽ More
Predicting patient-specific facial soft-tissue deformation is critical for iterative orthognathic surgery planning. However, current computational methods face a strict accuracy-efficiency trade-off: high-fidelity Finite Element Methods (FEM) are computationally prohibitive, whereas pure deep learning models often produce biomechanically inconsistent results. While Physics-Informed Neural Networks (PINNs) offer a promising avenue, learning the complex heterogeneous mechanics of bone--soft-tissue interactions with only partial clinical supervision (i.e., outer facial surfaces) remains highly unstable. To overcome these challenges, we present PINNOCHIO, a novel physics-informed framework for facial soft-tissue simulation. PINNOCHIO introduces a hybrid sequential decomposition that explicitly decouples discontinuous bone--soft-tissue interface movements from continuous volumetric hyperelastic deformation. This structural separation enables stable training and facilitates a physics-enabled sim-to-real adaptation strategy, ensuring internal biomechanical consistency without requiring volumetric ground truth. Evaluated on a 40-patient clinical cohort, PINNOCHIO outperforms existing baselines in both surface accuracy and physical validity. Furthermore, it achieves a substantial speedup over FEM, successfully resolving the accuracy-efficiency trade-off to provide a highly reliable and practical tool for interactive surgical planning.
△ Less
Submitted 31 May, 2026;
originally announced June 2026.
-
PhysSFI-Net: Physics-informed Geometric Learning of Skeletal and Facial Interactions for Orthognathic Surgical Outcome Prediction
Authors:
Jiahao Bao,
Huazhen Liu,
Yu Zhuang,
Leran Tao,
Xinyu Xu,
Yongtao Shi,
Mengjia Cheng,
Yiming Wang,
Congshuang Ku,
Ting Zeng,
Yilang Du,
Siyi Chen,
Shunyao Shen,
Suncheng Xiang,
Hongbo Yu
Abstract:
Orthognathic surgery repositions jaw bones to restore occlusion and enhance facial aesthetics. Accurate simulation of postoperative facial morphology is essential for preoperative planning. This study aims to develop and validate a physics-informed geometric deep learning framework named PhysSFI-Net for precise prediction of soft tissue deformation following…
▽ More
Orthognathic surgery repositions jaw bones to restore occlusion and enhance facial aesthetics. Accurate simulation of postoperative facial morphology is essential for preoperative planning. This study aims to develop and validate a physics-informed geometric deep learning framework named PhysSFI-Net for precise prediction of soft tissue deformation following orthognathic surgery. The model integrates a hierarchical feature extraction module with attention mechanisms to capture skeletal-facial interactions, an LSTM-based sequential predictor for incremental deformation, and a biomechanics-inspired reconstruction module for high-resolution facial modeling. The model was trained on 135 patients and externally validated on an independent cohort of 33 patients. Model performance was assessed using point cloud shape error, surface deviation error and landmark error between predicted facial shapes with corresponding ground truths. Quantitative analysis demonstrated that PhysSFI-Net achieved a global shape error of 1.070 +/- 0.088 mm, a surface deviation error of 1.296 +/- 0.349 mm and a landmark error of 2.445 +/- 1.326 mm. Comparative experiments indicated that PhysSFI-Net outperformed the state-of-the-art method ACMT-Net and baseline models. External validation further confirmed its robustness with a global HD of 1.431 +/- 0.087 mm and consistently lower subregional and mesh-based errors. In conclusion, PhysSFI-Net enables interpretable, high-resolution prediction of postoperative facial morphology, showing strong potential for clinical application in orthognathic surgical planning.
△ Less
Submitted 19 August, 2026; v1 submitted 5 January, 2026;
originally announced January 2026.
-
NICE: Neural Implicit Craniofacial Model for Orthognathic Surgery Prediction
Authors:
Jiawen Yang,
Yihui Cao,
Xuanyu Tian,
Yuyao Zhang,
Hongjiang Wei
Abstract:
Orthognathic surgery is a crucial intervention for correcting dentofacial skeletal deformities to enhance occlusal functionality and facial aesthetics. Accurate postoperative facial appearance prediction remains challenging due to the complex nonlinear interactions between skeletal movements and facial soft tissue. Existing biomechanical, parametric models a…
▽ More
Orthognathic surgery is a crucial intervention for correcting dentofacial skeletal deformities to enhance occlusal functionality and facial aesthetics. Accurate postoperative facial appearance prediction remains challenging due to the complex nonlinear interactions between skeletal movements and facial soft tissue. Existing biomechanical, parametric models and deep-learning approaches either lack computational efficiency or fail to fully capture these intricate interactions. To address these limitations, we propose Neural Implicit Craniofacial Model (NICE) which employs implicit neural representations for accurate anatomical reconstruction and surgical outcome prediction. NICE comprises a shape module, which employs region-specific implicit Signed Distance Function (SDF) decoders to reconstruct the facial surface, maxilla, and mandible, and a surgery module, which employs region-specific deformation decoders. These deformation decoders are driven by a shared surgical latent code to effectively model the complex, nonlinear biomechanical response of the facial surface to skeletal movements, incorporating anatomical prior knowledge. The deformation decoders output point-wise displacement fields, enabling precise modeling of surgical outcomes. Extensive experiments demonstrate that NICE outperforms current state-of-the-art methods, notably improving prediction accuracy in critical facial regions such as lips and chin, while robustly preserving anatomical integrity. This work provides a clinically viable tool for enhanced surgical planning and patient consultation in orthognathic procedures.
△ Less
Submitted 5 December, 2025;
originally announced December 2025.
-
TCFNet: Bidirectional face-bone transformation via a Transformer-based coarse-to-fine point movement network
Authors:
Runshi Zhang,
Bimeng Jie,
Yang He,
Junchen Wang
Abstract:
Computer-aided surgical simulation is a critical component of orthognathic surgical planning, where accurately simulating face-bone shape transformations is significant. The traditional biomechanical simulation methods are limited by their computational time consumption levels, labor-intensive data processing strategies and low accuracy. Recently, deep learn…
▽ More
Computer-aided surgical simulation is a critical component of orthognathic surgical planning, where accurately simulating face-bone shape transformations is significant. The traditional biomechanical simulation methods are limited by their computational time consumption levels, labor-intensive data processing strategies and low accuracy. Recently, deep learning-based simulation methods have been proposed to view this problem as a point-to-point transformation between skeletal and facial point clouds. However, these approaches cannot process large-scale points, have limited receptive fields that lead to noisy points, and employ complex preprocessing and postprocessing operations based on registration. These shortcomings limit the performance and widespread applicability of such methods. Therefore, we propose a Transformer-based coarse-to-fine point movement network (TCFNet) to learn unique, complicated correspondences at the patch and point levels for dense face-bone point cloud transformations. This end-to-end framework adopts a Transformer-based network and a local information aggregation network (LIA-Net) in the first and second stages, respectively, which reinforce each other to generate precise point movement paths. LIA-Net can effectively compensate for the neighborhood precision loss of the Transformer-based network by modeling local geometric structures (edges, orientations and relative position features). The previous global features are employed to guide the local displacement using a gated recurrent unit. Inspired by deformable medical image registration, we propose an auxiliary loss that can utilize expert knowledge for reconstructing critical organs.Compared with the existing state-of-the-art (SOTA) methods on gathered datasets, TCFNet achieves outstanding evaluation metrics and visualization results. The code is available at https://github.com/Runshi-Zhang/TCFNet.
△ Less
Submitted 19 August, 2025;
originally announced August 2025.
-
Facial Surgery Preview Based on the Orthognathic Treatment Prediction
Authors:
Huijun Han,
Congyi Zhang,
Lifeng Zhu,
Pradeep Singh,
Richard Tai Chiu Hsung,
Yiu Yan Leung,
Taku Komura,
Wenping Wang,
Min Gu
Abstract:
Orthognathic surgery consultation is essential to help patients understand the changes to their facial appearance after surgery. However, current visualization methods are often inefficient and inaccurate due to limited pre- and post-treatment data and the complexity of the treatment. To overcome these challenges, this study aims to develop a fully automated…
▽ More
Orthognathic surgery consultation is essential to help patients understand the changes to their facial appearance after surgery. However, current visualization methods are often inefficient and inaccurate due to limited pre- and post-treatment data and the complexity of the treatment. To overcome these challenges, this study aims to develop a fully automated pipeline that generates accurate and efficient 3D previews of postsurgical facial appearances for patients with orthognathic treatment without requiring additional medical images. The study introduces novel aesthetic losses, such as mouth-convexity and asymmetry losses, to improve the accuracy of facial surgery prediction. Additionally, it proposes a specialized parametric model for 3D reconstruction of the patient, medical-related losses to guide latent code prediction network optimization, and a data augmentation scheme to address insufficient data. The study additionally employs FLAME, a parametric model, to enhance the quality of facial appearance previews by extracting facial latent codes and establishing dense correspondences between pre- and post-surgery geometries. Quantitative comparisons showed the algorithm's effectiveness, and qualitative results highlighted accurate facial contour and detail predictions. A user study confirmed that doctors and the public could not distinguish between machine learning predictions and actual postoperative results. This study aims to offer a practical, effective solution for orthognathic surgery consultations, benefiting doctors and patients.
△ Less
Submitted 14 April, 2025; v1 submitted 14 December, 2024;
originally announced December 2024.
-
Revisiting Cephalometric Landmark Detection from the view of Human Pose Estimation with Lightweight Super-Resolution Head
Authors:
Qian Wu,
Si Yong Yeo,
Yufei Chen,
Jun Liu
Abstract:
Accurate localization of cephalometric landmarks holds great importance in the fields of orthodontics and orthognathics due to its potential for automating key point labeling. In the context of landmark detection, particularly in cephalometrics, it has been observed that existing methods often lack standardized pipelines and well-designed bias reduction proc…
▽ More
Accurate localization of cephalometric landmarks holds great importance in the fields of orthodontics and orthognathics due to its potential for automating key point labeling. In the context of landmark detection, particularly in cephalometrics, it has been observed that existing methods often lack standardized pipelines and well-designed bias reduction processes, which significantly impact their performance. In this paper, we revisit a related task, human pose estimation (HPE), which shares numerous similarities with cephalometric landmark detection (CLD), and emphasize the potential for transferring techniques from the former field to benefit the latter. Motivated by this insight, we have developed a robust and adaptable benchmark based on the well-established HPE codebase known as MMPose. This benchmark can serve as a dependable baseline for achieving exceptional CLD performance. Furthermore, we introduce an upscaling design within the framework to further enhance performance. This enhancement involves the incorporation of a lightweight and efficient super-resolution module, which generates heatmap predictions on high-resolution features and leads to further performance refinement, benefiting from its ability to reduce quantization bias. In the MICCAI CLDetection2023 challenge, our method achieves 1st place ranking on three metrics and 3rd place on the remaining one. The code for our method is available at https://github.com/5k5000/CLdetection2023.
△ Less
Submitted 29 September, 2023;
originally announced September 2023.
-
Automatic 3D Registration of Dental CBCT and Face Scan Data using 2D Projection Images
Authors:
Hyoung Suk Park,
Chang Min Hyun,
Sang-Hwy Lee,
Jin Keun Seo,
Kiwan Jeon
Abstract:
…(CBCT) and face scan data. It can be used for a digital platform of 3D jaw-teeth-face models in a variety of applications, including 3D digital treatment planning and orthognathic surgery. Difficulties in accurately merging facial scans and CBCT images are due to the different image acquisition methods and limited area of correspondence between the two facia…
▽ More
This paper presents a fully automatic registration method of dental cone-beam computed tomography (CBCT) and face scan data. It can be used for a digital platform of 3D jaw-teeth-face models in a variety of applications, including 3D digital treatment planning and orthognathic surgery. Difficulties in accurately merging facial scans and CBCT images are due to the different image acquisition methods and limited area of correspondence between the two facial surfaces. In addition, it is difficult to use machine learning techniques because they use face-related 3D medical data with radiation exposure, which are difficult to obtain for training. The proposed method addresses these problems by reusing an existing machine-learning-based 2D landmark detection algorithm in an open-source library and developing a novel mathematical algorithm that identifies paired 3D landmarks from knowledge of the corresponding 2D landmarks. A main contribution of this study is that the proposed method does not require annotated training data of facial landmarks because it uses a pre-trained facial landmark detection algorithm that is known to be robust and generalized to various 2D face image models. Note that this reduces a 3D landmark detection problem to a 2D problem of identifying the corresponding landmarks on two 2D projection images generated from two different projection angles. Here, the 3D landmarks for registration were selected from the sub-surfaces with the least geometric change under the CBCT and face scan environments. For the final fine-tuning of the registration, the Iterative Closest Point method was applied, which utilizes geometrical information around the 3D landmarks. The experimental results show that the proposed method achieved an averaged surface distance error of 0.74 mm for three pairs of CBCT and face scan datasets.
△ Less
Submitted 26 July, 2023; v1 submitted 17 May, 2023;
originally announced May 2023.
-
Deep Learning-based Facial Appearance Simulation Driven by Surgically Planned Craniomaxillofacial Bony Movement
Authors:
Xi Fang,
Daeseung Kim,
Xuanang Xu,
Tianshu Kuang,
Hannah H. Deng,
Joshua C. Barber,
Nathan Lampen,
Jaime Gateno,
Michael A. K. Liebschner,
James J. Xia,
Pingkun Yan
Abstract:
Simulating facial appearance change following bony movement is a critical step in orthognathic surgical planning for patients with jaw deformities. Conventional biomechanics-based methods such as the finite-element method (FEM) are labor intensive and computationally inefficient. Deep learning-based approaches can be promising alternatives due to their high…
▽ More
Simulating facial appearance change following bony movement is a critical step in orthognathic surgical planning for patients with jaw deformities. Conventional biomechanics-based methods such as the finite-element method (FEM) are labor intensive and computationally inefficient. Deep learning-based approaches can be promising alternatives due to their high computational efficiency and strong modeling capability. However, the existing deep learning-based method ignores the physical correspondence between facial soft tissue and bony segments and thus is significantly less accurate compared to FEM. In this work, we propose an Attentive Correspondence assisted Movement Transformation network (ACMT-Net) to estimate the facial appearance by transforming the bony movement to facial soft tissue through a point-to-point attentive correspondence matrix. Experimental results on patients with jaw deformity show that our proposed method can achieve comparable facial change prediction accuracy compared with the state-of-the-art FEM-based approach with significantly improved computational efficiency.
△ Less
Submitted 4 October, 2022;
originally announced October 2022.
-
SCULPTOR: Skeleton-Consistent Face Creation Using a Learned Parametric Generator
Authors:
Zesong Qiu,
Yuwei Li,
Dongming He,
Qixuan Zhang,
Longwen Zhang,
Yinghao Zhang,
Jingya Wang,
Lan Xu,
Xudong Wang,
Yuyao Zhang,
Jingyi Yu
Abstract:
…the fossils of one of the oldest known human ancestors, our LUCY dataset contains high-quality Computed Tomography (CT) scans of the complete human head before and after orthognathic surgeries, critical for evaluating surgery results. LUCY consists of 144 scans of 72 subjects (31 male and 41 female) where each subject has two CT scans taken pre- and post-…
▽ More
Recent years have seen growing interest in 3D human faces modelling due to its wide applications in digital human, character generation and animation. Existing approaches overwhelmingly emphasized on modeling the exterior shapes, textures and skin properties of faces, ignoring the inherent correlation between inner skeletal structures and appearance. In this paper, we present SCULPTOR, 3D face creations with Skeleton Consistency Using a Learned Parametric facial generaTOR, aiming to facilitate easy creation of both anatomically correct and visually convincing face models via a hybrid parametric-physical representation. At the core of SCULPTOR is LUCY, the first large-scale shape-skeleton face dataset in collaboration with plastic surgeons. Named after the fossils of one of the oldest known human ancestors, our LUCY dataset contains high-quality Computed Tomography (CT) scans of the complete human head before and after orthognathic surgeries, critical for evaluating surgery results. LUCY consists of 144 scans of 72 subjects (31 male and 41 female) where each subject has two CT scans taken pre- and post-orthognathic operations. Based on our LUCY dataset, we learn a novel skeleton consistent parametric facial generator, SCULPTOR, which can create the unique and nuanced facial features that help define a character and at the same time maintain physiological soundness. Our SCULPTOR jointly models the skull, face geometry and face appearance under a unified data-driven framework, by separating the depiction of a 3D face into shape blend shape, pose blend shape and facial expression blend shape. SCULPTOR preserves both anatomic correctness and visual realism in facial generation tasks compared with existing methods. Finally, we showcase the robustness and effectiveness of SCULPTOR in various fancy applications unseen before.
△ Less
Submitted 2 December, 2022; v1 submitted 14 September, 2022;
originally announced September 2022.
-
A Self-Supervised Deep Framework for Reference Bony Shape Estimation in Orthognathic Surgical Planning
Authors:
Deqiang Xiao,
Hannah Deng,
Tianshu Kuang,
Lei Ma,
Qin Liu,
Xu Chen,
Chunfeng Lian,
Yankun Lang,
Daeseung Kim,
Jaime Gateno,
Steve Guofang Shen,
Dinggang Shen,
Pew-Thian Yap,
James J. Xia
Abstract:
Virtual orthognathic surgical planning involves simulating surgical corrections of jaw deformities on 3D facial bony shape models. Due to the lack of necessary guidance, the planning procedure is highly experience-dependent and the planning results are often suboptimal. A reference facial bony shape model representing normal anatomies can provide an objectiv…
▽ More
Virtual orthognathic surgical planning involves simulating surgical corrections of jaw deformities on 3D facial bony shape models. Due to the lack of necessary guidance, the planning procedure is highly experience-dependent and the planning results are often suboptimal. A reference facial bony shape model representing normal anatomies can provide an objective guidance to improve planning accuracy. Therefore, we propose a self-supervised deep framework to automatically estimate reference facial bony shape models. Our framework is an end-to-end trainable network, consisting of a simulator and a corrector. In the training stage, the simulator maps jaw deformities of a patient bone to a normal bone to generate a simulated deformed bone. The corrector then restores the simulated deformed bone back to normal. In the inference stage, the trained corrector is applied to generate a patient-specific normal-looking reference bone from a real deformed bone. The proposed framework was evaluated using a clinical dataset and compared with a state-of-the-art method that is based on a supervised point-cloud network. Experimental results show that the estimated shape models given by our approach are clinically acceptable and significantly more accurate than that of the competing method.
△ Less
Submitted 11 September, 2021;
originally announced September 2021.
-
Automatic Registration between Cone-Beam CT and Scanned Surface via Deep-Pose Regression Neural Networks and Clustered Similarities
Authors:
Minyoung Chung,
Jingyu Lee,
Wisoo Song,
Youngchan Song,
Il-Hyung Yang,
Jeongjin Lee,
Yeong-Gil Shin
Abstract:
…between maxillofacial cone-beam computed tomography (CT) images and a scanned dental model is an essential prerequisite in surgical planning for dental implants or orthognathic surgery. We propose a novel method that performs fully automatic registration between a cone-beam CT image and an optically scanned model. To build a robust and automatic initial regi…
▽ More
Computerized registration between maxillofacial cone-beam computed tomography (CT) images and a scanned dental model is an essential prerequisite in surgical planning for dental implants or orthognathic surgery. We propose a novel method that performs fully automatic registration between a cone-beam CT image and an optically scanned model. To build a robust and automatic initial registration method, our method applies deep-pose regression neural networks in a reduced domain (i.e., 2-dimensional image). Subsequently, fine registration is performed via optimal clusters. Majority voting system achieves globally optimal transformations while each cluster attempts to optimize local transformation parameters. The coherency of clusters determines their candidacy for the optimal cluster set. The outlying regions in the iso-surface are effectively removed based on the consensus among the optimal clusters. The accuracy of registration was evaluated by the Euclidean distance of 10 landmarks on a scanned model which were annotated by the experts in the field. The experiments show that the proposed method's registration accuracy, measured in landmark distance, outperforms other existing methods by 30.77% to 70%. In addition to achieving high accuracy, our proposed method requires neither human-interactions nor priors (e.g., iso-surface extraction). The main significance of our study is twofold: 1) the employment of light-weighted neural networks which indicates the applicability of neural network in extracting pose cues that can be easily obtained and 2) the introduction of an optimal cluster-based registration method that can avoid metal artifacts during the matching procedures.
△ Less
Submitted 29 July, 2019;
originally announced July 2019.
-
Development of a computer-aided design software for dental splint in orthognathic surgery
Authors:
Xiaojun Chen,
Xing Li,
Lu Xu,
Yi Sun,
Constantinus Politis,
Jan Egger
Abstract:
In the orthognathic surgery, dental splints are important and necessary to help the surgeon reposition the maxilla or mandible. However, the traditional methods of manual design of dental splints are difficult and time-consuming. The research on computer-aided design software for dental splints is rarely reported. Our purpose is to develop a novel special so…
▽ More
In the orthognathic surgery, dental splints are important and necessary to help the surgeon reposition the maxilla or mandible. However, the traditional methods of manual design of dental splints are difficult and time-consuming. The research on computer-aided design software for dental splints is rarely reported. Our purpose is to develop a novel special software named EasySplint to design the dental splints conveniently and efficiently. The design can be divided into two steps, which are the generation of initial splint base and the Boolean operation between it and the maxilla-mandibular model. The initial splint base is formed by ruled surfaces reconstructed using the manually picked points. Then, a method to accomplish Boolean operation based on the distance filed of two meshes is proposed. The interference elimination can be conducted on the basis of marching cubes algorithm and Boolean operation. The accuracy of the dental splint can be guaranteed since the original mesh is utilized to form the result surface. Using EasySplint, the dental splints can be designed in about 10 minutes and saved as a stereo lithography (STL) file for 3D printing in clinical applications. Three phantom experiments were conducted and the efficiency of our method was demonstrated.
△ Less
Submitted 9 March, 2017;
originally announced March 2017.
-
Biomechanical models to simulate consequences of maxillofacial surgery
Authors:
Yohan Payan,
Matthieu Chabanas,
Xavier Pelorson,
Coriandre Vilain,
Patrick Levy,
Vincent Luboz,
Pascal Perrier
Abstract:
…a brief overview of the continuous elastic modelling method, two models are introduced and their use for computer-assisted applications discussed. The first model deals with orthognathic surgery and aims at predicting the facial consequences of maxillary and mandibular osteotomies. For this, a generic three-dimensional model of the face is automatically adap…
▽ More
This paper presents the biomechanical finite element models that have been developed in the framework of the computer-assisted maxillofacial surgery. After a brief overview of the continuous elastic modelling method, two models are introduced and their use for computer-assisted applications discussed. The first model deals with orthognathic surgery and aims at predicting the facial consequences of maxillary and mandibular osteotomies. For this, a generic three-dimensional model of the face is automatically adapted to the morphology of the patient by the mean of elastic registration. Qualitative simulations of the consequences of an osteotomy of the mandible can thus be provided. The second model addresses the Sleep Apnoea Syndrome. Its aim is to develop a complete modelling of the interaction between airflow and upper airways walls during respiration. Dynamical simulations of the interaction during a respiratory cycle are computed and compared with observed phenomena.
△ Less
Submitted 16 June, 2006;
originally announced June 2006.
