An in-depth description of the state-of-the-art of 3D shape analysis techniques and their applications This book discusses the different topics that come under the title of "3D shape analysis". It covers the theoretical foundations and the major solutions that have been presented in the literature. It also establishes links between solutions proposed by different communities that studied 3D shape, such as mathematics and statistics, medical imaging, computer vision, and computer graphics. The first part of 3D Shape Analysis: Fundamentals, Theory, and Applications provides a review of the background concepts such as methods for the acquisition and representation of 3D geometries, and the fundamentals of geometry and topology. It specifically covers stereo matching, structured light, and intrinsic vs. extrinsic properties of shape. Parts 2 and 3 present a range of mathematical and algorithmic tools (which are used for e.g., global descriptors, keypoint detectors, local feature descriptors, and algorithms) that are commonly used for the detection, registration, recognition, classification, and retrieval of 3D objects. Both also place strong emphasis on recent techniques motivated by the spread of commodity devices for 3D acquisition. Part 4 demonstrates the use of these techniques in a selection of 3D shape analysis applications. It covers 3D face recognition, object recognition in 3D scenes, and 3D shape retrieval. It also discusses examples of semantic applications and cross domain 3D retrieval, i.e. how to retrieve 3D models using various types of modalities, e.g. sketches and/or images. The book concludes with a summary of the main ideas and discussions of the future trends. 3D Shape Analysis: Fundamentals, Theory, and Applications is an excellent reference for graduate students, researchers, and professionals in different fields of mathematics, computer science, and engineering. It is also ideal for courses in computer vision and computer graphics, as well as for those seeking 3D industrial/commercial solutions.

Table of Contents:
Preface xv Acknowledgments xvii 1 Introduction 1 1.1 Motivation 1 1.2 The 3D Shape Analysis Problem 2 1.3 About This Book 5 1.4 Notation 9 Part I Foundations 11 2 Basic Elements of 3D Geometry and Topology 13 2.1 Elements of Differential Geometry 13 2.1.1 Parametric Curves 13 2.1.2 Continuous Surfaces 15 2.1.2.1 Differential Properties of Surfaces 17 2.1.2.1.1 First Fundamental Form 17 2.1.2.1.2 Second Fundamental Form and Shape Operator 18 2.1.2.2 Curvatures 19 2.1.2.3 Laplace and Laplace–Beltrami Operators 21 2.1.3 Manifolds, Metrics, and Geodesics 22 2.1.4 Discrete Surfaces 24 2.1.4.1 Representations of Discrete Surfaces 24 2.1.4.2 Mesh Data Structures 28 2.1.4.3 Discretization of the Differential Properties of Surfaces 29 2.2 Shape, Shape Transformations, and Deformations 30 2.2.1 Shape-Preserving Transformations 31 2.2.1.1 Normalization for Translation 32 2.2.1.2 Normalization for Scale 32 2.2.1.3 Normalization for Rotation 32 2.2.1.3.1 Rotation Normalization Using Principal Component Analysis (PCA) 33 2.2.1.3.2 Rotation Normalization Using Planar Reflection Symmetry Analysis 34 2.2.2 Shape Deformations 35 2.2.3 Bending 35 2.2.4 Stretching 37 2.3 Summary and Further Reading 38 3 3D Acquisition and Preprocessing 41 3.1 Introduction 41 3.2 3D Acquisition 41 3.2.1 Contact 3D Acquisition 43 3.2.1.1 Coordinate Measuring Machine (CMM) 43 3.2.1.2 Arm-Based 3D Scanner 44 3.2.2 Noncontact 3D Acquisition 44 3.2.2.1 Time-of-Flight 44 3.2.2.1.1 Pulse-Based TOF 44 3.2.2.1.2 Phase Shift-Based TOF 45 3.2.2.2 Triangulation 45 3.2.2.3 Stereo 47 3.2.2.4 Structured Light 50 3.2.2.4.1 Temporal Coded Patterns 51 3.2.2.4.2 Spatial Coded Patterns 52 3.2.2.4.3 Direct Coded Patterns 55 3.2.2.5 Shape from X 55 3.3 Preprocessing 3D Models 56 3.3.1 Surface Smoothing and Fairing 57 3.3.1.1 Laplacian Smoothing 57 3.3.1.2 Taubin Smoothing 58 3.3.1.3 Curvature Flow Smoothing 58 3.3.2 Spherical Parameterization of 3D Surfaces 58 3.4 Summary and Further Reading 62 Part II 3D Shape Descriptors 65 4 Global Shape Descriptors 67 4.1 Introduction 67 4.2 Distribution-Based Descriptors 69 4.2.1 Point Sampling 69 4.2.2 Geometric Features 70 4.2.2.1 Geometric Attributes 70 4.2.2.2 Differential Attributes 71 4.2.3 Signature Construction and Comparison 72 4.3 View-Based 3D Shape Descriptors 73 4.3.1 The Light Field Descriptors (LFD) 74 4.3.2 Feature Extraction 75 4.3.3 Properties 76 4.4 Spherical Function-Based Descriptors 77 4.4.1 Spherical Shape Functions 78 4.4.2 Comparing Spherical Functions 80 4.4.2.1 Spherical Harmonic Descriptors 80 4.4.2.2 SphericalWavelet Transforms 81 4.4.2.2.1 Wavelet Coefficients as a Shape Descriptor 82 4.4.2.2.2 SphericalWavelet Energy Signatures 82 4.5 Deep Neural Network-Based 3D Descriptors 83 4.5.1 CNN-Based Image Descriptors 84 4.5.2 Multiview CNN for 3D Shapes 85 4.5.2.1 Network Architecture 86 4.5.2.2 View Aggregation using CNN 86 4.5.3 Volumetric CNN 87 4.6 Summary and Further Reading 89 5 Local Shape Descriptors 93 5.1 Introduction 93 5.2 Challenges and Criteria 94 5.2.1 Challenges 94 5.2.2 Criteria for 3D Keypoint Detection 95 5.2.3 Criteria for Local Feature Description 96 5.3 3D Keypoint Detection 96 5.3.1 Fixed-Scale Keypoint Detection 97 5.3.1.1 Curvature-Based Methods 97 5.3.1.1.1 Local Surface Patch (LSP) 98 5.3.1.2 Other Surface Variation-Based Methods 98 5.3.1.2.1 Matei’s Method 99 5.3.1.2.2 Intrinsic Shape Signatures (ISS) 99 5.3.1.2.3 Harris 3D 99 5.3.2 Adaptive-Scale Keypoint Detection 101 5.3.2.1 Extrinsic Scale-Space Based Methods 101 5.3.2.1.1 3D Shape Filtering 101 5.3.2.1.2 Multiscale Surface Variation 104 5.3.2.2 Intrinsic Scale-Space Based Methods 106 5.3.2.2.1 Scale-Space on 2D Parameterized Images 106 5.3.2.2.2 Scale-Space on 3D Shapes 109 5.3.2.2.3 Scale-Space on Transformed Domains 112 5.4 Local Feature Description 113 5.4.1 Signature-Based Methods 114 5.4.1.1 Splash 114 5.4.1.2 Point Signature 115 5.4.2 Histogram Based Methods 115 5.4.2.1 Histogram of Spatial Distributions 115 5.4.2.1.1 Spin Images 116 5.4.2.1.2 3D Shape Context 117 5.4.2.1.3 Intrinsic Shape Signature (ISS) 118 5.4.2.1.4 Rotational Projection Statistics (RoPS) 118 5.4.2.2 Histogram of Geometric Attributes 122 5.4.2.2.1 Point Feature Histograms (PFH) 122 5.4.2.2.2 Fast Point Feature Histograms (FPFH) 123 5.4.2.2.3 Signature of Histograms of Orientations (SHOT) 123 5.4.2.3 Histogram of Oriented Gradients 124 5.4.3 Covariance-Based Methods 124 5.5 Feature Aggregation Using Bag of Feature Techniques 126 5.5.1 Dictionary Construction 127 5.5.1.1 Feature Extraction 127 5.5.1.2 Codebook Construction 127 5.5.2 Coding and Pooling Schemes 128 5.5.2.1 Sparse Coding 128 5.5.2.2 Fisher Vectors 129 5.5.3 Vector of Locally Aggregated Descriptors (VLAD) 129 5.5.4 Vector of Locally Aggregated Tensors (VLAT) 130 5.6 Summary and Further Reading 131 5.6.1 Summary of 3D Keypoint Detection 131 5.6.2 Summary of Local Feature Description 132 5.6.3 Summary of Feature Aggregation 133 Part III 3D Correspondence and Registration 135 6 Rigid Registration 137 6.1 Introduction 137 6.2 Coarse Registration 138 6.2.1 Point Correspondence-Based Registration 138 6.2.1.1 The Typical Pipeline 139 6.2.1.2 Transformation Estimation from a Group of Correspondences 139 6.2.1.3 Transformation Estimation fromThree Correspondences 140 6.2.1.4 Transformation Estimation from Two Correspondences 141 6.2.1.5 Transformation Estimation from One Correspondence 142 6.2.2 Line-Based Registration 143 6.2.2.1 Line Matching Method 143 6.2.2.2 Line Clustering Method 144 6.2.2.2.1 Rotation Estimation 145 6.2.2.2.2 Translation Estimation 146 6.2.3 Surface-Based Registration 146 6.2.3.1 Principal Component Analysis (PCA) 146 6.2.3.2 RANSAC-Based DARCES 147 6.2.3.3 Four-Points Congruent Sets (4PCS) 149 6.2.3.3.1 Affine Invariants of 4-Points Set 149 6.2.3.3.2 Congruent 4-Points Extraction 151 6.2.3.3.3 The 4PCS Algorithm 151 6.3 Fine Registration 152 6.3.1 Iterative Closest Point (ICP) 153 6.3.1.1 Closest Point Search 153 6.3.1.2 Transformation Estimation 153 6.3.1.3 Summary of the ICP Method 154 6.3.2 ICP Variants 155 6.3.2.1 Point Selection 155 6.3.2.2 Point Matching 156 6.3.2.3 Point PairWeighting 156 6.3.2.4 Point Pair Rejection 156 6.3.2.5 Error Metrics 157 6.3.3 Coherent Point Drift 157 6.4 Summary and Further Reading 160 7 Nonrigid Registration 161 7.1 Introduction 161 7.2 Problem Formulation 162 7.3 Mathematical Tools 165 7.3.1 The Space of Diffeomorphisms 165 7.3.2 Parameterizing Spaces 166 7.4 Isometric Correspondence and Registration 168 7.4.1 Möbius Voting 168 7.4.2 Examples 170 7.5 Nonisometric (Elastic) Correspondence and Registration 171 7.5.1 Surface Deformation Models 171 7.5.1.1 Linear Deformation Model 171 7.5.1.2 Elastic Deformation Models 172 7.5.2 Square-Root Normal Fields (SRNF) Representation 173 7.5.3 Numerical Inversion of SRNF Maps 174 7.5.3.1 SRNF Inversion Algorithm 176 7.5.4 Correspondence 177 7.5.4.1 Optimization Over SO(3) 178 7.5.4.2 Optimization Over Γ 178 7.5.4.3 Differential of 𝜙 [184] 179 7.5.4.4 Initialization of the Gradient [184] 179 7.5.5 Elastic Registration and Geodesics 181 7.5.6 Coregistration 181 7.6 Summary and Further Reading 184 8 Semantic Correspondences 187 8.1 Introduction 187 8.2 Mathematical Formulation 188 8.3 Graph Representation 191 8.3.1 Characterizing the Local Geometry and the Spatial Relations 191 8.3.1.1 Unary Descriptors 192 8.3.1.2 Binary Descriptors 192 8.3.2 Cross Mesh Pairing of Patches 192 8.4 Energy Functions for Semantic Labeling 194 8.4.1 The Data Term 194 8.4.2 Smoothness Terms 194 8.4.2.1 Smoothness Constraints 194 8.4.2.2 Geometric Compatibility 195 8.4.2.3 Label Compatibility 196 8.4.3 The Intermesh Term 196 8.5 Semantic Labeling 196 8.5.1 Unsupervised Clustering 197 8.5.2 Learning the Labeling Likelihood 199 8.5.2.1 GentleBoost Classifier 199 8.5.2.2 Training GentleBoost Classifiers 200 8.5.3 Learning the Remaining Parameters 201 8.5.4 Optimization Using Graph Cuts 202 8.6 Examples 202 8.7 Summary and Further Reading 204 Part IV Applications 207 9 Examples of 3D Semantic Applications 209 9.1 Introduction 209 9.2 Semantics: Shape or Status 209 9.3 Semantics: Class or Identity 212 9.4 Semantics: Behavior 216 9.5 Semantics: Position 219 9.6 Summary and Further Reading 221 10 3D Face Recognition 223 10.1 Introduction 223 10.2 3D Face Recognition Tasks, Challenges and Datasets 224 10.2.1 3D Face Verification 224 10.2.2 3D Face Identification 225 10.2.3 3D Face Recognition Challenges 225 10.2.3.1 Intrinsic Transformations 225 10.2.3.2 Acquisition Conditions 226 10.2.3.3 Data Acquisition 226 10.2.3.4 Computation Time 227 10.2.4 3D Face Datasets 227 10.3 3D Face Recognition Methods 228 10.3.1 Holistic Approaches 232 10.3.1.1 Eigenfaces and Fisherfaces 232 10.3.1.1.1 Eigenfaces 232 10.3.1.1.2 Fisherfaces 233 10.3.1.2 Iterative Closest Point 234 10.3.1.3 Hausdorff Distance 234 10.3.1.4 Canonical Form 234 10.3.2 Local Feature-Based Matching 235 10.3.2.1 Keypoint-Based Methods 235 10.3.2.1.1 Landmark-Based Methods 235 10.3.2.1.2 SIFT-Like Keypoints 236 10.3.2.2 Curve-Based Features 237 10.3.2.3 Patch-Based Features 238 10.3.2.4 Other Features 239 10.4 Summary 239 11 Object Recognition in 3D Scenes 241 11.1 Introduction 241 11.2 Surface Registration-Based Object Recognition Methods 241 11.2.1 Feature Matching 242 11.2.2 Hypothesis Generation 242 11.2.2.1 Geometric Consistency-Based Hypothesis Generation 243 11.2.2.2 Pose Clustering-Based Hypothesis Generation 244 11.2.2.3 Constrained Interpretation Tree-Based Hypothesis Generation 244 11.2.2.4 RANdom SAmple Consensus-Based Hypothesis Generation 245 11.2.2.5 GameTheory-Based Hypothesis Generation 246 11.2.2.5.1 Preliminary on Game Theory 246 11.2.2.5.2 Matching Game for Transformation Hypothesis Generation 247 11.2.2.6 Generalized Hough Transform-Based Hypothesis Generation 248 11.2.3 Hypothesis Verification 249 11.2.3.1 Individual Verification 249 11.2.3.2 Global Verification 251 11.3 Machine Learning-Based Object Recognition Methods 255 11.3.1 Hough Forest-Based 3D Object Detection 255 11.3.1.1 3D Local Patch Extraction 255 11.3.1.2 3D Local Patch Representation 256 11.3.1.3 Hough Forest Training and Testing 256 11.3.1.3.1 Offline Training 256 11.3.1.3.2 Online detection 258 11.3.2 Deep Learning-Based 3D Object Recognition 260 11.3.2.1 Hand-crafted Feature-Based Methods 262 11.3.2.2 2D View-Based Methods 262 11.3.2.3 3D Voxel-Based Methods 263 11.3.2.4 3D Point Cloud-Based Methods 265 11.4 Summary and Further Reading 265 12 3D Shape Retrieval 267 12.1 Introduction 267 12.2 Benchmarks and Evaluation Criteria 270 12.2.1 3D Datasets and Benchmarks 270 12.2.2 Performance Evaluation Metrics 271 12.2.2.1 Precision 272 12.2.2.2 Recall 272 12.2.2.3 Precision-Recall Curves 273 12.2.2.4 F- and E-Measures 273 12.2.2.5 Area under Curve (AUC) or Average Precision (AP) 273 12.2.2.6 Mean Average Precision (mAP) 274 12.2.2.7 Cumulated Gain-Based Measure 274 12.2.2.8 Nearest Neighbor (NN), First-Tier (FT), and Second-Tier (ST) 275 12.3 Similarity Measures 275 12.3.1 Dissimilarity Measures 275 12.3.2 Hashing and Hamming Distance 277 12.3.3 Manifold Ranking 278 12.4 3D Shape Retrieval Algorithms 280 12.4.1 Using Handcrafted Features 280 12.4.2 Deep Learning-Based Methods 282 12.5 Summary and Further Reading 284 13 Cross-domain Retrieval 285 13.1 Introduction 285 13.2 Challenges and Datasets 287 13.2.1 Datasets 288 13.2.2 Training Data Synthesis 289 13.2.2.1 Photo Synthesis from 3D Models 289 13.2.2.2 2D Sketch Synthesis from 3D Models 290 13.3 Siamese Network for Cross-domain Retrieval 290 13.4 3D Shape-centric Deep CNN 292 13.4.1 Embedding Space Construction 293 13.4.1.1 Principal Component Analysis 295 13.4.1.2 Multi-dimensional Scaling 296 13.4.1.3 Kernel-Based Analysis 296 13.4.2 Learning Shapes from Synthesized Data 298 13.4.3 Image and Sketch Projection 298 13.5 Summary and Further Reading 300 14 Conclusions and Perspectives 301 References 303 Index 337