Graph spectral image processing is the study of imaging data from a graph frequency perspective. Modern image sensors capture a wide range of visual data including high spatial resolution/high bit-depth 2D images and videos, hyperspectral images, light field images and 3D point clouds. The field of graph signal processing – extending traditional Fourier analysis tools such as transforms and wavelets to handle data on irregular graph kernels – provides new flexible computational tools to analyze and process these varied types of imaging data. Recent methods combine graph signal processing ideas with deep neural network architectures for enhanced performances, with robustness and smaller memory requirements. The book is divided into two parts. The first is centered on the fundamentals of graph signal processing theories, including graph filtering, graph learning and graph neural networks. The second part details several imaging applications using graph signal processing tools, including image and video compression, 3D image compression, image restoration, point cloud processing, image segmentation and image classification, as well as the use of graph neural networks for image processing.

Table of Contents:
Introduction to Graph Spectral Image Processing xi Gene CHEUNG and Enrico MAGLI Part 1. Fundamentals of Graph Signal Processing 1 Chapter 1. Graph Spectral Filtering 3 Yuichi TANAKA 1.1. Introduction 3 1.2. Review: filtering of time-domain signals 4 1.3. Filtering of graph signals 5 1.3.1. Vertex domain filtering 6 1.3.2. Spectral domain filtering 8 1.3.3. Relationship between graph spectral filtering and classical filtering 10 1.4. Edge-preserving smoothing of images as graph spectral filters 11 1.4.1. Early works 11 1.4.2. Edge-preserving smoothing 12 1.5. Multiple graph filters: graph filter banks 15 1.5.1. Framework 16 1.5.2. Perfect reconstruction condition 17 1.6. Fast computation 20 1.6.1. Subdivision 20 1.6.2. Downsampling 21 1.6.3. Precomputing GFT 22 1.6.4. Partial eigendecomposition 22 1.6.5. Polynomial approximation 23 1.6.6. Krylov subspace method 26 1.7. Conclusion 26 1.8. References 26 Chapter 2. Graph Learning 31 Xiaowen DONG, Dorina THANOU, Michael RABBAT and Pascal FROSSARD 2.1. Introduction 31 2.2. Literature review 33 2.2.1. Statistical models 33 2.2.2. Physically motivated models 35 2.3. Graph learning: a signal representation perspective 36 2.3.1. Models based on signal smoothness 38 2.3.2. Models based on spectral filtering of graph signals 43 2.3.3. Models based on causal dependencies on graphs 48 2.3.4. Connections with the broader literature 50 2.4. Applications of graph learning in image processing 52 2.5. Concluding remarks and future directions 55 2.6. References 57 Chapter 3. Graph Neural Networks 63 Giulia FRACASTORO and Diego VALSESIA 3.1. Introduction 63 3.2. Spectral graph-convolutional layers 64 3.3. Spatial graph-convolutional layers 66 3.4. Concluding remarks 71 3.5. References 72 Part 2. Imaging Applications of Graph Signal Processing 73 Chapter 4. Graph Spectral Image and Video Compression 75 Hilmi E. EGILMEZ, Yung-Hsuan CHAO and Antonio ORTEGA 4.1. Introduction 75 4.1.1. Basics of image and video compression 77 4.1.2. Literature review 78 4.1.3. Outline of the chapter 79 4.2. Graph-based models for image and video signals 79 4.2.1. Graph-based models for residuals of predicted signals 81 4.2.2. DCT/DSTs as GFTs and their relation to 1D models 87 4.2.3. Interpretation of graph weights for predictive transform coding 88 4.3. Graph spectral methods for compression 89 4.3.1. GL-GFT design 89 4.3.2. EA-GFT design 92 4.3.3. Empirical evaluation of GL-GFT and EA-GFT 97 4.4. Conclusion and potential future work 100 4.5. References 101 Chapter 5. Graph Spectral 3D Image Compression 105 Thomas MAUGEY, Mira RIZKALLAH, Navid MAHMOUDIAN BIDGOLI, Aline ROUMY and Christine GUILLEMOT 5.1. Introduction to 3D images 106 5.1.1. 3D image definition 106 5.1.2. Point clouds and meshes 106 5.1.3. Omnidirectional images 107 5.1.4. Light field images 109 5.1.5. Stereo/multi-view images 110 5.2. Graph-based 3D image coding: overview 110 5.3. Graph construction 115 5.3.1. Geometry-based approaches 117 5.3.2. Joint geometry and color-based approaches 121 5.3.3. Separable transforms 125 5.4. Concluding remarks 126 5.5. References 128 Chapter 6. Graph Spectral Image Restoration 133 Jiahao PANG and Jin ZENG 6.1. Introduction 133 6.1.1. A simple image degradation model 133 6.1.2. Restoration with signal priors 135 6.1.3. Restoration via filtering 137 6.1.4. GSP for image restoration 140 6.2. Discrete-domain methods 141 6.2.1. Non-local graph-based transform for depth image denoising 141 6.2.2. Doubly stochastic graph Laplacian 142 6.2.3. Reweighted graph total variation prior 145 6.2.4. Left eigenvectors of random walk graph Laplacian 150 6.2.5. Graph-based image filtering 155 6.3. Continuous-domain methods 155 6.3.1. Continuous-domain analysis of graph Laplacian regularization 156 6.3.2. Low-dimensional manifold model for image restoration 163 6.3.3. LDMM as graph Laplacian regularization 165 6.4. Learning-based methods 167 6.4.1. CNN with GLR 169 6.4.2. CNN with graph wavelet filter 171 6.5. Concluding remarks 172 6.6. References 173 Chapter 7. Graph Spectral Point Cloud Processing 181 Wei HU, Siheng CHEN and Dong TIAN 7.1. Introduction 181 7.2. Graph and graph-signals in point cloud processing 183 7.3. Graph spectral methodologies for point cloud processing 185 7.3.1. Spectral-domain graph filtering for point clouds 185 7.3.2. Nodal-domain graph filtering for point clouds 188 7.3.3. Learning-based graph spectral methods for point clouds 189 7.4. Low-level point cloud processing 190 7.4.1. Point cloud denoising 191 7.4.2. Point cloud resampling 193 7.4.3. Datasets and evaluation metrics 198 7.5. High-level point cloud understanding 199 7.5.1. Data auto-encoding for point clouds 199 7.5.2. Transformation auto-encoding for point clouds 206 7.5.3. Applications of GraphTER in point clouds 211 7.5.4. Datasets and evaluation metrics 211 7.6. Summary and further reading 213 7.7. References 214 Chapter 8. Graph Spectral Image Segmentation 221 Michael NG 8.1. Introduction 221 8.2. Pixel membership functions 222 8.2.1. Two-class problems 222 8.2.2. Multiple-class problems 226 8.2.3. Multiple images 227 8.3. Matrix properties 230 8.4. Graph cuts 232 8.4.1. The Mumford–Shah model 234 8.4.2. Graph cuts minimization 235 8.5. Summary 237 8.6. References 237 Chapter 9. Graph Spectral Image Classification 241 Minxiang YE, Vladimir STANKOVIC, Lina STANKOVIC and Gene CHEUNG 9.1. Formulation of graph-based classification problems 243 9.1.1. Graph spectral classifiers with noiseless labels 243 9.1.2. Graph spectral classifiers with noisy labels 246 9.2. Toward practical graph classifier implementation 247 9.2.1. Graph construction 247 9.2.2. Experimental setup and analysis 249 9.3. Feature learning via deep neural network 255 9.3.1. Deep feature learning for graph construction 258 9.3.2. Iterative graph construction 260 9.3.3. Toward practical implementation of deep feature learning 262 9.3.4. Analysis on iterative graph construction for robust classification 267 9.3.5. Graph spectrum visualization 269 9.3.6. Classification error rate comparison using insufficient training data 270 9.3.7. Classification error rate comparison using sufficient training data with label noise 270 9.4. Conclusion 271 9.5. References 272 Chapter 10. Graph Neural Networks for Image Processing 277 Giulia FRACASTORO and Diego VALSESIA 10.1. Introduction 277 10.2. Supervised learning problems 278 10.2.1. Point cloud classification 278 10.2.2. Point cloud segmentation 281 10.2.3. Image denoising 283 10.3. Generative models for point clouds 286 10.3.1. Point cloud generation 286 10.3.2. Shape completion 291 10.4. Concluding remarks 294 10.5. References 294 List of Authors 299 Index 301