Image understanding is an attempt to extract knowledge about a 3D scene from 20 images. The recent development of computers has made it possible to automate a wide range of systems and operations, not only in the industry, military, and special environments (space, sea, atomic plants, etc.), but also in daily life. As we now try to build ever more intelligent systems, the need for "visual" control has been strongly recognized, and the interest in image under- standing has grown rapidly. Already, there exists a vast body of literature-ranging from general philosophical discourses to processing techniques. Compared with other works, however, this book may be unique in that its central focus is on "mathematical" principles-Lie groups and group representation theory, in particular. In the study of the relationship between the 3D scene and the 20 image, "geometry" naturally plays a central role. Today, so many branches are inter- woven in geometry that we cannot truly regard it as a single subject. Neverthe- less, as Felix Klein declared in his Erlangen Program, the central principle of geometry is group theory, because geometrical concepts are abstractions of properties that are "invariant" with respect to some group of transformations. In this text, we specifically focus on two groups of transformations. One is 20 rotations of the image coordinate system around the image origin. Such coordi- nate rotations are indeed irrelevant when we look for intrinsic image properties.

Table of Contents:
1. Introduction.- 1.1 What is Image Understanding?.- 1.2 Imaging Geometry of Perspective Projection.- 1.3 Geometry of Camera Rotation.- 1.3.1 Projective Transformation.- 1.4 The 3D Euclidean Approach.- 1.5 The 2D Non-Euclidean Approach.- 1.5.1 Relative Geometry and Absolute Geometry.- 1.6 Organization of This Book.- 2. Coordinate Rotation Invariance of Image Characteristics.- 2.1 Image Characteristics and 3D Recovery Equations.- 2.2 Parameter Changes and Representations.- 2.2.1 Rotation of the Coordinate System.- 2.3 Invariants and Weights.- 2.3.1 Signs of the Weights and Degeneracy.- 2.4 Representation of a Scalar and a Vector.- 2.4.1 Invariant Meaning of a Position.- 2.5 Representation of a Tensor.- 2.5.1 Parity.- 2.5.2 Weyl’s Theorem.- 2.6 Analysis of Optical Flow for a Planar Surface.- 2.6.1 Optical Flow.- 2.6.2 Translation Velocity, Rotation Velocity, and Egomotion.- 2.6.3 Equations in Terms of Invariants.- 2.7 Shape from Texture for Curved Surfaces.- 2.7.1 Curvatures of a Surface.- Exercises.- 3. 3D Rotation and Its Irreducible Representations.- 3.1 Invariance for the Camera Rotation Transformation.- 3.1.1 SO(3) Is Three Dimensional and Not Abelian.- 3.2 Infinitesimal Generators and Lie Algebra.- 3.3 Lie Algebra and Casimir Operator of the 3D Rotation Group.- 3.3.1 Infinitesimal Rotations Commute.- 3.3.2 Reciprocal Representations.- 3.3.3 Spinors.- 3.4 Representation of a Scalar and a Vector.- 3.5 Irreducible Reduction of a Tensor.- 3.5.1 Canonical Form of Infinitesimal Generators.- 3.5.2 Alibi vs Alias.- 3.6 Restriction of SO(3) to SO(2).- 3.6.1 Broken Symmetry.- 3.7 Transformation of Optical Flow.- 3.7.1 Invariant Decomposition of Optical Flow.- Exercises.- 4. Algebraic Invariance of Image Characteristics.- 4.1 Algebraic Invariants and Irreducibility.- 4.2 Scalars, Points, and Lines.- 4.3 Irreducible Decomposition of a Vector.- 4.4 Irreducible Decomposition of a Tensor.- 4.5 Invariants of Vectors.- 4.5.1 Interpretation of Invariants.- 4.6 Invariants of Points and Lines.- 4.6.1 Spherical Geometry.- 4.7 Invariants of Tensors.- 4.7.1 Symmetric Polynomials.- 4.7.2 Classical Theory of Invariants.- 4.8 Reconstruction of Camera Rotation.- Exercises.- 5. Characterization of Scenes and Images.- 5.1 Parametrization of Scenes and Images.- 5.2 Scenes, Images, and the Projection Operator.- 5.3 Invariant Subspaces of the Scene Space.- 5.3.1 Tensor Calculus.- 5.4 Spherical Harmonics.- 5.5 Tensor Expressions of Spherical Harmonics.- 5.6 Irreducibility of Spherical Harmonics.- 5.6.1 Laplace Spherical Harmonics.- 5.7 Camera Rotation Transformation of the Image Space.- 5.7.1 Parity of Scenes.- 5.8 Invariant Measure.- 5.8.1 First Fundamental Form.- 5.8.2 Fluid Dynamics Analogy.- 5.9 Transformation of Features.- 5.10 Invariant Characterization of a Shape.- 5.10.1 Invariance on the Image Sphere.- 5.10.2 Further Applications.- Exercises.- 6. Representation of 3D Rotations.- 6.1 Representation of Object Orientations.- 6.2 Rotation Matrix.- 6.2.1 The nD Rotation Group SO(n).- 6.3 Rotation Axis and Rotation Angle.- 6.4 Euler Angles.- 6.5 Cayley—Klein Parameters.- 6.6 Representation of SO(3) by SU(2).- 6.6.1 Spinors.- 6.7 Adjoint Representation of SU(2).- 6.7.1 Differential Representation.- 6.8 Quaternions.- 6.8.1 Quaternion Field.- 6.9 Topology of SO(3).- 6.9.1 Universal Covering Group.- 6.10 Invariant Measure of 3D Rotations.- Exercises.- 7. Shape from Motion.- 7.1 3D Recovery from Optical Flow for a Planar Surface.- 7.2 Flow Parameters and 3D Recovery Equations.- 7.2.1 Least Squares Method.- 7.3 Invariants of Optical Flow.- 7.4 Analytical Solution of the 3D Recovery Equations.- 7.5 Pseudo-orthographic Approximation.- 7.5.1 Robustness of Computation.- 7.6 Adjacency Condition of Optical Flow.- 7.7 3D Recovery of a Polyhedron.- 7.7.1 Noise Sensitivity of Computation.- 7.8 Motion Detection Without Correspondence.- 7.8.1 Stereo Without Correspondence.- Exercises.- 8. Shape from Angle.- 8.1 Rectangularity Hypothesis.- 8.2 Spatial Orientation of a Rectangular Corner.- 8.2.1 Corners with Two Right Angles.- 8.3 Interpretation of a Rectangular Polyhedron.- 8.3.1 Huffman—Clowes Edge Labeling.- 8.4 Standard Transformation of Corner Images.- 8.5 Vanishing Points and Vanishing Lines.- 8.5.1 Spherical Geometry and Projective Geometry.- Exercises.- 9. Shape from Texture.- 9.1 Shape from Texture from Homogeneity.- 9.2 Texture Density and Homogeneity.- 9.2.1 Distributions.- 9.3 Perspective Projection and the First Fundamental Form.- 9.4 Surface Shape Recovery from Texture.- 9.5 Recovery of Planar Surfaces.- 9.5.1 Error Due to Randomness of the Texture.- 9.6 Numerical Scheme of Planar Surface Recovery.- 9.6.1 Technical Aspects of Implementation.- 9.6.2 Newton Iterations.- Exercises.- 10. Shape from Surface.- 10.1 What Does 3D Shape Recovery Mean?.- 10.2 Constraints on a $$2\frac{1} {2}\text{D}$$ Sketch.- 10.2.1 Singularity of the Incidence Structure.- 10.2.2 Integrability Condition.- 10.3 Optimization of a $$2\frac{1}{2}\text{D}$$ Sketch.- 10.3.1 Regularization.- 10.4 Optimization for Shape from Motion.- 10.5 Optimization of Rectangularity Heuristics.- 10.6 Optimization of Parallelism Heuristics.- 10.6.1 Parallelogram Test and Computational Geometry.- Exercises.- Appendix. Fundamentals of Group Theory.- A.1 Sets, Mappings, and Transformations.- A.2 Groups.- A.3 Linear Spaces.- A.4 Metric Spaces.- A.5 Linear Operators.- A.6 Group Representation.- A.7 Schur’s Lemma.- A.8 Topology, Manifolds, and Lie Groups.- A.9 Lie Algebras and Lie Groups.- A.10 Spherical Harmonics.