Generalized Principal Component Analysis

Book Summary

This book provides a comprehensive introduction to the latest advances in the mathematical theory and computational tools for modeling high-dimensional data drawn from one or multiple low-dimensional subspaces (or manifolds) and potentially corrupted by noise, gross errors, or outliers. This challenging task requires the development of new algebraic, geometric, statistical, and computational methods for efficient and robust estimation and segmentation of one or multiple subspaces. The book also presents interesting real-world applications of these new methods in image processing, image and video segmentation, face recognition and clustering, and hybrid system identification etc.

This book is intended to serve as a textbook for graduate students and beginning researchers in data science, machine learning, computer vision, image and signal processing, and systems theory. It contains ample illustrations, examples, and exercises and is made largely self-contained with three Appendices which survey basic concepts and principles from statistics, optimization, and algebraic-geometry used in this book.

Rene Vidal is a Professor of Biomedical Engineering and Director of the Vision Dynamics and Learning Lab at The Johns Hopkins University, USA.

Yi Ma is Executive Dean and Professor at the School of Information Science and Technology at the ShanghaiTech University, China.

S. Shankar Sastry is Dean of the College of Engineering, Professor of Electrical Engineering and Computer Science and Professor of Bioengineering at the University of California at Berkeley, USA.

This book is intended to serve as a textbook for graduate students and beginning researchers in data science, machine learning, computer vision, image and signal processing, and systems theory. It contains ample illustrations, examples, and exercises and is made largely self-contained with three Appendices which survey basic concepts and principles from statistics, optimization, and algebraic-geometry used in this book.

Rene Vidal is a Professor of Biomedical Engineering and Director of the Vision Dynamics and Learning Lab at The Johns Hopkins University, USA.

Yi Ma is Executive Dean and Professor at the School of Information Science and Technology at the ShanghaiTech University, China.

S. Shankar Sastry is Dean of the College of Engineering, Professor of Electrical Engineering and Computer Science and Professor of Bioengineering at the University of California at Berkeley, USA.

Table of Contents

- Introduction
**Part I: Modeling Data with a Single Subspace**- Principal Component Analysis
- Robust Principal Component Analysis
- Nonlinear and Nonparametric Extensions
**Part II: Modeling Data with Multiple Subspaces**- Algebraic-Geometric Methods
- Statistical Methods
- Spectral Methods
- Sparse and Low-Rank Methods
**Part III: Applications**- Image Representation
- Image Segmentation
- Motion Segmentation
- Hybrid System Identification
- Final Words

Slides

Slides for Algebraic, Sparse and Low Rank Subspace Clustering, CVPR June 2015.

Code

Chapter 2: Principal Component Analysis

Chapter 3: Robust Principal Component Analysis

Chapter 4: Nonlinear and Nonparametric Extensions

Chapter 6: Statistical Methods

Chapter 7: Spectral Methods

Chapter 8: Sparse and Low Rank Methods

Toolboxes:

Examples 2.7, 2.10, 2.11: Modeling face images under varying illuminations; model selection for face images.

Chapter 3: Robust Principal Component Analysis

Examples 3.3, 3.6: Completing face images.

Examples 3.7, 3.12: Face shadow removal.

Example 3.16: Outlier dection among face images.

Chapter 4: Nonlinear and Nonparametric Extensions

Examples 4.1, 4.10, 4.12, 4.15: Embedding face images under varying poses.

Examples 4.16: Embedding face images of two different subjects.

Examples 4.17, 4.23: Kmeans and Spectral Clustering of Face Images under Varying Pose.

Examples 4.18, 4.24: Kmeans and Spectral Clustering of Face Images under Varying Illumination.

Chapter 6: Statistical Methods

Experiment: Clustering Face Image under Varying Illumination.

Chapter 7: Spectral Methods

Experiment: Spectral Methods on Face Clustering.

Chapter 8: Sparse and Low Rank Methods

Experiment: Face Clustering Affinities for Two Subjectes from the Extended Yale B Data Set.

Experiment: Face Clustering Errors on All Subjects from the Extended Yale B Data Set.

Toolboxes:

Plottoolbox: A toolbox for plotting images.

loadimage_EYaleB: Code for reading images from EYaleB dataset.

RPCAtoolbox: Code for several RPCA algorithms.

loadimage_Outlier.BestMap: For evaluating clustering result.

SCtoolbox: Spectral clustering toolbox.

loadimage_ATT: Code for reading images from ATT dataset.

statisticalSC: Code for several statistical subspace clustering methods.

ASC: Code for algebraic subspace clustering methods.

EYaleB: Preprocessed Extended Yale B.