Advanced Topics in Machine Learning: Modeling and Segmentation of Multivariate Mixed Data

Course Description

This class will cover machine learning techniques for modeling and segmentation of multivariate mixed data. Topics will include subspace learning (PCA, Probabilistic PCA, Robust PCA, Sparse representation, Rank minimization), manifold learning (Kernel PCA, LLE, Isomap), subspace clustering (K-subspaces, Mixtures of PPCAs, Generalized PCA, Sparse subspace clustering), and manifold clustering (LLMC). These methods will be applied to several problems in computer vision, biomedical imaging, computational neuroscience, and computational biology.

Syllabus

**Introduction** (Chapter 1)

- 01/25 Course overview

**Subspace Learning**: Linear Dimensionality Reduction (Chapter 2)

- 01/25-29 Principal Component Analysis (PCA)
- 01/29 Application of PCA (slides)
- 02/01-03 Factor Analysis (FA) and Probabilistic PCA (PPCA)
- 02/05 Robust PCA: missing data (Power Factorization)
- 02/08-12: Shoveling
- 02/15 Robust PCA: outliers
- 02/17 Robust PCA via sparse representation and rank minimization: (RPCA) (slides)

**Manifold Learning**: Nonlinear Dimensionality Reduction (Chapter 2)

- 02/19 Nonlinear and Kernel PCA (KPCA)
- 02/22 Multidimensional Scaling (MDS), Isometric Embedding (Isomap) and Locally Linear Embedding (LLE) (slides)

**Subspace Clustering**: Iterative Methods (Chapter 4)

- 02/26 K-means and K-Subspaces
- 03/01 K-Subspaces and RANSAC

**Subspace Clustering**: Algebraic Methods (Chapter 3)

- 03/03 Line, plane, and hyperplane clustering (slides)
- 03/05 Midterm 1
- 03/08 Generalized Principal Component Analysis (GPCA)
- 03/10 Local Subspace Affinity (LSA) and Spectral Curvature Clustering (SCC)

**Subspace Clustering**: Robust Methods based on Sparse Representation (Chapter 5)

- 03/12 Sparse Subspace Clustering (SSC) (slides)

**Applications in Computer Vision**

- 03/22-24 Motion Segmentation from Multiple Affine Views (Chapter 8)
- 03/26-04/02 Motion Segmentation from Two Perspective Views (Chapter 8)
- 04/09 Spatial and Temporal Video Segmentation (Chapter 9) (slides)
- 04/16 Image Representation and Segmentation (Chapter 6) (slides)

**Manifold Clustering** (Chapter 12)

- 04/23-30 Locally Linear Manifold Clustering (LLMC) (slides)
- 05/07 Midterm 2

**References**

Administrative

**Grading policy**

- Homework
(30%): Homework problems will include both analytical exercises as well
as programming assignments in MATLAB.
- Exams
(70%): There will be two exams on
February 26
^{th}March 5^{th}and April 30^{th}May 7^{th. }

**Honor system**

Homeworks, midterms and projects will be individual. The strength of the university depends on academic and personal integrity. In this course, you must be honest and truthful. Ethical violations include cheating on exams, plagiarism, reuse of assignments, improper use of the Internet and electronic devices, unauthorized collaboration, alteration of graded assignments, forgery and falsification, lying, facilitating academic dishonesty, and unfair competition. All these will be severely penalized.

**Announcements**

- Class will meet MWF till March 12th and F thereafter.

**Homeworks**

Please submit your homework in *one single ZIP file* to the
Homework Submission Website.

- Homework 1. Due Wednesday February 17th, 2010. Solutions. Solutions code.
- Homework 2. Due Wednesday March 3rd, 2010. Download the dataset here. Solutions.
- Homework 3. Due Wednesday March 31st, 2010. Solution. Solution code for GPCA and k-subspaces.
- Homework 4. Due Friday May 7th, 2010. Motion sequences dataset. Videos of some of the motion sequences (just for reference, they are not needed for the homework). Face dataset.

**Midterms:**