Computer Vision

CSE P576 // Spring 2020

ai rendered image

Meeting Information

Lectures: Tuesdays 6:30-9:20pm

Office Hours: Thursdays 5:30-6:30pm, or by appointment


Matthew Brown, Jonathan Huang

TAs: Nishat Khan, Dianqi Li

Course Description

A masters course in computer vision, emphasizing fundamentals of geometry and image formation as well as deep learning and image understanding.

Grading: The grade is based on four projects (equally weighted). Each project will be a mix of coding and written answers. See the course overview below for handin dates. Late policy is as follows: Late assignments will be accepted up to 5 days after the deadline, but with 10% subtracted from the mark per day.

Collaboration Policy: Discussing ideas or general strategies is fine, but students should not share solutions or code.


  1. Feature Extraction and Matching: Build an image feature matcher, starting with simple convolution operations.
  2. Panoramic Stitching: Implement a panorama stitcher using features, RANSAC and rotation estimation.
  3. Image Classification: Classify CIFAR10 images using Linear models and CNNs. Part 1 and Part 2.
  4. Pixel Labelling: Implement and experiment with a per-pixel classification or regression algorithm.


Course Overview

Date Lecture Description Notes and Resources
3/31 Introduction Week 1 Notes
Image Formation Geometric and Photometric Image Formation, Pinhole Camera, Lenses, Sensors, Colour, Gamma, DCT, Image Coding
4/7 Filtering and Pyramids Linear + Non-Linear Filtering, Correlation, Convolution, Gaussian + Laplacian Pyramids, Sampling and Aliasing Week 2 Notes
Project 1 start
Features and Matching Detection, Correspondence, Edges, Corners, Regions, Patch Matching, SIFT, Shape Context, Learning Features
4/14 Planar Geometry 2D Transforms: Euclidean, Similarity, Affine, Projective, Camera Models: Perspective, Projective, Linear, Viewing planes, Lines and Camera Rotation Week 3 Notes
RANSAC Least Squares 2-view Alignment, Outliers, Robust Line Fitting, RANSAC, Minimal Subsets
4/21 Epipolar Geometry Epipolar Lines, Plane Constraint, Fundamental/Essential Matrix, 8 point algorithm, Triangulation, 2-view SFM Week 4 Notes
Project 2 start
Multiview Alignment and SFM Multiview Alignment, Residuals, Error Function, Structure from Motion, Bundle Adjustment, Pose Estimation, Triangulation
4/26 Project 1 due
4/28 Stereo Stereo matching, local + global, multiview stereo, plane sweep, volumetric, depth map merging, photometric stereo Week 5 Notes
Depth + Flow Depth imaging + fusion, signed distance functions, non-rigid matching, optical flow, Lucas Kanade algorithm PlaneSweep ipynb, LucasKanade ipynb.
Notebooks by Steven Lovegrove, Richard Newcombe
5/5 Linear Classification Visual classification intro, object recognition, instance, category, classification vs detection, linear classification, 2-class, N-class, linear and softmax regression Week 6 Notes
Project 3 start, Part 1 and Part 2
Visual Classification 2 Fundamentals and Pre-Deep Learning Classification, Bayesian classifiers, Gaussian distributions, PCA, LDA, Decision Forests, Visual words, SVMs
5/10 Project 2 due
5/12 Neural Networks Feature extraction, end to end learning, multiple linear layers, activation functions, biological neurons, space warping, universal approximation, convex optimization Week 7 Notes
Slides for Week 7 by Justin Johnson
Backpropagation Chain rule, computational gradients, forward/reverse mode autodiff, upstream/local gradients, flat backprop, modular design, scalar/vector/tensor backprop, matrix multiplication example
Convolutional Networks Convolutional layers, activation maps, dimension mappings, receptive fields, strides, pooling, LeNet5 example
5/19 Advanced CNNs CNN building blocks, dropout, batch norm, factorized convolutions, residual connections, popular architectures: AlexNet, VGG, GoogLeNet, Resnet, MobileNet, SE-Net Project 4 start
Object Detection Motivation + applications, sliding windows, anchor based detection, single-stage and two-stage architectures, evaluation metrics, IoU, precision-recall, mAP, practical tips
5/24 Project 3 due
5/26 Segmentation Dense prediction, semantic, instance, panoptic segmentation, keypoint estimation, fully convolutional nets, atrous, transpose convolution
Single-View Depth, Superres, Colorization Pixel labelling, single-view depth estimation, direct, self-supervision, super-resolution, colorization, image translation
6/2 Deep Learning in 3D Single-view, 2-view, multi-view depth, deep learning with points, meshes, voxels, SDFs, neural scene representation and rendering Week 10 Notes
Image Generation and GANs Loss functions: L2, VGG, adversarial, texture synthesis, style transfer, generative adversarial nets, image generation, conditional GANs, image translation, pix2pix
6/7 6/10 Project 4 due