![Title image: The Ancient Secrets of Computer Vision](images/title.jpg) ## Course Information ## This class is a general introduction to computer vision. It covers standard techniques in image processing like filtering, edge detection, stereo, flow, etc. (old-school vision), as well as newer, machine-learning based computer vision. Course will be offered in a variety of modalities: - In-person: [CSE2 G20](https://www.washington.edu/classroom/CSE2+G20) - Remotely: [Zoom](https://washington.zoom.us/j/95085664968) - Asynchronously: See below for lecture recordings Participate in whatever way best suits your needs this quarter! **Please do not come to in-person class if you are sick or have reason to suspect you may be sick.** ### Instructor ### Joseph Redmon - Email: pjreddie@cs.washington.edu ### TAs ### Varich Boonsanong: varicb@cs.washington.edu \ Minh Hoang: minh257@cs.washington.edu \ Zucks Zixuan Liu: zucksliu@cs.washington.edu \ Shane Lyu: alrope@cs.washington.edu \ Hamsa Shankar: hamsas@cs.washington.edu \ Adam Wang: adamwang@cs.washington.edu \ Guangyin Ye: gy020306@cs.washington.edu ### Office Hours ### - Monday - Varich 5:30-6:30pm at https://washington.zoom.us/j/97281584812 - Tuesday - Minh 11:30am-12:30pm at CSE 2 - 276 - Wednesday - Hamsa 2:00-3:00pm at https://washington.zoom.us/j/99049939766 - Zucks 3:30-4:30pm at https://washington.zoom.us/j/7826771790 - Thursday - Shane 1:30-2:30pm at https://washington.zoom.us/j/5673599975 - Joe 4:00-5:00pm at https://washington.zoom.us/j/3362756951 - Friday - Guangyin 11:30am-12:30pm at https://washington.zoom.us/j/2845110562 - Adam 1:00-2:00pm at CSE2 - 153 ### Resources ### - Ed Discussion Board: https://edstem.org/us/courses/21400/discussion/ - Canvas: https://canvas.uw.edu/courses/1612146 - Zoom: https://washington.zoom.us/j/95085664968 Slides are a mishmash of lots of other people's work. Special thanks to: Rob Fergus, Linda Shapiro, Harvey Rhody, Rick Szeliski, Ali Farhadi, Robert Collins. Lectures 8 and 9 on Flow, 3d, and stereo are by [Connor Schenck](https://homes.cs.washington.edu/~schenckc/). All of the slides, videos, and homeworks are free to use, modify, redistribute as you like without permission. Just make your own copy of the slides on Google Docs, don't ask to modify mine! ## Homeworks ## The class has 6 homeworks where you will build out a computer vision library in C. We cover basic image manipulations, filtering, features, stitching, optical flow, machine learning, and convolutional neural networks. Most of the homeworks will use [this repository](https://github.com/pjreddie/uwimg/). The individual homeworks can be found in the `src/` folder. - [Homework 0: Fun with Color!](https://github.com/pjreddie/uwimg/tree/main/src/hw0), Due January 12 - [Homework 1: Resizing](https://github.com/pjreddie/uwimg/tree/main/src/hw1), Due January 19 - [Homework 2: Filtering and Convolutions](https://github.com/pjreddie/uwimg/tree/main/src/hw2), Due January 26 - [Homework 3: Panoramas!](https://github.com/pjreddie/uwimg/tree/main/src/hw3), Due February 2 - [Homework 4: Optical Flow](https://github.com/pjreddie/uwimg/tree/main/src/hw4), Due February 9 - [Homework 5: Neural Networks and Machine Learning](https://github.com/pjreddie/uwimg/tree/main/src/hw5), Due February 16 **Note:** due date subject to change if we haven't covered relevant material in time for the assignment. **COVID Policy:** If you get COVID don't worry about doing your homework, rest, recover, do what you need to do to get better. If you feel like doing computer vision while sick go for it but also know you can take some time off. Once you are well please reach out to the course staff and we can figure out how to get you back on track with assingments and any missed classes. You will not be penalized for turning in assingments late due to COVID (or if you're having trouble getting caught back up afterward). **Late Policy:** - One “free” late day per assignment - If you still cannot turn in on time: - email me to tell me what day you will turn in - turn it in by that day ## Final Project ## There is a final project worth 20% of the final grade. Pick any area of computer vision that interests you and pursue some independent work in that area. Each project should have a significant technical component, software implementation, or large-scale study. Projects can focus on developing new techniques or tools in computer vision or applying existing tools to a new domain. If you don't have an idea you can train a classifier on birds and compete in the Kaggle competition posted on the Ed discussion board. ## Lectures ## ### Week 1: Image Basics ### This week we cover the basics of computer vision. There's an introduction to the three levels of vision, **low-level** vision mostly concerns the pixels or groups of nearby pixels, **mid-level** vision starts to connect images to each other and the real world, and **high-level** vision connects images to semantics and meaning. There's background information on the human visual system, color, light, what an image actually is, and how it's stored in a computer. All fun stuff! Once you've learned the basics you should be ready for [Homework 0](https://github.com/pjreddie/uwimg/tree/main/src/hw0), which is mostly an introduction to the codebase we'll be using for the assignments. #### Lecture 1: Introduction - [Slides](https://docs.google.com/presentation/d/1vZuncM3rrJZakza94UU8FvpEd9Ph-9E9I3Hj4Xkr91U/edit?usp=sharing) - [Video](https://washington.zoom.us/rec/play/hDuuixsCwyZlTnx-Mll8dxdSTtztP4oNx6zEfM-eDGY2sPP-59IVt0789ZlvKb3nbEZuQ7WkRJD91_Op.QcrKudEirz33VF-G) #### Lecture 2: Human Vision, Color Spaces, Transforms #### - [Slides](https://docs.google.com/presentation/d/1kTvnMCG7qZ8eoA4NF79Q-Qyirt-ZMq6RNv4xAkQh2bY/edit?usp=sharing) - [Video](https://washington.zoom.us/rec/play/9vA0O2pLTU15OocftCbgaz6w08fVDLdjrvruvK2BRB1pVZ9G-X2GVox6EAkJH4OM8UfmzPKWBcpXyNPK.Nb9uVBOBp_aKyWPq) -------- ### Week 2: Image Transformations ### In week 2 we start to dive into low-level vision and image processing. You learn how to manipulate images and perform operations like resizing, sharpening, smoothing, and more. You'll apply this knowledge as you get started on [Homework 1](https://github.com/pjreddie/uwimg/tree/main/src/hw1). #### Lecture 3: Image Coordinates, Resizing - [Slides](https://docs.google.com/presentation/d/1imYQe7kDCahP2YO69L7FBiwqcEc4LgVEDXW7pDMFhOs/edit?usp=sharing) - [Video](https://washington.zoom.us/rec/play/t8KIgf2fA963fNxAAYB1Vq7sAcPvIqRd0UjRXrIDu6i548NOzHKvXYTB-3Lc_AYwxUxCfvRJJq8uo-Fs.eD5vbOFaCZANUxDj) #### Lecture 4: Resizing, Filters, Convolutions - [Slides](https://docs.google.com/presentation/d/1owZEUBbFp-Iz5kLNo-KvppjtzJGF91wEzfviUphLWqs/edit?usp=sharing) - [Video](https://washington.zoom.us/rec/play/2qM_0IJr4Bvf0BOwuFO47a515u6T_ttK0t8Z0QclZAgSu7bUPGS58Y5K-7REBE_YKpNGuRuZbwEQlNam.C86GkhumiPtv6yhf) -------- ### Week 3: Edges and Features Time to put those convolutions to use! For week 3 we delve in to what makes images interesting, what makes them unique, how to find correspondences between images, and how to fit models with a large number of outliers in the data. #### Lecture 5: Edges and Features - [Slides](https://docs.google.com/presentation/d/1WqcYCxTnYzbPmhAsi5N7heb1GACzYd5Rt441Z36cp28/edit?usp=sharing) - [Video](https://washington.zoom.us/rec/play/4cv1ZCRrARmL_TEa22RNgpXLZ6ACQm7QcxbFfJ_Z4tEq53b8wxiEA3qUapJwE8T9qQY5rHnw57Gcxe7P.-fcyDcV-v1PbbzNj) #### Lecture 6: Harris, Matching, RANSAC - [Slides](https://docs.google.com/presentation/d/1_HFh3SdmdyZ_j-sFS4Tw17DmhKfjmYZqvSp7TmfdD_M/edit?usp=sharing) - [Video](https://washington.zoom.us/rec/play/6Hc5H14Zem9gEpYqStTR4Ns3wt5suvrobfCHps2TG3Sl_vH0ae2dBTxplLJC5WdFsI5WKB_Qv512qfiu.DNQTqSYHQHUAKyU7) ------- ### Week 4: Features and Flow #### Lecture 7: Matching, RANSAC, HOG, and SIFT - [Slides](https://docs.google.com/presentation/d/1Ti2o4HPX8xEIYpfgEm6W3EvGc2bWjiz0ZuyAZfTAuD4/edit?usp=sharing) - [Video](https://washington.zoom.us/rec/play/9By2tXAYKvGYC0w99zbrWYbAhjA5EcncNYCbkUX69owC_8TKcA449k4I_6QOt5v9xMVGtPiGGLnOMbvR.FnewimRA4oJKBEX7) #### Tutorial: Debugging With GDB and Valgrind - [Video](https://washington.zoom.us/rec/play/4--JlNyNkBJrGSJwSn2RRvnp0ueVtMfU3k9PA3lm9_oUy6ypv0wR8w6_6f0Mr0Q82_Ht7V42ZWOZ7gu9.77U9umhN53K6q-LY) #### Lecture 8: Optical Flow - [Slides](https://docs.google.com/presentation/d/1Z6xyKI5SWJ3Qs4aWcbP_bOlj34VW40WIHfhf3b4EN10/edit?usp=sharing) - [Video](https://washington.zoom.us/rec/play/p08ofOsl4Z9Sev8CuXQzV0PfhPhC_BiGfuxm62qMeDoC9VduCTxv8fMZNvxZ_brgman3zyUJRs9iJ_GE.7a7WiGM8Bmh8rycv) --------- ### Week 5: Depth and ML Review #### Lecture 9: 3D, Depth, and Stereo - [Slides](https://docs.google.com/presentation/d/1e8RSHMYzrRtA3s4wfb7xXoZ1CTH8lKXFlNhPFQVD9KI/edit?usp=sharing) - [Video](https://washington.zoom.us/rec/play/CGtKFog09KHRM9w5ebFeUGBbaUNE2rGVjAoxrqKcY-wZEaaUB2Np3D2sUzI_wFFTukbeR5uT4vMCPSg_.fDEqx3jDGcIeyZEa) #### Lecture 10: Machine Learning for Computer Vision - [Slides](https://docs.google.com/presentation/d/1vYhq30b9BEDzzl3rwP8uY6VF1x4jmtzJaxQGK0P2XAI/edit?usp=sharing) - [Video](https://washington.zoom.us/rec/play/9ShN5Po3RkIy-Diap5c7eUjqoFSMWBbiIl5FR-Miia8QI4UUL5FtfSiw2Y8l3eWdrE0lBS4vpuvbn_yb.7DkH-FtthjR3uhgc) -------- ### Week 6: Machine Learning and Neural Networks #### Lecture 11: More Machine Learning for Computer Vision - [Slides](https://docs.google.com/presentation/d/1c5qjBEQVhcwejWY6I88BwHGOdZiPvkDD6su1bbqqd2c/edit?usp=sharing) - [Video](https://washington.zoom.us/rec/play/dt-gYXP3G25h6WDvOjrwQYzgQUV5RahsTY6VlJVLeV1XrrCbeNKUK1G5NkN0uVfBxJSMdtsfG-aoG9xe.eNAqUeJS909lLq-K) #### Lecture 12: Neural Networks - [Slides](https://docs.google.com/presentation/d/1HQOET7oM3fzQba2KAcw20yEtqe_3ZLE9VTtfoPBgb8A/edit?usp=sharing) - [Video](https://washington.zoom.us/rec/play/H1Ngn7LXY_tTY9Qell7ovuL85miUZyF1bTRtlKrxHx1mlrsRqGWuoPNYT84OzBaNivLK2eZQ13SaABr-.hexHbYjp3rldcpnI) -------- ### Week 7: Convolutional Neural Networks #### Lecture 13: Convolutional Neural Networks - [Slides](https://docs.google.com/presentation/d/1szC_xsXx4kBtVaM-6yVjdJRPc2SGgOq3rcpzKM06-UM/edit?usp=sharing) - [Video](https://washington.zoom.us/rec/play/R76csVdMob8TtOse35KxbLLpXSPA9-qtVJrMgpNwZcvuaYRZk-5UQsEx9Fe5EuJE94qnXapVKAOIQjb4.79bBAs-4FvcqW_cZ) #### Tutorial 1: Neural Networks in PyTorch - [iPynb](https://colab.research.google.com/drive/1JR9vh7NpFB8f95cixhT0dQ9ky4xXY8Hi?usp=sharing) - [Video](https://washington.zoom.us/rec/play/j6FI5Y1h-rdMUXd9Gc9icgyCT_kq_MtSi12fj6WY7RxwzcGBEZn6ZTmOHYxPqkGwu1M2mDaCheScQqZZ.39xwF0C4jQraQlcR) #### Lecture 14: Network Architectures - [Slides](https://docs.google.com/presentation/d/1IIJ6uxpwT3NdOzO2L-KDVsSNOfQM0uBcf4F0WFy-jVc) - [Video](https://washington.zoom.us/rec/play/iM6pnu3PO93pakgW7K1yenCSOa1cBUR7kvmL_Eu2zTA-qibrhBdU8k_ysitho6J21ut6lmxXu9-_LxuY.cIiAHq59SxSxqP3W) - [iPynb](https://colab.research.google.com/drive/1DA4M7mINp3XqYYEpN-ec2KrJeZ_Ziddp#scrollTo=xVsagkSRifjk&line=6&uniqifier=1) --------- ### Week 8 #### Lecture 15: More Network Architecture Stuff - [Slides](https://docs.google.com/presentation/d/1XFr96QGJFHoRr0HyydnrLntulH7vNlIQd_SHm88yLNk/edit?usp=sharing) - [Video](https://washington.zoom.us/rec/play/eQMD03Qo94YK9gEGl6I2UdYoUdldEIDyzoOkV6isWAInADPC3rTchdVQ5poJcVgbmWgF9DdgS5NPpI_3.O5bNEimPIDzZx-Wa) #### Lecture 16: Semantic Segmentation - [Slides](https://docs.google.com/presentation/d/1XFr96QGJFHoRr0HyydnrLntulH7vNlIQd_SHm88yLNk/edit?usp=sharing) - [Video](https://washington.zoom.us/rec/play/NbyT7A5nGsSFh5oP2eZQB-wG_hpOMGuAns2_XUW9XlYdM1_9LWGC_-PDaDoMkNyU8hngc1heQ40BYxUj.PX5PqaU7aixi-tn4) #### Tutorial: CNNs in PyTorch - [iPynb](https://colab.research.google.com/github/pjreddie/uwimg/blob/main/tutorial2-cnns-in-pytorch.ipynb) - [Video](https://washington.zoom.us/rec/play/zXB50Z68TqyvV4xsyNFJjLpEUfzALUBiCQHz_Uqnn0lQthKNkFqoAO43AGUlnrpVKGEmBmxJAfJbWhZO.-b4eKkjgnC3RMZ0k) ---------- ### Week 9 #### Lecture 16: Object Detection - [Slides](https://docs.google.com/presentation/d/1CLVryBPNddEn8TwM2ngy1bHnlJC4t33OhERzVixFGj4/edit?usp=sharing) - [Video](https://washington.zoom.us/rec/play/k5J4eHOMVbb9bj-45AlEszUWKlNyNaXg0-jXZuyae8KhQT0sqPeMwYL-gj2D0tKUZ-dLQRQiK7GzRvk.iWsC1F_e3XrFHivm) #### Lecture 18: Vision and Language - [Slides](https://docs.google.com/presentation/d/1UJOl-C3qDUMw7ftgKJh7GEELxuNh0KRAtQpzhg3PWrc/edit?usp=sharing) - [Video](https://washington.zoom.us/rec/play/SOnGpP14KPFMfLQHwz1HbU6BS4ofRFmXcWduyJCTywmoPMp4Yk-uDYRiiXRdz8KtI0CMkENp8fGwF38s._Ikh9291fqoL156S) #### Tutorial: Transfer Learning in PyTorch - [iPynb - ImageNet and Transfer Learning](https://colab.research.google.com/drive/1EBz4feoaUvz-o_yeMI27LEQBkvrXNc_4?usp=sharing) - [iPynb - Transfer Learning to Birds](https://colab.research.google.com/drive/1kHo8VT-onDxbtS3FM77VImG35h_K_Lav?usp=sharing) ----------- ### Week 10 #### Lecture 19: Generative Adversarial Networks - [Slides](https://docs.google.com/presentation/d/1NgicaHQhluKQ0r39U4IWo8AtOBJUAQ2aBGCDq_DpgkA/edit?usp=sharing) - [Video](https://washington.zoom.us/rec/play/fTcXcOFwc3w8hvLpONvlDKoRBHEIYxW2HJz_yFMc_V4cTiBEeYDzkBx-QQ6tu5AaWjGOHQLPZps5QVCg.LlTBF_s2XxNwGSYx) #### Lecture 20: Diffusion Models - [Slides](https://docs.google.com/presentation/d/1vm-nQ7igsO_GerOmGjAbTDP58MvH_sgf04Q7bxxvTKM/edit?usp=sharing) - [Video](https://washington.zoom.us/rec/play/TfgJzcpf6w4fhaV1bywIRXNfKneFpdDXcurEToOcU-x9viu8UoF3MsFi7elnZRGhFmcCtgEN60xNWhld.kC7rSuxiuJr8I8J4)