![Title image: The Ancient Secrets of Computer Vision](images/title.jpg) ## Course Information ## This class is a general introduction to computer vision. It covers standard techniques in image processing like filtering, edge detection, stereo, flow, etc. (old-school vision), as well as newer, machine-learning based computer vision. Course will be offered in a variety of modalities: - In-person: [CSE2 G20](https://www.google.com/maps?q=47.652915,-122.304806+(CSE2)&z=18) **temporarily on hold due to high cases** - Remotely: [Zoom](https://washington.zoom.us/j/92412405037) - Asynchronously: See below for lecture recordings Participate in whatever way best suits your needs this quarter! **Please do not come to in-person class if you are sick or have reason to suspect you may be sick.** ### Instructor ### Joseph Redmon - Email: pjreddie@cs.washington.edu ### TAs ### - Rehaan Minh Bhimani - bhimar@cs.washington.edu - Greg Guo - grgrggtr@cs.washington.edu - Zucks Zixuan Liu - zucksliu@cs.washington.edu - Mino Nakura - nakuram@cs.washington.edu - Mark Theeranantachai - stheera@cs.washington.edu ### Office Hours ### - Monday - 12:00-1:00pm with Mino: https://washington.zoom.us/my/minonakura - Tuesday - 1:30-2:30pm with Rehaan: https://washington.zoom.us/j/94507887953 - Wednesday - 8:30-9:30pm with Mark: https://washington.zoom.us/j/96962845976 - Thursday - 11:00am-12:00pm with Zucks: https://washington.zoom.us/j/99378370744 - 1:00-2:00pm with Joe: https://washington.zoom.us/j/3362756951 - Friday - 2:30-3:30pm with Greg: https://washington.zoom.us/j/99306313827 ### Resources ### - Ed Discussion Board: https://edstem.org/us/courses/17033/discussion/ - Canvas: https://canvas.uw.edu/courses/1515312 - Zoom: https://washington.zoom.us/j/92412405037 Slides are a mishmash of lots of other people's work. Special thanks to: Rob Fergus, Linda Shapiro, Harvey Rhody, Rick Szeliski, Ali Farhadi, Robert Collins. Lectures 8 and 9 on Flow, 3d, and stereo are by [Connor Schenck](https://homes.cs.washington.edu/~schenckc/). All of the slides, videos, and homeworks are free to use, modify, redistribute as you like without permission. Just make your own copy of the slides on Google Docs, don't ask to modify mine! ## Homeworks ## The class has 6 homeworks where you will build out a computer vision library in C. We cover basic image manipulations, filtering, features, stitching, optical flow, machine learning, and convolutional neural networks. Most of the homeworks will use [this repository](https://github.com/pjreddie/uwimg/). The individual homeworks can be found in the `src/` folder. - [Homework 0: Fun with Color!](https://github.com/pjreddie/uwimg/tree/main/src/hw0), Due January 18th - [Homework 1: Resizing](https://github.com/pjreddie/uwimg/tree/main/src/hw1), Due January 20th - [Homework 2: Filtering and Convolutions](https://github.com/pjreddie/uwimg/tree/main/src/hw2), Due January 27th - [Homework 3: Panoramas!](https://github.com/pjreddie/uwimg/tree/main/src/hw3), Due February 3rd - [Homework 4: Optical Flow](https://github.com/pjreddie/uwimg/tree/main/src/hw4), Due February 10th - [Homework 5: Neural Networks and Machine Learning](https://github.com/pjreddie/uwimg/tree/main/src/hw5), Due February 17th **Note:** due date subject to change if we haven't covered relevant material in time for the assignment. You have 8 late day to use throughout the quarter. Each day late counts as one late day. Any number can be used on any assingment. After you have used your late days late assingments will be penalized up to 10% per day late. **COVID Policy:** If you get COVID don't worry about doing your homework, rest, recover, do what you need to do to get better. If you feel like doing computer vision while sick go for it but also know you can take some time off. Once you are well please reach out to the course staff and we can figure out how to get you back on track with assingments and any missed classes. You will not be penalized for turning in assingments late due to COVID (or if you're having trouble getting caught back up afterward). ## Final Project ## There is a final project worth 20% of the final grade. Pick any area of computer vision that interests you and pursue some independent work in that area. Each project should have a significant technical component, software implementation, or large-scale study. Projects can focus on developing new techniques or tools in computer vision or applying existing tools to a new domain. If you don't have an idea you can train a classifier on birds and compete in the Kaggle competition posted on the Ed discussion board. ## Lectures ## ### Week 1: Image Basics ### This week we cover the basics of computer vision. There's an introduction to the three levels of vision, **low-level** vision mostly concerns the pixels or groups of nearby pixels, **mid-level** vision starts to connect images to each other and the real world, and **high-level** vision connects images to semantics and meaning. There's background information on the human visual system, color, light, what an image actually is, and how it's stored in a computer. All fun stuff! Once you've learned the basics you should be ready for [Homework 0](https://github.com/pjreddie/uwimg/tree/main/src/hw0), which is mostly an introduction to the codebase we'll be using for the assignments. #### Lecture 1: Introduction - [Slides](https://docs.google.com/presentation/d/1VqTJEVC0gwxfY5TbINgBKyWqAmdxLYxgj_yIiphJj34/edit?usp=sharing) - [Video](https://washington.zoom.us/rec/play/tYNUctCKVo1LVfWMPPg4mDtdXW-4MGsbW9lTM1UzWVKZdxTmf-T7pGT68WHQBfAMOuBEXs1k5YA6BHi2.Nk__SZIJB-JVkbZ9) #### Lecture 2: Human Vision, Color Spaces, Transforms #### - [Slides](https://docs.google.com/presentation/d/1kTvnMCG7qZ8eoA4NF79Q-Qyirt-ZMq6RNv4xAkQh2bY/edit?usp=sharing) - [Video](https://washington.zoom.us/rec/play/EAK1XZFflNoK5leN9eW3Lg41XCOyF1Es9liatTaR3yGVvnAcaZgg7UzKGhGV9q8QgNM5WzX-SNYSqZ1Z.sZRvu_ELCbXVOspx) -------- ### Week 2: Image Transformations ### In week 2 we start to dive into low-level vision and image processing. You learn how to manipulate images and perform operations like resizing, sharpening, smoothing, and more. You'll apply this knowledge as you get started on [Homework 1](https://github.com/pjreddie/uwimg/tree/main/src/hw1). #### Lecture 3: Image Coordinates, Resizing - [Slides](https://docs.google.com/presentation/d/1imYQe7kDCahP2YO69L7FBiwqcEc4LgVEDXW7pDMFhOs/edit?usp=sharing) - [Video](https://washington.zoom.us/rec/play/AOxtNY1Tbbtdw0REhg6DqDroR_cPFTgYWyJuqY2ENbxcNUHj1DH8EaNpmqfgHoWfm8r4bvybvK1qBB_3.p9MAoLNCTt12kdVE) #### Lecture 4: Resizing, Filters, Convolutions - [Slides](https://docs.google.com/presentation/d/1owZEUBbFp-Iz5kLNo-KvppjtzJGF91wEzfviUphLWqs/edit?usp=sharing) - [Video](https://washington.zoom.us/rec/play/KnElM8uyNyL4ykuB_A6o5uGlDuNtAHl45ze2X2uVwS_qXsOVtcQL0sSoOw4vVQ3u9t5qiOpj6jy0pgSo.HhK-UhXh3VmTAhpe) -------- ### Week 3: Edges and Features Time to put those convolutions to use! For week 3 we delve in to what makes images interesting, what makes them unique, how to find correspondences between images, and how to fit models with a large number of outliers in the data. #### Lecture 5: Edges and Features - [Slides](https://docs.google.com/presentation/d/1WqcYCxTnYzbPmhAsi5N7heb1GACzYd5Rt441Z36cp28/edit?usp=sharing) - [Video](https://washington.zoom.us/rec/play/_x1at-Ws_Zdijp4yvJPeunXgdv_VQkH366heJ_KgNlw0dvUnFOr-DVSReKFwY9hC_oPRn_NpFCaYSALQ.Pk0smcV2EUEfeNfK) #### Lecture 6: Harris, Matching, RANSAC - [Slides](https://docs.google.com/presentation/d/1_HFh3SdmdyZ_j-sFS4Tw17DmhKfjmYZqvSp7TmfdD_M/edit?usp=sharing) - [Video](https://washington.zoom.us/rec/play/Mik0ko_UQOQaQJLGfTzMFFVEY0bFSTr_HkNCp2GnKvZXaiyHExHLkUY4CxABeS_L7oufT_ptU2bBnFtN.f2Pon0jUnM0Lt24G) #### Tutorial: Debugging Your Homework With GDB and Valgrind - [Video](https://washington.zoom.us/rec/play/mY6OeGKzpfR4T5R0myACNUlT-Bdp64pwKn3oo2B0jUZUjGfMLFXg-sW1Vksvxn_enqgW8C8rYAFl6xRM.YW2xTku3xciCIPrU) ------- ### Week 4: Features and Flow #### Lecture 7: Matching, RANSAC, HOG, and SIFT - [Slides](https://docs.google.com/presentation/d/1Ti2o4HPX8xEIYpfgEm6W3EvGc2bWjiz0ZuyAZfTAuD4/edit?usp=sharing) - [Video](https://washington.zoom.us/rec/play/0kgikeVq4yrh4BgwFElj8UfOUnauKUadXUN-9Ka5QCN-TvMXQSdXJIN6z1-vjWOSEMeJPDxQqlDT5Q.OHHmwFImVQ9IcQhg) #### Lecture 8: Optical Flow - [Slides](https://docs.google.com/presentation/d/1Z6xyKI5SWJ3Qs4aWcbP_bOlj34VW40WIHfhf3b4EN10/edit?usp=sharing) - [Video](https://washington.zoom.us/rec/play/i53DK5D8fDPcALMRjkxYpHscmHNT3rBn6WTwEc1NWp3gx-M1VT4I3IwtzoeXNRlrRawVc9oGOHF6D3i_.JJA_GsMIPgmkbC5a) --------- ### Week 5: Depth and ML Review #### Lecture 9: 3D, Depth, and Stereo - [Slides](https://docs.google.com/presentation/d/1e8RSHMYzrRtA3s4wfb7xXoZ1CTH8lKXFlNhPFQVD9KI/edit?usp=sharing) - [Video](https://washington.zoom.us/rec/play/oEFjxxjvD4G0CmpgbecZiIOmUlfc1z9CMhCBTf0E3n82C2FM7qfEHcMYJlAK35DpgTHTvnlaAm8COxjO.UndcIUpbwedZtCjG) #### Lecture 9 Part 2: - [Video](https://washington.zoom.us/rec/play/zd8NDZBIA0sclAMecbxEBhppp3zaTf5EDN5MV6KQefNq_z-yo4JvFsMfV0tqq6r7FB4sPrp1D1OVbKEI.-biEyTFAvoc2cMxX) #### PyTorch Tutorial: - [iPynb](https://colab.research.google.com/drive/1CYD8uaxc_J5xmkJWT3cKnaF4cnUASWJP?usp=sharing) - [Video](https://washington.zoom.us/rec/play/9x0_6aaPZnN3AGYMCmZOjbyrZk1_ES1ccntZ69EGfVYVhyhbX_YhaIvw4qosAuMBejI-dFoD_WwbriKZ.BAfaiSt7IRn7o0a5) -------- ### Week 6: Machine Learning and Neural Networks #### Lecture 10: Machine Learning for Computer Vision - [Slides](https://docs.google.com/presentation/d/1vYhq30b9BEDzzl3rwP8uY6VF1x4jmtzJaxQGK0P2XAI/edit?usp=sharing) - [Video](https://www.youtube.com/watch?v=AIL5PuvRAPI) - Sorry, I forgot to record the zoom meeting today! Here's the video from an earlier version of the class, we only made it to the end of Viola-Jones face detection (minute 50 or so in the recording) #### Lecture 11: More Machine Learning for Computer Vision - [Slides](https://docs.google.com/presentation/d/1c5qjBEQVhcwejWY6I88BwHGOdZiPvkDD6su1bbqqd2c/edit?usp=sharing) - [Video](https://washington.zoom.us/rec/play/zmcijy1yow0wW0cZaouBuI6PoXg-s3uskeUJsoCnQlAzJ8bCuUDnsB_qANNfi7gf87CLpXA1Dw0l3MLn.fBBKt0wlogKiJQN6) -------- ### Week 7: Convolutional Neural Networks #### Lecture 12: Neural Networks - [Slides](https://docs.google.com/presentation/d/1HQOET7oM3fzQba2KAcw20yEtqe_3ZLE9VTtfoPBgb8A/edit?usp=sharing) - [Video](https://washington.zoom.us/rec/play/LH4NR23EvQaamgEa8fxhS_q0pC7nrmMSmzASGQ6wvX6MaVyNtLRYh-UfmdMM20g-VEHR4VKoi6uxNzEx._cfc3CvwU94Ll3hv) #### Lecture 13: Convolutional Neural Networks - [Slides](https://docs.google.com/presentation/d/1szC_xsXx4kBtVaM-6yVjdJRPc2SGgOq3rcpzKM06-UM/edit?usp=sharing) - [Video](https://washington.zoom.us/rec/play/c1bPPlyHomN65rTozTqCDIbL-5nT2-SCJA8RwU_OhZrfjqwPhhvLIURF1PBQRfUo5iQh4MJQrCI2OWGX.0zRqsWEiEYBVCjmj) --------- ### Week 8 #### Lecture 14: Network Architectures - [Slides](https://docs.google.com/presentation/d/1IIJ6uxpwT3NdOzO2L-KDVsSNOfQM0uBcf4F0WFy-jVc/edit?usp=sharing) - [Video](https://washington.zoom.us/rec/play/BFDRb7Ehn490VSXx36fmgNc_nU4neIRfTlgnc6UEj8yZAJQSWix-zyWYsTMliitLmjdq8dT7Tn37geDc.SXN1uaKjYTfdQ7CX) #### Lecture 15: Semantic Segmentation - [Slides](https://docs.google.com/presentation/d/1XFr96QGJFHoRr0HyydnrLntulH7vNlIQd_SHm88yLNk/edit?usp=sharing) - [Video](https://washington.zoom.us/rec/play/B_kiqrDiFMLYibLXDpj7PpshlWSAkcSJh9BXbPRRQN_fJPj6RHB8AYWV2xhYRd9RAgMJJgTMhdmYdFJy.Z1b6Sy-Pd28SvJ-G) #### Tutorial: Neural Networks in PyTorch - [iPynb](https://colab.research.google.com/drive/1CYD8uaxc_J5xmkJWT3cKnaF4cnUASWJP?usp=sharing#scrollTo=mKmBp4tMaYro) - [Video](https://washington.zoom.us/rec/share/G0ZHZkp1rp-2953l4hDHWJJIjXf68Us9DqfWIJxeNV15Hbrlh9OrhdIBuNJ6ZXJH.nVz0-Mc8FXhnuzco?startTime=1645560349000) ---------- ### Week 9 #### Lecture 16: Object Detection - [Slides](https://docs.google.com/presentation/d/1CLVryBPNddEn8TwM2ngy1bHnlJC4t33OhERzVixFGj4/edit?usp=sharing) - [Video](https://washington.zoom.us/rec/play/5LqHYn3bw6G4F6rrAvVkyoTXjAMSWtZlfy3j3B-lrXz86KOT0Yx5FtS_KVeiWcHYLw7bZOlBqltG6gzp.Ug6A9yqvMVlhocUb) #### Lecture 17: Instance Segmentation - [Slides](https://docs.google.com/presentation/d/1QmwqTeQZuvoXo_pIYbLbS2dgiNsKWNn1eZXLxFnkZo0/edit?usp=sharing) - [Video](https://washington.zoom.us/rec/play/Ay3_RRNDHzqv2VMwWe24r5023JJbKviJvyR5HOahvR8trECC-yXe_9mto32E0E8zd0lMKSCImRLFjggf.Arv3shVWNX-_yESE) #### Tutorial: Convolutional Neural Networks in PyTorch - [iPynb](https://colab.research.google.com/drive/1k4SpEurwjaG3a4AAIQraDFg_7Qa1Cat5#scrollTo=rQ1-ePKDcQod) - [Video](https://washington.zoom.us/rec/play/qS9-tlvd1H7EY7N04B4UiE3Oz5ZBi2CbwlesO4Kyp9UhCc9NtAz9CuJDY3T-vWdtM2H6_S5Pbkm4l1BH.UEFB-QLCBoJ5RpBY) #### Tutorial: Transfer Learning in PyTorch - [iPynb](https://colab.research.google.com/drive/1EBz4feoaUvz-o_yeMI27LEQBkvrXNc_4?usp=sharing) - [Video (starts at 1:03:47)](https://washington.zoom.us/rec/play/Ay3_RRNDHzqv2VMwWe24r5023JJbKviJvyR5HOahvR8trECC-yXe_9mto32E0E8zd0lMKSCImRLFjggf.Arv3shVWNX-_yESE) ----------- ### Week 10 #### Lecture 19: Vision and Language - [Slides](https://docs.google.com/presentation/d/1UJOl-C3qDUMw7ftgKJh7GEELxuNh0KRAtQpzhg3PWrc/edit?usp=sharing) #### Lecture 20: Generative Adversarial Networks - [Slides](https://docs.google.com/presentation/d/1NgicaHQhluKQ0r39U4IWo8AtOBJUAQ2aBGCDq_DpgkA/edit?usp=sharing) ### [Course Evaluation](https://uw.iasystem.org/survey/253751)