Monday / Wednesday / Friday 9:30-10:20am, CSE2 G20

Instructor: Byron Boots
email: bbootscs.washington.edu
office hours: 10:30-11:30am Monday and Wednesday (after class), CSE2 131

TA Office Hours:

• Romain Camilleri: Thursday 1:00pm-3:00pm, CSE2 150
• Anirudh Canumalla: Tuesday 11:00am-1:00pm, CSE2 121
• Leo Du: Thursday 4:00pm-6:00pm, CSE2 152
• Ivan Montero: Monday 1:30pm-3:30pm, CSE2 131
• Jifan Zhang: Wednesday 5:30pm-7:30pm, CSE2 131
• Sujie Zhou: Friday 3:00pm-5:00pm, CSE2 151

Contact: cse446-staff@cs.washington.edu PLEASE COMMUNICATE TO THE INSTUCTOR AND TAS ONLY THROUGH THIS EMAIL (unless there is a reason for privacy).

Announcements and Resources will be posted via Canvas. Discussions and questions related to the material and assignemnts will take place on Piazza.

Course Description: This course will introduce the fundamental concepts and algorithms that enable computational artifacts to modify and improve their performance through experience. We will cover a variety of topics, including decision trees, logistic regression, support vector machines, ensemble methods, Bayesian methods, neural networks, clustering, and dimensionality reduction.

Objectives: There are four objectives for the course:

• To provide a broad survey of approaches and techniques in machine learning
• To develop a deeper understanding of several major topics in machine learning
• To develop programming skills that will help you to build intelligent, adaptive artifacts
• To develop the basic skills necessary to pursue research in ML

Lectures: Lectures are meant to summarize the readings and stress the important points. You are expected to come to class having already critically read any assigned material. Your active participation in class is crucial in making the course successful. I completely expect to be interrupted throughout a lecture with questions and maybe even your deep insights into the material. This is less about my teaching than about your learning. My role is merely to assist you in the process of learning more about the area.

Prerequesits: Students entering the class should be comfortable with programming in python and should have a pre-existing working knowledge of probability and statistics (MATH 394, STAT390, STAT 391, or CSE 312), data structures and algorithms (CSE 332). Additionally, students will find linear algebra (MATH 308) and vector calculus (MATH 324) to be very useful. Students who are weak in these areas should either: take the course at a later date, when better prepared, or expect to put in substantially more effort to catch up.

Main Readings From (relevant chapters are available for class members on Canvas):

• Machine Learning -- Tom Mitchell
• Learning from Data -- Yaser S. Abu-Mostafa, Malik Magdon-Ismail, and Hsuan-Tien Lin
• Machine Learning -- Peter Flach

Additional Resources: Throughout the course, you may find it useful to consult the following resources:

• Linear Algebra Review & Reference -- Zico Kolter
• Dive into Deep Learning -- Aston Zhang, Zack Lipton, Mu Li, ALex Smola
• Deep Learning -- Ian Goodfellow, Yoshua Bengio, and Aaron Courville
• Mathematics for Machine Learning -- Marc Deisenroth, A. Aldo Faisal, and Cheng Soon Ong
• An Introduction to Statistical Learning -- Gareth James, Daniela Witten, Trevor Hastie, and Robert Tibshirani
• A Course in Machine Learning -- Hal Daumé III
• Machine Learning Lecture Notes -- Andrew Ng

For a more advanced treatment of machine learning topics, I recommend:

• The Elements of Statistical Learning -- Trevor Hastie, Robert Tibshirani, and Jerome Friedman
• Information Theory, Inference, and Learning Algorithms -- David MacKay
• Pattern Recognition and Machine Learning -- Christopher Bishop
• Probabilistic Graphical Models -- Daphne Koller and Nir Friedman
• Convex Optimization -- Stephen Boyd and Lieven Vandenberghe

Software Resources:

• Python -- Python is the tool of choice for many machine learning practitioners.
• Google Developer Python Tutorial (highly recommended as a way to master python in just a few hours!)
• Learn Python 3 in Y Minutes (highly recommended as a way to master python in just a few minutes!!)
• NumPy Tutorial (also highly recommended!)
• scikit-learn -- A popular python library for supervised and unsupervised learning algorithms

Sections (starting week of 1/13/20):

• AA: Thursday 8:30am-9:20am, LOW 105 (Romain Camilleri)
• AB: Thursday 9:30am-10:20am, LOW 105 (Ivan Montero, Sujie Zhou)
• AC: Thursday 11:30am-12:20pm, LOW 105 (Ivan Montero, Jifan Zhang)
• AD: Thursday 12:30pm-1:20pm, THO 202 (Ani Canumalla, Leo Du)

Date	Due	Topic	Readings	Assignments
01/06/20		Introduction	Student Conduct Code: Academic Misconduct
01/08/20		Machine Learning, Framing a Learning Problem	The Discipline of Machine Learning, Flach: Prologue & Ch.1	Homework 0 (8%)
01/10/20		Supervised Learning & Decision Trees	Mitchell: Ch. 3
01/13/20	HW0	Decision Trees & Information Gain		Homework 1 (16%)
01/15/20		Decision Trees & Overfitting, K-Nearest Neighbors	Mitchell: Ch. 8
01/16/20		Section: Probability, Linear Algebra	Probability: exercise, solutions Linear Algebra: notebook, html
01/17/20		Evaluation of Learning Algorithms	LfD: 4.3
01/20/20		No Class (Martin Luther King Day)
01/22/20		Linear Regression & Gradient Descent	LfD: 3.1-3.2.1
01/23/20		Section: Convexity & Gradient Descent, HW0	Notes, Solution
01/24/20		Basis Functions	Flach 7-7.1, LfD: 3.4
01/27/20		Vectorization, Regularization	LfD: 4.1-4.2
01/29/20		Linear Classification: the Perceptron	Flach 7.2
01/30/20		Section: The Maximum Likelihood Estimator	Notes, Solution
01/31/20		Logistic Regression	LfD: 3.3, Flach: 7.4
02/03/20	HW1	Learning Theory, Bias & Variance	LfD: 2, Mitchell: Ch. 7, Handout: No Free Lunch, Learning Theory Notes
02/05/20		VC Dimension & Generalization Bounds
02/06/20		Section: Bias-Variance Tradeoff	Notes	Homework 2 (16%)
02/07/20		No Class (Instructor Travel)
02/10/20		Maximizing the Margin, Support Vector Machines	Flach: 7.3
02/12/20		Support Vector Machines (Continued)	Burges Tutorial
02/13/20		Section: Lagrange Multipliers & Duality	Notes
02/14/20		The Kernel Trick	Flach: 7.5
02/17/20		No Class (Presidents Day)
02/19/20	HW2	Ensemble Methods	Flach: 11-11.3, Adaboost Slides
02/20/20		Section: Kernelized Linear Regression, Random Forests	Notes
02/21/20		Ensemble Methods (continued)	Boosting the Margin	Homework 3 (16%)
02/24/20		Bayes' Rule, Independence, Density Estimation	Optional Reading: Graphical Models Tutorial
02/26/10		Naive Bayes	Generative Model Notes Flach: 9.2
02/27/20		Section: Naive Bayes	Notes
02/28/20		Neural Networks	Mitchell: Ch. 4
03/02/20		Neural Networks (continued), Backpropagation
03/04/20	HW3	Backpropagation		Homework 4 (16%)
03/05/20		Section: Neural Networks	Notes, Slides
03/06/20		Unsupervised Learning, Clustering, K-means	Flach: 8.4-8.6, 10.3, An Impossibility Theorem for Clustering
03/09/20		Gaussian Mixture Models, Expectation Maximization
03/11/20		Feature Selection
03/13/20	HW4	Dimensionality Reduction
03/18/20 8:30am-10:30am		Final Exam Cancelled

Your final grade is divided into three components: class participation, assignments, and a final exam.

• Assignments (72%). There will be five graded assignments. Generally, they are designed to give you practical working knowledge of fundamental machine learning algorithms as well as deeper insight into the material. They will also help to prepare you for the exams.
• Final Exam (25%). There will be a written, closed-book, closed note, closed everything, final exam in March.
• Class Participation (3%). Participation means attending classes, participating in class discussions, asking relevant questions, volunteering to provide answers to questions, and providing constructive criticism and creative suggestions that improve the course.

All graded assignments are due by the time and date indicated and all work must be turned in via electronic submission, per the instructions given in the assignment. E-mailed submissions will not be accepted. You should also treat assigned readings as assignments that are due at the beginning of each class.

Late homework policy: Assignments are due at the beginning of class on the day that they are due. You will be allowed 3 total late days without penalty for the entire semester. Please use these wisely, and plan ahead for job interviews, conferences, travel, deadlines, etc. Once those days are used, you will get zero credit for any late assignment. The only exceptions will require: a note from an appropriate authority and immediate notification of the problem when it arises. Naturally, your excuse must be acceptable. If a meteor landed on your bed and destroyed your assignment, I need a signed note from the meteor.

Attendance is expected, no video recording of the lecture will be provided.

Previous Exams and Assignment Solutions Will Not Be Posted.

Disclaimer: I reserve the right to modify any of these plans as need be during the course of the class; however, I won't do anything capriciously, anything I do change won't be too drastic, and you'll be informed as far in advance as possible.

Collaboration Policy: I want to encourage you to discuss the material and work together to understand it, here are some rules and guidelines for collaborating with others.

• The readings and lecture topics are group work. Please discuss the readings and associated topics with each other. Work together to understand the material. I highly recommend forming a reading group to discuss the material -- we will explore many challenging ideas and it helps to have multiple people working together to understand them.
• It is fine to discuss the topics covered in the homeworks, to discuss approaches to problems, and to sketch out general solutions. However, you MUST write up the homework answers, solutions, and programs individually. You are not permitted to share specific solutions, mathematical results, program code, knowledge representations, experimental results, etc. If you made any notes or worked out something on a white board with another person while discussing the homework, you may not use those notes while writing up your answer, however tempted you may be to do so.
• Under ABSOLUTELY NO circumstances should you share computer code with another student. While it is acceptable for students to discuss coding questions with others; it is NOT acceptable for students to look at another students code.
• We expect the students not to copy, refer to, or seek out solutions in published material on the web or from other textbooks (or solutions from previous years or other courses) when preparing their answers. Students are certainly encouraged to read extra material for a deeper understanding. If you do happen to find an assignment's answer, it must be acknowledged clearly with an appropriate citation on the submitted solution.
• If you discussed homework with another student, must also indicate on each homework with whom you collaborated with.
• Exams, of course, must be your own individual work.

If you have any questions as to what types of collaborations are allowed and which are dishonest, please ask me before you make an irreversable mistake.

Academic Integrity: I expect that each student will conduct themself with integrity. At least some of you are researchers-in-training, and I expect that you understand proper attribution and the importance of intellectual honesty. Unauthorized use of any previous semester course materials, such as tests, quizzes, homework, projects, lectures, and any other coursework, is prohibited in this course. My policy is strict: using these materials will be considered a direct violation of academic policy and will be dealt with accordingly.

Acknowledgements