Monday / Wednesday / Friday 9:30-10:20am, CSE2 G20

**Instructor**:
Byron Boots

email: bbootscs.washington.edu

office hours: 10:30-11:30am Monday and Wednesday (after class), CSE2 131

**TA Office Hours**:

- Romain Camilleri: Thursday 1:00pm-3:00pm, CSE2 150
- Anirudh Canumalla: Tuesday 11:00am-1:00pm, CSE2 121
- Leo Du: Thursday 4:00pm-6:00pm, CSE2 152
- Ivan Montero: Monday 1:30pm-3:30pm, CSE2 131
- Jifan Zhang: Wednesday 5:30pm-7:30pm, CSE2 131
- Sujie Zhou: Friday 3:00pm-5:00pm, CSE2 151

**Contact**: ** cse446-staff@cs.washington.edu** PLEASE COMMUNICATE TO THE INSTUCTOR AND TAS ONLY THROUGH THIS EMAIL (unless there is a reason for privacy).

Announcements and Resources will be posted via Canvas. Discussions and questions related to the material and assignemnts will take place on Piazza.

**Course Description**: This course will introduce the fundamental concepts and algorithms that enable computational artifacts to modify and improve their performance through experience. We will cover a variety of topics, including decision trees, logistic regression, support vector machines, ensemble methods, Bayesian methods, neural networks, clustering, and dimensionality reduction.

**Objectives**:
There are four objectives for the course:

- To provide a broad survey of approaches and techniques in machine learning
- To develop a deeper understanding of several major topics in machine learning
- To develop programming skills that will help you to build intelligent, adaptive artifacts
- To develop the basic skills necessary to pursue research in ML

**Lectures**:
Lectures are meant to summarize the readings and stress the important points. You are expected to come to class having already critically read any assigned material. Your active participation in class is crucial in making the course successful. I completely expect to be interrupted throughout a lecture with questions and maybe even your deep insights into the material. This is less about my teaching than about your learning. My role is merely to assist you in the process of learning more about the area.

**Prerequesits**: Students entering the class should be comfortable with programming in python and should have a pre-existing working knowledge of probability and statistics (MATH 394, STAT390, STAT 391, or CSE 312), data structures and algorithms (CSE 332). Additionally, students will find linear algebra (MATH 308) and vector calculus (MATH 324) to be very useful. Students who are weak in these areas should either: take the course at a later date, when better prepared, or expect to put in substantially more effort to catch up.

**Main Readings From** (relevant chapters are available for class members on Canvas):

- Machine Learning -- Tom Mitchell
- Learning from Data -- Yaser S. Abu-Mostafa, Malik Magdon-Ismail, and Hsuan-Tien Lin
- Machine Learning -- Peter Flach

**Additional Resources**:
Throughout the course, you may find it useful to consult the following resources:

- Linear Algebra Review & Reference -- Zico Kolter
- Dive into Deep Learning -- Aston Zhang, Zack Lipton, Mu Li, ALex Smola
- Deep Learning -- Ian Goodfellow, Yoshua Bengio, and Aaron Courville
- Mathematics for Machine Learning -- Marc Deisenroth, A. Aldo Faisal, and Cheng Soon Ong
- An Introduction to Statistical Learning -- Gareth James, Daniela Witten, Trevor Hastie, and Robert Tibshirani
- A Course in Machine Learning -- Hal DaumÃ© III
- Machine Learning Lecture Notes -- Andrew Ng

- The Elements of Statistical Learning -- Trevor Hastie, Robert Tibshirani, and Jerome Friedman
- Information Theory, Inference, and Learning Algorithms -- David MacKay
- Pattern Recognition and Machine Learning -- Christopher Bishop
- Probabilistic Graphical Models -- Daphne Koller and Nir Friedman
- Convex Optimization -- Stephen Boyd and Lieven Vandenberghe

**Software Resources**:

- Python -- Python is the tool of choice for many machine learning practitioners.
- Google Developer Python Tutorial (highly recommended as a way to master python in just a few hours!)
- Learn Python 3 in Y Minutes (highly recommended as a way to master python in just a few minutes!!)
- NumPy Tutorial (also highly recommended!)
- scikit-learn -- A popular python library for supervised and unsupervised learning algorithms

**Sections** (starting week of 1/13/20):

- AA: Thursday 8:30am-9:20am, LOW 105 (Romain Camilleri)
- AB: Thursday 9:30am-10:20am, LOW 105 (Ivan Montero, Sujie Zhou)
- AC: Thursday 11:30am-12:20pm, LOW 105 (Ivan Montero, Jifan Zhang)
- AD: Thursday 12:30pm-1:20pm, THO 202 (Ani Canumalla, Leo Du)

Date | Due | Topic | Readings | Assignments |
---|---|---|---|---|

01/06/20 | Introduction | Student Conduct Code: Academic Misconduct | ||

01/08/20 | Machine Learning, Framing a Learning Problem | The Discipline of Machine Learning, Flach: Prologue & Ch.1 | Homework 0 (8%) | |

01/10/20 | Supervised Learning & Decision Trees | Mitchell: Ch. 3 | ||

01/13/20 | HW0 |
Decision Trees & Information Gain | Homework 1 (16%) | |

01/15/20 | Decision Trees & Overfitting, K-Nearest Neighbors | Mitchell: Ch. 8 | ||

01/16/20 | Section: Probability, Linear Algebra | Probability: exercise, solutions Linear Algebra: notebook, html |
||

01/17/20 | Evaluation of Learning Algorithms | LfD: 4.3 | ||

01/20/20 | No Class (Martin Luther King Day) |
|||

01/22/20 | Linear Regression & Gradient Descent | LfD: 3.1-3.2.1 | ||

01/23/20 | Section: Convexity & Gradient Descent, HW0 | Notes, Solution | ||

01/24/20 | Basis Functions | Flach 7-7.1, LfD: 3.4 | ||

01/27/20 | Vectorization, Regularization | LfD: 4.1-4.2 | ||

01/29/20 | Linear Classification: the Perceptron | Flach 7.2 | ||

01/30/20 | Section: The Maximum Likelihood Estimator | Notes, Solution | ||

01/31/20 | Logistic Regression | LfD: 3.3, Flach: 7.4 | ||

02/03/20 | HW1 |
Learning Theory, Bias & Variance | LfD: 2, Mitchell: Ch. 7, Handout: No Free Lunch, Learning Theory Notes | |

02/05/20 | VC Dimension & Generalization Bounds | |||

02/06/20 | Section: Bias-Variance Tradeoff | Notes | Homework 2 (16%) | |

02/07/20 | No Class (Instructor Travel) |
|||

02/10/20 | Maximizing the Margin, Support Vector Machines | Flach: 7.3 | ||

02/12/20 | Support Vector Machines (Continued) | Burges Tutorial | ||

02/13/20 | Section: Lagrange Multipliers & Duality | Notes | ||

02/14/20 | The Kernel Trick | Flach: 7.5 | ||

02/17/20 | No Class (Presidents Day) |
|||

02/19/20 | HW2 |
Ensemble Methods | Flach: 11-11.3, Adaboost Slides | |

02/20/20 | Section: Kernelized Linear Regression, Random Forests | Notes | ||

02/21/20 | Ensemble Methods (continued) | Boosting the Margin | Homework 3 (16%) | |

02/24/20 | Bayes' Rule, Independence, Density Estimation | Optional Reading: Graphical Models Tutorial | ||

02/26/10 | Naive Bayes | Generative Model Notes Flach: 9.2 |
||

02/27/20 | Section: Naive Bayes | Notes | ||

02/28/20 | Neural Networks | Mitchell: Ch. 4 | ||

03/02/20 | Neural Networks (continued), Backpropagation | |||

03/04/20 | HW3 |
Backpropagation | Homework 4 (16%) | |

03/05/20 | Section: Neural Networks | Notes, Slides | ||

03/06/20 | Unsupervised Learning, Clustering, K-means | Flach: 8.4-8.6, 10.3, An Impossibility Theorem for Clustering | ||

03/09/20 | Gaussian Mixture Models, Expectation Maximization | |||

03/11/20 | Feature Selection | |||

03/13/20 | HW4 |
Dimensionality Reduction | ||

03/18/20 8:30am-10:30am |
Final Exam Cancelled |
|||

Your final grade is divided into three components: class participation, assignments, and a final exam.

**Assignments (72%).**There will be five graded assignments. Generally, they are designed to give you practical working knowledge of fundamental machine learning algorithms as well as deeper insight into the material. They will also help to prepare you for the exams.**Final Exam (25%).**There will be a written, closed-book, closed note, closed everything, final exam in March.**Class Participation (3%).**Participation means attending classes, participating in class discussions, asking relevant questions, volunteering to provide answers to questions, and providing constructive criticism and creative suggestions that improve the course.

**beginning**of each class.

**Late homework policy**: Assignments are due at the beginning of class on the day that they are due. You will be allowed 3 total late days without penalty for the entire semester. Please use these wisely, and plan ahead for job interviews, conferences, travel, deadlines, etc. Once those days are used, you will get zero credit for any late assignment. The only exceptions will require: a note from an appropriate authority and immediate notification of the problem when it arises. Naturally, your excuse must be acceptable. If a meteor landed on your bed and destroyed your assignment, I need a signed note from the meteor.

**Attendance is expected**, no video recording of the lecture will be provided.

Previous Exams and Assignment Solutions **Will Not Be Posted**.

**Disclaimer**: I reserve the right to modify any of these plans as need be during the course of the class; however, I won't do anything capriciously, anything I do change won't be too drastic, and you'll be informed as far in advance as possible.

**Collaboration Policy**: I want to encourage you to discuss the material and work together to understand it, here are some rules and guidelines for collaborating with others.

- The readings and lecture topics are group work. Please discuss the readings and associated topics with each other. Work together to understand the material. I highly recommend forming a reading group to discuss the material -- we will explore many challenging ideas and it helps to have multiple people working together to understand them.
- It is fine to discuss the topics covered in the homeworks, to discuss approaches to problems, and to sketch out general solutions. However, you MUST write up the homework answers, solutions, and programs individually. You are not permitted to share specific solutions, mathematical results, program code, knowledge representations, experimental results, etc. If you made any notes or worked out something on a white board with another person while discussing the homework, you may not use those notes while writing up your answer, however tempted you may be to do so.
- Under ABSOLUTELY NO circumstances should you share computer code with another student. While it is acceptable for students to discuss coding questions with others; it is NOT acceptable for students to look at another students code.
- We expect the students not to copy, refer to, or seek out solutions in published material on the web or from other textbooks (or solutions from previous years or other courses) when preparing their answers. Students are certainly encouraged to read extra material for a deeper understanding. If you do happen to find an assignment's answer, it must be acknowledged clearly with an appropriate citation on the submitted solution.
- If you discussed homework with another student, must also indicate on each homework with whom you collaborated with.
- Exams, of course, must be your own individual work.

**before**you make an irreversable mistake.

**Academic Integrity**:
I expect that each student will conduct themself with integrity. At least some of you are researchers-in-training, and I expect that you understand proper attribution and the importance of intellectual honesty.
Unauthorized use of any previous semester course materials, such as tests, quizzes, homework, projects, lectures, and any other coursework, is prohibited in this course. My policy is strict: using these materials will be considered a direct violation of academic policy and will be dealt with accordingly.

Some assignments, lectures, and ideas on this syllabus are partially adapted from Charles Isbell's ML course and Eric Eaton's ML course. I would like to thank Charles and Eric for helpful discussions and access to course materials.