Assignment  #9 Machine Learning - Neural Networks & Bayesian Learning

Due: December 6 (last class!)

You may work on this assign on your own or in teams of two.  Hand in hardcopy (one copy only if you are working in a team) in class on December 6.

This page makes extensive use of hyperlinks, please check the online version.

Readings for this week:

• Ch 20.1-20.3, skipping material on continuous models (Bayesian Learning)
  • Note: although the problems on this assignment only concern neural networks, there will be questions concerning Bayesian learning on the take-home final exam.
• Ch 20.5 (Neural Networks)

Download and install the Joone (Java Object Oriented Neural Network) GUI Editor on the computer of your choice.  You can find download links on the Joone homepage, http://www.jooneworld.com/ , or use the local copies of the Windows version here:

• Joone GUI Editor (Windows installer)
• Joone Complete Documentation (pdf)

1. The exclusive-OR function is the standard example of a non-linearly separable function that cannot be learned by a perceptron (a single neuron).  Perform the example of using the GUI to learn XOR from the Jooneworld page.  Then see what happens if your try to learn XOR using simpler networks.  Try:
   1. A single neuron
   2. Two input units connected directly to an output unit
   Write a paragraph describing what you saw during learning and testing for each case.
   
   
2. Create neural network that can distinguish just two characters, X and O.
   
   Section 8.3.1 of the Joone Complete Documentation describes how to create a character recognition network using unsupervised learning and a self-organizing map (SOM) structure.  For this exercise, however, I would prefer you implement a simple supervised, feed-forward, back-propagation network.  (If you get stuck and the due date is looming, then go ahead and replicate the SOM approach described in the documentation.)
   
   You can gather training and test examples in any of the following ways:
   1.  Digitize and downsample (to 10x10 or so) your own writing using any other image processing tools with which you are familiar.
   2. Use the Java digitizer and downsampler code that comes with Joone and is described in Section 8.3.1 of the Joone Complete Documentation.
   3. Use just the X and O characters from the optdigits.zip data set that comes from the UCI Machine Learning Repository.

   Hand in a print out (or hand sketch) of your neural network, and a 1 page writeup of your experiments and results.

Readings for next week:

• R&N Sections 4.3 and 4.4
• Section 5.3
• Page 222 (in 7.6, subsection beginning "Local-search algorithms") to 224