CSE576 - Project3 Writeup
David Richardson
Below are the results of my experiments for CSE576 - project3.
Extra features: For extra credit, I implemented the verifyFace routine.
The following are the first 10 eigenfaces and the average face computed from the nonsmiling_cropped images:
| average face |  |
eigen_face_5 |  |
| eigen_face_0 |  |
eigen_face_6 |  |
| eigen_face_1 |  |
eigen_face_7 |  |
| eigen_face_2 |  |
eigen_face_8 |  |
| eigen_face_3 |  |
eigen_face_9 |  |
| eigen_face_4 |  |
After constructing a user base using the same set of nonsmiling_cropped images, I experimented with using different numbers of eigenfaces to recognize the images in the smiling_cropped set of images. I tried the mean face plus 1 through 33 eigenfaces at a granularity of 2. Below is a plot of the number of smiling faces faces correctly recognized versus the number of eigenfaces used:
The plot essentially shows a leveling off effect as the number of eigenfaces increases. Initially, increasing the number of eigenfaces greatly increases the number of smiling faces correctly recognized. However, after about 10-15 eigenfaces, the upward trend seems to level off and stay pretty constant around 20-21 smiling faces recognized. Between 9-15 eigenfaces, there is a marked decrease in accuracy, which I wasn't sure how to explain. From the plot, it isn't obvious that there is a clear answer to how many eigenfaces to use. Presumably, 20 should suffice, but the difference in accuracy compared to 9 eigenfaces is almost minimal. I suspect that accuracy levels off eventually because since the eigenfaces I used were only of size 25 by 25 pixels, then after a certain number of eigenfaces, the remaining eigenfaces just aren't detailed enough to capture any more salient features of a face that would make a difference in recognition patterns. Indeed, some of the later eigenfaces appear a bit more jumbled and noisy. Oftentimes I found that as the number of eigenfaces increased, certain faces would cease to be correctly recognized and others that had previousl failed to be recognized would be successful. This trend was particularly true for eigenfaces number greater than 21 or so. I would routinely see certain sets of faces flip-flop from being correct to being incorrect, then back again. Presumably, the extra "noise" added in by too many eigenfaces may be randomly effecting the results.
In running the above recognition test, I came across a couple of images that routinely failed to be correctly recognized, regardless of the number of eigenfaces I used. A few of these images are shown below:
| hu.tga |  |
downey.tga |  |
hahn.tga |  |
For hu.tga, his results seemed to get closer as the number of eigenfaces increased, up until about 21 eigenfaces. Indeed, at 9 eigenfaces he was 5th in the list and by 21 eigenfaces he was 2nd. For downey.tga however, he wasn't even in the top 15 matches for any of the trial runs. I suspect that this may have been due to his particularly toothy smile in the photo? The same was true of hahn.tga, he wasn't in the top 15 matches for any of the trial runs, and was consistently recognized as gauthier.
Next, I cropped aseem.tga using the 10 computed eigenfaces file. I used min_scale,max_scale, step parameters of .45, .55, .01. The original image is shown with the cropped image:
| aseem.tga |  |
cropped image |  |
Next, I found a digital portrait of my friend Keith (I couldn't find a good one of myself). Using min_scale,max_scale, step parameters of .20, .30, .05, I was able to successfully crop the image to keith's face. Below are my results showing the original image and the cropped image:
| keith.tga |  |
cropped image |  |
I then ran my program on the group1.tga image to find the 3 faces in the image and mark them with green boxes. Using min_scale,max_scale, step parameters of .90, 1.025, .025, I was able to successfully mark the image. The results are given below, showing the original image and the marked image:
| group1.tga |  |
marked image |  |
Finally, I found a group photo of me and my friends out drinking at the Big Time Brewery. I cropped the image a bit to get rid of a bunch of clutter so as to have the best chance of find the faces in the image. I experimented with a whole bunch of different step sizes and min_scale and max_scale values. Slight fluctuations would oftentimes lead to big differences in the top 5 faces marked. The program was easily fooled by continuous, low-texture regions in the image, despite my implementation of the "trick" described in the assignment to adjust the mse to account for this problem. On one occasion, the program found a face on the reflection of the table. Ultimately, after much experimentation, the best results I was able to obtain was using min_scale,max_scale, step parameters of 1.0, 1.0, .01. That is, I just searched the original image size. Otherwise, I would get fits that returned, for instance, my friend mike's hand holding the beer glass (second person from the left of the image). The results aren't perfect, but I think it's pretty good. Below is the orignal image alongside of the marked version of that image:
| friends.tga |  |
marked image |  |
Overall, the program was effective when the images used had faces in similar orientations than those in the nonsmiling_cropped images set. Otherwise, the results you get can be painful and frustrating. Also, to get good results, I found that the images couldn't be too complicated but also couldn't contain large, low texture regions. I suppose though that ultimately PCA isn't the most robust way of detecting faces.