Spring 2003 CSE576 - Computer Vision - Eigenfaces

Recognition

Below is the average face and the 10 eigenfaces followed by a plot of the matches versus eigenfaces used.   

I created a routine in main.cpp that ran through the tests as described in the project assignment. As can be see from the plot above, there is a definite win through the first 5 eigenfaces, and no gain after 21 eigenfaces. Between those two values it tends to improve with more eigenfaces, but not monotonically so there is no clear knee in the graph. 21 is the lowest value that will optimize the solution and 10 seems to be a good value trading off number of eigenfaces with matches.
There were many incorrectly identified faces throughout the testing, and some that never did match. Here are a couple that didn't match and what the did match to.
matched to instead of
matched to instead of
The first mismatch seems easy to explain since the smiling image is so different from the non-smiling image. The second example seems like it should have matched very well. There were 3 or 4 like this; the smiling image is much the same as the non-smiling, but they didn't match. One clue here is that there were all wearing glasses. The PCA approach may have particular trouble with glasses.

Finding Faces

Cropped Aseem   

Cropped Photo #2   

Marked Group Photo #1  

Marked Group Photo #2   

Photo	min_scale	max_scale	step
Aseem	0.45	0.55	0.01
Photo #1	0.45	0.55	0.01
Group Photo #1	0.9	1.1	0.01
Group Photo #2	0.7	0.9	0.01

The first 3 photos went as planned. There were no false negatives or positives. Although I did try several images for the second solo photo that did not correctly identify the face in the picture. The system is still easily fooled by large non-textured areas even with the tip on changing the metric. The final group photo got most of the faces correct, but also had several false positive and false negatives as can be seen. But given the small training set it still did reasonably well.

Bells and Whistles

Speedup - it's in faces.cpp. There's actually very little code that needed to change for this, so I left turning it off and on as a define.
VerifyFace - see section below
Morphing - see section below

Verifying Faces

I set up a routine that ran verify face over the smiling faces for every MSE value from 1000 to 100,000 in increments of 1000. This is a very simplistic search method, but verification is fast enough that this still took very little time. The plot above shows the results. Up to a value of 17000 there were no false positives, but also very few verifications. After that point false positives began appearing, until the final mse limit of 100,000 where everything was matched, but a quarter of the matches were false positives. The optimal value would then depend on the purpose. If false positives were not acceptable then the limit would need to be set below 1700. If some error was acceptable then approximately 40,000 verifies over 2/3rds of the faces with a small number of false positives.

Morphing Faces

You can click on the image to run the avi. It came out reasonable well. There are 50 frames morphing in and then 50 morphing out to make it to look OK when it's looping. I also used 32 pixel faces to get a slightly larger video.