EE/CSE 576 Computer Vision - Project 3: Eigenfaces
Eigenfaces
Testing Recognition with Cropped Class Images
The first thing we have to do is create the eigenface database. We're have two sets of 33 cropped training images of people smiling
and not smiling. We create this database by finding the average face and computing the eigenvectors of the set of images.
We can compute the average face by finding the element wise average, that is stack all the training on top of one another
and since they're all automatically scaled to the same resolution we can find the average value of each pixel in the images by
simply adding all the pixel values for a given point in the image and dividing by the number of images. i.e.:
Which gives us the average face found here (enlarged 3x, the original was 25x25):
And we can get the the 10 largest eigenfaces by computing the jacobi of the images arranged in a
(imgWidth*imgHeight) x (imgWidth*imgHeight) matrix.: (enlarged 3x, the originals were 25x25)
After we compute the eigenFaces we can create a database of users with these images. Using this
database we can search for faces in other images, and even recognize faces in the database. For example
using a database of non-smiling images we can have the program attempt to recognize the smiling version
of the cropped faces by running the command:
--recognizeface user.tga base.user eig10.face 2
For example, if we feed in a smiling image, With the non-smiling version in the database already we'll get output similar to
main --recognizeface eckart.tga base.user eig10.face 5
Loaded faces from 'eig10.face'
Loaded users from 'base.user'
Face 'eckart.tga' recognized as being closest too:
0: nonsmiling_cropped/eckart; MSE: 8355.84
1: nonsmiling_cropped/lane; MSE: 16626.8
2: nonsmiling_cropped/simon; MSE: 22261.7
3: nonsmiling_cropped/lester; MSE: 23371.8
4: nonsmiling_cropped/moe; MSE: 32913.7
However, this only works correctly for 21 out of 33 images (63.7%) with 10 eigenfaces in our system. If we vary
the number of eigenFaces in the data base we end up with a success rate that maxes out at about 70%. Here is a graph
that shows the relationship between the number of eigenFaces and successful detection.
As you can see from the plot the sucess rate quickly climbs then plateaus. So it actually is better to use fewer eigenFaces
and save computation time than use the same number of eigenFaces as training images. Unfortunately, there isn't a definitive answer
from this plot since we get varying results - 11 eigenFaces looks good but 13 eigenFaces does worse. But, in general if you
pick a higher number of eigenFaces you seem more likely to do better, so half may be a good estimate of the number of eigenFaces to use.
The missed recognized images were somewhat consistent, for example the smiling version of downey.tga would only match up to downey's
ID after 14 other images were suggested. Placed side by side the images are somewhat different, so its clear to see why the program had
a difficult time.
A few other troublesome images were people with glasses. Since not everyone in the training set wore glasses the program has a difficult time
even when the image were relatively similar:
In about 2/3's of the cases where the program misidentified a person it was far off (the real person was listed as the 10th+ possibility), and in the other 1/3 it was usually
only a few people off (less than the 6th position).
Cropping and Finding Faces
Using Aseem's stylin picture as a test bed, here's how the program performs at finding faces in images. Even faces that aren't
in the training set.
Here's Aseem in the picture along with his cropped picture, found using:
main --findface aseem.tga eig10.face 0.45 0.55 0.01 crop 1 aseem_crop.tga
As luck would have my digital camera was all out of batteries. So went around the web and found this nice 'cooked' up image to try to
try on my executable. By chance the numbers I picked worked out and I got:
Actually, it took several runs to get the program to actually find something near the face. I actually had to be very specific with
this image to get it pick the real face as the primary 'face'. Otherwise it wanted to call his shoulder a face:
After a bit of work I was able to produce the marked and cropped versions using the commands:
main --findface cooked.tga eig10.face 0.55 0.55 0.01 crop 1 soma_crop.tga
main --findface cooked.tga eig10.face 0.55 0.55 0.01 mark 1 soma_mark.tga
Using the picture of group 1 we can run our program and mark the 3 faces in this image.
And after running the command:
--findface group1.tga eig10.face 0.8 0.9 0.1 mark 3 group_marked.tga
We end up with the marked faces of group 1:
My next attempt was somewhat unsuccessful. I found a group picture of some EE students and turned my program loose on the photo.
Unfortunately, the Linux Computing Cluster was down for the weekend so I wasn't able to find a perfect match with these images.
I think that this is due in part to the fact that the lighting isn't too terrific in this picture. The fact that the algorithm
is pretty simply probably has something to do with it. The weighing function I'm using is the same as was suggested but it still
seems like it has trouble on textured areas. And in truth is the likely reason why there were any false positives in these images
since they're relative face on and aren't noisy. Also again the program seems to have trouble with people with glasses, and since the
image resolution is a bit low it may not be able to see the middle guy's face. Here's the original image:
And here's the somewhat accurate marking from the program found using:
--findface samp10.tga eig10.face 0.9 0.9 0.1 mark 7 group_marked.tga
Interestingly enough, the first image is the guy on the left. But, the second is the girl on the right's shirt:
But hey, 2 our of 4 isn't bad, even the shotgun approach didn't work:
--findface samp10.tga eig10.face 0.7 1.1 0.05 mark 30 group_marked.tga
Extra Credit
The only extra credit that I implemented was to implement the verifyFace function.