CSE 455 Project 4: Eigenfaces

Ekaterina Nepomnyashchaya

CSE 455: Computer Vision

Project 4 Artifact

Original Image - NonSmiling

Original Image - Smiling

Testing Recognition with Cropped Class Images

1. Average Face & 10 Eigenfaces

25 X 25 :

2. Number of Eigenfaces Vs. Number of interesting face correctly recognized

3. Recognition Errors (When 10 EigenFaces are used)

Silly face 1 matched face 11
Silly face 16 matched face 8
Silly face 24 matched face 3
Silly face 25 matched face 26
Silly face 4 matched face 8
Silly face 7 matched face 5
Silly face 9 matched face 14
Face #1

0: Neutral\11; MSE: 34799.8
1: Neutral\3; MSE: 53397.9
2: Neutral\8; MSE: 60762.5
3: Neutral\25; MSE: 67517.4
4: Neutral\28; MSE: 69076.4
The correct result isn't even in the top 5 images.
Face #16

0: Neutral\8; MSE: 41579.1
1: Neutral\14; MSE: 42135.9
2: Neutral\20; MSE: 44804.7
3: Neutral\3; MSE: 44979.2
4: Neutral\16; MSE: 49514.2
Face #24

0: Neutral\3; MSE: 24676.3
1: Neutral\8; MSE: 25300.1
2: Neutral\24; MSE: 29061.4
Face #25

0: Neutral\26; MSE: 26647.5
1: Neutral\25; MSE: 29736.9
Face #4

0: Neutral\8; MSE: 32543.5
1: Neutral\4; MSE: 33842.4
Face #7

0: Neutral\5; MSE: 24848.8
1: Neutral\8; MSE: 43819.3
2: Neutral\3; MSE: 44995.6
3: Neutral\7; MSE: 47404
Face #9

0: Neutral\14; MSE: 48373.2
1: Neutral\6; MSE: 55066.2
2: Neutral\9; MSE: 62543.5

4. Questions

a. Describe the trends you see in your plots. Discuss the tradeoffs; how many eigenfaces should one use? Is there a clear answer?

Starting at 1 eigenface used there was a sharp increase in recognition as eigenfaces 2-7 were added. There was then first a slight drop and then moderate improvement as eignefaces 7-15 were added. There was no benefit to adding the last 12 eigenfaces.
When picking how many eigenfaces to use one must consider the accuracy vs. computation time tradeoff. As more eigenfaces are used, accuracy usually increases. However with more eigenfaces recognition time also increases. Aditionally, if way too many eigenfaces are used, we are likely to overfit. Then when new faces are introduced they may not be recognized as face because they aren't close enough to the specific 28 we trained on. Therefore we should shoot for using a high enough number of eigenfaces too get good recognition results, but not too many as to not overfit and get fast speeds.

c. Show images of recognition errors. How reasonable were the mistakes? Did the correct answer at least appear highly in the sorted results?

The seven recognition errors (when 10 Eigenfaces are used) are shown above. Many of these failures were pretty reaonsable and the correct result appeared high in the sorted results (correct answers are highlighted in orange in the results above). The notable exception is Face 1 which did not appear in the top 5 suggested matches.
It looks like raised eyebrows gave the algorithm serious trouble. Eyebrows are much higher intensity than the rest of the image causing MSE to be very high when they are moved. I think this was the cause of errors for face 11, 16 and 24. Additionally the algorithm tends to match people wearing glasses, and faces in similar orientations.

Cropping and Finding Faces

1. Finding and cropping one face

main --findface group/test.tga eig10.face .8 .86 .05 crop 1 testoutput.tga
main --findface yosemiteSmall.tga eig10.face .07 .25 .05 crop 1 yosemite_out.tga

2. Finding multiple faces

main --findface group/group7.tga eig10.face .8 .86 .05 mark 4 group7output.tga
main --findface show2.tga eig10.face .1 .3 .05 mark 4 showOut.tga

3. Questions

a. What min_scale, max_scale, and scale step did you use for each image?

See the subtitles.

b. Did your attempt to find faces result in any false positives and/or false negatives? Discuss why you think they occurred.

Yes. My results are nowhere near good.
In the group 7 picture I only find 2 faces. Changing around the weight of var() and distance from average would change the faces found (from the left two, to the middle two), but I was not able to figure out how to get three faces. Kicking up the weight on the variance should have worked, but then I would run into the problem where instead of finding faces, the algorithm would find edges of clothing with wall (where there is a ton of variance).
On the my own photos performance was even worse. I am guessing that's becuase of the much harsher lighting in those pictures than in the classroom. Also, in most of my photos my friends are smiling which causes the faces to be way different than the neutral faces we trained on.
Besides trying different window sizes I also experimented with adding large powers to var() and mse from average in the error calculations. You would think that the yosemite picture would be easy with high a high enough power on variance, but no increasing the variance didn't help. Nothing seemed to help get reasonable performance on either the yosemite picture or the group photo.




Verify Face

Plot of Threshold Vs. Face recognized

3. Questions

a. What MSE thresholds did you try? Which one worked best? What search method did you use to find it?

I tried thresholds starting at 25,000 up to 600,000 with increments of 25,000. In the region of 25,000 to 50,000 I sampled with increments of 5,000 (because false negative rates were decreasing very rapidly there). Take a look at the graph above for details (the plot does ommit very high MSE values tested for readiblity).
What threshold is "best" is unclear. That all depends on if the user places higher values on lower false posetive or false negatives rates. As MSE increases false negative rates go down and false posetive rates go up. Say we value both types of errors equally, then the best MSE is around 180,000.
To find these values I calculated MSEs for each intersting face compared to same person's neutral face (for posetive results) and each person's interesting face against someone else for negative results (I compared all excpet first face to first face and first face to second).

b. Using the best MSE threshold, what was the false negative rate? What was the false positive rate?

Using a good threshold of 200,000
False Negative Rate = 1 / 28
False Positive Rate = 2 / 28

Extra Credit

I implemented morph face. To check it out watch the video linked below where I morph face A into face B.
These two faces were chosen because of their difference in face shape (easier to see the transformation).
Face A and B (left to right)


Ten intermediate morph states between face A and face B
Gradual morph from Face A to Face B

Watch morph