Lars Bull
3/13/2009
CSE 455
Project 4

Eigenfaces for Recognition

Recognition

Face 'class_images\smiling_cropped\01.tga' recognized as being closest too:
0: class_images\nonsmiling_cropped\01; MSE: 472.355
1: class_images\nonsmiling_cropped\11; MSE: 635.804
2: class_images\nonsmiling_cropped\02; MSE: 821.308
3: class_images\nonsmiling_cropped\04; MSE: 866.838
4: class_images\nonsmiling_cropped\15; MSE: 986.612

Face 'class_images\smiling_cropped\02.tga' recognized as being closest too:
0: class_images\nonsmiling_cropped\20; MSE: 1122.82
1: class_images\nonsmiling_cropped\03; MSE: 1716.71
2: class_images\nonsmiling_cropped\08; MSE: 2104.1
3: class_images\nonsmiling_cropped\13; MSE: 2143.24
4: class_images\nonsmiling_cropped\12; MSE: 2148.56

Face 'class_images\smiling_cropped\03.tga' recognized as being closest too:
0: class_images\nonsmiling_cropped\03; MSE: 617.884
1: class_images\nonsmiling_cropped\04; MSE: 1034.54
2: class_images\nonsmiling_cropped\02; MSE: 1086.86
3: class_images\nonsmiling_cropped\14; MSE: 1102.92
4: class_images\nonsmiling_cropped\01; MSE: 1177.93

Face 'class_images\smiling_cropped\04.tga' recognized as being closest too:
0: class_images\nonsmiling_cropped\11; MSE: 1192.94
1: class_images\nonsmiling_cropped\01; MSE: 1301.2
2: class_images\nonsmiling_cropped\04; MSE: 1418.71
3: class_images\nonsmiling_cropped\02; MSE: 1563.45
4: class_images\nonsmiling_cropped\18; MSE: 1697

Face 'class_images\smiling_cropped\05.tga' recognized as being closest too:
0: class_images\nonsmiling_cropped\05; MSE: 1189.99
1: class_images\nonsmiling_cropped\02; MSE: 1298.74
2: class_images\nonsmiling_cropped\11; MSE: 1421.47
3: class_images\nonsmiling_cropped\01; MSE: 1459.47
4: class_images\nonsmiling_cropped\18; MSE: 1666.24

Face 'class_images\smiling_cropped\06.tga' recognized as being closest too:
0: class_images\nonsmiling_cropped\07; MSE: 1293.57
1: class_images\nonsmiling_cropped\06; MSE: 1738.22
2: class_images\nonsmiling_cropped\23; MSE: 1783.36
3: class_images\nonsmiling_cropped\03; MSE: 2240.26
4: class_images\nonsmiling_cropped\17; MSE: 2444.58

Face 'class_images\smiling_cropped\07.tga' recognized as being closest too:
0: class_images\nonsmiling_cropped\11; MSE: 1064.39
1: class_images\nonsmiling_cropped\02; MSE: 1281.76
2: class_images\nonsmiling_cropped\04; MSE: 1288.53
3: class_images\nonsmiling_cropped\01; MSE: 1331.33
4: class_images\nonsmiling_cropped\18; MSE: 1361.01

Face 'class_images\smiling_cropped\08.tga' recognized as being closest too:
0: class_images\nonsmiling_cropped\08; MSE: 2037.09
1: class_images\nonsmiling_cropped\09; MSE: 2118.29
2: class_images\nonsmiling_cropped\22; MSE: 2121.28
3: class_images\nonsmiling_cropped\12; MSE: 2559.15
4: class_images\nonsmiling_cropped\04; MSE: 2704.53

Face 'class_images\smiling_cropped\09.tga' recognized as being closest too:
0: class_images\nonsmiling_cropped\09; MSE: 1875.08
1: class_images\nonsmiling_cropped\22; MSE: 2039.88
2: class_images\nonsmiling_cropped\24; MSE: 2054.73
3: class_images\nonsmiling_cropped\12; MSE: 2094.53
4: class_images\nonsmiling_cropped\18; MSE: 2098.25

Face 'class_images\smiling_cropped\10.tga' recognized as being closest too:
0: class_images\nonsmiling_cropped\10; MSE: 498.817
1: class_images\nonsmiling_cropped\11; MSE: 687.074
2: class_images\nonsmiling_cropped\01; MSE: 1097.86
3: class_images\nonsmiling_cropped\02; MSE: 1237.35
4: class_images\nonsmiling_cropped\04; MSE: 1260.41

Face 'class_images\smiling_cropped\11.tga' recognized as being closest too:
0: class_images\nonsmiling_cropped\02; MSE: 753.829
1: class_images\nonsmiling_cropped\11; MSE: 885.088
2: class_images\nonsmiling_cropped\01; MSE: 1159.58
3: class_images\nonsmiling_cropped\15; MSE: 1191.09
4: class_images\nonsmiling_cropped\24; MSE: 1293.16

Face 'class_images\smiling_cropped\12.tga' recognized as being closest too:
0: class_images\nonsmiling_cropped\12; MSE: 941.203
1: class_images\nonsmiling_cropped\22; MSE: 1620.33
2: class_images\nonsmiling_cropped\09; MSE: 1699.94
3: class_images\nonsmiling_cropped\18; MSE: 1950.76
4: class_images\nonsmiling_cropped\08; MSE: 1960.02

Face 'class_images\smiling_cropped\13.tga' recognized as being closest too:
0: class_images\nonsmiling_cropped\19; MSE: 821.122
1: class_images\nonsmiling_cropped\13; MSE: 851.744
2: class_images\nonsmiling_cropped\12; MSE: 1102.8
3: class_images\nonsmiling_cropped\18; MSE: 1116.55
4: class_images\nonsmiling_cropped\22; MSE: 1259.43

Face 'class_images\smiling_cropped\14.tga' recognized as being closest too:
0: class_images\nonsmiling_cropped\18; MSE: 742.904
1: class_images\nonsmiling_cropped\16; MSE: 939.907
2: class_images\nonsmiling_cropped\22; MSE: 1093.89
3: class_images\nonsmiling_cropped\04; MSE: 1195.66
4: class_images\nonsmiling_cropped\19; MSE: 1318.82

Face 'class_images\smiling_cropped\15.tga' recognized as being closest too:
0: class_images\nonsmiling_cropped\15; MSE: 387.101
1: class_images\nonsmiling_cropped\02; MSE: 687.788
2: class_images\nonsmiling_cropped\01; MSE: 878.647
3: class_images\nonsmiling_cropped\11; MSE: 941.309
4: class_images\nonsmiling_cropped\24; MSE: 1200.96

Face 'class_images\smiling_cropped\16.tga' recognized as being closest too:
0: class_images\nonsmiling_cropped\16; MSE: 890.227
1: class_images\nonsmiling_cropped\22; MSE: 1047.52
2: class_images\nonsmiling_cropped\18; MSE: 1061.51
3: class_images\nonsmiling_cropped\02; MSE: 1105.44
4: class_images\nonsmiling_cropped\01; MSE: 1141.57

Face 'class_images\smiling_cropped\17.tga' recognized as being closest too:
0: class_images\nonsmiling_cropped\17; MSE: 1094.69
1: class_images\nonsmiling_cropped\03; MSE: 1417.89
2: class_images\nonsmiling_cropped\08; MSE: 1466.5
3: class_images\nonsmiling_cropped\23; MSE: 1534.87
4: class_images\nonsmiling_cropped\07; MSE: 1545.7

Face 'class_images\smiling_cropped\18.tga' recognized as being closest too:
0: class_images\nonsmiling_cropped\20; MSE: 1474.06
1: class_images\nonsmiling_cropped\09; MSE: 1516.92
2: class_images\nonsmiling_cropped\08; MSE: 2013.49
3: class_images\nonsmiling_cropped\07; MSE: 2042.68
4: class_images\nonsmiling_cropped\12; MSE: 2158.67

Face 'class_images\smiling_cropped\19.tga' recognized as being closest too:
0: class_images\nonsmiling_cropped\19; MSE: 435.952
1: class_images\nonsmiling_cropped\13; MSE: 917.619
2: class_images\nonsmiling_cropped\16; MSE: 1030.83
3: class_images\nonsmiling_cropped\18; MSE: 1166.8
4: class_images\nonsmiling_cropped\12; MSE: 1187.66

Face 'class_images\smiling_cropped\20.tga' recognized as being closest too:
0: class_images\nonsmiling_cropped\20; MSE: 851.827
1: class_images\nonsmiling_cropped\09; MSE: 2000.16
2: class_images\nonsmiling_cropped\23; MSE: 2095.34
3: class_images\nonsmiling_cropped\08; MSE: 2167.03
4: class_images\nonsmiling_cropped\12; MSE: 2169.82

Face 'class_images\smiling_cropped\21.tga' recognized as being closest too:
0: class_images\nonsmiling_cropped\21; MSE: 356.378
1: class_images\nonsmiling_cropped\24; MSE: 1052.18
2: class_images\nonsmiling_cropped\02; MSE: 1701.87
3: class_images\nonsmiling_cropped\18; MSE: 1738.9
4: class_images\nonsmiling_cropped\11; MSE: 2061.96

Face 'class_images\smiling_cropped\22.tga' recognized as being closest too:
0: class_images\nonsmiling_cropped\24; MSE: 1102.39
1: class_images\nonsmiling_cropped\22; MSE: 1159.38
2: class_images\nonsmiling_cropped\15; MSE: 1638.22
3: class_images\nonsmiling_cropped\18; MSE: 1725.97
4: class_images\nonsmiling_cropped\09; MSE: 1739.95

Face 'class_images\smiling_cropped\23.tga' recognized as being closest too:
0: class_images\nonsmiling_cropped\23; MSE: 738.349
1: class_images\nonsmiling_cropped\20; MSE: 1452.23
2: class_images\nonsmiling_cropped\07; MSE: 1599.14
3: class_images\nonsmiling_cropped\03; MSE: 1786.53
4: class_images\nonsmiling_cropped\17; MSE: 2046.81

Face 'class_images\smiling_cropped\24.tga' recognized as being closest too:
0: class_images\nonsmiling_cropped\15; MSE: 1048.92
1: class_images\nonsmiling_cropped\02; MSE: 1103.7
2: class_images\nonsmiling_cropped\12; MSE: 1152.44
3: class_images\nonsmiling_cropped\24; MSE: 1166.25
4: class_images\nonsmiling_cropped\22; MSE: 1251.07

I achieve 58% (14/24) accuracy with ten 25x25 non-smiling eigenfaces for recognition of smiling faces. I found that the more eigenfaces used, the greater the accuracy, as shown below in the graph of positive results for numbers of eigenfaces from 1 to 23. The tradeoff is speed; more eigenfaces means more computation and thus longer execution time, which are already somewhat substantial for some functions (such as findFaces). The curve is much steeper at the start than the end, so there are clearly some diminishing returns when adding more eigenfaces. Nine looks like it may be a good compromise, as that's just before the slope of the plot starts leveling off.

The recognition mistakes made by the function are reasonable in most cases, as can be seen above (first image is the smiling query, second is the top result, and the third the correct result (if a mismatch)). Sometimes it seemed to be contributed to by blur while other times it seemed to be a result of simply very similar faces. Image 22's results stick out as the most odd, but even they offer some similarities that could explain it (such as nose shape).

Finding Faces

crop, min scale: 0.45, max scale: 0.55, scale step: 0.01, Eigenfaces: 10 non-smiling, 25x25 pixels

min scale: 0.33, max scale: 0.33, scale step: 0.01, Eigenfaces: 10 non-smiling, 25x25 pixels

Eigenfaces: 10 non-smiling, 25x25 pixels

min scale: 0.45, max scale: 0.55, scale step: 0.01, Eigenfaces: 10 non-smiling, 25x25 pixels


Click for more group images

min scale: 0.79, max scale: 0.87, scale step: 0.04, Eigenfaces: 10 non-smiling, 20x20 pixels

There were certainly a lot of false positives in face detection, unfortunately. I can't say I know exactly why, but a lot of things just look like faces. Some modifications I made allowed the program to avoid most of the areas where it would come across these pitfalls, but it also allowed it to focus more on areas that had face-like qualities, so oftentimes hands, throats, ears, and flesh-colored objects come back as false positives. The thing that confused me the most is when its face would partially overlap a real face; sometimes I'd get half a face, or eyes, or a chin, but it didn't seem to always want to settle on a face. Some of this was extremely sensitive to scale, so small modifications would cause a box to shift (like with the box that only contains a mouth on the left of the previous image).

Stepping from .45 to .55 by .01 seemed to be a pretty good way to pick up faces in pictures that were rather clean. Unfortunately, it seems like the detector can get easily hung up on imperfections in the image at times, so wide ranges like that wasn't perfect for just any image. It's especially important to take note of the size of the image and size of the eigenfaces before setting scales, since the scale will make the eigenface relatively bigger or smaller against the scaled image. One can get a good idea of what size is needed for the scale simply by glancing at the picture.

Verifying Faces

I started off using a value around 60000 for the threshold and had some decent success immediately; there were only 6 false negatives and 4 false positives. My method for learning which direction to move in was simply looking through the list for the remaining incorrect results and taking into account which had the most nearby values, as well as how much of a buffer there was between the current threshold and nearby MSE values that could cause another bad result. I adjusted it upward a bit, trying 66000 and then 70000, eventually settling there (at 70000) with 5 false negatives and 3 false positives, leaving 40/48 correct results.

Extra Credit

I implemented the first whistle, the faster eigenvector algorithm that replaces the very slow and bloated default algorithm. It successfully allows me quick creation of eigenfaces at any size I desire.

I made improvements to the finder's ability to locate faces by using colors; I find the individual averages for red, green, and blue in a subimage and check to see if those values meet certain conditions (> or < certain values or even equations involving relationships to the other color averages), which kept the program from finding 'faces' in areas that are in no way colored like a face.

Finally, I also tested the face detector on animals (third whistle) and found that is indeed capable of locating animal faces:

min scale: 0.5, max scale: 0.5, scale step: 0.01, Eigenfaces: 10 non-smiling, 45x45 pixels

Unfortunately, I did not have the images (nor time to get or use them) to train on an animal set.

Other Pictures

This was one of my better attempts at IMG_0001. Unfortunately, it was not all that successful; the walls and lights were flesh colored (and thus weren't bypassed by my color checks) and it just so happens that the detector is very attracted to them. In order to start finding any normal amount of faces, I had to display a large number of faces to get beyond the walls/lights. Furthermore, the depth also made it hard to get to all of the faces. On the whole, though, I'm fairly happy with the number of faces I managed to tag in this image.