This project aimed to create algorithms for finding, recognizing, and verifying faces in images. By the dimensionality of a face using principle component analysis (PCA), we turn the rather difficult problem into a much simpler one. Limiting ourselves to the space of all 'faces' allows us to more easily decide what is a face and what is not, and to recognize faces we've already seen based on previously-computed data.
Unfortunately, I never was succesful in generating a set of correct eigenfaces from the input data. Thus, my results aren't very helpful in terms of evaluating the success or failure of this algorithm to identify faces. I will do my best to justify my results in the context of the bad data.
The first step was to generate a set of eigenfaces from the set of input data. This would allow the program to quickly compare a given image to the set of eigenfaces to determine if it was indeed a face. The idea is that through combining the various computed eigenfaces, any image of a face can be created. This, of course, isn't completely true; we have a very limited number of eigenfaces, and the approximation only works for faces of similar orientation and without any occlusions. Regardless, this method works very well in theory in reducing the dimentionality of the face problem.
As you can see below, the generated eigenfaces seem to be random data, rather than appearing facelike themselves. I haven't really determined the cause, but I assume it was from incorrectly generating the matrix used to solve for the eigenfaces themselves. Because these eigenfaces are incorrect, the rest of my algorithms have very limited success.
| Average | Eigenfaces
|
|
|
|---|
Obviously, as a result of the bad eigenfaces, my algorithm was virtually useless for identifying smiling faces from unsmiling faces. Virtually every face was incorrectly identified as being smiling face 11.tga. (Face 11.tga was correctly identified as face 11.tga.) What is interesting to note is that the same 5 or 6 faces appeared in the recognition order, regardless of which face I was testing. The order of these faces was very close to independent of the face being tested; 13.tga was second for the majority of the faces, 25.tga was third, 05.tga was fourth, and finally 10.tga was fifth. I hypothesize that this was due to the fact that basically any combination of eigenfaces resulted in an image that was essentially just random noise, and this ordering of faces is consistent across most random noise patterns.
| Input (nonsmiling) | Output (smiling) | Input (nonsmiling) | Output (smiling) | Input (nonsmiling) | Output (smiling) | Input (nonsmiling) | Output (smiling)  
| 
| 
| 
01.tga | 11.tga |
| 02.tga | 11.tga |
| 03.tga | 11.tga |
| 04.tga | 11.tga
| MSE: 10171.8 |
| MSE: 12074.5 |
| MSE: 12024.3 |
| MSE: 10573.9
| | |||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
The five faces and the order in which they most commonly occured
|
| 
| 
| 
| 
|
|---|---|---|---|---|
| 11.tga | 13.tga | 25.tga | 05.tga | 10.tga |
Examining the number of eigenfaces used in the computation for face recognition provides more interesting insight as to the problems with the eigenfaces. Using just one eigenface, all the images were identified as 25.tga; using three or five resulted in every image being identified as 13.tga; for 7 or more eigenfaces, the result was always 11.tga. This indicates that 25.tga differs the least from the eigenface with the highest eigenvalue, while 13.tga is closer to the various combinations of 3 and 5 eigenfaces, and 11.tga is closest to combinations of 5 or more eigenfaces.
Regardless of the number of eigenfaces used, the algorithm always only identified a single face correctly. There were several close calls, however. For isntance, at 5 or more eigenfaces, 13.tga was always the 2nd choice for identification of user 13. There was also similar correspondance between 25.tga and identification of user 25, though the algorithm was somewhat less accurate in this case. Thus, there appears to be at least a small amount of correspondance between the eigenfaces and the images from which they were generated.
The final part of this project was to use the program to find faces in arbitrary images. The program could either crop them out of the image or mark them with a green box. Again, due to the poorly-generated eigenfaces, this algorithm couldn't be thoroughly tested. While it does what it is supposed to, it can't find faces without good data.
As you can see below, my algorithm tends to find the top center of the input image. The size of the box it finds is determined by the minimum scale passed to the findFace function. This makes sense because the MSE between an arbitrary subimage and a small random eigenface will be smaller than that of a larger eigenface and subimage. The marks furthermore tend to occur in a row, indicating that the MSE for these areas is very similar or the same. This makes sense as well, since the areas marked by the squares are all nearly identical in color composition and texture. As to why it seems to find the center of each image, I can only guess. Perhaps the MSE for these regions is simply lower. I'm more inclined to believe, however, that I have an indexing problem that causes my algorithm to only check the right part of the image, or to favor these subimages more.
| elf.tga | elf.cropped.tga
| (0.45 to 0.55, step 0.01)
|  
| |||
|---|---|---|---|---|---|
| jaime.tga | jaime.cropped.tga
| (1.00 to 1.50, step 0.05)
|  
| |||
|---|---|---|---|---|---|
| graduation.tga | graduation.marked.tga
| (0.45 to 0.75, step 0.05)
|  
| |||
|---|---|---|---|---|---|
| group1.sm.tga | group1.sm.marked.tga
| (0.50 to 1.00, step 0.05)
|  
| |||
|---|---|---|---|---|---|
| IMG_0031.tga | IMG_0031.marked.tga
| (0.50 to 1.00, step 0.05)
|  
| |||
|---|---|---|---|---|---|
It is very hard to draw any kind of useful conclusion from the data I attained in this project. Without a good set of eignefaces to test against, the majority of the program's functions weren't useful or even testable. With a set of good eigenfaces, the experiments performed would be a lot more conclusive.
That being said, it was evident, even from running these tests, that the various algorithms were fairly slow at computing their results (particularly eigenFaces and findFaces, both of which involved doing a great deal of number crunching). Future improvements to this experiement would definitely include some way to increase the efficiency of the algorithm. One method that comes to mind, for instance, would be using some sort of skin classifier, such as a Baysian test, to determine whether a given pixel should even be considered for evaluation via the face detection algorithm.