CSEP 576 Project 4 -- Eigen FacesAdditional Methods Implemented1. Recognize face with skin color informationUsing only eigen faces result in not so satisfying performance on the test group images. A lot of times it chooses some area around clothes, walls etc that accidentally look like a face from its view. Those areas usually differs greatly in color as to skin color. Thus I decided it might be helpful if I could use skin color information, together with eigen faces to recognize faces. To start with, for each pixel, I assign it a color vector: CV = (R, G, B)If a pixel's color is too dark (My threshold is |CV| < 10), I throw it away, because it might be very difficult to know its real color in that case. Then I denote each pixel's adjust color vector as: CV' = CV / |CV|That is, CV' is a normalized unit vector. But its direction in 3D space reflects the actual color of the pixel, regardless the lighting condition. I define the average normalized color vector in an area as normalized sum of all CV' of pixels. For all training faces, compute the average color. For the training examples in this class (use argument: --facecolor nonsmiling_cropped/list.txt), I got: AverageFaceColorVec = (0.80210, 0.52109, 0.29170) Then in findFace, for each possible face area, I compute its average color as FaceColorVec, and compute the cosine theta between AverageFaceColorVec and FaceColorVec, times the MSE computed by using eigen faces with: (1 - AverageFaceColorVec * FaceColorVec) + 0.001 The 0.001 is an experimental number to avoid relying too much on the color when the difference between FaceColorVec and AverageFaceColorVec is too small. This method greatly improved face finding accuracy. I have done some comparison in below for group face recognition. The method above successfully recognized all class group images, while eigen faces alone usually detects only 1-2 faces out of 4 in a picture. To calculate average face color from a group of picture, use following arguments: --facecolor nonsmiling_cropped/list.txt 2. Verify FaceI have implemented verifyFace. The implementation is pretty straightforward. To find which MSE is best, I added a command verifyfacemse, given the userbase and a list of smiling iamges, find the best MSE that minimizes sum of false positives and false negatives. --verifyfacemse smiling_cropped/list.txt base6.user eig6.faceI compute a matrix of MSEs, each row is the smiling image's MSE against all non-smiling images. Thus the diagonal of the matrix is the MSE of the smiling image against its own non-smiling image. To find out the best MSE, I use every possible MSE in the matrix, as the threshold to verify face, and find the one that maximizes total correct result. Following is the result:
3. Morph FaceI have implemented morphFace. To speed up batch generating, I add a command morphfacesbatch. Following is an example of this command: --morphfacesbatch smiling_cropped/s20.tga smiling_cropped/s08.tga eig10_40.face -1000 1000 20 morph\morphIt will generate distance between -1000 to 1000, with step = 20, and file name will be something like morph\morph_00000.tga, like morph\morph_00001.tga etc. Following is some pictures and Video I generated:
Test Report 1 - Recognition1. Eigen Faces
2. Face recognition accuracyI have run face recogintion over different configurations. Following is the table lists the number of correctly recognized faces over number of eigen faces and size of eigen face: 
I have also added a command to recognize batch faces, as following format: --recognizefacebatch <textfile image list> <userbase file> <eigenfaces file> <matches>
Example:
--recognizefacebatch smiling_cropped/list.txt base.user eig10.face 1
It will print out how many faces were correctly recognized at the end.
3. Trend analysis (Question 1)From the chart, we could see that accuracy reaches optimal while eigen faces are more than 9. Also, face size of 20X20 or 15X15 have better accuracy than 10X10 or 25X25 face sizes. It appears that choosing 20X20 with 9 eigen faces is good enough for this test. However, because we have a very limited user base, this may not always be good for general cases. 4. Some error examples (Question 2)
The first incorrectly identified image as shown above, the big open mouth is probably the cause of the mistake. Even from a human eye, without the glasses, it's hard to identify this picture. The second one is not such an unforgivable mistake, consider both of them have beards. The third one might be caused by different head position. Test Report 2 - Face finding1. Finding Bush's face
2. Finding my own faceMin scale: 0.61. Max scale: 0.61. Following are the top 3 faces found. Actually the top 1 face it finds is not a face. However this is the best I could get.
3. Other group imageAll group images are generated by Min: 0.60. Max: 0.70. Step: 0.03. I have done two tests for each picture, one with the method mentioned above, and one with eigen faces only. We can see that with eigen faces alone it sometimes decides that some low textured area on clothes and wall as faces. While with skin color information used, all class group images are recognized correctly.
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
