CSE/EE 576 Spring 2011: Project 1
Color Clustering and Skin Finding

Date released:  Wednesday, March 30, 2011

Date due: Friday, April 15, 2011 11.59pm

(Late policy: 5% off per day late till Sunday, April 17, 2011)
Download the C++ skeleton code for project 1.
Matlab users create your own code, do not use Matlab K-Means function.

Download the images you need:
    1. Faces training set
    2. Faces testing set
    3. Faces training groundtruth set
    4. Faces testing groundtruth
    5. Scenes

Download .pdf template for the report (proj1-report.pdf). You are free to use any text processing tools like latex etc, however make sure that you have the same sections in your report.

Download  project grading guidelines for the project here.

In this assignment, you will segment images by color, using the K-means algorithm and some variants. You will also try to classify the skin areas on the face images using the generated clusters.

face image segmented by color skin pixels highlighted

What You Should Do

Part I: Color Clustering

  1. First implement a basic K-Means Clustering Algorithm (your own code; do not use the MatLab K-Means). Try several color spaces: RGB, normalized RGB, HSV or any other color space.
    1. First with randomly selected cluster seeds
    2. Next try sampling pixels from the image to find the seeds. Choose a pixel and make its value a seed if it is sufficiently different from already-selected seeds. Repeat till you get K different seeds.
    3. Next with a method that you develop for selecting the seeds intelligently from the image using its color histogram (again your code). The seed selection should be automatic, given the histogram and the number of seeds to be selected. One way to go is to find the peaks in the color histogram as candidates for seeds.
  2. Now develop and implement a smarter K-Means variant (your own code again) that determines the best value for K by evaluating statistics of the clusters. Some possible methods to try:
    1. You can start from the color histogram, as K is closely related to the number of peaks in the histogram. Not all the peaks are necessary as you want only the dominant ones, so you should pick the ones that occupies a certain portion of the image in terms of pixels.
    2. You can also try clustering using different Ks, and pick the best one. The metric could be related to the distance between clusters and the variance within each cluster.
    3. You are free to come up with your own ways.

  3. Test each variant of the above on both the following face images and scene images and report your results: face01, face04, face05, face08, face10, face23, face28, s03, s06, s08, s09, s12.
Your k-means code should output an image that can be used to show your clusters. This can be a grayscale image where each pixel's value is the number of the cluster to which it has been assigned, which the provided autocolor function will transform into something more easily interpretable. It could also be a ppm where each pixel has the mean color of the cluster it was assigned to (this generally makes a prettier picture, but it can be harder to tell the number of clusters), or better yet output both.

Part II: Skin Classification

The goal of this part is to develop a very simple skin detector from the results of Part I. For this part of the homework, we recommend that you work in normalized RGB space.

The common RGB representation of color images is not suitable for characterizing skin-color. In the RGB space, the triple component (r, g, b) represents not only color, but also luminance. Luminance may vary across a person's face due to the ambient lighting and is not a reliable measure in separating skin from non-skin regions. Luminance can be removed from the color representation in the normalized RGB space or chromatic color space. Chromatic colors, also known as "pure" colors in the absence of luminance, are defined by the simple normalization process shown below:

r = R/(R+G+B)

g = G/(R+G+B)

Note: r+g+b = 1.

  1. Start with the face training image set (face-training.zip).
  2. Run your K-means algorithm on the face training set to get K clusters with small K, ie K < 9.
  3. Keep the information of the cluster index for each pixel and the cluster centroid  (average r and g value).
  4. Use the groundtruth images (face-training-grountruth.zip) to assign the final cluster label (skin or non skin) for training. You can use majority vote of the pixels in each cluster as the cluster label.
  5. Use a classifier to learn the skin model. You can use WEKA  and test out your model using a Naive Bayes classifier and a number of other classifier. A short WEKA tutorial is provided here  You will need to generate your training and testing data file in the proper ARFF format. The training and testing data file will contain the cluster centroid and the  cluster label (Sample of training and testing ARFF file). You want to obtain a skin model with high classification accuracy.
  6. Test your skin model on the face test image set 
  7. Report on its performance: classification accuracy and include some images as well.

What You Should Turn In

1. All of your code that is created for the above Part I and II. Your code must be well commented and in the ASCII format so that the grader can compile it to working binaries.    
    You should put headers on all your routines with the following information:
2. Write a brief report on the performance of:

Your report must clearly describe and explain the algorithms you developed.Also include some discussions on failure examples or limitations for your approach; this will shed light on future improvements.
This report can be a Word document or  pdf document. HTML or webpages are not accepted. Your report must include output images of your algorithm.

Please email your code and report to Bilge (bilge@cs) in a zip file with your name as the zip file name e.g. JohnDoe.zip, by Friday, April 15, 2011 11.59pm.