Computer Vision (CSE 490CV/EE 400B), Winter 2002

Project 0: Image filtering

Assigned: Friday, Jan 11, 2002
Due: Thursday, Jan 17, 2002 (by 11:59pm)

Goal

To implement and get experience with image filtering--a basic image processing operation.
To get comfortable with image data structures and user-interface programming with FLKT.

Description

You will implement a simple window-based image filtering tool, called imgflt. This tool allows you to load an image and apply any 5x5 cross-correlation filter to it. The filter may be applied to the entire image, or "brushed" on to any selected region.

You will be given a working skeleton program, which provides the user interface that you'll need for this program. Specifically, it allows you to load in an image and specify any 5x5 filter by typing in the filter coefficients, a scale factor which multiplies each filter coefficient, and an offset which is added to the pixel before displaying. The interface supports loading and saving of both images and filters. You can also change the brush size. For a complete description of which mouse button does what, run the program and select the "Help" menu item.

We have provided a sample solution executable and test images. Try this out to see how your program should run. Pay special attention to the behavior of the brush. It should apply the filter to each pixel only once, even if the brush is moved over that pixel multiple times. This may be accomplished by copying the image to a temporary image which is the one currently displayed. Filtering operations affect the temporary image but leave the original image intact. When you click "Accept" in the filter window, the changes are committed to the original image.

You will write the code to do cross-correlation filtering. First, however, you should check out the online documentation for FLTK 1.0, a cross-platform user interface and graphics API that we'll be using in this class.

Skeleton Code

You can download the skeleton files. The code is compiled in Visual C++ 6.0, and organized by the workspace file imgflt.dsw. All the source files (.h/.cpp) are in the subdirectory src. There are 21 files total, but many are just user interface files that are automatically generated by fluid, the FLTK user interface building tool. Here is a description of what's in there:

ImageLib/*.cpp/h are files taking care of image file I/O. The only image format we support in this tool is targa (.tga) format, which is also readable in Photoshop and most other image viewing and editing tools. One reason we like tga is that it's possible to store a transparency (alpha) channel with the RGB image data, in 32 bit mode. You shouldn't have to change these files.
BrushConfigUI.h/cpp, FltDesignUI.h/cpp, ImgFilterUI.h/cpp, and HelpPageUI.h/cpp define four types of windows used in the tool. They are generated by fluid through BrushConfigUI.fl, FltDesignUI.fl, ImgFilterUI.fl, and HelpPageUI.fl. You don't have to worry too much about them if you don't want to change the structures of the windows. If you do decide to change the UI, you may prefer using fluid to do so.
imgflt.h is a header file used in most of the files.
FltAux.h defines a few auxiliary structures/routines for the application.
ImgFltMain.cpp is the main file where you want to start reading.
correlation.h/cpp are the files where your image filtering code will go.
ImgView.h/cpp are files where most of the UI requests are implemented. In ImgView.cpp, you want to finish handle() so that it applies the brush filter when the left mouse button is pressed and moved.

To Do

Get familiar with FLTK.
Read the skeleton files, especially ImgFltMain.cpp, correlation.h/cpp, and imgView.h/cpp.
In correlation.cpp, implement image_filter(...) function, which carries out 2D image filtering.
In correlation.cpp, implement pixel_filter(...) function, which carries out 2D image filtering only for a specified pixel (used for the brush). You could call pixel_filter from image_filter while you are debugging, but it will run slowly since you're making a function call for every pixel. In your final version, you will want to inline the pixel filtering code in image_filter.
In ImgView.cpp, finish the handle(...) function, such that when you drag the mouse with left button down, the filter is applied wherever the mouse moves. Make sure that if the mouse moves to the same position twice, the filter is applied only once. After the result is accepted any region may be filtered again.
In ImgFltMain.cpp, generate help info about how to use your application in my_img_filter(...) if you added any new features.

Data Structure

The main data structures for this projects are four memory buffers:
unsigned char *imgBuf;
imgBuf points to a memory of size [3*W*H], which saves an image of W columns and H rows sequentially from left to right and from top to bottom. Each pixel has three color R,G,B. The arrangement of the buffer is a sequence of R,G,B colors. The RGB of top left pixel occupies the first 3 bytes in imgBuf, and the top row of W pixels occupy the first 3*W bytes, and so on. For example, the pixel at column i and row j is :
imgBuf[3*(j*W+i)+0],imgBuf[3*(j*W+i)+1],imgBuf[3*(j*W+i)+2].

double *origImg,*curImg;
origImg and curImg save two images of the same size as imgBuf does. The difference is they two represent colors in floating point type for filtering while imgBuf uses unsigned char for displaying. origImg initially is loaded with an original image. Any following filter operations will cache the results in curImg for preview purpose. If the operations are accepted (by a button in Filter Design Panel), the content in curImg is copied to origImg. If the operations are cancelled, the content in origImg is copied to curImg. imgBuf is kept consistent with curImg, up to rounding errors after clampping to [0,255], for displaying purpose.

unsigned char *brushSelection;
brushSeletion is a buffer of size W*H. It is used to indicate where the filter has been applied in brush mode. For example, if the filter has been applied to pixel (col,row), then
brushSelection[row*imgWidth+col] = 1;
Otherwise,
brushSelection[row*imgWidth+col] = 0.
About Brush Mode:
Square Brush
When the mouse moves to position (col,row), you should apply filter to all the pixels (i,j) inside the square around the mouse, defined as:
|i-col| < brushSize AND |j-row| < brushSize.

Round Brush
When the mouse moves to position (col,row), you should apply filter to all the pixels (i,j) inside the cirle around the mouse, of radius brushSize, defined as: (i-col)*(i-col)+(j-row)*(j-row) < brushSize*brushSize.

Brush Opacity
brushOpacity is a floating number between 0 and 1, which is used to attenuate the pixel colors that have been filtered in brush mode. For example, if pixel (i,j) has color (R,G,B) and is filtered in brush mode, e.g., brushSelection[j*imgWidth+i] = 1, then pixel (i,j) will be drawn as (R*brushOpacity, G*brushOpacity, B*brushOpacity) instead of its original color (R,G,B).
About TARGA image format:
If you look at the ImageLib files, images can contain tags that control how border padding is handled (zero, replicate, reflect). You only need to support one of these options and can hard-code it into your program.

We're not expecting anything fancy beyond the basic requirements on this project, although you're welcome to add to it if you'd like. We probably won't be giving out extra credit on this project since it's meant to help people get up to speed quickly with the tools and data structures.

Correction

For those of you who have already started working on the skeleton codes and are confused about the square and round brush features, I have updated a new ImgView.cpp file for you to use. This file is almost the same as its old version, except that I put more comments and hints for TO DO 3. You can download it and insert it your imgflt workspace. (The current link above to skeleton codes has also been updated to contain this new imgView.cpp)

For those of you who read the code deeply, do not worry about the attributes of TARGA image format if they bug you. The current code is doing fine.

Last modified on January 09, 2002