This page is designed to assist you in completing project 1, Impressionist. The following is a quick tutorial for using a subset of OpenGL as well as to help you understand the basics of image processing -- enough to get you through project 1.
OpenGL - 2D/3D Graphics Library developed at Silicon Graphics Inc. OpenGL has gained widespread industry acceptance in the high-end 3D markets, as well as in the low-end consumer markets. OpenGL's main thrust is as a 3D programming interface. Although it is targeted to the 3D market, it is more than adequate for use in 2D. OpenGL provides fine grained control over images as well as 3D objects. If you plan on continuing on in the computer graphics field after 457, knowing OpenGL is a must.
Basic features:
Other features:
OpenGL has its own definitions for variable types, and you should use these whenever you're writing in OpenGL. They are simply re-definitions of the basic types; GLint is simply an int. Why do you want to use them then? OpenGL programmers reserve the right to change the way OpenGL handles data internally. If you use their variables, then they ensure that the changes will be invisible to your program. If you don't, then the float that you send to glColor3f might not work in a future release. The following table shows the OpenGL variables and their corresponding data types.
| OpenGL Type Definition | Data Type |
| GLbyte | signed char |
| GLshort | short |
| GLint, GLsizei | int or long |
| GLfloat, GLclampf | double |
| GLubyte, GLboolean | unsigned char |
| GLushort | unsigned short |
| GLuint, GLenum, GLbitfield | unsigned int or unsigned long |
All the functions in the OpenGL library have names beginning with "gl". Defined constants have names beginning with "GL_".
Since C doesn't allow function overloading, it common for there to be a family of functions for performing the same operation, differing only in the number and/or types of arguments accepted by each. The names of these functions have tags at the end to indicate the type of argument. The whole family will be referred to with a star syntax. For instance, glColor*() refers to any of the 32 functions available within OpenGL for setting the current color, including:
| glColor3f( GLfloat, GLfloat, GLfloat ) -Takes 3 floats. |
| glColor4d( GLdouble, GLdouble, GLdouble, GLdouble ) - Takes 4 doubles (the fourth is the alpha value) |
| glColor3ubv( GLubyte* ) - Takes a vector (or array) containing 3 unsigned bytes. |
OpenGL comes with a library of convenience functions called the glu library. All of the function names begin with glu.
We will use OpenGL in the Impressionist program to draw the various brush strokes. We will mainly stick to drawing lines and points. If you choose to do so, you can make fancier brushes using some of the other primitives (such as triangles, squares, and general polygons) for extra bells and whistles. Here's a brief introduction (you'll find more in the man pages and reference manuals)
void glBegin( GLenum mode )
Example Code -- primitives: The following bits of code show you how draw single-pixel dots, a triangle outline (see OpenGL manual for other ways to do this), and a filled triangle (as a three-sided polygon).
glColor3f( red, green, blue );
// drawing a single-pixel dot to the
// window, at pixel coordinate (x,y)
glBegin( GL_POINTS );
glVertex2i( x, y );
glEnd();
// drawing an outlined triangle (many ways to
// do this) having vertices A, B, and C
glBegin( GL_LINE_STRIP );
glVertex2i( Ax, Ay );
glVertex2i( Bx, By );
glVertex2i( Cx, Cy );
glVertex2i( Ax, Ay );
glEnd();
// drawing a filled triangle having
// vertices A, B, and C
glBegin( GL_POLYGON );
glVertex2i( Ax, Ay );
glVertex2i( Bx, By );
glVertex2i( Cx, Cy );
glEnd();
glFlush(); // don't forget this!
In the Impressionist project, you will have to control various aspects of the brush strokes, such as their position and orientation. Since you will be using OpenGL to complete this project, you will find it very beneficial to know a little bit about how "transformations" (shifting, rotation, scaling, etc,) are done in OpenGL.
Recall that OpenGL is basically a state machine. For many aspects of it, you setup certain parameters, and until you change them, GL will use those parameters for everything it draws.
You have probably already seen how this is used for things like object color (via glColor) and drawing mode (via glBegin / glEnd). However, there are also state variables for things like position (accomplished via "translation", or shifting) and direction (accomplished via rotation).
For example, you can call glRotate* to set the rotation state. If you tell OpenGL to rotate 45 degrees around the z axis (with glRotate3f( 45, 0.0, 0.0, 1.0) ), then everything you draw will be rotated 45 degrees before it's drawn to the screen. This is a quick and easy way of changing the location and orientation of an object.
So how does this apply to Impressionist? Recall that you'll be drawing various brush strokes on a digital canvas, each at a different position and in a different direction. By setting the GL state variables for position and orientation before drawing your brush, you can use the same code regardless of the brush's position or orientation. The stroke will automatically be drawn at the correct position and in the correct direction!
Here are the OpenGL calls needed to do some simple image transformations.
There are several matrices in OpenGL. Amongst them are the projection matrix, to control the camera position, and the modelview matrix, to control drawing. We want to use the modelview matrix, so we need to explicitly tell that to OpenGL with a call to glMatrixMode:
glMatrixMode(GL_MODELVIEW);
Image transformations are accomplished using matrices. A series of matrices are multiplied to produce a given image transformation. Without going into too much detail, let's just say that you'll want to save your current transformation matrix ("push" it onto a matrix stack) before doing your brush-specific translation/rotation, and restore the original matrix ("pop" it off the matrix stack) when you're done with that brush stroke.
Here are the calls you need to use.
glPushMatrix(); <&<Do the translation, rotation>> <&<Draw the brush stroke>> glPopMatrix();
To draw at a certain point, you'll want to "translate" your origin to that position. Here is the call you use in OpenGL to do 2D translation:
glTranslate*( startX, startY, 0.0 );
(Note: the * is replaced by a letter that depends on the type of the parameters you pass.)
You can also take advantage of OpenGL to automatically rotate the things you draw. Here is the OpenGL call you'll want to use:
glRotate*( angle, 0.0, 0.0, 1.0 );
(Note: we use 0.0, 0.0, 1.0 because we want to rotate around the z-axis.)
To help you wade through all of the files we give you for Impressionist, here is a brief list of the more important classes and what they're used for:
To implement some of the requirements of this project, you will need to know a little about image processing. More specifically, this document will cover some of the aspects of edge detection and image gradients.
Often in image processing, we apply an operation to just a small region of an image at a time. One of the most common techniques used for applying such operations is by using something called a filter. Diffenent image filters are distinguished by their kernel and their support. Basically, you can think of a filter kernel as a grid of values that we overlay on top of the pixels of an image. Here is an example of a kernel:
| 0 | 1 | 1 | 1 | 0 |
| 1 | 1 | 2 | 1 | 1 |
| 1 | 2 | 3 | 2 | 1 |
| 1 | 1 | 2 | 1 | 1 |
| 0 | 1 | 1 | 1 | 0 |
We apply an operation to a pixel in the image as follows. First, imagine placing the center of the kernel on top of the pixel of interest. Multiply the value at each position in the kernel by the color of the pixel underneath that position. Then, add up all these products, and divide by the sum of the values in the kernel (if it's nonzero).
We can do many interesting things by changing the values in the kernel. The rest of this section will explain how kernels and other image manipulation methods are used to find edges and gradients in an image.
The first step in finding edges and gradients in an image is to blur the image. (Note: for finding edges and image gradients, we use a grayscale version of the original image.) Blurring reduces the "noise" (randomness) in the picture, and allows us to find edges and determine image gradients more accurately.
To get a grayscale version of the image, we take the average of the red, green, and blue channels. However we use the following weights:
Intensity = 0.299R + 0.587G + 0.144B
Why do we use these weights? Well, the human eye is better at detecting some colors than others. It picks up green the best, followed by red then blue. We're interested in producing an image that represents what the eye thinks the intensity of light is at each pixel, and this weighting achieves that effect.
There are several choices for the filter kernel to use for blurring image. We will present three of them here.
In general, which filter you should use depends on the image you're manipulating and what you want to accomplish. The box filter is the easiest to implement, but the Bartlett and Gaussian filters often produce "better" blurring results.
Once you have blurred the grayscale version of the original image, you are ready to do edge detection. The method of edge detection we present here is known as Sobel edge detection.
To do Sobel edge detection, we once again use filters. This time we make two passes over the image, using a different kernel each time. Here are the filter kernels that are used.
1 |
2 |
1 |
0 |
0 |
0 |
-1 |
-2 |
-1 |
1 |
0 |
-1 |
2 |
0 |
-2 |
1 |
0 |
-1 |
By using both these filter kernels on each pixel in the image, we get two new values at each pixel location (call them temp1 and temp2). We then produce a single value at each location using this formula: sqrt( temp1*temp1 + temp2*temp2 ). (You may notice a striking similarity to the distance formula used in mathematics.) Finally, we say that if this single value is above a certain threshold (usually specified by the user), then there is an edge at that location.
Given a blurred image, we can also produce the gradients for an image. To intuitively understand what the image gradient is, recall what a gradient in is in mathematics --- the direction of maximum rate of change. For a surface (such as an array of pixels), the gradient is the direction in which the color changes the fastest (and perpendicular to the gradient, the color changes the slowest --- i.e., it's close to the same color).
Why is this important to us (and important for the Impressionist program, in particular)? Well, if we know (at each pixel) the direction in which the color is changing the least, we can orient our brush strokes in that direction to make it more realistic. (Because if we paint a stroke in a certain direction, the color of the paint stays the same during that stroke!)
So how do we implement this? If you answer contained the word "filters," you're on the right track. We will actually produce two gradients, the x-gradient (telling how fast the color changes along each row), and the y-gradient (telling how fast the color changes along each column). Here are the kernels that are used.
x-gradient filter kernel:
-1 |
1 |
y-gradient filter kernel:
1 |
-1 |
Using these kernels gives us two separate images, one of which tells us how fast the color changes (at each pixel) in the x-direction, and one telling us how fast it changes in the y-direction. Finally, we use this information to determine the direction of the maximum rate of change in color, and we orient our brush strokes perpendicular to that direction.
Send questions or comments to Kevin Audleman (kforbes@cs.washington.edu)