created 11/25/2007

Image Puzzles — Image Smoothing

This section discusses functions that smooth, sharpen, and find patterns in images.

22. Image Smoothing

A common image processing operation is image smoothing. Smoothing softly blurs the image, rounding off sharp corners, and smoothing over noise. The image becomes slightly out of focus. This is often a visually desirable effect. Here, for example, is the original Kernighan image on the left and a smoothed version on the right:

 

For some applications, the smoothed version looks better. The sharp line of the glasses frame is less intrusive, for example.

There are many ways to smooth an image. A common method is to replace each pixel of the image with the average of the pixel itself and the eight pixels that surround it:

The smoothing operation inputs an original image (which will not be changed) and produces a smoothed image, a new image based on the original. In general, each pixel of the smoothed image is the average of nine pixels from the original image.

For example, in the picture, the pixel at row 2 column 3 in the original image has gray level f. The pixel at the corresponding location in the smoothed image has gray level f' , which is the average of nine pixels from the original.

The neighborhoods are overlapping; that is, each pixel of the original image has a neighborhood centered on it which determines the value of the corresponding pixel in the smoothed image. Think of the neighborhood as moving through the original image one pixel at a time. For example, the next pixel of the smoothed image is the average of the neighborhood centered at row 3 column 3:

 

Details (which you could probably skip):

A more detailed formula for computing the value of a pixel f' in the smoothed image based on a 3x3 neighborhood is:

This may look awful but it is really saying the same thing as above. Let the first sigma (also called a summation sign) pick its first value, row=r-1. Then for that value of row the nested sigma runs through all three values for col producing:

f(r-1, c-1) + f(r-1,c) + f(r-1, c+1)

Next, the first sigma picks another value, row=r and the nested sigma runs through all three values for col again:

+ f(r, c-1) + f(r,c) + f(r, c+1)

Finally, the first sigma picks its last value, row=r+1 and the nested sigma runs through all three values for col yet again:

+ f(r+1, c-1) + f(r+1,c) + f(r+1, c+1)

The net result is that the double-summation gives this sum:

f(r-1, c-1) + f(r-1,c) + f(r-1, c+1) +
f(r,   c-1) + f(r,  c) + f(r,   c+1) +
f(r+1, c-1) + f(r+1,c) + f(r+1, c+1)

You may recognize this as a nested for-loop:

sum = 0;
for ( row=r-1; row<=r+1; row++ )
  for ( col=c-1; col<=c+1; col++ )
    sum += image[row][col];

In a working program you will need to use getPixel() to get each pixel, and you will need to deal with incomplete neighborhoods.

 

23. Incomplete Neighborhoods

The nine pixels from the original image that go into each average are called a 3x3 neighborhood. In general, each pixel f'(row, col) in the smoothed image is the average of a 3x3 neighborhood centered about the location (row, col). But not all locations have a complete 3x3 neighborhood with the location at the center. The picture shows several incomplete neighborhoods:

A complete 3x3 neighborhood cannot be formed for the pixels of the edges of an image. There are three things that can be done about this:

  1. Base the smoothed image on only the complete 3x3 neighborhoods, so that the smoothed image will have two less rows and two less columns that the original.
  2. Use the unchanged values from the original image for the edge pixels of the smoothed image, and use the neighborhood averages for the rest. The smoothed image will be the same size as the original.
  3. Use the average of incomplete neighborhoods for the edge pixels of the smoothed image, and use the averages of complete neighborhoods for the rest.

 

24. Neighborhoods of Various Sizes

For a larger smoothing effect, use a larger neighborhood. In the picture, the neighborhood is now 5x5. The formula shows that in general the value of a pixel f' in the smoothed image is based on a neighborhood centered on a pixel f in the original image. The sigma symbol means to add up all the pixels in a neighborhood. That sum (of 25 values) is divided by the number of pixels that went into the sum (25 in this case).

 

 

It is convenient to make the neighborhood a square, with an odd number of pixels per side so that there is a center pixel. For example, neighborhoods are often 3x3 or 5x5 or 7x7.

 

25. Another View of Smoothing

Another way to view the 3x3 smoothing operation is that it is done by centering a 3x3 mask over each pixel of the original image. Here is a picture that shows this. The mask is a 3x3 region where each pixel holds the value one:

 

The value for each pixel f'(row, col) of the output image is calculated by centering the mask over (row,col) of the input image. Then each pixel of the input image is multiplied by the value in the mask that covers it. For example, the pixel that contains value a has a 1 over it, so a is multiplied by 1. Now all of these products are added up. This sum is then divided by the sum of the values in the mask.

The result of all of this is the same number as with the original view of smoothing explained in section 22. What is the point?

The point is that now each pixel in the output image is the weighted average of the pixels in a neighborhood. This often produces a better looking smoothed image than the original version. For example, in the following, the central pixel of each neighborhood has just as much influence in the output pixel as the remaining 8 pixels of the neighborhood:

Larger smoothing masks can be used to produce various kinds of smoothing. For example, with the following mask

 

The central pixel, the middle ring of pixels and the outer ring have equal weight (20) in the weighted sum. The sum of weighted pixels now would be divided by 60, the sum of the weights in the mask.

 

26. Repeated Smoothing

Rather than smooth an image once with a big mask, repeadedly smooth the image with a small mask. For example, smooth an image I 0 with a 3x3 mask to produce image I 1 , then smooth I 1 with the same mask to produce I 2.

— Return to the main contents page