How Image Filtering is used to Improve Picture Quality

Image Filtering is used everywhere nowadays. Right from social media applications and news articles to various training models to improve image data. It can be used to remove noise from images as well as smoothen edges.

To understand image filtering, let us first understand image blurring. Starting with Gaussian Blur

Gaussian Blur

In image processing, a Gaussian blur (also known as Gaussian smoothing) is the result of blurring an image by a Gaussian function. The name is derived from the mathematician and scientist Carl Friedrich Gauss and is also known as Gaussian Blurring.

As I stated before, the Gaussian blur is a type of image-blurring filter that uses a Gaussian function (which also expresses the normal distribution in statistics) for calculating the transformation to apply to each pixel in the image. The formula of a Gaussian function in one dimension is

In two dimensions, it is the product of two such Gaussian functions, one in each dimension:

where x is the distance from the origin in the horizontal axis, y is the distance from the origin in the vertical axis, and σ is the standard deviation of the Gaussian distribution. When applied in two dimensions, this formula produces a surface whose contours are concentric circles with a Gaussian distribution from the centre point.

In order to save computational time and power while applying gaussian blur on 2-D images, instead of applying the gaussian matrix directly on the 2-D matrix, it is first applied in one dimension horizontally and then a series of 1-D gaussian matrices are applied vertically. In computational terms, this is a useful property, since the calculation can be performed in O(w(kernel)*w(Image)*h(Image)) + O(h(kernel)*w(Image)*h(Image)) time, as opposed to O(w(kernel)*h(kernel) * w(Image)*h(Image)) for a non-separable kernel. (h is height and w is width)

Gaussian blurring is commonly used when reducing the size of an image.

Box Blur

Box blur is also known as box linear filter. Box blurs are frequently used to approximate Gaussian blur.

A box blur is generally implemented as an image effect that affects the whole screen. The blurred colour of the current pixel is the average of the current pixel’s colour and its 8 neighbouring pixels. Here, the weight is constant and lies within a square (box).

It can be derived from this formula:

br denotes the box radius, which is half the size of the square.

So, if we had to show the box blur visually it would look like this :

In order to implement Box Blur we can use the openCV function boxFilter()

import numpy as np
import cv2
from matplotlib import pyplot as plt
img = cv2.imread("outdoor.jpg", -1);
blurred = np.uint8(cv2.boxFilter(img, cv2.CV_64F, ksize=(5, 5), normalize=True));
f = plt.figure(figsize=(15,15));
f.add_subplot(1, 2, 1).set_title('Original Image');
plt.imshow(img[:,:,::-1])
f.add_subplot(1, 2, 2).set_title('Blurred Image');
plt.imshow(blurred[:,:,::-1]);
plt.show();

The above code yields the following results:

Min and Max Filters

The Minimum filter enhances ‘minimum’ values in the image by increasing its area. Similar to a dilate function each 3x3 (or other window size) is processed for the darkest surrounding pixel. That darkest pixel then becomes the new pixel value at the centre of the window.

Here is a visualisation:

We can implement it is using the PIL library as follows

import numpy as np
from PIL import Image, ImageFilter
from matplotlib import pyplot as plt
img = Image.open("indoor.jpg");
size = 5;
blurred = img.filter(ImageFilter.MinFilter(size=size));
f = plt.figure(figsize=(15,15))
f.add_subplot(1, 2, 1).set_title('Original Image');
plt.imshow(img)
f.add_subplot(1, 2, 2).set_title('Blurred Image');
plt.imshow(blurred)
plt.show();

Similarly, when we enhance the ‘highest’ or ‘maximum’ values of the image, it is called maximum filtering.

An interesting use case of minimum filtering is in astrophotographical images. Many wide field deep sky images have so many, so small, and so bright stars, that non-stellar objects as diffuse nebulae and Milky Way condensations, can be greatly obscured by them. A modulated minimum filter (Amount parameter less than 1) can be applied in these cases to decrease the visual impact of stars, giving non-stellar objects their deserved importance in the scene.

Median and Mode Filtering

Median and Mode filtering are also very useful tools for noise removal. As you probably guessed, it takes the median and mode respectively of the square sections of the images taken.

import numpy as np
import cv2
from matplotlib import pyplot as plt
img = cv2.imread(“imgs/chapter5/outdoor.jpg”, -1);
blurred = cv2.medianBlur(img, 5);
f = plt.figure(figsize=(15,15);
f.add_subplot(1, 2, 1).set_title('Original Image');
plt.imshow(img[:,:,::-1])
f.add_subplot(1, 2, 2).set_title('Blurred Image');
plt.imshow(blurred[:,:,::-1]);
plt.show();

The code above yields the following results:

Similarly, we can implement a mode Filter using the PIL library:

img = img.filter(ImageFilter.ModeFilter(size=size));

Use Case: Pepper and Salt Noise Removal

Now that I have explained various types of filtering/blurring, let us try to understand which one is the best to use in various situations.

Let’s take a use case of removing salt and pepper noise.

import numpy as np
import random
import cv2
from matplotlib import pyplot as plt
from PIL import Image, ImageFilter
#Function to add salt and pepper noise to the image
def sp_noise(image,prob):
output = np.zeros(image.shape,np.uint8)
thres = 1 - prob
for i in range(image.shape[0]):
for j in range(image.shape[1]):
rdn = random.random()
if rdn < prob:
output[i][j] = [0, 0, 0]
elif rdn > thres:
output[i][j] = [255, 255, 255]
else:
output[i][j] = image[i][j]
return output
img = cv2.imread('outdoor.jpg',1) 
noise_img = sp_noise(image,0.05)
#box filter
kernel_size=5;
box_filtered = cv2.blur(img, (kernel_size, kernel_size));
# max filter
temp = Image.fromarray(noise_img, mode="RGB");
max_filtered = temp.filter(ImageFilter.MinFilter(size=kernel_size));
# min filter
temp = Image.fromarray(noise_img, mode="RGB");
min_filtered = temp.filter(ImageFilter.MaxFilter(size=kernel_size));
# median filter
median_filtered = cv2.medianBlur(img, kernel_size);
# mode filter
temp = Image.fromarray(noise_img, mode="RGB");
mode_filtered = temp.filter(ImageFilter.ModeFilter(size=kernel_size));

The code above will take the original image and add salt and pepper noise to it. After adding the noise, we have used various filtering techniques to discover which one achieves best results.

f = plt.figure(figsize=(15,15))
f.add_subplot(4, 2, 1).set_title('Original Image');

plt.imshow(image[:,:,::-1]);
f.add_subplot(4, 2, 2).set_title('Noisy Image');

plt.imshow(noise_img[:,:,::-1]);
f.add_subplot(4, 2, 3).set_title('Box Filtered Image');

plt.imshow(box_filtered[:,:,::-1]);
f.add_subplot(4, 2, 4).set_title('Min Filtered Image');

plt.imshow(min_filtered);
f.add_subplot(4, 2, 5).set_title('Max Filtered Image');

plt.imshow(max_filtered);
f.add_subplot(4, 2, 6).set_title('Median Filtered Image');

plt.imshow(median_filtered[:,:,::-1]);
f.add_subplot(4, 2, 7).set_title('Mode Filtered Image');

plt.imshow(mode_filtered);
plt.show();

Now using matplotlib.pyplot() , we have been able to see the results of the various images as follows:

Hence, we can see that the best results are achieved from median filtered image.

This is one of many ways in which filtering is used for noise removal and to improve image quality. Various filtering techniques can be used to remove various kinds of noise from images.

References:

• http://blog.ivank.net/fastest-gaussian-blur.html
• https://pixinsight.com/doc/legacy/LE/19_morphological/minmax_filter/minmax_filter.html
• https://www.geeksforgeeks.org/box-blur-algorithm-with-python-implementation/
• https://en.wikipedia.org/wiki/Box_blur
• https://warwick.ac.uk/fac/sci/statistics/staff/research_students/ip/postphd/