Image Filtering is used everywhere nowadays. Right from social media applications and news articles to various training models to improve image data. It can be used to remove noise from images as well as smoothen edges.
To understand image filtering, let us first understand image blurring. Starting with Gaussian Blur
In image processing, a Gaussian blur (also known as Gaussian smoothing) is the result of blurring an image by a Gaussian function. The name is derived from the mathematician and scientist Carl Friedrich Gauss and is also known as Gaussian Blurring.
As I stated before, the Gaussian blur is a type of image-blurring filter that uses a Gaussian function (which also expresses the normal distribution in statistics) for calculating the transformation to apply to each pixel in the image. The formula of a Gaussian function in one dimension is
In two dimensions, it is the product of two such Gaussian functions, one in each dimension:
where x is the distance from the origin in the horizontal axis, y is the distance from the origin in the vertical axis, and σ is the standard deviation of the Gaussian distribution. When applied in two dimensions, this formula produces a surface whose contours are concentric circles with a Gaussian distribution from the centre point.
In order to save computational time and power while applying gaussian blur on 2-D images, instead of applying the gaussian matrix directly on the 2-D matrix, it is first applied in one dimension horizontally and then a series of 1-D gaussian matrices are applied vertically. In computational terms, this is a useful property, since the calculation can be performed in O(w(kernel)*w(Image)*h(Image)) + O(h(kernel)*w(Image)*h(Image)) time, as opposed to O(w(kernel)*h(kernel) * w(Image)*h(Image)) for a non-separable kernel. (h is height and w is width)
Gaussian blurring is commonly used when reducing the size of an image.
Box blur is also known as box linear filter. Box blurs are frequently used to approximate Gaussian blur.
A box blur is generally implemented as an image effect that affects the whole screen. The blurred colour of the current pixel is the average of the current pixel’s colour and its 8 neighbouring pixels. Here, the weight is constant and lies within a square (box).
It can be derived from this formula:
br denotes the box radius, which is half the size of the square.
So, if we had to show the box blur visually it would look like this :
In order to implement Box Blur we can use the openCV function
import numpy as npimport cv2from matplotlib import pyplot as pltimg = cv2.imread("outdoor.jpg", -1);blurred = np.uint8(cv2.boxFilter(img, cv2.CV_64F, ksize=(5, 5), normalize=True));f = plt.figure(figsize=(15,15));f.add_subplot(1, 2, 1).set_title('Original Image');plt.imshow(img[:,:,::-1])f.add_subplot(1, 2, 2).set_title('Blurred Image');plt.imshow(blurred[:,:,::-1]);plt.show();
The above code yields the following results:
Min and Max Filters
The Minimum filter enhances ‘minimum’ values in the image by increasing its area. Similar to a dilate function each 3x3 (or other window size) is processed for the darkest surrounding pixel. That darkest pixel then becomes the new pixel value at the centre of the window.
Here is a visualisation:
We can implement it is using the PIL library as follows
import numpy as npfrom PIL import Image, ImageFilterfrom matplotlib import pyplot as pltimg = Image.open("indoor.jpg");size = 5;blurred = img.filter(ImageFilter.MinFilter(size=size));f = plt.figure(figsize=(15,15))f.add_subplot(1, 2, 1).set_title('Original Image');plt.imshow(img)f.add_subplot(1, 2, 2).set_title('Blurred Image');plt.imshow(blurred)plt.show();
Similarly, when we enhance the ‘highest’ or ‘maximum’ values of the image, it is called maximum filtering.
An interesting use case of minimum filtering is in astrophotographical images. Many wide field deep sky images have so many, so small, and so bright stars, that non-stellar objects as diffuse nebulae and Milky Way condensations, can be greatly obscured by them. A modulated minimum filter (Amount parameter less than 1) can be applied in these cases to decrease the visual impact of stars, giving non-stellar objects their deserved importance in the scene.
Median and Mode Filtering
Median and Mode filtering are also very useful tools for noise removal. As you probably guessed, it takes the median and mode respectively of the square sections of the images taken.
import numpy as np
from matplotlib import pyplot as pltimg = cv2.imread(“imgs/chapter5/outdoor.jpg”, -1);
blurred = cv2.medianBlur(img, 5);f = plt.figure(figsize=(15,15);
f.add_subplot(1, 2, 1).set_title('Original Image');
plt.imshow(img[:,:,::-1])f.add_subplot(1, 2, 2).set_title('Blurred Image');
The code above yields the following results:
Similarly, we can implement a mode Filter using the PIL library:
img = img.filter(ImageFilter.ModeFilter(size=size));
Use Case: Pepper and Salt Noise Removal
Now that I have explained various types of filtering/blurring, let us try to understand which one is the best to use in various situations.
Let’s take a use case of removing salt and pepper noise.
import numpy as npimport randomimport cv2from matplotlib import pyplot as pltfrom PIL import Image, ImageFilter#Function to add salt and pepper noise to the imagedef sp_noise(image,prob):output = np.zeros(image.shape,np.uint8)thres = 1 - probfor i in range(image.shape):for j in range(image.shape):rdn = random.random()if rdn < prob:output[i][j] = [0, 0, 0]elif rdn > thres:output[i][j] = [255, 255, 255]else:output[i][j] = image[i][j]return outputimg = cv2.imread('outdoor.jpg',1) noise_img = sp_noise(image,0.05)#box filterkernel_size=5;box_filtered = cv2.blur(img, (kernel_size, kernel_size));# max filtertemp = Image.fromarray(noise_img, mode="RGB");max_filtered = temp.filter(ImageFilter.MinFilter(size=kernel_size));# min filtertemp = Image.fromarray(noise_img, mode="RGB");min_filtered = temp.filter(ImageFilter.MaxFilter(size=kernel_size));# median filtermedian_filtered = cv2.medianBlur(img, kernel_size);# mode filtertemp = Image.fromarray(noise_img, mode="RGB");
mode_filtered = temp.filter(ImageFilter.ModeFilter(size=kernel_size));
The code above will take the original image and add salt and pepper noise to it. After adding the noise, we have used various filtering techniques to discover which one achieves best results.
f = plt.figure(figsize=(15,15))f.add_subplot(4, 2, 1).set_title('Original Image');
plt.imshow(image[:,:,::-1]);f.add_subplot(4, 2, 2).set_title('Noisy Image');
plt.imshow(noise_img[:,:,::-1]);f.add_subplot(4, 2, 3).set_title('Box Filtered Image');
plt.imshow(box_filtered[:,:,::-1]);f.add_subplot(4, 2, 4).set_title('Min Filtered Image');
plt.imshow(min_filtered);f.add_subplot(4, 2, 5).set_title('Max Filtered Image');
plt.imshow(max_filtered);f.add_subplot(4, 2, 6).set_title('Median Filtered Image');
plt.imshow(median_filtered[:,:,::-1]);f.add_subplot(4, 2, 7).set_title('Mode Filtered Image');
matplotlib.pyplot() , we have been able to see the results of the various images as follows:
Hence, we can see that the best results are achieved from median filtered image.
This is one of many ways in which filtering is used for noise removal and to improve image quality. Various filtering techniques can be used to remove various kinds of noise from images.