| OpenCV 5.0.0alpha Open Source Computer Vision

Loading...
Searching...
No Matches
Smoothing Images

Prev Tutorial: Random generator and text with OpenCV
Next Tutorial: Eroding and Dilating

Original author | Ana Huamán
Compatibility | OpenCV >= 3.0

# Goal

In this tutorial you will learn how to apply diverse linear filters to smooth images using OpenCV functions such as:

- blur()
- GaussianBlur()
- medianBlur()
- bilateralFilter()

# Theory

Note
  The explanation below belongs to the book Computer Vision: Algorithms and Applications by Richard Szeliski and to LearningOpenCV

- Smoothing, also called blurring, is a simple and frequently used image processing operation.
- There are many reasons for smoothing. In this tutorial we will focus on smoothing in order to reduce noise (other uses will be seen in the following tutorials).

- To perform a smoothing operation we will apply a filter to our image. The most common type of filters are linear, in which an output pixel's value (i.e. \(g(i,j)\)) is determined as a weighted sum of input pixel values (i.e. \(f(i+k,j+l)\)) :

  \[g(i,j) = \sum_{k,l} f(i+k, j+l) h(k,l)\]

  \(h(k,l)\) is called the kernel, which is nothing more than the coefficients of the filter.

  It helps to visualize a filter as a window of coefficients sliding across the image.

- There are many kind of filters, here we will mention the most used:

## Normalized Box Filter

- This filter is the simplest of all! Each output pixel is the mean of its kernel neighbors ( all of them contribute with equal weights)

- The kernel is below:

  \[K = \dfrac{1}{K_{width} \cdot K_{height}} \begin{bmatrix} 1 & 1 & 1 & ... & 1 \\ 1 & 1 & 1 & ... & 1 \\ . & . & . & ... & 1 \\ . & . & . & ... & 1 \\ 1 & 1 & 1 & ... & 1 \end{bmatrix}\]

## Gaussian Filter

- Probably the most useful filter (although not the fastest). Gaussian filtering is done by convolving each point in the input array with a Gaussian kernel and then summing them all to produce the output array.
- Just to make the picture clearer, remember how a 1D Gaussian kernel look like?

Assuming that an image is 1D, you can notice that the pixel located in the middle would have the biggest weight. The weight of its neighbors decreases as the spatial distance between them and the center pixel increases.

Note
  Remember that a 2D Gaussian can be represented as :

  \[G_{0}(x, y) = A e^{ \dfrac{ -(x - \mu_{x})^{2} }{ 2\sigma^{2}_{x} } + \dfrac{ -(y - \mu_{y})^{2} }{ 2\sigma^{2}_{y} } }\]

  where \(\mu\) is the mean (the peak) and \(\sigma^{2}\) represents the variance (per each of the variables \(x\) and \(y\))

## Median Filter

The median filter run through each element of the signal (in this case the image) and replace each pixel with the median of its neighboring pixels (located in a square neighborhood around the evaluated pixel).

## Bilateral Filter

- So far, we have explained some filters which main goal is to smooth an input image. However, sometimes the filters do not only dissolve the noise, but also smooth away the edges. To avoid this (at certain extent at least), we can use a bilateral filter.
- In an analogous way as the Gaussian filter, the bilateral filter also considers the neighboring pixels with weights assigned to each of them. These weights have two components, the first of which is the same weighting used by the Gaussian filter. The second component takes into account the difference in intensity between the neighboring pixels and the evaluated one.
- For a more detailed explanation you can check this link

# Code

- What does this program do?
  - Loads an image
  - Applies 4 different kinds of filters (explained in Theory) and show the filtered images sequentially

C++

- Downloadable code: Click here
- Code at glance:
  #include <iostream>
  #include "opencv2/imgproc.hpp"
  #include "opencv2/imgcodecs.hpp"
  #include "opencv2/highgui.hpp"
  using namespace std;
  using namespace cv;
  int DELAY_CAPTION = 1500;
  int DELAY_BLUR = 100;
  int MAX_KERNEL_LENGTH = 31;
  Mat src; Mat dst;
  char window_name[] = "Smoothing Demo";
  int display_caption( const char* caption );
  int display_dst( int delay );
  int main( int argc, char ** argv )
  {
  namedWindow( window_name, WINDOW_AUTOSIZE );
  const char* filename = argc >=2 ? argv[1] : "lena.jpg";
  src = imread( samples::findFile( filename ), IMREAD_COLOR );
  if (src.empty())
  {
  printf(" Error opening image\n");
  printf(" Usage:\n %s [image_name-- default lena.jpg] \n", argv[0]);
  return EXIT_FAILURE;
  }
  if( display_caption( "Original Image" ) != 0 )
  {
  return 0;
  }
  dst = src.clone();
  if( display_dst( DELAY_CAPTION ) != 0 )
  {
  return 0;
  }
  if( display_caption( "Homogeneous Blur" ) != 0 )
  {
  return 0;
  }
  for ( int i = 1; i < MAX_KERNEL_LENGTH; i = i + 2 )
  {
  blur( src, dst, Size( i, i ), Point(-1,-1) );
  if( display_dst( DELAY_BLUR ) != 0 )
  {
  return 0;
  }
  }
  if( display_caption( "Gaussian Blur" ) != 0 )
  {
  return 0;
  }
  for ( int i = 1; i < MAX_KERNEL_LENGTH; i = i + 2 )
  {
  GaussianBlur( src, dst, Size( i, i ), 0, 0 );
  if( display_dst( DELAY_BLUR ) != 0 )
  {
  return 0;
  }
  }
  if( display_caption( "Median Blur" ) != 0 )
  {
  return 0;
  }
  for ( int i = 1; i < MAX_KERNEL_LENGTH; i = i + 2 )
  {
  medianBlur ( src, dst, i );
  if( display_dst( DELAY_BLUR ) != 0 )
  {
  return 0;
  }
  }
  if( display_caption( "Bilateral Blur" ) != 0 )
  {
  return 0;
  }
  for ( int i = 1; i < MAX_KERNEL_LENGTH; i = i + 2 )
  {
  bilateralFilter ( src, dst, i, i*2, i/2 );
  if( display_dst( DELAY_BLUR ) != 0 )
  {
  return 0;
  }
  }
  display_caption( "Done!" );
  return 0;
  }
  int display_caption( const char* caption )
  {
  dst = Mat::zeros( src.size(), src.type() );
  putText( dst, caption,
  Point( src.cols/4, src.rows/2),
  FONT_HERSHEY_COMPLEX, 1, Scalar(255, 255, 255) );
  return display_dst(DELAY_CAPTION);
  }
  int display_dst( int delay )
  {
  imshow( window_name, dst );
  int c = waitKey ( delay );
  if( c >= 0 ) { return -1; }
  return 0;
  }
  cv::Mat
  n-dimensional dense array class
  Definition mat.hpp:951
  cv::Mat::clone
  CV_NODISCARD_STD Mat clone() const
  Creates a full copy of the array and the underlying data.
  cv::Mat::size
  MatSize size
  Definition mat.hpp:2448
  cv::Mat::cols
  int cols
  Definition mat.hpp:2425
  cv::Mat::empty
  bool empty() const
  Returns true if the array has no elements.
  cv::Mat::rows
  int rows
  the number of rows and columns or (-1, -1) when the matrix has more than 2 dimensions
  Definition mat.hpp:2425
  cv::Mat::type
  int type() const
  Returns the type of a matrix element.
  cv::Point_< int >
  cv::Scalar_< double >
  cv::Size_
  Template class for specifying the size of an image or rectangle.
  Definition types.hpp:338
  cv::imshow
  void imshow(const String &winname, InputArray mat)
  Displays an image in the specified window.
  cv::waitKey
  int waitKey(int delay=0)
  Waits for a pressed key.
  cv::putText
  void putText(InputOutputArray img, const String &text, Point org, int fontFace, double fontScale, Scalar color, int thickness=1, int lineType=LINE_8, bool bottomLeftOrigin=false)
  Draws a text string.
  cv::medianBlur
  void medianBlur(InputArray src, OutputArray dst, int ksize)
  Blurs an image using the median filter.
  cv::blur
  void blur(InputArray src, OutputArray dst, Size ksize, Point anchor=Point(-1,-1), int borderType=BORDER_DEFAULT)
  Blurs an image using the normalized box filter.
  cv::bilateralFilter
  void bilateralFilter(InputArray src, OutputArray dst, int d, double sigmaColor, double sigmaSpace, int borderType=BORDER_DEFAULT)
  Applies the bilateral filter to an image.
  cv::GaussianBlur
  void GaussianBlur(InputArray src, OutputArray dst, Size ksize, double sigmaX, double sigmaY=0, int borderType=BORDER_DEFAULT, AlgorithmHint hint=cv::ALGO_HINT_DEFAULT)
  Blurs an image using a Gaussian filter.
  highgui.hpp
  main
  int main(int argc, char *argv[])
  Definition highgui_qt.cpp:3
  imgcodecs.hpp
  imgproc.hpp
  cv
  Definition core.hpp:107
  std
  STL namespace.

Java

- Downloadable code: Click here
- Code at glance:
  import org.opencv.core.*;
  import org.opencv.highgui.HighGui;
  import org.opencv.imgcodecs.Imgcodecs;
  import org.opencv.imgproc.Imgproc;
  class SmoothingRun {
  int DELAY_CAPTION = 1500;
  int DELAY_BLUR = 100;
  int MAX_KERNEL_LENGTH = 31;
  Mat src = new Mat(), dst = new Mat();
  String windowName = "Filter Demo 1";
  public void run(String[] args) {
  String filename = ((args.length > 0) ? args[0] : "../data/lena.jpg");
  src = Imgcodecs.imread(filename, Imgcodecs.IMREAD_COLOR);
  if( src.empty() ) {
  System.out.println("Error opening image");
  System.out.println("Usage: ./Smoothing [image_name -- default ../data/lena.jpg] \n");
  System.exit(-1);
  }
  if( displayCaption( "Original Image" ) != 0 ) { System.exit(0); }
  dst = src.clone();
  if( displayDst( DELAY_CAPTION ) != 0 ) { System.exit(0); }
  if( displayCaption( "Homogeneous Blur" ) != 0 ) { System.exit(0); }
  for (int i = 1; i < MAX_KERNEL_LENGTH; i = i + 2) {
  Imgproc.blur(src, dst, new Size(i, i), new Point(-1, -1));
  displayDst(DELAY_BLUR);
  }
  if( displayCaption( "Gaussian Blur" ) != 0 ) { System.exit(0); }
  for (int i = 1; i < MAX_KERNEL_LENGTH; i = i + 2) {
  Imgproc.GaussianBlur(src, dst, new Size(i, i), 0, 0);
  displayDst(DELAY_BLUR);
  }
  if( displayCaption( "Median Blur" ) != 0 ) { System.exit(0); }
  for (int i = 1; i < MAX_KERNEL_LENGTH; i = i + 2) {
  Imgproc.medianBlur(src, dst, i);
  displayDst(DELAY_BLUR);
  }
  if( displayCaption( "Bilateral Blur" ) != 0 ) { System.exit(0); }
  for (int i = 1; i < MAX_KERNEL_LENGTH; i = i + 2) {
  Imgproc.bilateralFilter(src, dst, i, i * 2, i / 2);
  displayDst(DELAY_BLUR);
  }
  displayCaption( "Done!" );
  System.exit(0);
  }
  int displayCaption(String caption) {
  dst = Mat.zeros(src.size(), src.type());
  Imgproc.putText(dst, caption,
  new Point(src.cols() / 4, src.rows() / 2),
  Imgproc.FONT_HERSHEY_COMPLEX, 1, new Scalar(255, 255, 255));
  return displayDst(DELAY_CAPTION);
  }
  int displayDst(int delay) {
  HighGui.imshow( windowName, dst );
  int c = HighGui.waitKey( delay );
  if (c >= 0) { return -1; }
  return 0;
  }
  }
  public class Smoothing {
  public static void main(String[] args) {
  // Load the native library.
  System.loadLibrary(Core.NATIVE_LIBRARY_NAME);
  new SmoothingRun().run(args);
  }
  }
  cv::Point
  Point2i Point
  Definition types.hpp:209
  cv::Size
  Size2i Size
  Definition types.hpp:373
  cv::Scalar
  Scalar_< double > Scalar
  Definition types.hpp:712

Python

- Downloadable code: Click here
- Code at glance:
  import sys
  import cv2 as cv
  import numpy as np
  # Global Variables
  DELAY_CAPTION = 1500
  DELAY_BLUR = 100
  MAX_KERNEL_LENGTH = 31
  src = None
  dst = None
  window_name = 'Smoothing Demo'
  def main(argv):
  cv.namedWindow(window_name, cv.WINDOW_AUTOSIZE)
  # Load the source image
  imageName = argv[0] if len(argv) > 0 else 'lena.jpg'
  global src
  src = cv.imread(cv.samples.findFile(imageName))
  if src is None:
  print ('Error opening image')
  print ('Usage: smoothing.py [image_name -- default ../data/lena.jpg] \n')
  return -1
  if display_caption('Original Image') != 0:
  return 0
  global dst
  dst = np.copy(src)
  if display_dst(DELAY_CAPTION) != 0:
  return 0
  # Applying Homogeneous blur
  if display_caption('Homogeneous Blur') != 0:
  return 0
  for i in range(1, MAX_KERNEL_LENGTH, 2):
  dst = cv.blur(src, (i, i))
  if display_dst(DELAY_BLUR) != 0:
  return 0
  # Applying Gaussian blur
  if display_caption('Gaussian Blur') != 0:
  return 0
  for i in range(1, MAX_KERNEL_LENGTH, 2):
  dst = cv.GaussianBlur(src, (i, i), 0)
  if display_dst(DELAY_BLUR) != 0:
  return 0
  # Applying Median blur
  if display_caption('Median Blur') != 0:
  return 0
  for i in range(1, MAX_KERNEL_LENGTH, 2):
  dst = cv.medianBlur(src, i)
  if display_dst(DELAY_BLUR) != 0:
  return 0
  # Applying Bilateral Filter
  if display_caption('Bilateral Blur') != 0:
  return 0
  for i in range(1, MAX_KERNEL_LENGTH, 2):
  dst = cv.bilateralFilter(src, i, i * 2, i / 2)
  if display_dst(DELAY_BLUR) != 0:
  return 0
  # Done
  display_caption('Done!')
  return 0
  def display_caption(caption):
  global dst
  dst = np.zeros(src.shape, src.dtype)
  rows, cols, _ch = src.shape
  cv.putText(dst, caption,
  (int(cols / 4), int(rows / 2)),
  cv.FONT_HERSHEY_COMPLEX, 1, (255, 255, 255))
  return display_dst(DELAY_CAPTION)
  def display_dst(delay):
  cv.imshow(window_name, dst)
  c = cv.waitKey(delay)
  if c >= 0 : return -1
  return 0
  if __name__ == "__main__":
  main(sys.argv[1:])
  cv::samples::findFile
  cv::String findFile(const cv::String &relative_path, bool required=true, bool silentMode=false)
  Try to find requested data file.
  cv::namedWindow
  void namedWindow(const String &winname, int flags=WINDOW_AUTOSIZE)
  Creates a window.
  cv::imread
  CV_EXPORTS_W Mat imread(const String &filename, int flags=IMREAD_COLOR_BGR)
  Loads an image from a file.

# Explanation

Let's check the OpenCV functions that involve only the smoothing procedure, since the rest is already known by now.

## Normalized Block Filter:

- OpenCV offers the function blur() to perform smoothing with this filter. We specify 4 arguments (more details, check the Reference):
  - src: Source image
  - dst: Destination image
  - Size( w, h ): Defines the size of the kernel to be used ( of width w pixels and height h pixels)
  - Point(-1, -1): Indicates where the anchor point (the pixel evaluated) is located with respect to the neighborhood. If there is a negative value, then the center of the kernel is considered the anchor point.

C++
for ( int i = 1; i < MAX_KERNEL_LENGTH; i = i + 2 )
{
blur( src, dst, Size( i, i ), Point(-1,-1) );
if( display_dst( DELAY_BLUR ) != 0 )
{
return 0;
}
}
Java
for (int i = 1; i < MAX_KERNEL_LENGTH; i = i + 2) {
Imgproc.blur(src, dst, new Size(i, i), new Point(-1, -1));
displayDst(DELAY_BLUR);
}
Python
for i in range(1, MAX_KERNEL_LENGTH, 2):
dst = cv.blur(src, (i, i))
if display_dst(DELAY_BLUR) != 0:
return 0

## Gaussian Filter:

- It is performed by the function GaussianBlur() : Here we use 4 arguments (more details, check the OpenCV reference):
  - src: Source image
  - dst: Destination image
  - Size(w, h): The size of the kernel to be used (the neighbors to be considered). \(w\) and \(h\) have to be odd and positive numbers otherwise the size will be calculated using the \(\sigma_{x}\) and \(\sigma_{y}\) arguments.
  - \(\sigma_{x}\): The standard deviation in x. Writing \(0\) implies that \(\sigma_{x}\) is calculated using kernel size.
  - \(\sigma_{y}\): The standard deviation in y. Writing \(0\) implies that \(\sigma_{y}\) is calculated using kernel size.

C++
for ( int i = 1; i < MAX_KERNEL_LENGTH; i = i + 2 )
{
GaussianBlur( src, dst, Size( i, i ), 0, 0 );
if( display_dst( DELAY_BLUR ) != 0 )
{
return 0;
}
}
Java
for (int i = 1; i < MAX_KERNEL_LENGTH; i = i + 2) {
Imgproc.GaussianBlur(src, dst, new Size(i, i), 0, 0);
displayDst(DELAY_BLUR);
}
Python
for i in range(1, MAX_KERNEL_LENGTH, 2):
dst = cv.GaussianBlur(src, (i, i), 0)
if display_dst(DELAY_BLUR) != 0:
return 0

## Median Filter:

- This filter is provided by the medianBlur() function: We use three arguments:
  - src: Source image
  - dst: Destination image, must be the same type as src
  - i: Size of the kernel (only one because we use a square window). Must be odd.

C++
for ( int i = 1; i < MAX_KERNEL_LENGTH; i = i + 2 )
{
medianBlur ( src, dst, i );
if( display_dst( DELAY_BLUR ) != 0 )
{
return 0;
}
}
Java
for (int i = 1; i < MAX_KERNEL_LENGTH; i = i + 2) {
Imgproc.medianBlur(src, dst, i);
displayDst(DELAY_BLUR);
}
Python
for i in range(1, MAX_KERNEL_LENGTH, 2):
dst = cv.medianBlur(src, i)
if display_dst(DELAY_BLUR) != 0:
return 0

## Bilateral Filter

- Provided by OpenCV function bilateralFilter() We use 5 arguments:
  - src: Source image
  - dst: Destination image
  - d: The diameter of each pixel neighborhood.
  - \(\sigma_{Color}\): Standard deviation in the color space.
  - \(\sigma_{Space}\): Standard deviation in the coordinate space (in pixel terms)

C++
for ( int i = 1; i < MAX_KERNEL_LENGTH; i = i + 2 )
{
bilateralFilter ( src, dst, i, i*2, i/2 );
if( display_dst( DELAY_BLUR ) != 0 )
{
return 0;
}
}
Java
for (int i = 1; i < MAX_KERNEL_LENGTH; i = i + 2) {
Imgproc.bilateralFilter(src, dst, i, i * 2, i / 2);
displayDst(DELAY_BLUR);
}
Python
for i in range(1, MAX_KERNEL_LENGTH, 2):
dst = cv.bilateralFilter(src, i, i * 2, i / 2)
if display_dst(DELAY_BLUR) != 0:
return 0

# Results

- The code opens an image (in this case lena.jpg) and display it under the effects of the 4 filters explained.
- Here is a snapshot of the image smoothed using medianBlur: