CN103632354A - Multi focus image fusion method based on NSCT scale product - Google Patents
Multi focus image fusion method based on NSCT scale product Download PDFInfo
- Publication number
- CN103632354A CN103632354A CN201210305604.8A CN201210305604A CN103632354A CN 103632354 A CN103632354 A CN 103632354A CN 201210305604 A CN201210305604 A CN 201210305604A CN 103632354 A CN103632354 A CN 103632354A
- Authority
- CN
- China
- Prior art keywords
- mrow
- msub
- msubsup
- msup
- prime
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000007500 overflow downdraw method Methods 0.000 title claims abstract description 13
- 230000004927 fusion Effects 0.000 claims abstract description 66
- 238000000354 decomposition reaction Methods 0.000 claims description 12
- 230000009466 transformation Effects 0.000 claims description 6
- 238000001914 filtration Methods 0.000 claims description 3
- 238000007781 pre-processing Methods 0.000 claims description 2
- 230000000694 effects Effects 0.000 abstract description 20
- 238000000034 method Methods 0.000 abstract description 14
- 238000002474 experimental method Methods 0.000 abstract description 5
- 238000005070 sampling Methods 0.000 abstract description 3
- 238000011156 evaluation Methods 0.000 description 7
- 238000004364 calculation method Methods 0.000 description 6
- 238000012545 processing Methods 0.000 description 6
- 230000000007 visual effect Effects 0.000 description 4
- 238000003709 image segmentation Methods 0.000 description 2
- 238000013519 translation Methods 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 239000003814 drug Substances 0.000 description 1
- 238000003708 edge detection Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
Images
Landscapes
- Image Processing (AREA)
Abstract
The invention discloses a multi focus image fusion method based on NSCT scale product. The method comprises the steps that non-sampling contourlet transform (NSCT) is used to decompose a multi focus image; laplace energy and the fusion rule of a low frequency sub-band are used; scale product, local laplace energy and the fusion rule of a high frequency direction sub-band are used; and NSCT inverse operation is used to acquire a fusion image. According to an experiment results, the algorithm can fully extract source image information and can inject the information into the fusion image; the influence of noise can be effectively restrained; and the fusion effect is better than the fusion effects of other methods.
Description
Technical Field
The invention belongs to the field of image processing, and particularly relates to a multi-focus image fusion method based on NSCT (non-subsampled Contourlet transform) scale products.
Background
Image fusion is one of the research hotspots in the current image processing field, and is widely applied to the fields of remote sensing, machine vision, medicine, military, judicial arts, manufacturing industry and the like. When an image sensor such as a CCD or a CMOS is used for acquiring an image, due to the depth of field of a lens, a scene positioned on a focusing plane can obtain a clear projection on the image, and scenes positioned at other positions are blurred on the image to different degrees. A focused image is a precondition for a plurality of subsequent processing, and the main method for solving the problem is a multi-focus image fusion technology, namely, a series of images are shot by adopting different focal length settings, and then the images are fused to obtain a clear fused image.
At present, the commonly used multi-focus image fusion method is mainly divided into two main categories, namely a spatial domain method and a transform domain method. The spatial domain fusion method is mainly divided into 3 fusion modes with the unit of pixel, image block and area. In the case of multi-focus image fusion based on pixels, it is usually necessary to determine whether each pixel is focused, which has the disadvantage of large calculation amount and error. The multi-focus image fusion method based on the image blocks is high in calculation efficiency, but how to select the proper size of the image blocks needs to be further researched. The multi-focus image fusion method based on the region increases the amount of calculation because image segmentation processing must be performed first, and the fusion effect greatly depends on the quality of image segmentation. Transform domain based image fusion includes pyramid transform, wavelet transform and multi-scale geometric analysis methods. Do and Vetterli put forward Contourlet transform, combine Laplacian pyramid decomposition with directional filter bank, have good many resolutions and directionality, it is a better image representation method. However, in the process of performing the Contourlet transform on the image, the down-sampling operation needs to be performed on the image, so that the Contourlet transform does not have translation invariance, and a Gibbs phenomenon is generated in the image processing. In order to solve the problem, Arthur L provides a non-sampling Contourlet transform (NSCT) which has translation invariance and redundant information, so that the direction information in the image to be fused can be effectively extracted, and a better fusion effect is obtained.
As can be seen from a large amount of literature, the algorithm for multi-focus image fusion does not consider the influence of noise on the image and the visual effect of human eyes, which will cause the fused image to be inconsistent with the effect perceived by human eyes.
Disclosure of Invention
The invention aims to provide a multi-focus image fusion algorithm which can inhibit noise and meet the visual sense of human eyes, and particularly provides a multi-focus image fusion algorithm based on NSCT (non-subsampled Contourlet transform) scale product.
Assuming active images A and B, the specific steps of the multi-focus image fusion algorithm based on contrast are as follows:
step 1: image pre-processing
Because of the interference of factors such as noise and the like, the multi-focus images A and B need to be preprocessed, and the image A and B are filtered by adopting an average filter to obtain processed images A 'and B'.
Step 2: image decomposition
The invention adopts NSCT to decompose the images A 'and B' to respectively obtain the low-frequency subband coefficients of the focused image A
And high frequency directional subband coefficients
Low frequency subband coefficients of the focused image B
And high frequency directional subband coefficientsWhere l represents the number of scales of decomposition and k represents the number of directional decomposition steps.
And step 3: fusion rules
Numerous references indicate that: the fusion rule of multi-focus image fusion directly influences the image fusion effect. Considering that the purpose of the invention is to design a multi-focus image fusion algorithm which can restrain noise and meet human vision, the invention adopts NSCT scale product and Laplace energy Sum (SML) as a fusion rule, which is specifically expressed as follows:
1) fusion strategy for low frequency sub-bands
The low-frequency sub-band obtained after NSCT decomposition of the image is an approximate description of the source image and contains most of the capability features in the image. The method adopts Laplace energy Sum (SML) to reflect the edge characteristics of the image, and can properly represent the focusing characteristic and definition of the image to a certain extent.
Since the contrast is considered in a certain region, assume a window size of m1×n1Low frequency coefficients of source images A' and B
And
corresponding local laplace energy sum
And
the calculation formula (2) is as shown in formula (1):
obviously, the larger the local laplacian energy sum, the more abundant the image information is in the region. According to the size of the Laplace energy sum of the source image, the fusion rule of the low-frequency coefficients is as the formula (3):
2) fusion strategy for high-frequency directional subbands
The high-frequency sub-band of the image after NSCT decomposition reflects the edge detail information of the image. If the source image is mixed with noise, the noise of the image after multi-scale decomposition is mixed with edge detail information of the image. If the fusion rule is directly applied to the wavelet coefficient, the misselection of the fusion coefficient can be caused. Since the presence of noise may cause the metric values calculated according to the fusion rule to be erroneous. The invention multiplies the high-frequency direction sub-band coefficients of two adjacent NSCTs to form important structural information of an NSCT multi-scale product reinforced image, weakens noise, and then uses the Laplace operator as a measurement standard of pixel definition. The specific process is as follows:
firstly, for the high-frequency direction sub-bands of the source images A' and B
Andperforming a scale product operation, see formulas (4) and (5):
wherein k islThe sum of the decomposition levels in the direction of the dimension l is represented.
Second, in a window size of m1×n1Next, a source image scale product is calculated
And
local laplace energy sum of
And
the calculation formula is formula (6) and formula (7):
wherein,
the fusion rule of the high-frequency direction sub-band is shown in formula (8):
and 4, step 4: image reconstruction
And performing NSCT inverse transformation on each sub-band coefficient of the fused image to obtain a final fused image.
According to the method, firstly, NSCT is adopted to decompose a source image, different fusion rules are respectively adopted according to the characteristics of a low-frequency sub-band and a high-frequency direction sub-band, and finally, NSCT inverse transformation is carried out to obtain a multi-focus fusion image. Experimental results show that the algorithm can effectively inhibit the influence of noise without fully extracting source image information and injecting the source image information into a fusion image, and the fusion effect is better than that of other methods.
Drawings
FIG. 1 is a flow chart of the present invention;
fig. 2 is a diagram of the fusion effect of different fusion methods on a multi-focus image without noise interference, wherein:
FIG. 2(a) left focused image;
FIG. 2(b) right focused image;
FIG. 2(c) is an effect graph based on the DWT fusion algorithm;
FIG. 2(d) an effect graph based on the LSWT fusion algorithm;
FIG. 2(e) an effect graph based on NSCT fusion algorithm;
FIG. 2(f) uses the fusion effect of the present invention.
FIG. 3 is a graph of the fusion effect of different fusion methods on multi-focus images under noise interference, wherein: .
FIG. 3(a) left focused image;
FIG. 3(b) right focused image;
FIG. 3(c) is an effect graph based on the DWT fusion algorithm;
FIG. 3(d) an effect graph based on the LSWT fusion algorithm;
FIG. 3(e) an effect graph based on NSCT fusion algorithm;
FIG. 3(f) uses the fusion effect of the present invention.
Detailed Description
Referring to fig. 1, the specific process of the present invention includes:
step 1: image filtering process
Carrying out mean filtering processing on the two source images A and B to obtain filtered source images A 'and B';
step 2: image decomposition
Decomposing the source images A 'and B' by using NSCT to respectively obtain a low-frequency subband and a high-frequency directional subband of the source image A ', and a low-frequency subband and a high-frequency directional subband of the source image B';
and step 3: fusion rules
1) Adopting a formula (3) as a fusion rule of low-frequency sub-bands;
2) firstly, the scale product operation is carried out on the high-frequency sub-band, and a formula (8) is adopted as the fusion rule of the high-frequency direction sub-band.
And 4, step 4: image reconstruction
And performing NSCT inverse transformation on each sub-band coefficient of the fused image to obtain a final fused image.
In order to verify the performance of the algorithm, fusion experiments are respectively carried out on noiseless and noisy multi-focus images. In the experiment, besides the visual effect, Mutual Information (MI) and Q are also adoptedAB/FAs an objective evaluation index. These two indicators are used because the purpose of image fusion is to fuse information, and they do not necessarily require knowledge of the ideal fused image. Where MI is used to measure how much information the source image has transferred into the fused result, QAB FSobel edge detection is utilized to measure how much edge detail information is transferred from a source imageMove to the fused image. The larger the value of both, the better the effect of fusion.
In the experiment, a DWT (discrete wavelet transform) -based image fusion algorithm, an LSWT-based image fusion algorithm and an NSCT-based image fusion algorithm are respectively adopted to fuse noise-free multi-focus images, and the fusion results are shown in FIG. 2 and Table 1.
TABLE 1 comparison of Performance evaluation of different fusion methods
Table 1 shows the observable evaluation indexes MI and QAB FThe numerical value of (c). The effectiveness and superiority of the algorithm herein are further confirmed, consistent with the visually obtained conclusions.
In the experiment, an image fusion algorithm based on DWT transformation, an image fusion algorithm based on LSWT transformation and an image fusion algorithm based on NSCT are respectively adopted to fuse multi-focus images containing noise. The multi-focus image carries white gaussian noise (variance is 0.01), and the fusion result is shown in fig. 3. And adopting the improved peak signal-to-noise ratio (VPSNR) as an objective evaluation index, wherein the calculation formula of the VPSNR is as shown in a formula (13):
wherein,the variances of the fused image and the noise source image, respectively. Obviously, the smaller the noise contained in the fused image, the larger the VPSNR value. When the VPSNR is close to 0, the noise content of the fused image is close to that of the source image; and if the VPSNR value is less than 0, the fused image has higher noise content than the source image.
The fusion evaluation index values of these four fusion algorithms are shown in table 2.
TABLE 2 Performance evaluation comparison of different fusion methods for noise-containing multi-focus images
As can be seen from the effects of fig. 2 and 3 and the evaluation index values in tables 1 and 2, the fused image obtained by the method of the present invention contains the most image information, and has not only a good visual effect, but also rich information and a good fusion effect.
Claims (1)
1. A multi-focus image fusion method based on NSCT scale product comprises the following steps:
step 1: image pre-processing
And filtering the images A and B by adopting an average filter to obtain processed images A 'and B'.
Step 2: image decomposition
Decomposing the images A 'and B' by NSCT to respectively obtain the low-frequency subband coefficient of the focused image A
And high frequency directional subband coefficients
Low frequency subband coefficients of the focused image B
And high frequency directional subband coefficients
Where l represents the number of scales of decomposition and k represents the number of directional decomposition steps.
And step 3: fusion rules
1) Fusion rule of low frequency sub-band
(1) Calculating the low-frequency coefficients of the source images A 'and B' by applying the formula (1) and the formula (2)
And
is/are as follows
And
(2) the fusion rule of the low-frequency sub-band is formula (3)
2) Fusion rule of high-frequency direction sub-bands
(1) For high frequency direction sub-band of source images A' and B
And
performing a scale product operation, see formulas (4) and (5):
(2) calculating the source image scale product using equations (6) and (7)
And
is/are as follows
And
wherein,
the fusion rule of the high-frequency direction sub-band is shown in formula (8):
and 4, step 4: image reconstruction
And performing NSCT inverse transformation on each sub-band coefficient of the fused image to obtain a final fused image.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210305604.8A CN103632354A (en) | 2012-08-24 | 2012-08-24 | Multi focus image fusion method based on NSCT scale product |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210305604.8A CN103632354A (en) | 2012-08-24 | 2012-08-24 | Multi focus image fusion method based on NSCT scale product |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN103632354A true CN103632354A (en) | 2014-03-12 |
Family
ID=50213371
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201210305604.8A Pending CN103632354A (en) | 2012-08-24 | 2012-08-24 | Multi focus image fusion method based on NSCT scale product |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103632354A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104463822A (en) * | 2014-12-11 | 2015-03-25 | 西安电子科技大学 | Multi-focus image fusing method and device based on multi-scale overall filtering |
| CN104574268A (en) * | 2014-12-30 | 2015-04-29 | 长春大学 | Cloud and fog removing method based on non-subsample contourlet transform and non-negative matrix factorization |
| CN104992426A (en) * | 2015-07-15 | 2015-10-21 | 中国科学院广州生物医药与健康研究院 | Multilayer image fusion algorithm for bright-field microscopic imaging |
| CN104182955B (en) * | 2014-09-05 | 2016-09-14 | 西安电子科技大学 | Image interfusion method based on steerable pyramid conversion and device thereof |
| CN113379660A (en) * | 2021-06-15 | 2021-09-10 | 深圳市赛蓝科技有限公司 | Multi-dimensional rule multi-focus image fusion method and system |
-
2012
- 2012-08-24 CN CN201210305604.8A patent/CN103632354A/en active Pending
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104182955B (en) * | 2014-09-05 | 2016-09-14 | 西安电子科技大学 | Image interfusion method based on steerable pyramid conversion and device thereof |
| CN104463822A (en) * | 2014-12-11 | 2015-03-25 | 西安电子科技大学 | Multi-focus image fusing method and device based on multi-scale overall filtering |
| CN104463822B (en) * | 2014-12-11 | 2017-08-25 | 西安电子科技大学 | Multi-focus image fusing method and its device based on multiple dimensioned global filtering |
| CN104574268A (en) * | 2014-12-30 | 2015-04-29 | 长春大学 | Cloud and fog removing method based on non-subsample contourlet transform and non-negative matrix factorization |
| CN104574268B (en) * | 2014-12-30 | 2017-06-16 | 长春大学 | Cloud and mist method is gone based on non-down sampling contourlet transform and Non-negative Matrix Factorization |
| CN104992426A (en) * | 2015-07-15 | 2015-10-21 | 中国科学院广州生物医药与健康研究院 | Multilayer image fusion algorithm for bright-field microscopic imaging |
| CN104992426B (en) * | 2015-07-15 | 2018-04-20 | 中国科学院广州生物医药与健康研究院 | A kind of multi-layer image blending algorithm for light field micro-imaging |
| CN113379660A (en) * | 2021-06-15 | 2021-09-10 | 深圳市赛蓝科技有限公司 | Multi-dimensional rule multi-focus image fusion method and system |
| CN113379660B (en) * | 2021-06-15 | 2022-09-30 | 深圳市赛蓝科技有限公司 | Multi-dimensional rule multi-focus image fusion method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Narang et al. | Graph-wavelet filterbanks for edge-aware image processing | |
| CN101630405B (en) | Multi-focusing image fusion method utilizing core Fisher classification and redundant wavelet transformation | |
| CN105096280A (en) | Method and device for processing image noise | |
| CN104504652A (en) | Image denoising method capable of quickly and effectively retaining edge and directional characteristics | |
| CN103632354A (en) | Multi focus image fusion method based on NSCT scale product | |
| Hiremath et al. | Performance comparison of wavelet transform and contourlet transform based methods for despeckling medical ultrasound images | |
| Mustafa et al. | Medical image de-noising schemes using wavelet transform with fixed form thresholding | |
| CN117911267B (en) | Back-scattered X-ray image noise reduction method based on wavelet transformation | |
| Pradnya et al. | Wavelet based image fusion techniques | |
| CN104361570A (en) | Image fusing method based on fractional Fourier transformation | |
| CN103700071B (en) | A kind of depth map up-sampling edge enhancing method | |
| CN103778615B (en) | Multi-focus image fusing method based on region similitude | |
| Kannan et al. | Optimal decomposition level of discrete wavelet transform for pixel based fusion of multi-focused images | |
| Sumalatha et al. | Hierarchical lossless image compression for telemedicine applications | |
| CN103632353A (en) | Multi focus image fusion algorithm based on NSCT | |
| Kalaiselvi et al. | A novel wavelet thresholding technique to denoise magnetic resonance images‖ | |
| CN105096274A (en) | Infrared image noise reduction method based on non-subsampled contourlet domain mixed statistical model | |
| CN102622731B (en) | Contourlet domain Wiener filtering image denoising method based on two-dimensional Otsu | |
| Razman et al. | Filtering technique in ultrasound for kidney, liver and pancreas image using Matlab | |
| Chen et al. | Multifocus image fusion based on multiwavelet and DFB | |
| Liang et al. | Self-adaptive spatial image denoising model based on scale correlation and SURE-LET in the nonsubsampled contourlet transform domain | |
| Budhewar | Wavelet and Curvelet Transform based Image Fusion Algorithm | |
| Rajasekhar et al. | Multilevel medical image fusion using multi-level local extrema and non sub-sampled contourlet transformation | |
| Chen et al. | Adaptive video denoising using block matching 3-D filtering | |
| Wu et al. | The contourlet transform with multiple Cycles spinning for catenary image denoising |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| DD01 | Delivery of document by public notice |
Addressee: XI'AN YUANSHUO SCIENCE & TECHNOLOGY CO., LTD. Document name: Notification of Publication of the Application for Invention |
|
| DD01 | Delivery of document by public notice |
Addressee: XI'AN YUANSHUO SCIENCE & TECHNOLOGY CO., LTD. Document name: Notification of before Expiration of Request of Examination as to Substance |
|
| DD01 | Delivery of document by public notice |
Addressee: XI'AN YUANSHUO SCIENCE & TECHNOLOGY CO., LTD. Document name: Notification that Application Deemed to be Withdrawn |
|
| C02 | Deemed withdrawal of patent application after publication (patent law 2001) | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20140312 |