Zhonghua Wang*, Xiaoming Huang** and Faliang Huang***
A New Image Enhancement Algorithm Based on Bidirectional Diffusion
Abstract: To solve the edge ringing or block effect caused by the partial differential diffusion in image enhancement domain, a new image enhancement algorithm based on bidirectional diffusion, which smooths the flat region or isolated noise region and sharpens the edge region in different types of defect images on aviation composites, is presented. Taking the image pixel’s neighborhood intensity and spatial characteristics as the attribute descriptor, the presented bidirectional diffusion model adaptively chooses different diffusion criteria in different defect image regions, which are elaborated are as follows. The forward diffusion is adopted to denoise along the pixel’s gradient direction and edge direction in the pixel’s smoothing area while the backward diffusion is used to sharpen along the pixel’s gradient direction and the forward diffusion is used to smooth along the pixel’s edge direction in the pixel’s edge region. The comparison experiments were implemented in the delamination, inclusion, channel, shrinkage, blowhole and crack defect images, and the comparison results indicate that our algorithm not only preserves the image feature better but also improves the image contrast more obviously.
Keywords: Backward Diffusion, Forward Diffusion, Image Enhancement, Local Feature
1. Introduction
The goal of image enhancement is to highlight useful information and remove useless information, that is to say, we hope that the edges are sharpened and the noises are suppressed [1-3]. Since the noise and edge are mainly concentrated in the high frequency component of image, it is difficult to denoise the noise and sharpen the edge simultaneously.
As is known to all, the partial differential equation has been widely used in image processing [4,5]. Introduced the anisotropic diffusion concept into the thermal diffusion equation, the PM model [6] is extensively used in image enhancement. Since its diffusion rate is only controlled by image gradient magnitude, the image edges are blurred. Luo et al. [7] proposed the coupled anisotropic diffusion model, but the model particularly relies on the artificial experience and the image weak edges are lost. Nadernejad [8] presented an improved image enhancement method using relaxed geometric mean filter. Though its edge preserving ability is strong, the block effect on image is introduced. For the first time, Osher and Rudin [9] put forward the shock filter model to enhance the image edge features. The anisotropic diffusion with shock filter model (ADSFM) was presented by Alvarez and Mazorra [10], combined the shock filter model and PM model. However, the block effect is prone to occur in the processed image. Fu et al. [11] put forward the coupled bidirectional flow model (CBDFM), but it causes the ringing effect at the image edge. The adaptive bidirectional diffusion filter model (ABDFM) was proposed by Xiong et al. [12]. Though the ABDFM model avoids the ringing effect, the image weak edges are lost.
In order to solve the edge ringing or block effect caused by the partial differential diffusion, in this paper, the image is partitioned into three areas: the edge region, flat region, and isolated noise region. On the base, the image can be further described as the non-edge and edge areas with different features, then we propose a new image enhancement method, which is based on bidirectional diffusion with local feature. The comparison experiments indicate that our method possesses better ability for denoising and preserving edges in defect image, which make the image contrast more obvious.
The remainder of this paper is organized as follows. In Section 2, the partial differential diffusion concept is reviewed. Our method is proposed in Section 3 and its performance is validated in Section 4. Section 5 gives a summary
2. Partial Differential Diffusion Model
2.1 PM Model
The linear scale space concept-based nonlinear anisotropic diffusion model was presented by Perona and Malik [6] (PM model), which was defined as follows.
(1)
[TeX:] $$\left\{\begin{array}{l} \frac{\partial u(i, j, t)}{\partial t}=\operatorname{div}(g(i, j, t) \nabla u) \\ u(i, j, 0)=u^{0} \end{array}\right.$$where[TeX:] $$d i v, \nabla \text { and } u^{0}$$represent the divergence operator, gradient operator and original image, respectively. is the monotonic decreasing function, shown in the formula (2) or (3). The smaller a pixel’s gradient magnitude is, the larger the pixel's diffusion coefficient is, and vice versa.
where TH is the global image gradient threshold,[TeX:] $$\nabla u$$is the image gradient vector. Although the PM model is used to preserve the image edges and suppress the image noises, it still has two main problems. One problem is that when the noise belongs to the region with intense gray value change, it cannot be suppressed effectively, the other problem is that the single threshold TH causes the image nonuniformity.
2.2 Anisotropic Diffusion with Shock Filter Model
Since the flat region, isolated noise region and edge region in image have different pixel intensity and spatial features, Osher and Rudin [9] proposed the shock filter model, which is expressed by the formula (4), to highlight the image detail. In the shock filter model, the backward diffusion was first introduced to enhance the image edges, which reduced the image edge blurring caused by the forward diffusion.
(4)
[TeX:] $$\frac{\partial u}{\partial t}=-\operatorname{sign}\left(u_{N N}\right)|\nabla u(x, y, t)|$$where[TeX:] $$sign$$defines the symbolic function, and[TeX:] $$u_{N N}$$represents the second order derivative along image gradient direction.
In order to further lessen the edge blurring, the ADSFM was presented by Alvarez and Mazorra [10], shown in the expression (5).
(5)
[TeX:] $$\frac{\partial u}{\partial t}=-\operatorname{sign}\left(G_{\sigma} * u_{N N}\right)|\nabla u|+c u_{T T}$$where[TeX:] $$G_{\sigma}$$is defined as the Gaussian function with standard deviation, is regarded as the smoothing factor, and[TeX:] $$u_{T T}$$represents the second order derivative along image edge direction. In the right side of the expression (5), the first item is used to sharpen the image edges and the second item is adopted to smooth the image noises. However, the artificial saw-tooth phenomenon is generated in the processed image since the inconsistent diffusion rates of edge pixels possessing the same attributes.
2.3 Bidirectional Diffusion Model
Introducing the image structure information into the ADSFM, Fu et al. [11] proposed the CBDFM. The CBDFM was defined by the formulas (6) and (7), and the corresponding diffusion coefficients were expressed in Table 1.
Table 1.
| [TeX:] $$c_{T}$$ | [TeX:] $$\mathcal{C}_{N}$$ | [TeX:] $$w$$ | |
|---|---|---|---|
| [TeX:] $$\left|\left(u_{G}^{0}\right)_{N}\right|>T_{1}$$ | [TeX:] $$\frac{1}{1+l_{1} \kappa^{2}}$$ | 0 | 1 |
| [TeX:] $$T_{2}\left|\left(u_{G}^{0}\right)_{N}\right| \leq T_{1}$$ | [TeX:] $$\frac{1}{1+l_{1} \kappa^{2}}$$ | 0 | [TeX:] $$\left|t h\left(l_{2} u_{N N}\right)\right|$$ |
| Otherwise | 1 | 1 | 0 |
(6)
[TeX:] $$\frac{\partial u}{\partial t}=c_{N} u_{N N}+c_{T} u_{T T}-w\left(v_{N N}^{n}\right) \operatorname{sign}\left(\left(u_{G}\right)_{N N}\right)|\nabla u|$$
(7)
[TeX:] $$\left\{\begin{array}{l} v^{0}=u^{0}, u_{G}=G_{\sigma} * u \\ v^{n+1}=u^{n}+\Delta t\left(-w\left(v_{N v}^{n}\right) \operatorname{sign}\left(\left(u_{G}\right)_{N N}\right)|\nabla u|\right) \\ u^{n+1}=v^{n+1}+\Delta t\left(c_{N} v_{N N}^{n+1}+c_{T} v_{T T}^{n+1}\right) \end{array}\right.$$where[TeX:] $$\mathcal{C}_{N}$$is the diffusion coefficient along image gradient direction,[TeX:] $$C_{T}$$is the diffusion coefficient along image edge direction, th represents the hyperbolic tangent function,[TeX:] $$T_{1} \text { and } T_{2}$$are the image gradient thresholds, k is the image level-set curvature,[TeX:] $$l_{1} \text { and } l_{2}$$are the accommodation coefficients.
Although the CBDFM smooths the image noises and sharpens the image edges better, it not only increases the computational complexity but also introduces the image ringing effect. Introducing the Laplacian rotation invariance, Xiong et al [12] simplified the CBDFM and then proposed the ABDFM, indicated by the formulas from (8) to (10).
(9)
[TeX:] $$c_{N}=\left\{\begin{array}{ll} \alpha |\nabla u| \leq T H \\ -\beta |\nabla u|>T H^{\prime}, c_{T}=\alpha \end{array}\right.$$
(10)
[TeX:] $$\frac{\partial u}{\partial t}=\left\{\begin{array}{ll} \alpha \Delta u |\nabla u| \leq T H \\ -\beta \Delta u+(\alpha+\beta) u_{T T} |\nabla u|>T H \end{array}\right.$$where and represent the forward diffusion rate and backward diffusion rate, respectively. represents the Laplacian operator. If the image pixel gradient magnitude is smaller than the global threshold TH, the homogeneous diffusion is carried out on the image datum while the image pixel gradient magnitude is greater than the TH, the nonhomogeneous diffusion is implemented on the image datum. Although the ABDFM model reduces the image ringing effect, the single threshold TH causes the weak edge loss.
3. Local Feature-based Bidirectional Diffusion Model
The aim of image enhancement is to highlight the image details. In this paper, we present a new image enhancement method which is based on bidirectional diffusion. The proposed algorithm flow is described as follows. First, according to the image pixel’s neighborhood intensity and spatial characteristics, it is determined that the pixel is a non-edge point or an edge point. Second, in the different pixel’s characteristic regions of the defect image, the different diffusion criteria are chosen, that is, the non-edge region is denoised by the forward diffusion along the pixel’s gradient direction and edge direction while the edge region is sharpened along the pixel's gradient direction by the backward diffusion and is smoothed along the pixel’s edge direction by the forward diffusion. Third, the pixel grayscale is adjusted by the proposed bidirectional diffusion function to achieve the defect image enhancement. Therefore, the bidirectional diffusion-based image enhancement algorithm is implemented as follows.
Notation:[TeX:] $$Pixel(i, $j)$$represents the pixel corresponding to (i, j) image plane coordinate and f represents the bidirectional diffusion function.
Here[TeX:] $$u(i, j)$$indicates the pixel grayscale corresponding to the image plane coordinate[TeX:] $$(i, j)$$and the pixel’s neighborhood window is defined as 3×3 size, shown in Fig. 1. For[TeX:] $$u(i, j)$$, its gradient magnitude is computed by Eq. (13) and its gradient direction is expressed by the formulas (14). Provided that a pixel is regarded as an edge point, two conditions should be satisfied simultaneously. First, the pixel must be a salient point. Second, in the pixel’s neighborhood window, other pixels have the same gradient direction as the pixel.
In this paper, in order to represent the local feature of the defect image, p is defined as a pixel’s gradient magnitude to its neighborhood median ratio, shown in the formula (15).
where T represents the pixel's neighborhood median, and[TeX:] $$\varepsilon$$is any small constant.
In a pixel’s flat region, p is relatively small since the pixel’s gradient magnitude closes to zero, but p is relatively large in the pixel’s edge region or isolated noise region. If p is greater than the global image saliency threshold , the pixel can be judged as a salient point. Then, in the neighborhood of the salient point, if other pixels have the same gradient direction as the salient point, the salient point is determined as an edge point. On the contrary, the pixel is judged as a non-edge point. Therefore, combined with the pixel’s edge statistical information, a new bidirectional diffusion-based image enhancement model is presented, which is indicated by Eq. (16).
(16)
[TeX:] $$\left\{\begin{array}{l} G=\operatorname{grad}(u) \\ \frac{\partial u}{\partial t}=-\beta \times \operatorname{symbol} \times \operatorname{sign}(\Delta u)|\nabla u|+(1-s y m b o l) \times \alpha u_{N N}+\alpha u_{T T} \end{array}\right.$$
(17)
[TeX:] $$\text {symbol}=\left\{\begin{array}{l} 1 \text { meets thetwo predefined conditions of edge pixel} \\ 0 \ other \end{array}\right.$$where G are the pixel's gradient magnitude calculated by the formula (13), symbol is the pixel’s edge denoter, and are the weighting diffusion coefficients.
If symbol is equal to 1, the pixel’s neighborhood window is determined to be in the image edge region, then the backward diffusion is used to sharpen along the pixel’s gradient direction and the forward diffusion is used to smooth along the pixel's edge direction while symbol is 0, the pixel’s neighborhood window is regarded as the image flat area or noise area, the forward diffusion is adopted for smoothing the region along the pixel's gradient direction and edge direction.
4. Experimental Analysis
In this paper, the MATLAB 2014b is selected as the simulation tool and the comparison experiments are implemented in the delamination, inclusion, channel, shrinkage, blowhole and crack defect images on aviation components to validate our method performance.
In this paper, the PM model, ADSFM, CBDFM, ABDFM and our method are respectively used to enhance the defect images and their experimental results are shown in Figs. 2–7. In our algorithm, the parameter definitions of TR, and are derived from the statistics of the following optimal experiment results. Here TR is set to 0.05, and are 0.015 and 0.03, respectively. In the meantime, for the PM model, ADSFM, CBDFM and ABDFM, their parameter values are from these cited papers to obtain their optimal enhancement images.
As seen from Fig. 2(b) to Fig. 7(b), the image edges are blurred and the image ringing effects are generated, especially from Figure 3 (b) to Figure 6 (b). As shown from Fig. 2(c) to Fig. 7(c), the image block effects are obvious. As seen from Fig. 2(d) to Fig. 7(d), the salt and pepper noises appear. As shown from Fig. 2(e) to Fig. 7(e), the artificial saw-tooth phenomena occur as well, especially from Fig. 4(e) to Fig. 7(e). However, as seen from Fig. 2(f) to Fig. 7(f), in the aspects of the image blocking, saw-tooth and edge ringing suppression, our proposed method has better performance trade-offs.
Fig. 2.
Fig. 3.
Fig. 4.
Fig. 5.
Fig. 6.
Fig. 7.
In order to further evaluate the performances of the above-mentioned five algorithms, the peak signal-to-noise ratio (PSNR) and Global Structural Similarity Index (GSSI) are selected as the parameter indexes [13,14], defined by the formulas (18) and (19), respectively. The smaller the GSSI, the worse the image structure preservation, and the larger the PSNR, the better the noise removal.
(18)
[TeX:] $$P N S R=10 \log _{I 0} \frac{255 \times 255}{\frac{1}{m \times n} \sum_{i=0}^{m-1} \sum_{j=0}^{n-1}(I(i, j)-R(i, j))^{2}}$$
(19)
[TeX:] $$G S S I=\frac{\sum_{x=1}^{m} \sum_{y=1}^{n} \frac{\left(2 \mu_{x} \mu_{y}+C_{1}\right)\left(2 \mu_{x y}+C_{2}\right)}{\left(\mu_{x}^{2}+\mu_{y}^{2}+C_{1}\right)\left(\sigma_{x}^{2}+\sigma_{y}^{2}+C_{2}\right)}}{m \times n}$$According to the different defect images, the PSNR and GSSI of the corresponding enhanced images are shown in Tables 2 and 3.
As seen in the PSNR of Table 2 and the GSSI of Table 3, the two parameter indexes, acquired by adopting our proposed algorithm, are greater than by using other four models. It is indicated that the abilities of the noise suppression and the edge preservation are better than other four models.
Table 2.
| Image defect | PM model | ADSF model | CBDF model | ABDF model | Proposed algorithm |
|---|---|---|---|---|---|
| delamination | 33.47 | 26.22 | 24.26 | 26.61 | 34.11 |
| inclusion | 35.22 | 24.31 | 27.88 | 36.74 | 37.95 |
| channel | 39.06 | 29.02 | 25.30 | 40.74 | 45.10 |
| shrinkage | 38.66 | 28.83 | 24.72 | 41.28 | 44.42 |
| blowhole | 38.63 | 26.10 | 24.97 | 35.30 | 41.84 |
| crack | 37.35 | 29.35 | 26.58 | 34.82 | 42.77 |
Table 3.
| Image defect | PM model | ADSF model | CBDF model | ABDF model | Proposed algorithm |
|---|---|---|---|---|---|
| delamination | 0.89 | 0.93 | 0.76 | 0.91 | 0.96 |
| inclusion | 0.86 | 0.94 | 0.87 | 0.96 | 0.96 |
| channel | 0.90 | 0.87 | 0.73 | 0.95 | 0.98 |
| shrinkage | 0.91 | 0.85 | 0.72 | 0.96 | 0.99 |
| blowhole | 0.90 | 0.85 | 0.82 | 0.94 | 0.98 |
| crack | 0.88 | 0.89 | 0.78 | 0.93 | 0.97 |
5. Conclusions
Image enhancement is to highlight useful image information and remove useless image information, which are conducive to the analysis and treatment task of human or machine. Aiming at the image enhancement problem of edge ringing or block effect caused by the partial differential diffusion, a bidirectional diffusion-based image enhancement method is presented. Through using this model, the image edge region can be sharpened and the image flat region or isolated noise region can be smoothed. The focus of defect the image enhancement in this paper is to determine a pixel’s neighborhood intensity and spatial characteristics, then to adopt different diffusion criteria for the pixel’s edge or non-edge regions, that is to say, the backward diffusion along the pixel’s gradient direction and the forward diffusion along the pixel’s edge direction are used for the pixel’s edge area to sharpen while the forward diffusion in the pixel’s gradient direction and edge direction is adopted for the pixel’s non-edge area to smooth. Compared with other models, the experiment comparisons show that our method not only enhances the image edge better but also improves the image contrast better.
Biography
Zhonghua Wang
https://orcid.org/0000-0003-0265-6671
He was born in China in 1977. He received Ph.D. degree from Huazhong University of Science and Technology, China, in 2011. He is an associate professor of Nanchang Hangkong University, China. His research interests include image processing, pattern recognition and artificial intelligence. He has hosted or attended several National Natural Science Foundations of China.
Biography
Xiaoming Huang
https://orcid.org/0000-0003-1265-7467
He was born in China in 1977. He received Ph.D. degree from Huazhong University of Science and Technology, China, in 2011. He is an associate professor of Nanchang Hangkong University, China. His research interests include image processing, pattern recognition and artificial intelligence. He has hosted or attended several National Natural Science Foundations of China. He was born in China in 1993. He is a master’s student in Nanchang Hangkong University, China. His research interests include pattern recognition and artificial intelligence.
Biography
Faliang Huang
https://orcid.org/0000-0001-5561-4411
He was born in China in 1977. He received Ph.D. degree from Huazhong University of Science and Technology, China, in 2011. He is an associate professor of Nanchang Hangkong University, China. His research interests include image processing, pattern recognition and artificial intelligence. He has hosted or attended several National Natural Science Foundations of China. He was born in China in 1993. He is a master’s student in Nanchang Hangkong University, China. His research interests include pattern recognition and artificial intelligence. He was born in China in 1987. He received the master’s degree from Nanchang Hangkong University, China, in 2016. He is an engineer of Beijing Xinyihua Technology Co., Ltd. His research interests include image processing and pattern recognition. He has attended several National Natural Science Foundations of China.
References
- 1 S. C. Nercessian, K. A. Panetta, S. S. Agaian, "Non-linear direct multi-scale image enhancement based on the luminance and contrast masking characteristics of the human visual system," IEEE Transactions on Image Processing, vol. 22, no. 9, pp. 3549-3561, 2013.doi:[[[10.1109/TIP.2013.2262287]]]
- 2 A. Nandal, V. Bhaskar, A. Dhaka, "Contrast-based image enhancement algorithm using grey-scale and colour space," IET Signal Processing, vol. 12, no. 4, pp. 514-521, 2018.doi:[[[10.1049/iet-spr.2017.0272]]]
- 3 S. U. Khan, W. Y. Chai, C. S. See, A. Khan, "X-ray image enhancement using a boundary division wiener filter and wavelet-based image fusion approach," Journal of Information Processing System, vol. 12, no. 1, pp. 35-45, 2016.doi:[[[10.3745/JIPS.02.0029]]]
- 4 C. Lopez-Molina, M. Galar, H. Bustince, B. De Baets, "On the impact of anisotropic diffusion on edge detection," Pattern Recognition, vol. 47, no. 1, pp. 270-281, 2014.doi:[[[10.1016/j.patcog.2013.07.009]]]
- 5 S. Tebini, Z. Mbarki, H. Seddik, E. B. Braiek, "Rapid and efficient image restoration technique based on new adaptive anisotropic diffusion function," Digital Signal Processing, vol. 48, pp. 201-215, 2016.doi:[[[10.1016/j.dsp.2015.09.013]]]
- 6 P. Perona, J. Malik, "Scale-space and edge detection using anisotropic diffusion," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 12, no. 7, pp. 629-639, 1990.doi:[[[10.1109/34.56205]]]
- 7 H. Luo, L. Zhu, H. Ding, "Coupled anisotropic diffusion for image selective smoothing," Signal Processing, vol. 86, no. 7, pp. 1728-1736, 2006.doi:[[[10.1016/j.sigpro.2005.09.019]]]
- 8 E. Nadernejad, "Improvement of nonlinear diffusion equation using relaxed geometric mean filter for low PSNR images," Electronics Letters, vol. 49, no. 7, pp. 457-458, 2013.custom:[[[-]]]
- 9 S. Osher, L. I. Rudin, "Feature-oriented image enhancement using shock filters," SIAM Journal on Numerical Analysis, vol. 27, no. 4, pp. 919-940, 1990.custom:[[[-]]]
- 10 L. Alvarez, L. Mazorra, "Signal and image restoration using shock filters and anisotropic diffusion," SIAM Journal on Numerical Analysis, vol. 31, no. 2, pp. 590-605, 1994.custom:[[[-]]]
- 11 S. Fu, Q. Ruan, C. Mu, W. Q. WANG, "Feature preserving coupled bidirectional flow for edge sharpening and image enhancement," Chinese Journal of Computers, vol. 31, no. 3, pp. 529-535, 2008.custom:[[[-]]]
- 12 J. T. Xiong, Q. S. Sun, L. C. Li, J. Y. Yang, "An adaptive bidirectional diffusion process for passive millimeter-wave image denoising and enhancement," Journal of Infrared and Millimeter Waves, vol. 30, no. 6, pp. 556-560, 2011.doi:[[[10.3724/sp.j.1010.2011.00556]]]
- 13 A. S. Tolba, H. M. Raafat, "Multiscale image quality measures for defect detection in thin films," The International Journal of Advanced Manufacturing Technology, vol. 79, pp. 113-122, 2015.custom:[[[-]]]
- 14 D. Tang, D. Lu, B. Yang, D. Xu, "Similarity metric between mural images with constraints of the overall structure of contours," Journal of Image and Graphics, vol. 18, no. 8, pp. 968-975, 2013.custom:[[[-]]]