Shunfeng Wang* , Jiacen Xie* , Yuhui Zheng** , Jin Wang*** and Tao Jiang*A Method of Coupling Expected Patch Log Likelihood and Guided Filtering for Image De-noisingAbstract: With the advent of the information society, image restoration technology has aroused considerable interest. Guided image filtering is more effective in suppressing noise in homogeneous regions, but its edge-preserving property is poor. As such, the critical part of guided filtering lies in the selection of the guided image. The result of the Expected Patch Log Likelihood (EPLL) method maintains a good structure, but it is easy to produce the ladder effect in homogeneous areas. According to the complementarity of EPLL with guided filtering, we propose a method of coupling EPLL and guided filtering for image de-noising. The EPLL model is adopted to construct the guided image for the guided filtering, which can provide better structural information for the guided filtering. Meanwhile, with the secondary smoothing of guided image filtering in image homogenization areas, we can improve the noise suppression effect in those areas while reducing the ladder effect brought about by the EPLL. The experimental results show that it not only retains the excellent performance of EPLL, but also produces better visual effects and a higher peak signal-to-noise ratio by adopting the proposed method. Keywords: Edge Preserving , Expected Patch Log Likelihood , Image De-noising , Guided Filtering 1. IntroductionThe image is the most direct form by which humans perceive the world, and it has been widely applied in various fields. However, images are unavoidably polluted by noise in the process of acquisition and transmission. Therefore, image de-noising plays an indispensable role in our daily life. In order to remove the noise from images effectively, numerous image de-noising approaches have been presented, such as the traditional Bayes method [1-4] and various regularization methods. Among these, total variation (TV) regularization [5], as a representative regularization de-noising method, has attracted considerable attention because of its low computational complexity and well-understood mathematical behavior. Since the introduction of TV regularization in the context of image processing, many researchers have recently presented various mature algorithms and extended the applications of TV functional [6-12]. With the observation that similar image patches are ubiquitous in the whole image, a non-local type of average filtering has been proposed [13-15]. Since then, a self-similarity-based non-local method has been widely studied for image regularization [16]. Protter et al. [17] proposed a non-local total variational regularization model.Gao et al. [18] used the Zernike moment to identify the similarity between image patches. A better image restoration effect can be obtained by using these methods. Other de-noising methods based on non-local information include K-SVD [19] and BM3D [20,21] among others. Beyond that, classical filters like Wiener filtering [22,23] and Kalman filtering [24] have been used till now. The principle of Wiener filtering is to transform the problem into the minimizing of the mean square error between the original and estimated values. The Expected Patch Log Likelihood (EPLL) image restoration techniques proposed by Zoran and Weiss [25], who utilized a mixture model to learn image patch prior, have attracted a lot of attention. As one of the most popular methods in the field of non-local information de-noising, it is characterized by a higher peak signal-to-noise ratio (PSNR) and a better visual effect. Many scholars have been keen to improve it [26-30], but there is still great potential for improvement in terms of the quality of denoising. For example, when the noise level is high, the de-noising results of EPLL tend to exhibit the ladder effect in the smooth regions. Guided image filtering [31] can establish the filter kernel explicitly, and its guided image can be the input image or the related of input, while the output image is a local linear transformation of the guided image. When using guided filtering to de-noise an image, we usually use the input image as the guided image to smooth and de-noise the image. But in the process of an experiment, it is found that when the noise level is low, guided filtering can remove the noise and keep the edge well. With the increase of noise, such details as the edge and texture of the image are polluted by noise, which has a great impact on the de-noising results rather than providing effective guidance information, when using this filtering. In summary, we consider that guided image filtering has a better noise suppression effect in homogeneous regions, whereas its edge-preserving property is poor. The key to guided filtering is the selection of the guided image. The result of EPLL de-noising helps achieve a good structure, but it is easy to cause the ladder effect in homogeneous areas. For this reason, we propose a method of coupling the EPLL and guided filtering for image de-noising that incorporates these two methods, which complement and promote each other, thus enhancing the effect of image de-noising. The remainder of this paper is organized as follows. Section 2 presents a brief summary of the basic theory of the EPLL model and guided filtering; Section 3 elaborates upon the details of how to integrate the EPLL with guided filtering and analyze the benefits of such a process; Section 4 presents the experimental results of image de-noising as supporting evidence for the effectiveness of the proposed method; and Section 5 presents some conclusions about the method of coupling EPLL and guided filtering for image de-noising, as well as directions for future studies. 2. Guided Image Filtering and Expected Patch Log Likelihood2.1 Guided Image FilteringThe guided filter supposes that the output image q has a local linear relationship with the guided image I[31]. That is, in the window ωk centered on the pixel k, q is satisfied with a linear transformation associated with I:
where (ak,bk) is the linear constant coefficient in the square window ωk, the radius of ωk is r. In order to determine the linear coefficient (ak,bk) , the following cost functions are introduced in [3]:
(2)[TeX:] $$\left( a _ { k } , b _ { k } \right) = \sum _ { i \in \omega _ { k } } \left( \left( a _ { k } I _ { i } + b _ { k } - p _ { i } \right) ^ { 2 } + \varepsilon a _ { k } ^ { 2 } \right)$$where ε is a regularization parameter, [TeX:] $$q _ { i } = p _ { i } - n _ { i } ; p _ { i }$$ is the pixel of the input image, and ni is the noise of this pixel. (ak,bk) can be obtained by minimizing Eq. (2):
(3)[TeX:] $$a _ { k } = \frac { \frac { 1 } { | \omega | } \sum _ { i \in \omega _ { k } } I _ { i } p _ { i } - \mu _ { k } \overline { p } _ { k } } { \sigma _ { k } ^ { 2 } + \varepsilon }$$
where [TeX:] $$\mu _ { k } \text { and } \sigma _ { k } ^ { 2 }$$ represent the mean and variance of the guided image I in the window ωk respectively; |ω| is the number of pixels in the window ωk, and pk is the mean of the input image p in the window ωk. The output value qi of the pixel i is related to all the windows containing the pixel i. Therefore, in order to obtain a stable qi, we need to average it. So the final output image qi is:
(5)[TeX:] $$q _ { i } = \overline { a } _ { i } I _ { i } + \overline { b } _ { i } = \frac { 1 } { | \omega | } \sum _ { k \in \omega _ { i } } \left( a _ { k } I _ { i } + b _ { k } \right) , \forall i \in I$$EPLL is a method of restoring an image by using the statistical information of the external image patch. The basic idea is to maximize the likelihood probability of the image patch, and make the restored image patch close to the prior. For a degraded image X and a known prior knowledge, EPLL is defined as:
where Pi is an operator to extract image patch on the i the pixel, then PiX is the extracted patch. logp(PiX) indicates the logarithmic value of the likelihood probability of the i th patch under a given prior distribution. For a given degraded image Y, the cost function i:
(7)[TeX:] $$f _ { p } ( X | Y ) = \frac { \lambda } { 2 } \| A X - Y \| ^ { 2 } - E P L L _ { p } ( X )$$where A is a degenerate matrix and λ is the regularization parameter. The “Half Quadratic Splitting” [25] method can be used to optimize Eq. (7). A set of auxiliary variables {zi} is introduced to be equal to PiX, yielding the following cost function:
(8)[TeX:] $$f _ { p , \beta } \left( X , \left\{ z _ { i } \right\} | Y \right) = \frac { \lambda } { 2 } \| A X - Y \| ^ { 2 } + \sum _ { i } \frac { \beta } { 2 } \left( \left\| P _ { i } X - z _ { i } \right\| ^ { 2 } - \log p \left( z _ { i } \right) \right)$$As β tends to infinity, we obtain that {zi} will be equal to PiX and the solutions to Eq. (8) and Eq. (7) converge. We know that many popular image priors can be regarded as a special case of Gaussian Mixture Model (GMM). The GMM can also be used in EPLL, and the non-Gaussian distribution is represented by the combination of several single Gaussian distributions. Thus, the log likelihood of a given patch PiX is:
(9)[TeX:] $$\log p \left( P _ { i } X \right) = \log \left( \sum _ { k = 1 } ^ { K } \pi _ { k } N \left( P _ { i } X | \mu _ { k } , \Sigma _ { k } \right) \right)$$where K is the number of mixture components, πk is the mixing weight for each mixture component, and μk and Σk are the corresponding mean and covariance matrix. To solve Eq. (6), we choose the most likely Gaussian mixing weight kmax for each patch PiX, and then Eq. (6) is minimized by alternatively updating zi and X:
(10)[TeX:] $$z _ { i } ^ { n + 1 } = \left( \Sigma _ { k _ { \max } } + \frac { 1 } { \beta } I \right) ^ { - 1 } \left( P _ { i } X ^ { n } \Sigma _ { k _ { \max } } + \frac { 1 } { \beta } I \mu _ { k _ { \max } } \right)$$
(11)[TeX:] $$X ^ { n + 1 } = \left( \lambda A ^ { T } A + \beta \sum _ { j } P _ { j } ^ { T } P _ { j } \right) ^ { - 1 } \left( \lambda A ^ { T } y + \beta \sum _ { j } P _ { j } ^ { T } z _ { j } ^ { n + 1 } \right)$$where μkmax and Σkmax are the corresponding mean and covariance matrix with the mixing weight kmax, and I is an identity matrix. 3. Proposed Method with the Combination of EPLL and Guided FilteringGuided image filtering has two advantages: it has good edge-preserving properties, so it will not cause a “gradient reversal”, and it can also be used for other purposes than smoothing. Moreover, with the help of the guiding image, the output image is more structured. The image obtained by EPLL has a better structure and can provide better auxiliary information for the guided filter. Based on the advantages of the two models, this study proposes a method of coupling the expected patch log likelihood and guided filtering for image de-noising, with the aim of improving the de-noising performance. The algorithm implementation steps (Table 1) are as follows: This method is mainly used to eliminate Gaussian additive noise. It has a very good smoothing effect while maintaining the edge information effectively, which may be attributable to the fact that it performs the second smoothing filtering for homogeneous regions on the basis of EPLL. When the guided image is fixed, the filtering results mainly depend on the coefficients (ak,bk). Table 1.
In the iteration process, the input image is the same as the guided image, and Eqs. (3) and (4) can be simplified as:
(12)[TeX:] $$a _ { k } = \frac { \sigma _ { k } ^ { 2 } } { \sigma _ { k } ^ { 2 } + \varepsilon }$$
The parameter ε determines whether a pixel is in a boundary region (Table 2). Table 2.
It can been seen from the above analysis that the edge-preserving effect of the guided filter depends on the guided image, while the EPLL model can present a better guiding image for guided filtering. In contrast, the second smoothing of flat regions obtained by the guided filtering can improve the noise suppression effect of the EPLL and avoid the ladder effect. These two methods complement and reinforce each other perfectly. The next sections prove the validity of the proposed method via experiments. 4. Experimental ResultsIn this section, we discuss the performance of the proposed method. In this study’s experiments, the GMM with 200 mixture components was learned from a set of 2 × 106 images patches, which were sampled from the Berkeley Segmentation Database Benchmark (BSDS300). In order to verify the effectiveness of the proposed method, it was compared with guided filtering and EPLL in terms of visual effects and numerical results. To the images used in this study’s experiments were added Gaussian noise with zero mean and standard variance, σ = 15 or σ = 30. The parameters for EPLL in the experiments were as follows: image patch size √L = 8, regularization parameter λ = L/σ2, and penalty parameter β = 1⁄σ2 ∗ [1 2 4 8 16]. Meanwhile, the parameters of guided filtering were as follows: radius r = 2, and regularization parameter ε = 0.022. The results are as follows: Fig. 1 displays the de-noised results of the three methods on Couple image with dimensions of 512×512. Here, Fig. 1(a) is the original clean image; Fig. 1(b) is the noisy image generated by adding Gaussian white noise with zero mean and standard variance σ = 30 to the original image; Fig. 1(c) shows the result of the guider filtering, but the edges, details, and other information have not been preserved well; Fig. 1(d) shows the de-noising results of the EPLL model, where it can be seen that mottling occurs in certain areas; and Fig. 1(e) shows the de-noising result of the proposed method. This shows a better visual effect at the boundary of the wall. The boundary is preserved and the transition is more natural in the smooth region. As shown in Fig. 2, the local enlargement of the results of the three algorithms confirms the above analysis. Figs. 3 and 4 show the same result. The proposed method makes the restoration result smoother and preserves more details. Thus, it is reasonable to conclude that the proposed method is superior to EPLL in terms of numerical results, as shown in Table 3. Table 3.
5. ConclusionBased on the complementarity of EPLL and guided filtering, this paper proposes a method of coupling the expected patch log likelihood and guided filtering for image de-noising. It uses the EPLL model to construct the guided image for guided filtering, which can provide better structural information for guided filtering. Meanwhile, by the secondary smoothing of guided image filtering in the image homogenization areas, we can improve the noise suppression effect in those areas, and reduce the ladder effect brought about by EPLL. The experimental results show that the proposed method is better than the previous two methods, in terms of both the visual effect and numerical performance. This combination makes full use of the advantages of the two methods while making up for their shortcomings, which makes the two complement and progress each other. Of course, there are still some shortcomings in this method. For example, the selection of the parameters ε and iteration times are all artificially set, and their values directly determine whether the algorithm will be overly smooth or not. As such, any future research will focus on this aspect. BiographyShunfeng Wanghttps://orcid.org/0000-0002-9482-7775She received the M.S. degree in the Nanjing University of Information Science and Technology, China in 2002. Now she is a Professor at the College of Mathematics and Statistics, Nanjing University of Information Science and Technology. Her research interests cover pattern recognition, and information processing. BiographyJin Wanghttps://orcid.org/0000-0002-6516-6787He received the B.S. and M.S. degrees from Nanjing University of Posts and Telecommunications, China in 2002 and 2005, respectively. He received Ph.D. degree from Kyung Hee University Korea in 2010. Now, he is a professor in the School of Computer Communication Engineering, Changsha University of Science Technology. His research interests mainly include routing protocol and algorithm design, performance evaluation and optimization for wireless ad hoc and sensor networks. He is a member of the IEEE and ACM. References
|