CN103810674B - Based on the image enchancing method relying on the reconstruct of perception object's position - Google Patents

Abstract

The present invention provides an image enhancement method based on position reconstruction of dependent perception objects. The method includes: initially extracting the foreground object of the image; object; formulate an optimization target that depends on the position of the perceived object according to the aesthetic criteria of photography and rely on the positional relationship between the perceived objects, and optimize the composition; complete the background of the missing part of the optimized image, and perform edge trimming to obtain the final image enhancement results. The invention can automatically analyze and extract dependent perception objects in images, and use the analysis results for image composition optimization.

Description

Image Enhancement Method Based on Perception-Dependent Object Position Reconstruction

technical field

The invention relates to the technical field of image processing, in particular to an image enhancement method relying on perceptual object analysis and position reconstruction.

Background technique

With the development of digital media technology, more and more image enhancement techniques are dedicated to improving the visual quality of images. These methods convert human perception of images into computable models, such as hue and sharpness, etc.

Image enhancement and editing technology is a very important tool in the field of computer graphics, such as the Possion blending method proposed by P'erez et al. for image fusion; the Alpha-matting method is used for edge trimming after image synthesis; shape-aware images Editing methods can be used for editing of image content. However, these methods only provide tools for image editing, and do not actively process images according to human senses and photographic viewpoints to make images more beautiful.

Many existing works have studied how to use computational models to evaluate whether an image is beautiful or not. These works are based on the extraction of global or local features of the image, and given quantifiable evaluation criteria. In order to improve the aesthetics of the image, many scholars have extracted Calculate the model and edit the image content accordingly. For example, Santella et al. proposed "Gaze-basedinteraction forsemi-automatic photo cropping", and Nishiyama et al. proposed "Sensation-based photo cropping". Both works are based on photo cropping to optimize image composition; Liu et al. proposed "Optimizing photo composition ”, which uses photo cropping and repositioning operations to optimize the composition of a photo. However, the above methods do not operate on the image at the object level, and the degree of composition that can be changed is limited.

Recently, Bhattacharya et al. proposed "A holistic approach to aestheticenhancement of photographs", which uses the composition rules of photography to move the position of the foreground image. However, this method does not consider the dependence of the foreground image, and cannot guarantee the correctness of the semantic information of the image.

Contents of the invention

(1) Technical problems to be solved

The present invention provides an image enhancement method based on the reconstruction of the position of the dependent perception object, operates the image on the object level, takes into account the dependency between the dependent perception objects, and ensures the correctness of the semantic information of the image.

(2) Technical solution

The present invention provides an image enhancement method based on reconstruction of dependent perception object position, the method comprising:

S1, initially extracting the foreground object of the image;

S2. Dividing the image into several regions, and performing dependency analysis on foreground objects in each region to obtain dependent perception objects;

S3. Formulate an optimization target that depends on the position of the perceived object according to the aesthetic criteria of photography and rely on the positional relationship between the perceived objects, and optimize the composition;

S4. Carry out background completion for the missing part of the optimized image, and perform edge trimming to obtain a final image enhancement result.

Preferably, in the step S1, a saliency segmentation method is used to initially extract the foreground object of the image.

Preferably, the step S2 specifically includes:

S21. The image is divided into several regions by means of image segmentation. If each region has 1/2 of the area covering the foreground object extracted in step S1, it is set as a dependent perception object. If the significance value of the region is low If it is lower than the threshold, it is set as a pure background;

S22. Extracting sharpness, clarity, and color harmony features from the rest of the segmented regions;

S23. Using a multi-label graph cut method to perform region dependency analysis to obtain a final dependency-aware object.

Preferably, the optimization target in step S3 includes the three-point distance of dependent perceptual objects, the diagonal distance of dependent perceptual objects, visual balance items, correlation items between dependent perceptual objects and constraint penalty items.

Preferably, the optimization target is optimized using a heuristic method.

Preferably, in the step S4, a content-aware method is used to complete the background of the missing part of the optimized image.

(3) Beneficial effects

The image enhancement method based on the position reconstruction of dependent perceptual objects proposed by the present invention obtains dependent perceptual objects by performing dependency analysis on foreground objects and other areas of the image, and reconstructs the position of dependent perceptual objects, so that the degree of composition that can be changed is no longer has limitations. The optimization objective considers the relationship between foreground objects on the basis of photography composition rules, making the semantic information more accurate and the image more beautiful.

Description of drawings

Fig. 1 is a flowchart of the steps of the method provided by the present invention.

detailed description

The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments.

The present invention discloses an image enhancement method based on reconstruction of dependent perception object position, as shown in Fig. 1, the method includes:

S1, initially extracting the foreground object of the image;

S2. Dividing the image into several regions, and performing dependency analysis on foreground objects in each region to obtain dependent perception objects;

S3. Formulate an optimization target that depends on the position of the perceived object according to the aesthetic criteria of photography and rely on the positional relationship between the perceived objects, and optimize the composition;

S4. Carry out background completion for the missing part of the optimized image, and perform edge trimming to obtain a final image enhancement result.

This method obtains dependent perceptual objects by performing dependency analysis on the foreground and other regions of the image. Composition optimization is performed on perception-dependent objects. The optimization goal considers the relationship between foreground objects on the basis of photographic composition rules, making the optimized semantic information more accurate.

In the step S1, the foreground object of the image is preliminarily extracted using a saliency segmentation method.

Described step S2 specifically comprises:

S21. The image is divided into several regions by means of image segmentation. If each region has 1/2 of the area covering the foreground object extracted in step S1, it is set as a dependent perception object. If the significance value of the region is low If it is lower than the threshold, it is set as a pure background;

S22. Extracting sharpness, clarity, and color harmony features from the rest of the segmented regions;

S23. Using a multi-label graph cut method to perform region dependency analysis to obtain a final dependency-aware object.

The optimization objective in step S3 includes the three-point distance of dependent perception objects, the diagonal distance of dependent perception objects, visual balance items, correlation items between dependent perception objects and constraint penalty items.

Wherein, the optimization target is optimized using a heuristic method.

In the step S4, a content-aware method is used to complete the background of the missing part of the optimized image.

The final composition-optimized image is better than traditional methods.

Specifically, the method is:

S1. Use the saliency segmentation method to perform preliminary extraction of foreground objects on an input image, specifically including:

S11. Calculate the regional salience value by using the regional contrast method, wherein the salience value interval is [0, 1]. If the salience value of the region is high, it is set as a foreground candidate area, and if the salience value is low, it is set as a background candidate area;

S12. Using an image segmentation method to extract the foreground object target according to the information of the foreground candidate area and the background candidate area. Repeat the above method to extract the foreground objects of the image one by one, wherein the foreground object area extracted last time is set as the background, and the saliency value of the area is reduced. After this step, the foreground objects in the image will be extracted one by one.

S2. Divide the image into several areas, and perform dependency analysis on the segmented areas to obtain dependent perception objects:

S21. The image is divided into several regions by image segmentation, and each segmented region is analyzed: if 1/2 of the area covers the foreground object extracted in step S1, then this region is set as a dependent perception object, if this If the significance value of the region is lower than the threshold t, it is set as pure background;

S22. Extract sharpness E _a , sharpness E _s , and color harmony ΔCH features for the remaining regions:

The sharpness E _a is: ${\mathrm{E.}}_a=G\left[\frac{1}{\mathrm{no}}\underset{i=11}{\overset{\mathrm{no}}{\mathrm{\Σ}}}\mathrm{\δ}\left(i\right)\mathrm{D.}\left(i\right)\right]---\left(1\right)$

Wherein D(i) is the second order derivative of the image, if D(i)>0.1, then the value of δ(i) is 1, n is the number of pixels in the area, and G is the Gaussian normalization function.

The sharpness E _s is: ${\mathrm{E.}}_S=G[{\mathrm{\Σ}}_{(u,v)\∈f_h}f(u,v)/{\mathrm{\Σ}}_{(u,v)\∈f_L}f(u,v)]--\left(2\right)$

Among them, F _H ＝{(u, v)|βW<|uu ₀ |≤αW, βH<|vv ₀ |≤αH}

F _L ＝{(u, v)||uu ₀ |≤βW, |vv ₀ |≤βH}

W and H are the width and height of the image respectively, F _H is the high frequency band, F _L is the low frequency band, (u ₀ , v ₀ ) is the center frequency, α=0.4, β=0.2.

The color harmony ΔCH is:

ΔCH=exp{(CH+5) ² /2} (3)

According to A Color Harmony Model for Two-ColourCombinations written by Ou et al. on color harmony, CH=HC+ _{H L} + _{H H ,} where HC is the color item, HL is the brightness item, and _{H H is} the hue item. After obtaining the characteristics of the region, use the Multi-label GraphCut method to perform region dependency analysis to obtain the final dependent perception object. The energy equation is:

$\mathrm{<span\; class="notranslate">\; E.</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; L</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}_{{\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; \&Element;</span>\mathrm{<span\; class="notranslate">\; \&Omega;</span>}}{\mathrm{<span\; class="notranslate">\; D.</span>}}_{{\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; L</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}_{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&Element;</span>\mathrm{<span\; class="notranslate">\; N</span>}}{\mathrm{<span\; class="notranslate">\; T</span>}}_{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; L</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; L</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 4</span>}<span\; class="notranslate">\; )</span>$

in,

${\mathrm{<span\; class="notranslate">\; D.</span>}}_{\mathrm{<span\; class="notranslate">\; r</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; L</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>{<span\; class="notranslate">\; \{</span><span\; class="notranslate">\; [</span>{\mathrm{<span\; class="notranslate">\; E.</span>}}_{\mathrm{<span\; class="notranslate">\; S</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; r</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; E.</span>}}_{\mathrm{<span\; class="notranslate">\; S</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; L</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ]</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>{<span\; class="notranslate">\; [</span>{\mathrm{<span\; class="notranslate">\; E.</span>}}_{\mathrm{<span\; class="notranslate">\; a</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; r</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; E.</span>}}_{\mathrm{<span\; class="notranslate">\; a</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; L</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ]</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; \&Delta;CH</span>}}_{\mathrm{<span\; class="notranslate">\; r</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; L</span>}}{<span\; class="notranslate">\; \}</span>}^{\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; 2</span>}}}$

${\mathrm{<span\; class="notranslate">\; T</span>}}_{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; L</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; L</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\left\{\begin{array}{cc}\mathrm{<span\; class="notranslate">\; 0</span>}<span\; class="notranslate">\; ,</span>& \mathrm{<span\; class="notranslate">\; if</span>}{\mathrm{<span\; class="notranslate">\; L</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; L</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}\\ {\mathrm{<span\; class="notranslate">\; D.</span>}}_{{\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ,</span>& \mathrm{<span\; class="notranslate">\; otherwise</span>}\end{array}\right.$

Among them, r represents the segmented region and L represents the label.

S3. Formulate an optimization target that depends on the position of the perceptual object based on the aesthetic criteria of photography and the positional relationship between the dependent perceptual objects, and use a heuristic method to reconstruct the position: first define the layout optimization target as:

$\mathrm{<span\; class="notranslate">\; E.</span>}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; D.</span>}}_{\mathrm{<span\; class="notranslate">\; P</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; D.</span>}}_{\mathrm{<span\; class="notranslate">\; L</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; D.</span>}}_{\mathrm{<span\; class="notranslate">\; V</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; no</span>}}{\mathrm{<span\; class="notranslate">\; P</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; \&omega;</span>}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; no</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; \&NotEqual;</span>\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; no</span>}}\mathrm{<span\; class="notranslate">\; R</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 5</span>}<span\; class="notranslate">\; )</span>$

Where n represents the number of perception-dependent objects, and the three-point point of the image means that if the length and width of the image are normalized to the [0, 1] interval, the coordinates on the image are (1/3, 1/3) , (1/3, 2/3), (2/3, 1/3), (2/3, 2/3) four points.

The energy term D _P of the third-point distance depending on the perceived object is:

${\mathrm{<span\; class="notranslate">\; D.</span>}}_{\mathrm{<span\; class="notranslate">\; P</span>}}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}^{\mathrm{<span\; class="notranslate">\; no</span>}}{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; c</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; P</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 6</span>}<span\; class="notranslate">\; )</span>$

Among them, m _i represents the number of pixels of the dependent perceptual object, c _i represents the centroid of the dependent perceptual object, and P _j represents the nearest third point to the dependent perceptual object.

The diagonal distance energy item D _L dependent on the perceived object is:

${\mathrm{<span\; class="notranslate">\; D.</span>}}_{\mathrm{<span\; class="notranslate">\; L</span>}}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}^{\mathrm{<span\; class="notranslate">\; no</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; \&NotEqual;</span>\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; no</span>}}\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; f</span>}}_{\mathrm{<span\; class="notranslate">\; a</span>}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; l</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; L</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; L</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; f</span>}}_{\mathrm{<span\; class="notranslate">\; d</span>}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; l</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; L</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; L</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 7</span>}<span\; class="notranslate">\; )</span>$

Among them, L ₁ and L ₂ are the two diagonal lines of the image, l _{i, j} are two straight lines depending on the centroid of the perceived object, M _{i, j} is the midpoint of the two centroids, θ _k is the distance between l _{i, j} and The included angle of L _k .

f _a (l _{i, j} , L ₁ , L ₂ )=|θ ₁ ||θ ₂ |/4π ² , $f_d(l_{i,j},L_1,L_2)=\sqrt{2}d(m_{i,j},L_1)\&Center\; Dot;\sqrt{2}d(m_{i,j},L_2);$ d(M, L) represents the distance from the point M to the straight line L.

The visual equalization energy item D _V is:

${\mathrm{<span\; class="notranslate">\; D.</span>}}_{\mathrm{<span\; class="notranslate">\; V</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; exp</span>}<span\; class="notranslate">\; \{</span><span\; class="notranslate">\; -</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; 2</span>}\mathrm{<span\; class="notranslate">\; \&sigma;</span>}}{\mathrm{<span\; class="notranslate">\; d</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; C</span>}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; m</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \}</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 8</span>}<span\; class="notranslate">\; )</span>$

Among them, m _i represents the number of pixels dependent on the perceived object, c _i represents the centroid of the dependent perceived object, and the visual balance point C is the center of the image, d(a, b) represents the distance between two points, σ=0.2.

The correlation item R between dependent perception objects is:

R(i,j)=S(i,j)||Δ _i,j -Δ′ _i,j || (9)

Where S(i, j)=λS _shape (i, j)+(1-λ)S _color (i, j); S _shape is the Shapecontext similarity between dependent perception objects, and S _color is the color histogram of dependent perception objects The chi-square distance of , Δ _{i, j} represents the distance between the two dependent perception objects before optimization, and Δ′ _{i, j} represents the distance between the two dependent perception objects after optimization.

The constraint penalty term P can constrain the final position to be the nearest optimal solution. Use the heuristic method to optimize E to obtain the optimized position.

S4. Use the content-aware method to complete the background of the missing part of the optimized image, and perform alpha-matting processing to obtain the final image enhancement result: after optimizing the position of the dependent perception object in the image, it is necessary to use the content The perceptual image completion method completes the missing part of the image. At the same time, in some cases, it needs to trim the boundary part that depends on the perceptual object, and use the result as the final image after the composition optimization of the output.

The above is only a preferred embodiment of the present invention, it should be pointed out that for those of ordinary skill in the art, without departing from the technical principle of the present invention, some improvements and replacements can also be made, these improvements and replacements It should also be regarded as the protection scope of the present invention.

Claims (6)

1. based on the image enhancement method that relies on perception object position reconstruction, it is characterized in that, the method comprises the following steps: S1, initially extracting the foreground object of the image; S2. Dividing the image into several regions, and performing dependency analysis on foreground objects in each region to obtain dependent perception objects; S3. Formulate an optimization target that depends on the position of the perceived object according to the aesthetic criteria of photography and rely on the positional relationship between the perceived objects, and optimize the composition; S4. Carry out background completion for the missing part of the optimized image, and perform edge trimming to obtain a final image enhancement result. 2. The method according to claim 1, characterized in that, in the step S1, a salient segmentation method is used to initially extract the foreground object of the image. 3. The method according to claim 1, wherein said step S2 specifically comprises: S21. The image is divided into several regions by means of image segmentation. If each region has 1/2 of the area covering the foreground object extracted in step S1, it is set as a dependent perception object. If the significance value of the region is lower than Threshold, it is set to pure background; S22. Extracting sharpness, clarity, and color harmony features from the rest of the segmented regions; S23. Using a multi-label graph cut method to perform region dependency analysis to obtain a final dependency-aware object. 4. The method according to claim 1, characterized in that, the optimization target in step S3 includes the three-point distance of dependent perceptual objects, the diagonal distance of dependent perceptual objects, visual balance items, and the correlation between dependent perceptual objects item and constraint penalty item. 5. The method according to claim 4, wherein a heuristic method is used to optimize the optimization target. 6. The method according to claim 1, characterized in that, in the step S4, a content-aware method is used to complete the background of the missing part of the optimized image.