CN110473245A - A kind of depth image document screening method and system - Google Patents

Abstract

The invention discloses a kind of depth image document screening method and system, it will real-time collected depth image file division be currently multiple subgraphs, and the characteristics of image for extracting subgraph carries out abnormal point estimation, to which the depth image file for filtering out qualified is stored, the less depth image of abnormal point can be quickly and effectively filtered out to be saved, it saves memory space and improves the collection effect and collecting efficiency of depth image acquisition equipment, improve the precision of collected depth image, it effectively increases and obtains accurate colouring information and depth information from depth image and then the reduction degree of the threedimensional model that restores object out or scene.

Description

Depth image file screening method and system

Technical Field

The disclosure relates to the technical field of file storage and image processing, in particular to a depth image file screening method and system.

Background

The depth image is also called a distance image, and refers to an image taking the distance (depth) from an image collector to each point in a scene as a pixel value, and directly reflects the geometric shape of a visible surface of a scene. Therefore, a part of depth images have low storage value, and the storage space and the acquisition effect are seriously wasted, so that the acquisition efficiency, the image effect and the acquisition precision of the conventional depth image acquisition are completely unsatisfactory.

Disclosure of Invention

In order to solve the above problems, the present disclosure provides a technical solution of a depth image file screening method and system, which divides a depth image file acquired in real time at present into a plurality of sub-images, extracts image features of the sub-images to perform abnormal point estimation, and thereby screens out a qualified depth image file for storage.

In order to achieve the above object, according to an aspect of the present disclosure, there is provided a depth image file screening method including the steps of:

s100, recording the depth image as a first image;

s200, dividing the first image into a plurality of sub-images through a watershed algorithm, wherein the number of the obtained sub-images is n, and the sub-images are recorded as sigma_i,(0<i≤n)；

S300, extracting the image characteristics of the outlines of the sub-images;

s400, carrying out abnormal point estimation according to the image characteristics;

s500, discarding the first image when the number of the abnormal points in the first image is larger than an abnormal threshold;

s600, saving the first image when the number of the abnormal points in the first image is smaller than or equal to the abnormal threshold.

Further, in S100, the depth image is an image captured by any one of a laser radar depth imaging method, a computer stereo vision imaging method, a coordinate measuring machine method, a moire fringe method, and a structured light method at a depression angle of 30 ° with respect to the horizon.

Further, in S300, the method for extracting the image feature of each sub-image contour includes:

the method for extracting the outline of each sub-image comprises the steps of detecting the outline of the edge of the image through any one of a Canny operator, a Sobel operator and a Perwit operator; the image features are:

Feature＝{Gr_i,H_i,A_i,P_i,Nl_i,Vl_i} (1)

in the formula, Gr_iRepresenting sub-images ∑_iMean value of gray scale of_i＝{h_i,s_iDenotes the sub-image Σ_iThe component values of H and S with the largest occurrence frequency are the components of H and S in the HSV color space of the sub-image; a. the_iThe total number of pixels in the sub-image; p_iRepresenting Contour_iPosition of center of gravity of, Nl_iThe straight line, Vl, in the contour of the sub-image after the contour is disassembled is recorded_iIs a perpendicular line in the contour line after the contour of the subimage is disassembled;

Nl_iand Vl_iThe calculation method of (2) is as follows:

firstly, the edge point P on the contour line of the edge image of the sub-image is used_j(x_ij,y_ij),(H<j is less than or equal to m-H) as the center, and constructing a sliding window Slop with the size of L to the Contour_iCarrying out local direction coding, wherein the default setting is that L is 80 pixels, m is the total number of pixels of a contour line, and the contour line consists of edge points;

the local direction coding method comprises the following steps: let the coordinates of the end points on both sides of the contour line segment cut by Slop be (x)_i1,y_i1),(x_i2,y_i2) Then edge point P_jThe local direction θ of (a) is:

where the linear ratio factor s is D/L, D represents the endpoint (x)_i1,y_i1),(x_i2,y_i2) The linear distance between the two adjacent sliding windows, L is the length of the contour obtained by the sliding window, T is a set threshold value, and T is set to be 0.5 by default;

the edge of the encoded sub-image edge image is split into a plurality of mutually continuous straight line segments at the corner point, and each straight line segment is Nl_i＝{l₁,l₂,…,l_kTherein ofk is the total number of lines in the profile, m₁And m₂Respectively showing the number of pixels of the starting point and the end point of the kth straight line in the outline from the starting point of the outline.

Contour for sub-image_iStraight line l in_kTaking two endpoints as A and B, l_kThe angle of inclination of (a) is calculated as:

wherein,

if inclined at an angle of | theta_k-90|<σ (default setting σ of the present disclosure is 10), i.e., determine l_kIs a vertical line and is saved to Vl_i。

Further, in S400, the method for performing outlier estimation according to the image features includes:

vertical lines Vl in all sub-image feature vectors_iForm parallel vertical linear sequences of lines₁,line₂,...,line_i...,line_pWherein, a line of a straight line_iThe equation of the straight line of (1) is:

y＝a_ix+b_i (4)

line pair according to least square method_iAnd (3) carrying out linear fitting to obtain the minimum value of the error of the fitted straight line:

determining the slope a_iY-axis intercept b_iThen, solving the intersection point by simultaneously establishing any two fitting straight lines:

each cross point obtained by calculation is an abnormal point set; clustering each intersection point by a fuzzy c-means clustering algorithm, and selecting the center of the most various members in the clustering group as an abnormal point V from the fuzzy c-means clustering result_r＝{x_i,y_iAnd obtaining the number of the abnormal points in the first image.

Further, in S500 and S600, the abnormality threshold is an integer value set manually, and in the present disclosure, the abnormality threshold is set to be 30 by default.

The invention also provides a depth image file screening system, which comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the computer program to run in the units of the following system:

an image reading unit for recording the depth image as a first image;

an image segmentation unit for segmenting the first image into a plurality of sub-images by a watershed algorithm;

the characteristic extraction unit is used for extracting the image characteristics of the outlines of the sub-images;

an abnormal point estimation unit for estimating abnormal points according to the image characteristics;

the image abandoning unit is used for abandoning the first image and transferring the first image to the image reading unit when the number of the abnormal points in the first image is larger than the abnormal threshold value;

and the image saving unit is used for saving the first image when the number of the abnormal points in the first image is less than or equal to the abnormal threshold value.

The beneficial effect of this disclosure does: the invention provides a depth image file screening method and system, which can quickly and effectively screen out depth images with fewer abnormal points for storage, save storage space, improve the acquisition effect and acquisition efficiency of depth image acquisition equipment, improve the accuracy of the acquired depth images, and effectively improve the reduction degree of a three-dimensional model for recovering an object or a scene by obtaining accurate color information and depth information from the depth images.

Drawings

The foregoing and other features of the present disclosure will become more apparent from the detailed description of the embodiments shown in conjunction with the drawings in which like reference characters designate the same or similar elements throughout the several views, and it is apparent that the drawings in the following description are merely some examples of the present disclosure and that other drawings may be derived therefrom by those skilled in the art without the benefit of any inventive faculty, and in which:

FIG. 1 is a flow chart of a depth image file screening method;

fig. 2 is a diagram showing a structure of a depth image file screening system.

Detailed Description

The conception, specific structure and technical effects of the present disclosure will be clearly and completely described below in conjunction with the embodiments and the accompanying drawings to fully understand the objects, aspects and effects of the present disclosure. It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict.

Fig. 1 is a flowchart illustrating a depth image file screening method according to the present disclosure, and a depth image file screening method according to an embodiment of the present disclosure is described below with reference to fig. 1.

The disclosure provides a depth image file screening method, which specifically comprises the following steps:

s100, recording the depth image as a first image;

s200, dividing the first image into a plurality of sub-images through a watershed algorithm, wherein the number of the obtained sub-images is n, and the sub-images are recorded as sigma_i,(0<i≤n)；

S300, extracting the image characteristics of the outlines of the sub-images;

s400, carrying out abnormal point estimation according to the image characteristics;

s500, discarding the first image when the number of the abnormal points in the first image is larger than an abnormal threshold;

s600, saving the first image when the number of the abnormal points in the first image is smaller than or equal to the abnormal threshold.

Further, in S100, the depth image is an image captured by any one of a laser radar depth imaging method, a computer stereo vision imaging method, a coordinate measuring machine method, a moire fringe method, and a structured light method at a depression angle of 30 ° with respect to the horizon.

Further, in S300, the method for extracting the image feature of each sub-image contour includes:

the method for extracting the outline of each sub-image comprises the steps of detecting the outline of the edge of the image through any one of a Canny operator, a Sobel operator and a Perwit operator; the image features are:

Feature＝{Gr_i,H_i,A_i,P_i,Nl_i,Vl_i} (1)

in the formula, Gr_iRepresenting sub-images ∑_iMean value of gray scale of_i＝{h_i,s_iDenotes the sub-image Σ_iThe component values of H and S with the largest occurrence frequency are the components of H and S in the HSV color space of the sub-image; a. the_iThe total number of pixels in the sub-image; p_iRepresenting Contour_iPosition of center of gravity of, Nl_iThe straight line, Vl, in the contour of the sub-image after the contour is disassembled is recorded_iIs a perpendicular line in the contour line after the contour of the subimage is disassembled;

Nl_iand Vl_iThe calculation method of (2) is as follows:

firstly, the edge point P on the contour line of the edge image of the sub-image is used_j(x_ij,y_ij),(H<j is less than or equal to m-H) as the center, and constructing a sliding window Slop with the size of L to the Contour_iCarrying out local direction coding, wherein the default setting is that L is 80 pixels, m is the total number of pixels of a contour line, and the contour line consists of edge points;

the local direction coding method comprises the following steps: let the coordinates of the end points on both sides of the contour line segment cut by Slop be (x)_i1,y_i1),(x_i2,y_i2) Then edge point P_jThe local direction θ of (a) is:

where the linear ratio factor s is D/L, D represents the endpoint (x)_i1,y_i1),(x_i2,y_i2) The linear distance between the two adjacent sliding windows, L is the length of the contour obtained by the sliding window, T is a set threshold value, and T is set to be 0.5 by default;

the edge of the encoded sub-image edge image is split into a plurality of mutually continuous straight line segments at the corner point, and each straight line segment is Nl_i＝{l₁,l₂,…,l_kTherein ofk is the total number of lines in the profile, m₁And m₂Respectively showing the number of pixels of the starting point and the end point of the kth straight line in the outline from the starting point of the outline.

Contour for sub-image_iStraight line l in_kTaking two endpoints as A and B, l_kThe angle of inclination of (a) is calculated as:

wherein,

if inclined at an angle of | theta_k-90|<σ (default setting σ of the present disclosure is 10), i.e., determine l_kIs a vertical line and is saved to Vl_i。

Further, in S400, the method for performing outlier estimation according to the image features includes:

vertical lines Vl in all sub-image feature vectors_iForm parallel vertical linear sequences of lines₁,line₂,...,line_i...,line_pWherein, a line of a straight line_iThe equation of the straight line of (1) is:

y＝a_ix+b_i (4)

line pair according to least square method_iAnd (3) carrying out linear fitting to obtain the minimum value of the error of the fitted straight line:

determining the slope a_iY-axis intercept b_iThen, solving the intersection point by simultaneously establishing any two fitting straight lines:

each cross point obtained by calculation is an abnormal point set; clustering each intersection point by a fuzzy c-means clustering algorithm, and selecting a clustering group from fuzzy c-means clustering resultsThe center of the most various members is used as an abnormal point V_r＝{x_i,y_iAnd obtaining the number of the abnormal points in the first image.

Preferably, in S500, the first image is discarded as the currently acquired first image is not saved, that is, the defective image is discarded; saving the first image in S600 is saving the first image to a memory, i.e., saving the qualified image.

Further, in S500 and S600, the anomaly threshold is an integer value set manually, and in the present disclosure, the anomaly threshold is set to 30 by default according to test experience.

A depth image file screening system provided in an embodiment of the present disclosure is a depth image file screening system structure diagram of the present disclosure as shown in fig. 2, and a depth image file screening system of the embodiment includes: a processor, a memory and a computer program stored in the memory and executable on the processor, the processor implementing the steps in one of the embodiments of the depth image file screening system described above when executing the computer program.

The system comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the computer program to run in the units of the following system:

an image reading unit for recording the depth image as a first image;

an image segmentation unit for segmenting the first image into a plurality of sub-images by a watershed algorithm;

the characteristic extraction unit is used for extracting the image characteristics of the outlines of the sub-images;

an abnormal point estimation unit for estimating abnormal points according to the image characteristics;

the image abandoning unit is used for abandoning the first image and transferring the first image to the image reading unit when the number of the abnormal points in the first image is larger than the abnormal threshold value;

and the image saving unit is used for saving the first image when the number of the abnormal points in the first image is less than or equal to the abnormal threshold value.

The depth image file screening system can be operated in computing equipment such as desktop computers, notebooks, palm computers and cloud servers. The depth image file screening system can be operated by a system comprising, but not limited to, a processor and a memory. Those skilled in the art will appreciate that the example is merely an example of a depth image file filtering system and is not intended to limit a depth image file filtering system, and may include more or less components than, or in combination with, certain components, or different components, e.g., an input-output device, a network access device, a bus, etc.

The Processor may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic, discrete hardware components, etc. The general purpose processor may be a microprocessor or the processor may be any conventional processor or the like, the processor is a control center of the operating system of the depth image file screening system, and various interfaces and lines are used to connect various parts of the entire operating system of the depth image file screening system.

The memory may be used to store the computer programs and/or modules, and the processor may implement the various functions of the depth image file screening system by running or executing the computer programs and/or modules stored in the memory and calling the data stored in the memory. The memory may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required by at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may store data (such as audio data, a phonebook, etc.) created according to the use of the cellular phone, and the like. In addition, the memory may include high speed random access memory, and may also include non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), at least one magnetic disk storage device, a Flash memory device, or other volatile solid state storage device.

While the present disclosure has been described in considerable detail and with particular reference to a few illustrative embodiments thereof, it is not intended to be limited to any such details or embodiments or any particular embodiments, but it is to be construed as effectively covering the intended scope of the disclosure by providing a broad, potential interpretation of such claims in view of the prior art with reference to the appended claims. Furthermore, the foregoing describes the disclosure in terms of embodiments foreseen by the inventor for which an enabling description was available, notwithstanding that insubstantial modifications of the disclosure, not presently foreseen, may nonetheless represent equivalent modifications thereto.

Claims (5)

1. A depth image file screening method is characterized by comprising the following steps:

s100, recording the depth image as a first image;

s200, segmenting the first image into a plurality of sub-images through a watershed algorithm;

s300, extracting the image characteristics of the outlines of the sub-images;

s400, carrying out abnormal point estimation according to the image characteristics;

s500, discarding the first image when the number of the abnormal points in the first image is larger than an abnormal threshold;

s600, saving the first image when the number of the abnormal points in the first image is smaller than or equal to the abnormal threshold.

2. The method for screening depth image files according to claim 1, wherein in S100, the depth image is an image captured by any one of lidar depth imaging, computer stereo imaging, coordinate measuring machine, moire fringe, and structured light with a depression angle of 30 ° with respect to the horizon.

3. The method for screening depth image files according to claim 2, wherein in S300, the method for extracting the image features of the sub-image profiles comprises:

the method for extracting the outline of each sub-image comprises the steps of detecting the outline of the edge of the image through any one of a Canny operator, a Sobel operator and a Perwit operator;

the image features are:

Feature＝{Gr_i,H_i,A_i,P_i,Nl_i,Vl_i} (1)

in the formula, Gr_iRepresenting sub-images ∑_iMean value of gray scale of_i＝{h_i,s_iDenotes the sub-image Σ_iThe component values of H and S with the largest occurrence frequency are the components of H and S in the HSV color space of the sub-image; a. the_iThe total number of pixels in the sub-image; p_iRepresenting Contour_iPosition of center of gravity of, Nl_iThe straight line, Vl, in the contour of the sub-image after the contour is disassembled is recorded_iIs a perpendicular line in the contour line after the contour of the subimage is disassembled;

Nl_iand Vl_iThe calculation method of (2) is as follows:

firstly, the edge point P on the contour line of the edge image of the sub-image is used_j(x_ij,y_ij),(H<j is less than or equal to m-H) as the center, and constructing a sliding window Slop with the size of L to the Contour_iCarrying out local direction coding, wherein the default setting is that L is 80 pixels, m is the total number of pixels of a contour line, and the contour line consists of edge points;

the local direction coding method comprises the following steps: let the coordinates of the end points on both sides of the contour line segment cut by Slop be (x)_i1,y_i1),(x_i2,y_i2) Then edge point P_jThe local direction θ of (a) is:

where the linear ratio factor s is D/L, D represents the endpoint (x)_i1,y_i1),(x_i2,y_i2) The linear distance between the two adjacent sliding windows, L is the length of the contour obtained by the sliding window, T is a set threshold value, and T is set to be 0.5 by default;

the edge of the encoded sub-image edge image is split into a plurality of mutually continuous straight line segments at the corner point, and each straight line segment is Nl_i＝{l₁,l₂,…,l_kTherein ofk is the total number of lines in the profile, m₁And m₂Respectively representing the number of pixels of a starting point and an end point of a kth straight line in the outline from the starting point of the outline;

contour for sub-image_iStraight line l in_kTaking two endpoints as A and B, l_kThe angle of inclination of (a) is calculated as:

wherein,

if inclined at an angle of | theta_k-90|<σ, i.e. determining_kIs a vertical line and is saved to Vl_i。

4. The method for screening depth image files according to claim 3, wherein in S400, the method for estimating outliers according to image features comprises:

vertical lines Vl in all sub-image feature vectors_iForm parallel vertical linear sequences of lines₁,line₂,...,line_i...,line_pWherein, a line of a straight line_iThe equation of the straight line of (1) is:

y＝a_ix+b_i (4)

line pair according to least square method_iAnd (3) carrying out linear fitting to obtain the minimum value of the error of the fitted straight line:

determining the slope a_iY-axis intercept b_iThen, solving the intersection point by simultaneously establishing any two fitting straight lines:

each cross point obtained by calculation is an abnormal point set; clustering each intersection point by a fuzzy c-means clustering algorithm, and selecting the center of the most various members in the clustering group as an abnormal point V from the fuzzy c-means clustering result_r＝{x_i,y_i}。

5. A depth image file screening system, the system comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the computer program to run in the units of the following system:

an image reading unit for recording the depth image as a first image;

an image segmentation unit for segmenting the first image into a plurality of sub-images by a watershed algorithm;

the characteristic extraction unit is used for extracting the image characteristics of the outlines of the sub-images;

an abnormal point estimation unit for estimating abnormal points according to the image characteristics;

the image abandoning unit is used for abandoning the first image and transferring the first image to the image reading unit when the number of the abnormal points in the first image is larger than the abnormal threshold value;

and the image saving unit is used for saving the first image when the number of the abnormal points in the first image is less than or equal to the abnormal threshold value.