CN110728246A - A Cube Object Recognition Method Based on Depth Image - Google Patents

Abstract

The present invention proposes a method for recognizing a cube object based on a depth image, including: S1, acquiring a depth image of the target and its gradient maps in the X and Y directions; S2, classifying the pixel points in it according to the gradient map, and acquiring edge points Figure; S3, obtain the segmented images of the upper surface, the left vertical plane and the right vertical plane of the target in the depth image according to the above results; S4, carry out edge line detection to the edge point map, and cluster and merge all detected edge lines , obtain a set of collinear edge line groups; S5, extract corner points according to the longest edge line of each group of the collinear edge line group set; S6, pair the extracted corner points to obtain a quadrilateral; S7, determine the upper surface of the cubic object Surface and overall outline; S8, obtain the left and right vertical planes corresponding to each upper surface quadrangle, and realize the positioning of the cubic object. The invention is applied in the logistics industry, solves the problems of volume measurement limitation and time-consuming for objects such as cube-shaped packages, and improves the efficiency of package sorting.

Description

A Cube Object Recognition Method Based on Depth Image

technical field

The invention relates to the field of target recognition of depth images, in particular to a method for recognizing cubic objects based on depth images.

Background technique

Depth image, also known as range image, refers to an image whose pixel value is taken from the distance from the image collector to each point in the scene. It directly reflects the geometry of the visible surface of the target. Therefore, depth image-based applications are widely used. Especially in the logistics industry. At present, the throughput of domestic logistics, port shipping, and airport shipping is very large, and it is increasing. The volume measurement of parcels, goods, and containers by various sorting centers is relatively limited and time-consuming, which reduces transportation efficiency.

In the prior art, most of the packages are in the shape of cubic objects. The volume measurement of the package is mainly by comparing the depth images of the conveyor belt before and after loading, extracting the rectangular area where the package is located to know the length and width of the package, and using the difference feature to obtain the package. This method is limited to specific scenarios and has a small scope of application, which cannot meet the needs of dense accumulation of parcels in the actual logistics industry; Causes misjudgment of the length, width and height of the cuboid-shaped object, resulting in a large volume calculation error; and most of the existing technologies require the camera to shoot the object vertically, which limits the camera's shooting angle. Basically, only a single object can be detected. The object detection error is large, and it is difficult to accurately detect multiple faces of a cubic object, and the calculation accuracy is affected by the interference of external environmental noise. At present, the volume measurement of large goods and containers is basically manual measurement.

SUMMARY OF THE INVENTION

In order to solve the problems of limited and time-consuming volume measurement of objects in the shape of cubic objects such as packages, goods, and containers in the prior art, the present invention proposes a method for recognizing cubic objects based on depth images.

The technical scheme of the present invention is realized as follows:

A cube object recognition method based on depth image, comprising:

S1. Obtain the depth image of the target, and obtain the gradient maps in the X direction and the Y direction with the same size according to the depth image; S2. According to the gradient map, the pixels are divided into three categories: horizontal plane, left vertical plane and right vertical plane. Obtain the edge point map of the depth image; S3. Obtain the segmented images of the upper surface, the left vertical plane and the right vertical plane of the target in the depth image according to the classification result, the gradient map and the edge point map; S4, Perform edge line detection on the edge point map , clustering and merging all detected edge lines to obtain a set of collinear edge line groups; S5, extracting corner points according to the longest edge line of each group of the collinear edge line group set; S6, extracting corner points Pairing is performed to obtain a quadrilateral; S7, the upper surface and the overall outline of the cubic object are determined according to the obtained quadrilateral and the segmented image of the upper surface; S8, each upper surface quadrilateral is obtained according to the segmented image of the left vertical plane and the segmented image of the right vertical plane The corresponding left and right vertical planes are used to realize the positioning of cubic objects.

Preferably, according to the classification result, the gradient map and the edge point map, the specific method of obtaining the segmented images of the upper surface, the left vertical surface and the right vertical surface of the target in the depth image is as follows:

According to the classification result, obtain the binary image of the horizontal plane, left vertical plane and right vertical plane with the same size as the gradient image, define the segmented image of the upper surface with the same size as the gradient image, and set the initial value of the pixel point to 0;

Traverse the pixels in the first column of the binary image on the horizontal plane from bottom to top, obtain the credibility value of each pixel according to the depth map, and give the credibility threshold, if the credibility value of the pixel is less than the given credibility value If the reliability threshold is determined, the state of the pixel is unknown, and the next pixel is processed; otherwise, if the corresponding pixel value is 255, the state of the pixel is horizontal, and if the corresponding pixel is 0, Then the state of the pixel is vertical. If the number of pixels with a pixel value of 0 is greater than the given height threshold, then the pixels with a pixel value of 255 that appear later correspond to the segmented image on the upper surface. Update its pixel value to 255, and record the number of rows where the pixel state changes each time. After traversing the pixels of all rows in the first column, it will start from the number of recorded rows to the first row of the first row. The pixel values of all the pixel points in the column corresponding to the positions on the upper surface segmented image are set to 0. In the same way, all rows and columns of the parallel plane image are traversed, and the initial upper surface segmented image is obtained. The segmented image is patched, and the segmented image on the upper surface is updated;

The edge points in the edge point map are respectively corresponded one-to-one in the updated upper surface segmented image, the left vertical surface binary image and the right vertical surface binary image, and the updated segmented image of the upper surface is horizontally divided. The pixel value of the non-edge point of the state is updated to 0, the pixel value of the non-edge point in the left vertical plane is updated to 0, the pixel value of the non-edge point in the left vertical plane is 255, and the non-edge point in the right vertical plane is updated. The pixel value of 255 is updated to 0, and the final segmented image of the upper surface, the binary image of the left vertical plane and the binary image of the right vertical plane are obtained, and the binary image of the left vertical plane and the right vertical plane are obtained. The binary images of are denoted as the segmented image of the left vertical plane and the segmented image of the right vertical plane, respectively.

Preferably, the specific method for obtaining the edge point map of the depth image is:

Calculate the average gradient value Yp in the Y direction of all the pixels in the vertical plane, the gradient average value Yq in the Y direction of all the pixels in the horizontal plane, and the gradient average value Xp in the X direction of all the pixels in the left vertical plane. The average value Xq of the gradient in the X direction of all the pixels of the vertical plane; according to Yp, Yq, Xp, Xq, set the positive threshold in the Y direction, the negative threshold in the Y direction, the positive threshold in the X direction, and the negative threshold in the X direction; define a gradient map with For images of the same size, the initial value of the pixels in the image is set to 0; traverse all the pixels of the gradient map in the X direction and the gradient map in the Y direction, for each pixel, if the gradient value in the Y direction is The absolute value is greater than the absolute value of the gradient value in the X direction, and is smaller than the negative threshold value in the Y direction or greater than or equal to the positive threshold value in the Y direction, the pixel point is an edge point, the pixel point corresponds to the defined image and its pixel value is updated as 255; if the absolute value of the gradient value in the Y direction is less than the absolute value of the gradient value in the X direction, and the absolute value of the gradient value in the X direction is less than or equal to the negative threshold value in the X direction or greater than or equal to the positive threshold value in the X direction, the pixel is Edge point, correspond the pixel point to the defined image and update its pixel value to 255; get the edge point map.

Preferably, edge line detection is performed on the edge point graph, and all detected edge lines are clustered and merged to obtain a set of collinear edge line groups as follows:

Perform edge line detection on the edge point graph, obtain all edge lines and sort them into edge line sets according to length to short, and define an empty collinear edge line group set. The edge lines of A are collinear, and the edge lines between the two groups are not collinear. First, add the longest edge line to a set of collinear edge line sets. For each A edge line in the edge line set , judging whether it is collinear with the longest B edge line in each set of collinear edge line groups, and the judgment conditions are as follows:

C1. Calculate the angle difference between the A edge line and the B edge line, and determine whether the angle difference is less than or equal to a given angle difference threshold e;

C2. If the angle difference is less than or equal to the angle difference threshold e, calculate the distances T1 and T2 from the start point of the B edge line to the start point and end point of the A edge line, and the distances T3 and T4 from the end point of the B edge line to the start point and end point of the A edge line , given the distance threshold G between points, determine whether the minimum distance among the distances T1, T2, T3 and T4 is less than or equal to the distance threshold G between points;

C3. If min(min(T1, T2), min(T3, T4)) ≤ G, calculate the length L of the edge line of A and the length H of the edge line of B, and judge the maximum distance among the distances T1, T2, T3 and T4 Is it less than G+H+L;

C4. If max(max(T1, T2), max(T3, T4))<(G+H+L), calculate the distances S1 and S2 from the starting point and end point of the B edge line to the A edge line, and the distance of the A edge line The distances S3 and S4 from the starting point and the end point to the edge line of B, given the distance threshold J, if the maximum distance between T1 and T2 is less than L+G, judge whether S1 is less than or equal to J; if the maximum distance between T3 and T4 is less than L+G , judge whether S2 is less than or equal to J; if the maximum distance between T1 and T3 is less than L+G, judge whether S3 is less than or equal to J; if the maximum distance between T2 and T4 is less than L+G, judge whether S4 is less than or equal to J; if any If the maximum distance is greater than or equal to L+G, then directly execute the next step;

C5. If the conditions in C4 are met, the A edge line and the B edge line are collinear, and the A edge line is added to the group of the collinear edge line group set, and the edge line of the group is updated;

During the judgment process, if any of the conditions are not satisfied, the longest edge line of the next group in the set of collinear edge lines is directly processed, until all the groups of the set of collinear edge lines are processed, and the longest edge line of each set of edge lines is used. The edge line indicates that if the A edge line is not collinear with the longest edge line of all groups of the collinear edge line group set, the A edge line is directly added to the new group of the collinear edge line group set, and the collinear edge line is updated. The number of line group sets; after the A edge line and the B edge line are collinear, determine whether the A edge line and the B edge line meet the merging conditions. If T1≤H, T2≤H, T3≤H and T4≤H, then A The edge line and the B edge line do not meet the merging conditions, otherwise, the A edge line and the B edge line are merged, the starting point and the end point of the B edge line are updated, and the set of collinear edge line groups is updated;

Process the next edge line in the edge line set in the same way as processing the A edge line, and continuously update the collinear edge line group set. Line set collection.

Preferably, the specific method of extracting the corner points according to the longest edge line of each group of the collinear edge line group set is:

C1. Obtain the longest edge line of each group of the collinear edge line set, record it as the seed edge line set, sort all the edge lines of the seed edge line set from long to short, and traverse all the edge lines of the seed edge line set , start from the longest C edge line and calculate the angle a between it and any other D edge lines. The range of a is 0 to 180 degrees. Given the curvature threshold b, determine whether min(a, 180-a) greater than b;

C2. If min(a, 180-a)>b, calculate the intersection point W(Wx, Wy) of the C edge line and the D edge line, and judge whether Wx or Wy does not exceed the image range;

C3. If Wx or Wy does not exceed the image range, determine whether the intersection point W is a corner point. The conditions for judging whether the intersection point W is a corner point are: first calculate the distances d1 and d2 from the two endpoints of the C edge line to the intersection point W, and define the minimum distance min_d1, if the intersection W is between the two endpoints of the C edge line, then min_d1=0, otherwise min_d1=min(d1, d2), if the minimum distance min_d1 is not 0 or min(d1, d2) is less than the given arm Long threshold Z1, then calculate the distances d3 and d4 from the two endpoints of the D edge line to the intersection W, and define the minimum distance min_d2, if the intersection W is between the two endpoints of the D edge line, then min_d2=0, otherwise min_d2=min (d3, d4), if the minimum distance min_d2 is not 0 or min(d3, d4) is less than the given arm length threshold Z2, judge whether min(d1, d2) and min(d3, d4) are less than the given intersection radius threshold r;

C4. If the condition of C3 is met, given the threshold v of the gap between the arms of the quadrilateral, judge whether max(min(d1, d2), min(d3, d4)) is less than or equal to v, if so, the intersection point W is the corner point of the quadrilateral;

The conditions C1 to C4 are judged in turn, if any one of the conditions is not satisfied, the next edge line is processed;

Add the acquired first corner point to the corner point set, and for the rest of the acquired corner points, calculate the distance L _i from all the corner points in the corner point set, i is a variable, and judge all the corner points in the corner point set. Whether the corner points are similar, if L _i < r and not similar to other corner points, add the corner point to the corner point set;

The intersection point W corresponds to two arms, the first arm is the line connecting the end point of the C edge line, the second arm is the line connecting the end point of the D edge line, and the end point of the C edge line is the middle line of the C edge line. The larger corresponding endpoint of d1 and d2, the end point of the D edge line is the larger corresponding endpoint of d3 and d4 in the D edge line; the same way to deal with any other two edge lines in the seed edge line group set, get all the The corner points of , expand the two arms of each corner point according to the set of seed edge lines to obtain the final set of corner points to realize corner point extraction.

Preferably, the specific method for pairing the extracted corner points is:

C1. Pair all the corner points in the corner point set. If the coordinates of the two paired corner points are different, continue to judge whether the two corner points share an arm, and mark the two arms of one corner point as arm11 and arm12, the two arms of the other corner are recorded as arm21 and arm22, first determine whether arm11 and arm21 are the same arm of the two corners;

C2. The way of judging that arm11 and arm21 are the same arm is: if the distance between the two corner points is greater than one quarter of the minimum value of arm11 and arm21, first calculate the absolute value of the angle difference between arm11 and the two corner points. Value x1, the absolute value of the angle difference between arm21 and the line connecting the two corner points x2, the value range of x1 and x2 is [0, 180], given the angle similarity threshold f, if both x1 and x2 are less than the angle similarity threshold f , given the distance threshold u between the corner points and the length ratio threshold ratio, if the sum of the lengths of arm11 and arm21 is greater than or equal to the ratio times the distance between the two corner points, and is greater than or equal to the difference between the distance between the two corner points and u, then determine arm11 and arm21 are the same arm, and the corners are paired successfully;

Otherwise, use the same method to judge whether the other two arms of the two corners are the same arm, if not, the corner pairing fails, otherwise the corner pairing succeeds;

In the same way, pair the remaining corners in pairs to obtain all the successfully paired corner combinations. The same arm of each corner combination is the shared arm, and the other two arms are open arms. Calculate the two corners in each corner combination. Open the included angles u1 and u2 between the arm and the shared arm, the value range of the included angles u1 and u2 is [0, 180], and remove the combination of corners whose included angle u1 or u2 is less than the curvature threshold b; Corner combination, determine whether its two open arms are on the same side of the shared arm, remove the corner combination of the two open arms that are not on the same side of the shared arm, and finally determine a quadrilateral according to the currently paired corner combination; the same way Process all corner combinations to get all quads.

Preferably, the specific manner of extending the two arms of each corner point according to the set of seed edge lines is:

Traverse all edge lines in the set of seed edge lines, and for each corner point, calculate it to the current seed edge

The distance of the edge line, if the distance is less than the given threshold, it is judged whether the seed edge line can extend the two arms of the current corner point, first determine whether the seed edge line can extend the arm 1 of the current corner point, the specific method is:

C1. Calculate the distance from the current corner to the starting point and ending point of the seed edge line, denoted as dist1 and

dist2, determine whether the current corner point is between the two ends of the seed edge line;

C2. If the current corner point is between the two ends of the seed edge line, move the current corner point to the seed edge

The line segment where the starting point of the line is located is recorded as M, the starting point of the line segment M is the current corner point, the end point is the starting point of the seed edge line, the line segment where the arm 1 of the current corner point is located is recorded as N, the starting point of the line segment N is the current corner point, Calculate the angle angle1 of line segment M and the angle angle2 of line segment N, and judge whether the absolute value of the difference between angle1 and angle2 is less than the given angle threshold;

C3. If the absolute value of the difference between angle1 and angle2 is less than the given angle threshold, calculate the distance rM from the end point of the line M to the current corner point, calculate the length rN of the line segment N, and judge whether the difference between rM and rN is less than the given value The length difference threshold of ;

C4. If the difference between rM and rN is less than the given length difference threshold, calculate the end point of the line segment N

The distance to the current seed edge line, if the distance is less than the given point-to-line distance threshold and dist1 is greater than rN, then the current seed edge line can extend the arm 1 of the current corner, and the extended arm 1 of the current corner. The end point is the starting point of the seed edge line, and the length of the expanded arm 1 is dist1;

C5. If any of the conditions from C2 to C4 are not satisfied, it means that the current seed edge line cannot be extended

For arm 1 at the current corner point, the same method is used to determine whether the current seed edge line can expand the arm 2 of the current corner point. If it can be expanded, the end point of the expanded arm 2 is the starting point of the seed edge line. The length of 2 is dist1. If it cannot be extended, go to the next step directly;

C6. Update the end point of the line segment M in C2 to the end point of the current seed edge line, update the line segment N to the line segment where the arm 2 of the current corner point is located, the starting point of the line segment N is the current corner point, dist1 is updated to dist2, using the same The method judges again whether the current seed edge line can extend the arms 1 and 2 of the current corner, and realizes the expansion of the two arms of the current corner;

C7. At this time, if the conditions in C1 are not met, judge whether dist1 is less than or equal to dist2;

C8. If dist1 is less than or equal to dist2, update the starting point of line segment M in C2 to the current seed edge

The starting point of the edge line, the end point is updated to the end point of the current edge line, and the line segment N is the line segment where the arm 1 or arm 2 of the current corner point is located. Use the same method to determine whether the current seed edge line can extend the current corner point. Arm 1 and arm 2. Realize the expansion of the two arms at the current corner;

C9. If dist1 is greater than dist2, update the starting point of line segment M in C2 to the current seed edge line

The end point is updated to the starting point of the current edge line, the line segment N is the line segment where arm 1 or arm 2 of the current corner point is located, dist1 is updated to dist2, and the same method is used to determine whether the current seed edge line can extend the arm of the current corner point. 1 and arm 2, to realize the expansion of the two arms at the current corner.

Preferably, in S7, the specific manner of determining the upper surface and the overall outline of the cubic object according to the obtained segmented images of the quadrilateral and the upper surface is:

C1. Process the acquired quadrilateral first. If the distance between the end points of the two arms of the matched quadrilateral is less than the given distance ratio threshold times, the quadrilateral does not meet the requirement, process the next quadrilateral, obtain all the quadrilaterals that meet the conditions, and obtain For the contour and circumscribed rectangle of each quadrilateral, calculate its convex hull according to the contour coordinate points;

C2. Perform contour detection according to the segmented image of the upper surface, obtain all contours, calculate the convex hull according to the contour coordinate points, obtain the smallest rotated rectangle containing each contour and the circumscribed rectangle containing the rotated rectangle, and calculate the Area, to determine whether the area of the contour is greater than the given area threshold;

C3. If the area of the contour is greater than the area threshold, match the current contour with the qualified quadrilateral, calculate the minimum distances h1 and h2 from the two corner points of the matched quadrilateral to the convex hull of the current contour, and judge the quadrilateral for matching Whether the two corner points of is outside the convex hull of the matching quadrilateral and whether h1 and h2 are less than or equal to the given distance threshold;

C4. If the two corner points of the matching quadrilateral are outside the convex hull of the matching quadrilateral and both h1 and h2 are less than or equal to the distance threshold, judge whether the center point of the rotating rectangle outside the current contour is in the matching quadrilateral inside the convex hull;

C5. If the center point of the rotating rectangle outside the current contour is located inside the convex hull of the matching quadrilateral, calculate the absolute value of the minimum distance to the matching quadrilateral convex hull, if the value is less than or equal to the length and width of the rotating rectangle, And the center point is located on the lower side of any arm of the matching quadrilateral, calculate the overlapping area of the bounding rectangle of the matching quadrilateral and the bounding rectangle of the current contour, if the overlapping area is greater than the given ratio threshold, then the current contour The initial matching with the matching quadrilateral is successful, and the matching quadrilateral is recorded as the preliminary matching quadrilateral;

C6. Normalize the distance from the center point of the rotating rectangle of the current contour to the three arms of the preliminary matching quadrilateral. The normalized value is within the theoretical range, and the sum of h1 and h2 is less than or equal to the given threshold , denote the preliminary matching quadrilateral as an effective quadrilateral;

C7. Determine whether the two open arms of the effective quadrilateral face down, and record the effective quadrilateral with the two open arms facing down as the seed quadrilateral; if the next seed quadrilateral is obtained, calculate the distance from the two corners of the seed quadrilateral to its outline Whether the sum of the minimum distances is less than the sum of the minimum distances from the two corners of the previous seed quadrilateral to its contour, if it is less than, replace the previous seed quadrilateral with the current seed quadrilateral, otherwise remove the current seed quadrilateral, and continue processing until the current contour is obtained. the final seed quad;

C8. Determine whether the final seed quadrilateral of the current contour has a shared corner point with the effective quadrilateral;

C9. If there is a shared corner point, determine whether the line segment of the shared arm and the open arm of the effective quadrilateral overlaps with the line segment of the shared arm of the seed quadrilateral. Whether the overlapping number of the line segment of the open arm of the seed quadrilateral is greater than or equal to 1, if so, obtain a cubic object with the three arms of the seed quadrilateral as the upper surface, and the arms of the effective quadrilateral except the overlapping arms as the sides of the side vertical plane Contour, get all the contours of cubic objects, and record them as the set of contours of cubic objects;

C10. Obtain all point coordinates of the outline of the cubic object according to the endpoints of each side of the outline of the cubic object, and the outline of the cubic object takes the seed quadrangle as the upper surface;

C11. Traverse the set of cubic object contours, calculate the y-coordinate P.y of the center point of the contour of each cubic object and the y-coordinate t.y of the center point of its upper surface, and calculate the area P.area of the contour of each cubic object and the area t of its upper surface .area, if t.y>P.y or t.area>P.area, then remove the outline of the cube to obtain the final set of outlines of the cube.

Preferably, the left and right vertical planes corresponding to each upper surface quadrilateral are obtained according to the segmented image of the left vertical plane and the segmented image of the right vertical plane. Obtain all the contours of the left vertical plane and their corresponding rotating rectangles and circumscribed rectangles, and judge whether the y-coordinate of the center point of the rotating rectangle is greater than or equal to the y-coordinate of the center point of the quadrilateral on the upper surface; Whether the ratio of the contour area of the cube corresponding to the quadrilateral and the overlapping area of the circumscribed rectangle to the contour area of the cube is greater than or equal to the given area ratio threshold; The contour with the largest area is the left vertical contour of the current cubic object;

Obtain all the contours of the right vertical plane and their corresponding rotated rectangles and circumscribed rectangles according to the segmented image of the right vertical plane. The left vertical plane and the right vertical plane corresponding to the upper surface of each cubic object are obtained, and each group of the upper surface, the left vertical plane and the right vertical plane constitutes a cubic object to realize the detection of the cubic object.

Preferably, in C8, the specific method for judging whether the final seed quadrilateral of the current contour has a shared corner point with the effective quadrilateral is: calculating the distances from the two corner points of the final seed quadrilateral to the two corner points of the effective quadrilateral respectively, if If the four distances are all greater than the given threshold, it means that the seed quadrilateral and the effective quadrilateral do not have shared corners; if the four distances are all greater than the given threshold, it means that the seed quadrilateral and the effective quadrilateral share corner.

The beneficial effects of the present invention are as follows: the method for recognizing a cube object based on a depth image of the present invention, on the basis of quickly distinguishing the surface of the cube object by dividing the image, performs edge line detection on the edge point map, and all detected edge lines are detected. Perform clustering and merging to obtain all collinear edge line groups, the edge lines in each group are collinear, select the longest edge line to represent each group of edge lines, and only use its longest edge line in the calculation. Long edge lines can reduce the amount of calculation and improve the stability of detection; then use the intersections between the edge lines after clustering to extract corner points and pair them. The corner point pairing verifies whether there is a shared arm between the two corner points. Whether the two open arms are on the same side of their shared arms and whether the included angle between each corner point is greater than a given angle threshold, quadrilateral detection is realized according to the corner point pairing results, and then all the The upper surface contour is matched with all detected quadrilaterals to obtain the upper surface of the cube and the overall contour of the cube, and then combined with the left and right vertical planes to segment the image to obtain all left and right vertical surface contours, Then, the left vertical plane and the right vertical plane corresponding to each upper surface are determined according to the overall contour of the cube, so as to accurately realize the detection of the cubic object. Practical applications include at least an upper surface and a vertical plane in volume measurements.

The cubic object recognition method of the present invention is applied to the volume measurement of parcels, goods, containers and other targets, avoids comparing the depth images of the conveyor belt before and after loading, but directly recognizes the cube-shaped target, and obtains the corresponding length and width. The volume of the high calculation target solves the problem of limited and time-consuming package volume measurement, and helps the logistics industry to improve the efficiency of package sorting, which has important practical application significance. The cubic object recognition method of the present invention can also be applied in the field of image recognition.

Detailed ways

The technical solutions in the embodiments of the present invention will be described clearly and completely below. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

Embodiment: a method for recognizing a cube object based on a depth image, comprising: S1, acquiring a depth image of the target, and respectively obtaining gradient maps in the X direction and the Y direction of the same size according to the depth image; Classify the pixel points to obtain the edge point map of the depth image; S3, obtain the segmented images of the upper surface, left vertical plane and right vertical plane of the target in the depth image according to the classification result, gradient map and edge point map; S4, to the edge point Perform edge line detection on the graph, cluster and merge all detected edge lines, and obtain a set of collinear edge line groups; S5, extract corner points according to the longest edge line of each group of the collinear edge line set; S6, Pairing the extracted corner points to obtain a quadrilateral; S7, determining the upper surface and overall outline of the cubic object according to the obtained quadrilateral and the segmented image of the upper surface; S8, obtaining according to the segmented image of the left vertical plane and the segmented image of the right vertical plane The left and right vertical planes corresponding to each upper surface quadrilateral realize the positioning of cubic objects. The pixel point classification method is suitable for any object in any scene. In practice, when a cubic object is positioned, the cubic object contains at least a visible upper surface and a vertical plane in the image. The plane where the target is located is the lowest plane in the depth image or directly on the ground. It is convenient to realize the positioning of the cubic object more accurately.

The specific way of classifying the pixels in S2 according to the gradient map is: traverse each pixel value DX(r, c) in the gradient map in the X direction, and each pixel value DY in the gradient map in the Y direction (r, c), where r=0...m, c=0...n, m represents the number of rows in the gradient map, and n represents the number of columns in the gradient map; given a positive segmentation threshold p and a negative segmentation Threshold q, the pixels corresponding to DY(r, c) greater than or equal to p are located on the vertical plane, and the pixels corresponding to DY(r, c) less than or equal to q are located on the horizontal plane; for all points on the vertical plane, the pixels less than or equal to q The pixels corresponding to DX(r, c) are located on the left vertical plane, and the pixels corresponding to DX(r, c) greater than or equal to p are located on the right vertical plane.

The specific method of obtaining the edge point map of the depth image in S2 is: calculating the gradient mean value Yp in the Y direction of all the pixels in the vertical plane after classification, the gradient mean value Yq in the Y direction of all the pixels in the horizontal plane, and the left vertical plane. The gradient average value Xp of all pixels in the X direction and the gradient average value Xq of all pixels of the right vertical plane in the X direction; according to Yp, Yq, Xp, Xq set the Y direction positive threshold, Y direction negative threshold, X Positive threshold in the direction and negative threshold in the X direction; define an image with the same size as the gradient map, and the initial value of the pixels in the image is set to 0; traverse all the pixels of the gradient map in the X direction and the gradient map in the Y direction, For each pixel, if the absolute value of the gradient value in the Y direction is greater than the absolute value of the gradient value in the X direction, and less than the negative threshold value in the Y direction or greater than or equal to the positive threshold value in the Y direction, the pixel point is an edge point, the pixel The point corresponds to the defined image and its pixel value is updated to 255, if the absolute value of the gradient value in the Y direction is less than the absolute value of the gradient value in the X direction, and the absolute value of the gradient value in the X direction is less than or equal to the X direction. If the negative threshold is greater than or equal to the positive threshold in the X direction, the pixel is an edge point, and the pixel corresponds to the defined image and its pixel value is updated to 255; the edge point image is obtained, and the edge point image is a binary image.

The specific method of obtaining the segmented images of the upper surface, left vertical plane and right vertical plane of the target in the depth image according to the classification result, the gradient map and the edge point map in S3 is as follows: according to the classification result, the horizontal plane, left vertical plane and left vertical plane with the same size as the gradient map are obtained according to the classification result. The binary image of the surface and the right vertical plane, define the segmented image of the upper surface with the same size as the gradient image, and set the initial value of the pixel point to 0; traverse the pixel points in the first column of the binary image of the horizontal plane from bottom to top , obtain the credibility value of each pixel point according to the depth map, and give the credibility threshold value. If the credibility value of the pixel point is less than the given credibility threshold value, the state of the pixel point is unknown, and processing The next pixel; otherwise, if the corresponding pixel value is 255, the state of the pixel is horizontal; if the corresponding pixel is 0, the state of the pixel is vertical; if the pixel value is 0, the state of the pixel is vertical. The number of points is greater than the given height threshold, then the pixels with the pixel value of 255 that appear later correspond to the segmented image on the upper surface and update the pixel value to 255, and record the state of each pixel point. The number of rows where the pixel points are located. After traversing the pixels of all rows in the first column, the pixel values from the number of recorded rows to the first column of the first row correspond to the position of the segmented image on the upper surface. Set to 0, traverse all the rows and columns of the parallel plane image in the same way, and obtain the initial segmented image of the upper surface; repair the obtained initial segmented image of the upper surface. The specific method is: given the image slice size threshold t, Define the upper surface patch image with the same size as the upper surface segmented image, set the initial value of the pixel point to 0, and update the upper surface patch image from the tth column to the total number of columns minus the pixels in column t according to the segmented image of the upper surface. Point value, traverse all rows and columns 1 to 2*t of the segmented image on the upper surface, update each pixel value of the t-th column of each row of the patched image on the upper surface to the current row of the segmented image on the upper surface The sum of the pixel values of the 1st to 2*t columns of the 1st to 2nd*t columns, update the upper surface patch image; traverse all the rows and t+1th columns of the upper surface segmented image to the total number of columns minus t columns, and patch the upper surface Each pixel value of the image is updated to the pixel value of the patch image on the upper surface of the previous column of the current row and the current column plus the pixel value of the segmented image on the upper surface of the current row and current column plus the t column minus the current row. Subtract the pixel value of the segmented image on the upper surface of column t from the current column to obtain the final upper surface patch image; traverse all pixels of the upper surface patch image, and give the sum threshold of pixel values, if the pixel value of the current pixel is greater than Given the threshold of the sum of pixel values, update the pixel value corresponding to the segmented image on the upper surface to 255, otherwise update it to 0, and update the segmented image on the upper surface;

The edge points in the edge point map are respectively corresponded one-to-one in the updated upper surface segmented image, the left vertical surface binary image and the right vertical surface binary image, and the updated segmented image of the upper surface is horizontally divided. The pixel value of the non-edge point of the state is updated to 0, the pixel value of the non-edge point in the left vertical plane is updated to 0, the pixel value of the non-edge point in the left vertical plane is 255, and the non-edge point in the right vertical plane is updated. The pixel value of 255 is updated to 0, and the final segmented image of the upper surface, the binary image of the left vertical plane and the binary image of the right vertical plane are obtained, and the binary image of the left vertical plane and the right vertical plane are obtained. The binary images of are denoted as the segmented image of the left vertical plane and the segmented image of the right vertical plane, respectively.

In the calculation process, due to noise interference, individual points on the upper surface may be misjudged as points on the vertical plane, resulting in discontinuous points on the upper surface. In order to improve the stability of the algorithm, the segmented image of the upper surface is repaired.

Record the row number of the pixel point where the state of each pixel point changes, and obtain the depth image of the target in S1. The target can be fully seen in the depth image, and traverse each column of pixel points from the bottom to the top. If the cubic object appears successively The vertical plane and the horizontal plane, then the category of the subsequent pixel points will definitely change again, the last recorded pixel point state changes the number of rows corresponding to the position of the current column of the first row corresponds to the segmentation on the upper surface The pixel value on the image is set to 0 (the point value from the record line number to the first line may be 0 or 255, and set to 0 uniformly); otherwise, it is very likely that the upper surface of the current cube object does not completely appear in the picture. , culling such points on the upper surface, and setting the pixel value corresponding to the segmented image on the upper surface to 0.

In S4, edge line detection is performed on the edge point map, and all detected edge lines are clustered and merged to obtain a set of collinear edge line groups. The specific method is: perform edge line detection on the edge point map, obtain all edge lines and Sort the edge line sets from longest to shortest, and define an empty collinear edge line group set. The collinear edge line group set contains multiple sets of edge lines. The edge lines in each group are collinear, and the edge lines between the two groups are If it is not collinear, first add the longest edge line to a group of collinear edge line group sets, and for each A edge line in the edge line set, judge that it is the longest with each group of the collinear edge line group set. The judgment conditions are as follows: C1. Calculate the angle difference between the A edge line and the B edge line, and judge whether the angle difference is less than or equal to the given angle difference threshold e, the angle difference is greater than e, and the A edge The line and the B edge line must not be collinear, because the two collinear edge lines should be approximately parallel, and the angle difference is relatively small; C2. If the angle difference is less than or equal to e, calculate the starting point of the B edge line to the starting point and end point of the A edge line The distances T1 and T2, the end point of B edge line to the start point and end point of A edge line T3 and T4, given the distance threshold G between points, whether the minimum distance among the distances T1, T2, T3 and T4 is less than or equal to the distance threshold between points G, this part is to detect whether the minimum distance between the two endpoints of the A edge line and the B edge line is less than the given distance threshold G between points, if the minimum distance is greater than the given distance threshold between points, it means that the two edges If the lines are far apart, min(min(T1, T2), min(T3, T4))>G, then the edge lines of A and B are definitely not collinear; C3, if min(min(T1, T2), min( T3, T4)) ≤ G, calculate the length L of the edge line of A and the length H of the edge line of B, and judge whether the maximum distance among the distances T1, T2, T3 and T4 is less than G+H+L, max(max(T1, T2), max(T3, T4)) ≥ (G+H+L), A and B edge lines are not collinear; C4, if max(max(T1, T2), max(T3, T4)) < (G +H+L), calculate the distances S1 and S2 from the start and end points of the B edge line to the A edge line, and the distances S3 and S4 from the start and end points of the A edge line to the B edge line, given the distance threshold J, if T1 and If the maximum distance of T2 is less than L+G, judge whether S1 is less than or equal to J; if the maximum distance between T3 and T4 is less than L+G, judge whether S2 is less than or equal to J; if the maximum distance between T1 and T3 is less than L+G, judge Whether S3 is less than or equal to J; if the maximum distance between T2 and T4 is less than L+G, judge whether S4 is less than or equal to J; if any maximum distance is greater than or equal to L+G, directly execute the next step; C5, meet the conditions in C4 , then the A edge line and the B edge line are collinear, the A edge line is added to the group of the collinear edge line group set, and the edge line of the group is updated; in the sequential judgment process from C1 to C6, any condition Not satisfied, directly process the set of collinear edge line groups The longest edge line of the next group in the set, until all the groups of the collinear edge line group set are processed, each set of edge lines is represented by its longest edge line, if the A edge line is the same as the collinear edge line set of all groups of the set of edge lines. If the longest edge lines are not collinear, then the A edge line is directly added to the new group of the collinear edge line group set, and the group number of the collinear edge line group set is updated; after the A edge line and the B edge line are collinear, Determine whether the A edge line and the B edge line meet the merging conditions. If T1≤H, T2≤H, T3≤H, and T4≤H, then the A edge line and the B edge line do not meet the merging condition, otherwise, the A edge line and the B edge line do not meet the merge condition. Merge B edge lines, update the start and end points of B edge lines, update the set of collinear edge line groups; process the next edge line in the edge line set in the same way as processing A edge lines, and continuously update the set of collinear edge lines, The edge lines in the collinear edge line group set are sorted according to their length, and the final collinear edge line group set is obtained.

The collinear clustering and merging of edge lines here is mainly because in the process of edge line detection, the original continuous edge lines are often disconnected due to noise interference. Originally, only one edge line may detect multiple approximations during detection. Multiple edge lines with very close parallel distances but different lengths will reduce the stability and accuracy of the algorithm. The main purpose of collinear clustering and merging is to divide multiple line segments that may belong to the same line into the same group. Merge, merge multiple disconnected collinear edge lines, obtain the longest edge line of each group, and use the longest edge line to represent all edge lines of the group. On the one hand, it reduces the amount of calculation and improves the algorithm on the other hand. stability.

Merge the A edge line and the B edge line, and update the start and end points of the B edge line as follows: the B edge line is the longest edge line of the group in which it belongs. If the A edge line and the B edge line are collinear and meet the merging conditions , then update the start and end points of the B edge line according to the A edge line. If the x coordinates of the two ends of the edge line are different, define the left end point of the edge line as the starting point and the right end point as the end point. If the x coordinates of the two ends are the same, then Define the upper end point as the start point and the lower end point as the end point. The judgment method for updating the end point of the B edge line is: C1. If T1 > H and T2 > H, on this basis, T3 > T4, update the end point of the B edge line to the starting point of the A edge line, which does not satisfy T3 > T4 If T1 > H and T2 > H in C1, if T1 > H but T2 <= H, update the end point of B edge line to The starting point of the A edge line; C3, does not meet the conditions in C1 and C2, if T2 > H but T1 <= H, then update the B edge line end point to the A edge line end point. The judgment method for updating the starting point of the B edge line is: C1. If T3 > H and T4 > H, on this basis, T1 > T2, then update the starting point of the B edge line to the starting point of the A edge line. If it does not satisfy T1 > For the condition of T2, update the starting point of the B edge line to the end point of the A edge line; C2, do not meet the conditions in C1 T3>H and T4>H, if T3>H but T4 <=H, then update the starting point of the B edge line is the starting point of the edge line of A; C3, does not meet the conditions in C1 and C2, if T4 > H but T3 <= H, then update the starting point of the edge line of B to the end point of the edge line of A.

In S5, the specific method of extracting corner points according to the longest edge line of each set of collinear edge line groups is: C1. Obtain the longest edge line of each group of the collinear edge line set, and denote it as the seed edge line set, Sort all the edge lines of the seed edge line set from long to short, traverse all the edge lines of the seed edge line set, and start from the longest C edge line to calculate the angle a, a between it and any other D edge lines The range is 0 to 180 degrees, given the curvature threshold b, determine whether min(a, 180-a) is greater than b; C2, if min(a, 180-a)>b, calculate the C edge line and the D edge line If Wx or Wy does not exceed the image range, start processing the next edge line; C3. If Wx or Wy does not exceed the image range, judge whether the intersection point W is Corner point, the condition for judging that the intersection point W is a corner point is: first calculate the distances d1 and d2 from the two endpoints of the C edge line to the intersection point W, and define the minimum distance min_d1, if the intersection point W is between the two endpoints of the C edge line, Then min_d1=0, otherwise min_d1=min(d1, d2), if the minimum distance min_d1 is not 0 or min(d1, d2) is less than the given arm length threshold Z1, then calculate the two endpoints of the D edge line to the intersection point W The distances d3 and d4, define the minimum distance min_d2, if the intersection W is between the two endpoints of the D edge line, then min_d2=0, otherwise min_d2=min(d3, d4), if the minimum distance min_d2 is not 0 or min( d3, d4) is less than the given arm length threshold Z2, judge whether min(d1, d2) and min(d3, d4) are less than the given intersection radius threshold r; C4, if the conditions of C3 are met, the given quadrilateral arm Gap threshold v, judge whether max(min(d1, d2), min(d3, d4)) is less than or equal to v, if it does not satisfy the next edge line, if it is satisfied, the intersection point W is the corner of the quadrilateral; the condition C1 to Judging in turn in C4, if any of the conditions is not satisfied, the next edge line is processed; the first corner point obtained is added to the corner point set, and for the remaining corner points obtained, it and all corners in the corner point set will be calculated. The distance L _i of the point, i is a variable, judge whether it is similar to all the corner points in the corner point set, if _Li < r, and it is not similar to the other corner points, add the corner point to the corner point set; the intersection point W corresponds to two arms, the first arm is the line connecting it with the end point of the C edge line, the second arm is the line connecting it with the end point of the D edge line, and the end point of the C edge line is d1 and d1 in the C edge line. The larger corresponding endpoint in d2, the end point of the D edge line is the larger corresponding endpoint in d3 and d4 in the D edge line; the same way to deal with any other two edge lines in the seed edge line group set, get all the corners point, expand the two arms of each corner point according to the set of seed edge lines, and obtain the final set of corner points to realize corner point extraction.

The specific way of judging whether two corner points are similar is: respectively calculating the absolute values a11, a12, a21, a22 of the difference between the angle of each open arm of one corner point and the angle of each open arm of the other corner point, The value range of the absolute value of the angle difference is [0, 180], select min (a11, a12), min (a21, a22), given the angle similarity threshold f, if max (min (a11, a12), min(a21, a22)) < f, then the two corners are similar.

The similarity of two corner points means that the corresponding opening arms of the two corner points are approximately parallel. For example: if the two corner points respectively include a horizontal opening arm and a vertical opening arm, the horizontal opening arms of the two corner points are approximately parallel. parallel, the angle difference is very small, the two vertical open arms are approximately parallel, and the angle difference is also small, so the minimum value of min(a11, a12) and min(a21, a22) should also be smaller than the given similar angle threshold, Otherwise the two corners are not similar.

The specific method of extending the two arms of each corner point according to the set of seed edge lines is: traverse all edge lines of the set of seed edge lines, and for each corner point, calculate the distance from the current seed edge line, if the distance is less than the given edge , then judge whether the seed edge line can extend the two arms of the current corner. First, judge whether the seed edge line can extend the arm 1 of the current corner. The specific method is as follows:

C1. Calculate the distance from the current corner to the starting point and ending point of the seed edge line, denoted as dist1 and

dist2, determine whether the current corner point is between the two ends of the seed edge line;

C2. If the current corner point is between the two ends of the seed edge line, move the current corner point to the seed edge

The line segment where the starting point of the line is located is recorded as M, the starting point of the line segment M is the current corner point, the end point is the starting point of the seed edge line, the line segment where the arm 1 of the current corner point is located is recorded as N, the starting point of the line segment N is the current corner point, Calculate the angle angle1 of line segment M and the angle angle2 of line segment N, and judge whether the absolute value of the difference between angle1 and angle2 is less than the given angle threshold;

C3. If the absolute value of the difference between angle1 and angle2 is less than the given angle threshold, calculate the distance rM from the end point of the line M to the current corner point, calculate the length rN of the line segment N, and judge whether the difference between rM and rN is less than the given value The length difference threshold of ;

C4. If the difference between rM and rN is less than the given length difference threshold, calculate the end point of the line segment N

The distance to the current seed edge line, if the distance is less than the given point-to-line distance threshold and dist1 is greater than rN, then the current seed edge line can extend the arm 1 of the current corner, and the extended arm 1 of the current corner. The end point is the starting point of the seed edge line, and the length of the expanded arm 1 is dist1;

C5. If any of the conditions from C2 to C4 are not satisfied, it means that the current seed edge line cannot be extended

For arm 1 at the current corner point, the same method is used to determine whether the current seed edge line can expand the arm 2 of the current corner point. If it can be expanded, the end point of the expanded arm 2 is the starting point of the seed edge line. The length of 2 is dist1. If it cannot be extended, go to the next step directly;

C6. Update the end point of the line segment M in C2 to the end point of the current seed edge line, update the line segment N to the line segment where the arm 2 of the current corner point is located, the starting point of the line segment N is the current corner point, dist1 is updated to dist2, using the same The method judges again whether the current seed edge line can extend the arms 1 and 2 of the current corner, and realizes the expansion of the two arms of the current corner;

C7. At this time, if the conditions in C1 are not met, judge whether dist1 is less than or equal to dist2;

C8. If dist1 is less than or equal to dist2, update the starting point of line segment M in C2 to the current seed edge

The starting point of the edge line, the end point is updated to the end point of the current edge line, and the line segment N is the line segment where the arm 1 or arm 2 of the current corner point is located. Use the same method to determine whether the current seed edge line can extend the current corner point. Arm 1 and arm 2. Realize the expansion of the two arms at the current corner;

C9. If dist1 is greater than dist2, update the starting point of line segment M in C2 to the current seed edge line

The end point is updated to the starting point of the current edge line, the line segment N is the line segment where arm 1 or arm 2 of the current corner point is located, dist1 is updated to dist2, and the same method is used to determine whether the current seed edge line can extend the arm of the current corner point. 1 and arm 2, to realize the expansion of the two arms at the current corner.

The specific method of pairing the extracted corner points in S6 is as follows: C1. All corner points in the corner point set are paired in pairs. If the coordinates of the two paired corner points are different, continue to judge whether the two corner points are between them. Share one arm, mark the two arms of one corner as arm11 and arm12, and the two arms of the other corner as arm21 and arm22, first judge whether arm11 and arm21 are the same arm of the two corners; C2, judge arm11 The way that arm21 and arm21 are the same arm is: if the distance between the two corner points is less than a quarter of the minimum value of arm11 and arm21, then arm11 and arm21 are not the same arm; if the distance between the two corner points is greater than arm11 And one quarter of the minimum value of arm21, first calculate the absolute value x1 of the angle difference between arm11 and the line connecting the two corner points, the absolute value x2 of the angle difference between arm21 and the line connecting the two corner points, and take the values of x1 and x2 The value range is [0, 180], given the angular similarity threshold f, if both x1 and x2 are less than the angular similarity threshold f, given the distance threshold u between corners and the length ratio threshold ratio, if the sum of the lengths of arm11 and arm21 is greater than or equal to If the ratio of the distance between the two corner points is greater than or equal to the difference between the distance between the two corner points and u, then it is determined that arm11 and arm21 are the same arm, and the corner points are paired successfully; otherwise, the same method is used to determine the difference between the two corner points. Whether the other two arms are the same arm, if they are not, the corner point pairing fails, otherwise the corner point pairing is successful; pair the remaining corner points in the same way to obtain all the corner point combinations that are paired successfully, each corner point The same arm of the combination is the shared arm, and the other two arms are the open arms. Calculate the included angles u1 and u2 between the two open arms and the shared arm in each corner combination. The value range of the included angles u1 and u2 is [ 0, 180], remove the corner combination whose included angle u1 or u2 is less than the curvature threshold b; for the remaining corner combination, determine whether the two open arms are on the same side of the shared arm, and calculate the sharing of the corner combination. The center point of the arm, taking the center point as the starting point, calculate the unit normal vector of the shared arm, calculate the vector v1 where the line connecting the end point of the open arm of the first corner point and the center point is located, and the open arm of the second corner point. The vector v2 where the line connecting the end point and the center point is located. If the inner product of the vectors v1 and v2 and the unit normal vector of the shared arm is positive and negative, the two open arms are not on the same side of the shared arm, and the two open arms are not on the same side of the shared arm. The corner combinations on the same side of the shared arm are eliminated, and finally a quadrilateral is determined according to the currently paired corner combination; all corner combinations are processed in the same way to obtain all quadrilaterals.

In S7, the upper surface and the overall outline of the cubic object are determined according to the obtained segmented images of the quadrilateral and the upper surface, and then the left and right vertical surfaces corresponding to each upper surface quadrilateral are obtained according to the segmented images of the left and right vertical surfaces to realize the cubic object. position. The specific method of determining the upper surface and overall outline of the cubic object is as follows: C 1. Process the acquired quadrilateral first. If the distance between the end points of the two arms of the matched quadrilateral is less than 0.8 times the distance between the two corner points, the quadrilateral is not Meet, process the next quadrilateral, obtain all the quadrilaterals that meet the conditions, and obtain the contour and circumscribed rectangle of each quadrilateral, and calculate its convex hull according to the contour coordinate points; C2. Perform contour detection according to the segmented image of the upper surface, and obtain all contours , calculate its convex hull according to the contour coordinate points, and obtain the smallest rotated rectangle containing each contour and the circumscribed rectangle containing the rotated rectangle, calculate the area of each contour, and determine whether the area of the contour is greater than the given area threshold, if the contour If the area of is less than the given area threshold, the next contour is processed; C3. If the area of the contour is greater than the given area threshold, the current contour is matched with the qualified quadrilateral, and the two corner points of the matched quadrilateral are calculated respectively. The minimum distances h1 and h2 to the convex hull of the current contour, determine whether the two corners of the matching quadrilateral are outside the convex hull of the matching quadrilateral and whether h1 and h2 are less than or equal to the given distance threshold, if h1 and Any one of h2 is greater than the given distance threshold, it means that there is a corner point on the inside of the contour and close to its center point, and the next quadrilateral matching is performed; C4, if the two corners of the matched quadrilateral are in The outside of the convex hull of the matching quadrilateral and both h1 and h2 are less than or equal to the distance threshold, the two corners of the matching quadrilateral are located outside the current contour, and determine whether the center point of the rotating rectangle outside the current contour is matching. Inside the convex hull of the quadrilateral, if it is not inside, perform the next quadrilateral matching; C5. If the center point of the rotated rectangle outside the current contour is inside the convex hull of the matching quadrilateral, calculate the minimum distance to the matching quadrilateral contour The absolute value of , satisfies the condition that the value is less than or equal to the length and width of the rotated rectangle, and the center point is located on the lower side of any arm of the matching quadrilateral, calculate the overlap between the bounding rectangle of the matching quadrilateral and the bounding rectangle of the current contour Area, if the overlapping area is greater than the given ratio threshold, the current contour and the matching quadrilateral are initially matched successfully, and the matched quadrilateral is recorded as the preliminary matching quadrilateral; C6, the center point of the rotation rectangle of the current contour to the preliminary matching quadrilateral The distance of the three arms is normalized, the normalized value is within the theoretical range, and the sum of h1 and h2 is less than or equal to the given threshold, the preliminary matching quadrilateral is recorded as an effective quadrilateral; C7, judge valid Whether the two open arms of the quadrilateral are facing down, the effective quadrilateral with the two open arms facing down is recorded as the seed quadrilateral. The angle of the line segment on the axis is 0 degrees, and the angle of the line segment increases continuously according to the clockwise rotation. The value range of the angle is [0, 360]. Use this definition rule to calculate the angle of the two open arms. If the two angles are less than 15 degrees or greater than 165 degrees , it means that the two open arms of the effective quadrilateral are facing up, and the next one is processed; otherwise, if the end point of the two open arms of the effective quadrilateral is above the two corner points, the next one is processed, otherwise, the two open arms are processed. The effective quadrilateral with arms facing down is recorded as the seed quadrilateral; if the next seed quadrilateral that satisfies the above conditions is obtained, calculate whether the sum of the minimum distances from the two corners of the current seed quadrilateral to its convex hull is less than the two of the previous seed quadrilateral. The sum of the minimum distances from the corner point to its convex hull, if it is less than the current seed quadrilateral, replace the previous seed quadrilateral, otherwise remove the current seed quadrilateral, and continue processing to obtain the final seed quadrilateral of the current contour; C8, determine the current contour Whether the final seed quadrilateral has a shared corner point with the valid quadrilateral is judged by: among the seed quadrilaterals and valid quadrilaterals matching the current contour, each contour has only one seed quadrilateral but there may be multiple valid quadrilaterals, and calculate the final seed quadrilateral The distances between the two corner points of , respectively, to the two corner points of the effective quadrilateral, if the four distances are all greater than the given threshold, it means that there is no shared corner point between the seed quadrilateral and the effective quadrilateral, and the next effective quadrilateral is processed; C9, if If there is a shared corner point, determine whether the line segment of the shared arm and the open arm of the effective quadrilateral overlaps with the line segment of the shared arm of the seed quadrilateral. Whether the overlapping number of the line segment where the open arm is located is greater than or equal to 1, and if so, obtain the contour of the cubic object with the three arms of the seed quadrilateral as the upper surface, and the arms of the effective quadrilateral except the overlapping arms as the sides of the side vertical plane, and obtain All the contours of the cubic object are recorded as the set of contours of the cubic object, and each contour of the cubic object is a polygon; C10. Obtain all the point coordinates of the contour of the cubic object according to the endpoints of each side of the contour of the cubic object, and the contour of the cubic object takes the seed quadrilateral as Upper surface; C11. Traverse the set of cubic object contours, calculate the y-coordinate P.y of the center point of the contour of each cubic object and the y-coordinate t.y of the center point of its upper surface, and calculate the area P.area of the contour of each cubic object and its upper surface The area t.area of , if t.y>P.y or t.area>P.area, then remove the contour of the cubic object to obtain the final set of cubic object contours. 0.8 is the given distance scale threshold. 15 degrees and 165 degrees are given low and high angle thresholds.

The way to judge the overlap of two line segments is: given line segment E and line segment F, length L1 of E and length H1 of F; calculate the angle difference between line segment E and line segment F, if the angle difference is greater than the given threshold, then line segment E Does not overlap with the line segment F; otherwise, calculate the distances R1 and R2 from the start point of E to the start point and end point of F, and the distances R3 and R4 from the end point of E to the start point and end point of F, if max(max(R1, R2), max (R3, R4)) >= (max(L1, H1) + min(L1, H1) / 2), then line segment E and line segment F do not overlap; otherwise, if max(R1, R2) < H1, line segment E's The distance from the starting point to the line segment F is less than the given threshold. If max(R3, R4) < H1, the distance from the end point of the line segment E to the line segment F is less than the given threshold. If max(R1, R3) < L1, the distance of the line segment F The distance from the starting point to the line segment E is less than the given threshold. If max(R2, R4) < L1, and the distance from the end point of the line segment F to the line segment E is less than the given threshold, the line segment E and the line segment F overlap.

In S8, the left and right vertical planes corresponding to each upper surface quadrilateral are obtained according to the segmentation image of the left vertical plane and the segmentation image of the right vertical plane. The image obtains all the contours of the left vertical plane and their corresponding rotated rectangles and circumscribed rectangles, and determines whether the y-coordinate of the center point of the rotated rectangle is greater than or equal to the y-coordinate of the center point of the quadrilateral on the upper surface. If the y-coordinate of the center point of the rotated rectangle is smaller than the upper The y-coordinate of the center point of the quadrilateral on the surface is used to process the next contour; if the y-coordinate of the center point of the rotating rectangle is greater than or equal to the y-coordinate of the center point of the quadrilateral on the upper surface, determine the area of the contour of the cube corresponding to the quadrilateral on the upper surface and the overlapping area of the circumscribed rectangle. Whether the ratio of the contour area of the cube object is greater than or equal to 0.6; if it is greater than or equal to 0.6, find the contour with the largest area among all the left vertical contours that meet the above conditions, which is the left vertical contour of the current cubic object; obtain the right vertical contour according to the right vertical segmentation image All the contours of the surface and their corresponding rotated rectangles and circumscribed rectangles. The processing methods for all the contours of the right vertical segmented image are the same as the processing methods for all the contours of the left vertical surface, so as to obtain the left vertical surface corresponding to the upper surface of each cubic object. and the right vertical plane, each group of upper surface, left vertical plane and right vertical plane constitutes a cubic object to realize cubic object detection. 0.6 is the given area scale threshold.

The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the scope of the present invention. within the scope of protection.

Claims (10)

1. A cubic object recognition method based on a depth image is characterized by comprising the following steps:

s1, obtaining a depth image of the target, and respectively obtaining gradient maps in the X direction and the Y direction which have the same size with the depth image; s2, dividing pixel points in the image into a horizontal plane, a left vertical plane and a right vertical plane according to the gradient map, and acquiring an edge point map of the depth image; s3, acquiring segmented images of the upper surface, the left vertical surface and the right vertical surface of the target in the depth image according to the classification result, the gradient map and the edge point map; s4, performing edge line detection on the edge point image, clustering and combining all detected edge lines, and acquiring a collinear edge line group set; s5, extracting corner points according to each group of the longest edge lines of the collinear edge line group set; s6, pairing the extracted corner points to obtain a quadrangle; s7, determining the upper surface and the overall contour of the cubic object according to the obtained quadrangle and the segmentation image of the upper surface; and S8, acquiring a left vertical surface and a right vertical surface corresponding to each upper surface quadrangle according to the segmented image of the left vertical surface and the segmented image of the right vertical surface, and realizing the positioning of the cubic object.

2. The method for identifying a cubic object based on a depth image as claimed in claim 1, wherein the divided images of the upper surface, the left vertical surface and the right vertical surface of the object in the depth image are acquired by the classification result, the gradient map and the edge point map in a specific manner:

obtaining a horizontal plane, a left vertical plane and a right vertical plane which have the same size as the gradient map according to the classification result, defining a segmentation image of an upper surface which has the same size as the gradient map, and setting an initial value of a pixel point to be 0;

traversing pixel points in a first column in a binary image of a horizontal plane from bottom to top, acquiring the reliability value of each pixel point according to the depth image, giving a reliability threshold value, and if the reliability value of the pixel point is smaller than the given reliability threshold value, processing the next pixel point if the state of the pixel point is unknown; otherwise, if the corresponding pixel point value is 255, the state of the pixel point is horizontal, if the corresponding pixel point value is 0, the state of the pixel point is vertical, if the number of pixel points with a pixel point value of 0 is greater than a given height threshold, then the pixel point with the pixel point value of 255 appearing later is corresponded to the segmentation image on the upper surface, the pixel value is updated to be 255, the number of lines of the pixel point with the changed pixel point state each time is recorded, after the pixel points of all lines in the first row are traversed, setting the pixel values of the positions of all pixel points in the first column from the recorded line number to the 1 st line, which correspond to the upper surface segmentation image, as 0, traversing all the lines and rows of the parallel plane image in the same way to obtain an initial upper surface segmentation image, repairing the obtained initial segmented image of the upper surface, and updating the segmented image of the upper surface;

and respectively corresponding the edge points in the edge point graph to the updated upper surface divided image, the left vertical plane binary image and the right vertical plane binary image one by one, updating the pixel point value of the horizontal non-edge point in the updated upper surface divided image to be 0, updating the pixel point value of the non-edge point in the left vertical plane binary image to be 0, updating the pixel point value of the non-edge point in the right vertical plane binary image to be 0, acquiring the final upper surface divided image, the left vertical plane binary image and the right vertical plane binary image, and respectively recording the left vertical plane binary image and the right vertical plane binary image as the left vertical plane divided image and the right vertical plane divided image.

3. The method for identifying a cubic object based on a depth image as claimed in claim 1, wherein the specific manner of obtaining the edge point map of the depth image is as follows:

calculating the classified gradient average Yp of all the pixel points in the Y direction of the vertical plane, the classified gradient average Yq of all the pixel points in the Y direction of the horizontal plane, the gradient average XP of all the pixel points in the X direction of the left vertical plane and the gradient average Xq of all the pixel points in the X direction of the right vertical plane; setting a positive threshold value in the Y direction, a negative threshold value in the Y direction, a positive threshold value in the X direction and a negative threshold value in the X direction according to Yp, Yq, Xp and Xq; defining an image with the same size as the gradient image, and setting the initial value of a pixel point in the image to be 0; traversing all pixel points of the gradient map in the X direction and the gradient map in the Y direction, and if the absolute value of the gradient value in the Y direction of each pixel point is larger than the absolute value of the gradient value in the X direction and is smaller than a negative threshold value in the Y direction or is larger than or equal to a positive threshold value in the Y direction, the pixel point is an edge point, and the pixel point is corresponding to a defined image and the pixel value is updated to be 255; if the absolute value of the gradient value in the Y direction is smaller than the absolute value of the gradient value in the X direction, and the absolute value of the gradient value in the X direction is smaller than or equal to the negative threshold value in the X direction or larger than or equal to the positive threshold value in the X direction, the pixel point is an edge point, the pixel point is corresponding to a defined image, and the pixel value is updated to be 255; and obtaining an edge point diagram.

4. The depth-image-based cubic object recognition method according to claim 3,

the specific way of carrying out edge line detection on the edge point diagram, clustering and merging all detected edge lines and obtaining a collinear edge line group set is as follows:

performing edge line detection on an edge line diagram, acquiring all edge lines, sequencing the edge lines into an edge line set according to the length of the edge lines, defining an empty collinear edge line set, wherein the collinear edge line set comprises a plurality of groups of edge lines, the edge lines in each group are collinear, the edge lines between two groups are not collinear, firstly adding the longest edge line into one group of the collinear edge line set, judging whether each A edge line in the edge line set is collinear with each group of the longest B edge lines in the collinear edge line set or not, and sequentially performing the following judgment conditions:

c1, calculating the angle difference between the edge line A and the edge line B, and judging whether the angle difference is less than or equal to a given angle difference threshold value e;

c2, if the angle difference is less than or equal to the angle difference threshold e, calculating the distances T1 and T2 from the starting point of the B edge line to the starting point and the ending point of the A edge line, and the distances T3 and T4 from the ending point of the B edge line to the starting point and the ending point of the A edge line, and given the distance threshold G between points, judging whether the minimum distance among the distances T1, T2, T3 and T4 is less than or equal to the distance threshold G between points;

c3, if min (min (T1, T2), min (T3, T4)) ≦ G, calculating the length L of the edge line A and the length H of the edge line B, and judging whether the maximum distance among the distances T1, T2, T3 and T4 is smaller than G + H + L;

c4, if max (max (T1, T2), max (T3, T4)) < (G + H + L), calculating distances S1 and S2 from the starting point and the end point of the B edge line to the A edge line, and distances S3 and S4 from the starting point and the end point of the A edge line to the B edge line, giving a distance threshold J, and if the maximum distance between T1 and T2 is smaller than L + G, judging whether S1 is smaller than or equal to J; if the maximum distance between the distances T3 and T4 is smaller than L + G, judging whether S2 is smaller than or equal to J; if the maximum distance between the distances T1 and T3 is smaller than L + G, judging whether S3 is smaller than or equal to J; if the maximum distance between the distances T2 and T4 is smaller than L + G, judging whether S4 is smaller than or equal to J; if any maximum distance is larger than or equal to L + G, directly executing the next step;

c5, if the condition in C4 is satisfied, the A edge line and the B edge line are collinear, the A edge line is added into the group of the collinear edge line group set, and the edge line of the group is updated;

in the judging process, if any condition is not met, directly processing the longest edge line of the next group in the collinear edge line group set until all groups of the collinear edge line group set are processed, wherein each group of edge lines is represented by the longest edge line of the group, and if the A edge line is not collinear with the longest edge line of all groups of the collinear edge line group set, directly adding the A edge line into a new group of the collinear edge line group set, and updating the group number of the collinear edge line group set; after the A edge line and the B edge line are collinear, judging whether the A edge line and the B edge line meet a combination condition, if T1 is not less than H, T2 and not more than H, T3 and not more than H and T4 and not more than H, the A edge line and the B edge line do not meet the combination condition, otherwise, combining the A edge line and the B edge line, updating the starting point and the end point of the B edge line, and updating a collinear edge line group set;

and processing the next edge line in the edge line set in the same way as the edge line A, continuously updating the collinear edge line group set, and sequencing the edge lines of each group in the collinear edge line group set according to the length to obtain the final collinear edge line group set.

5. The method for identifying a cubic object based on a depth image as claimed in claim 4, wherein the specific manner of extracting the corner points from each group of the longest edge lines of the set of collinear edge line groups is:

c1, obtaining the longest edge line of each group of the collinear edge line group set, recording the longest edge line as a seed edge line set, sequencing all edge lines of the seed edge line set from long to short, traversing all edge lines of the seed edge line set, calculating an included angle a between the longest C edge line and any other D edge line from the longest C edge line, setting a curvature threshold value b, and judging whether min (a, 180-a) is greater than b;

c2, if min (a, 180-a) > b, calculating the intersection point W (Wx, Wy) of the C edge line and the D edge line, and judging whether Wx or Wy does not exceed the image range;

c3, if Wx or Wy does not exceed the image range, judging whether the intersection point W is an angular point, wherein the condition for judging whether the intersection point W is the angular point is as follows: first, distances D1 and D2 from two end points of the C edge line to the intersection point W are calculated, a minimum distance min _ D1 is defined, if the intersection point W is between the two end points of the C edge line, min _ D1=0, otherwise min _ D1= min (D1, D2), if the minimum distance min _ D1 is not 0 or min (D1, D2) is less than a given arm length threshold Z1, then distances D3 and D4 from the two end points of the D edge line to the intersection point W are calculated, a minimum distance min _ D2 is defined, if the intersection point W is between the two end points of the D edge line, min _ D2=0, otherwise min _ D56 = min (D3, D4), if the minimum distance min _ D2 is not 0 or min (D3, D4) is less than a given arm length threshold Z2, and it is determined whether the intersection point W2 is smaller than the given arm length threshold Z2, D2, 2 r;

c4, if the condition of C3 is met, giving a quadrilateral arm gap threshold v, judging whether max (min (d1, d2) and min (d3, d 4)) is less than or equal to v, and if the condition is met, determining that the intersection point W is the corner point of the quadrilateral;

sequentially judging the conditions C1 to C4, and if any condition is not met, processing the next edge line;

adding the first acquired corner point into the corner point set, and calculating the distances L between the other acquired corner points and all corner points in the corner point set_iI is a variable, judging whether all the angular points in the angular point set are similar, if L is the same_iR, and adding the corner point to the corner point set, wherein the corner point is not similar to the rest corner points;

the intersection point W corresponds to two arms, the first arm is a connecting line of the intersection point W and the terminal point of the C edge line, the second arm is a connecting line of the intersection point W and the terminal point of the D edge line, the terminal point of the C edge line is the larger corresponding terminal point of D1 and D2 in the C edge line, and the terminal point of the D edge line is the larger corresponding terminal point of D3 and D4 in the D edge line; and processing any two rest edge lines in the seed edge line group set in the same way to obtain all the corner points, expanding two arms of each corner point according to the seed edge line set to obtain a final corner point set, and realizing corner point extraction.

6. The method for identifying a cubic object based on a depth image as claimed in claim 5, wherein the extracted corner points are paired in a specific manner:

c1, pairing all the corner points in the corner point set in pairs, if the coordinates of the two paired corner points are different, continuously judging whether the two corner points share one arm, marking the two arms of one corner point as arm11 and arm12, marking the two arms of the other corner point as arm21 and arm22, and firstly judging whether arm11 and arm21 are the same arm of the two corner points;

c2, judging the mode that arm11 and arm21 are the same arm: if the distance between two corner points is greater than one fourth of the minimum value of arm11 and arm21, firstly calculating the absolute value x1 of the angle difference between arm11 and the connecting line of the two corner points, the absolute value x2 of the angle difference between arm21 and the connecting line of the two corner points, and the value ranges of x1 and x2 are [0, 180], giving an angle similarity threshold value f, if x1 and x2 are both smaller than the angle similarity threshold value f, giving an inter-corner distance threshold value u and a length proportion threshold value ratio, if the length sum of arm11 and arm21 is greater than or equal to the ratio times of the distance between the two corner points and is greater than or equal to the difference between the distance between the two corner points and u, judging that arm11 and arm21 are the same arm, and successfully pairing the corner points;

otherwise, judging whether any other two arms of the two angular points are the same arm by adopting the same method, if not, failing to pair the angular points, otherwise, successfully pairing the angular points;

pairwise pairing the rest corner points in the same manner to obtain all successfully paired corner point combinations, wherein the same arm of each corner point combination is a shared arm, the other two arms are opening arms, included angles u1 and u2 between the two opening arms and the shared arm in each corner point combination are calculated, the value ranges of the included angles u1 and u2 are [0, 180], and the corner point combinations with the included angle u1 or u2 smaller than the bending threshold b are removed; judging whether two opening arms of the remaining corner point combinations are on the same side of the shared arm or not, removing the corner point combinations of the two opening arms which are not on the same side of the shared arm, and finally determining a quadrangle according to the currently paired corner point combinations; all corner combinations are processed in the same way to obtain all quadrilaterals.

7. The depth image-based cubic object recognition method of claim 5, wherein the specific way of expanding the two arms of each corner point according to the set of seed edge lines is as follows:

traversing all edge lines of the seed edge line set, and calculating the edge of each corner point to the current seed edge

If the distance is smaller than a given threshold, judging whether the seed edge line can extend two arms of the current corner point, and firstly judging whether the seed edge line can extend the arm1 of the current corner point, wherein the specific mode is as follows:

c1, calculating the distance from the current corner point to the starting point and the ending point of the edge line of the seed, and recording as dist1 and

dist2, judging whether the current corner point is between two end points of the seed edge line;

c2, if the current corner point is between two end points of the edge line of the seed, the current corner point is connected to the edge of the seed

Marking a line segment where a starting point of the line is positioned as M, the starting point of the line segment M as a current corner point, the end point of the line segment M as a starting point of a seed edge line, the line segment where an arm1 of the current corner point is positioned as N, the starting point of the line segment N as the current corner point, calculating an angle1 of the line segment M and an angle2 of the line segment N, and judging whether the absolute value of the difference between the angle1 and the angle2 is smaller than a given angle threshold value or not;

c3, if the absolute value of the difference between angle1 and angle2 is smaller than a given angle threshold, calculating the distance rM from the end point of the line M to the current corner point, calculating the length rN of the line segment N, and judging whether the difference value between rM and rN is smaller than a given length difference threshold;

c4, if the difference between rM and rN is less than the given length difference threshold, calculating the end point of the line segment N

If the distance to the current seed edge line is smaller than the given point-to-line distance threshold and dist1 is greater than rN, the current seed edge line may extend the arm1 of the current corner point, the end point of the extended arm1 of the current corner point is the start point of the seed edge line, and the length of the extended arm1 is dist 1;

c5, if any condition from C2 to C4 is not satisfied, it indicates that the current seed edge line may not be expanded

Judging whether the current seed edge line can be expanded to the arm2 of the current angular point or not by adopting the same method for the arm1 of the current angular point, if so, taking the end point of the expanded arm2 as the starting point of the seed edge line, taking the length of the expanded arm2 as dist1, and if not, directly executing the next step;

c6, updating the terminal point of the line segment M in the C2 to the terminal point of the current seed edge line, updating the line segment N to the line segment where the arm2 of the current corner point is located, the starting point of the line segment N is the current corner point, updating dist1 to dist2, judging whether the current seed edge line can expand the arm1 and the arm2 of the current corner point again by adopting the same method, and realizing the expansion of the two arms of the current corner point;

c7, if the condition in C1 is not met, judging whether dist1 is less than or equal to dist 2;

c8, if dist1 is less than or equal to dist2, updating the starting point of the segment M in C2 to be the current seed edge

Updating the starting point of the edge line to be the end point of the current edge line, wherein the line segment N is the line segment where the arm1 or the arm2 of the current angular point is located, and judging whether the current seed edge line can expand the arm1 or the arm2 of the current angular point by adopting the same method to realize the expansion of the two arms of the current angular point;

c9, if dist1 is larger than dist2, updating the starting point of the line segment M in the C2 to be the current seed edge line

And (3) updating the terminal point to be the starting point of the current edge line, updating the line segment N to be the line segment where the arm1 or the arm2 of the current corner point is located, updating the dist1 to be the dist2, and judging whether the current seed edge line can expand the arm1 and the arm2 of the current corner point by adopting the same method to realize the expansion of the two arms of the current corner point.

8. The method for identifying a cubic object based on depth images as claimed in claim 6, wherein the specific way of determining the top surface and the overall contour of the cubic object from the obtained segmented images of the quadrangle and the top surface at S7 is:

c1, processing the obtained quadrangles, if the end point distance of the two arms of the quadrangles to be matched is smaller than the given distance proportion threshold value times, and the quadrangles do not accord with the end point distance, processing the next quadrangle, obtaining all quadrangles which accord with the conditions, obtaining the outline and the external rectangle of each quadrangle, and calculating the convex hull of each quadrangle according to the outline coordinate points;

c2, carrying out contour detection according to the segmented image of the upper surface to obtain all contours, calculating convex hulls of the contours according to contour coordinate points, obtaining a minimum rotating rectangle containing each contour and a circumscribed rectangle containing the rotating rectangle, calculating the area of each contour, and judging whether the area of the contour is larger than a given area threshold value;

c3, if the area of the contour is larger than the area threshold, matching the current contour with the quadrangle meeting the conditions, respectively calculating the minimum distances h1 and h2 from the two corner points of the quadrangle to be matched to the convex hull of the current contour, and judging whether the two corner points of the quadrangle to be matched are outside the convex hull of the quadrangle to be matched and whether h1 and h2 are smaller than or equal to a given distance threshold;

c4, if two corner points of the matched quadrangle are outside the convex hull of the matched quadrangle and h1 and h2 are both smaller than or equal to the distance threshold, judging whether the center point of the rotating rectangle outside the current outline is inside the convex hull of the matched quadrangle;

c5, if the center point of the rotating rectangle outside the current contour is positioned in the convex hull of the quadrangle for matching, calculating the absolute value of the minimum distance from the rotating rectangle to the convex hull of the quadrangle for matching, and meeting the condition that the value is less than or equal to the length and width of the rotating rectangle and the center point is positioned at the lower side of any arm of the quadrangle for matching, calculating the overlapping area of the external rectangle of the quadrangle for matching and the external rectangle of the current contour, if the overlapping area is greater than a given proportional threshold, the current contour and the quadrangle for matching are successfully primarily matched, and the quadrangle for matching is recorded as a primarily matched quadrangle;

c6, normalizing the distances from the center point of the rotating rectangle of the current contour to the three arms of the preliminary matching quadrangle, wherein the normalized value is in a theoretical range, the sum of h1 and h2 is less than or equal to a given threshold value, and the preliminary matching quadrangle is marked as an effective quadrangle;

c7, judging whether the two opening arms of the effective quadrangle face downwards or not, and recording the effective quadrangle with the two opening arms facing downwards as a seed quadrangle; if the next sub-quadrangle is obtained, calculating whether the sum of the minimum distances from the two corner points of the sub-quadrangle to the outline of the sub-quadrangle is smaller than the sum of the minimum distances from the two corner points of the previous sub-quadrangle to the outline of the sub-quadrangle, if so, replacing the previous sub-quadrangle with the current sub-quadrangle, otherwise, removing the current sub-quadrangle, and continuing processing until the final sub-quadrangle of the current outline is obtained;

c8, judging whether a final seed quadrangle of the current contour and the effective quadrangle have a shared corner point;

c9, if the sharing angular points exist, judging whether the line segment where the sharing arm and the opening arm of the effective quadrangle are located is overlapped with the line segment where the sharing arm of the seed quadrangle is located, if so, judging whether the number of the overlapping of the line segment where the sharing arm and the opening arm of the effective quadrangle are located and the line segment where the opening arm of the seed quadrangle is located is more than or equal to 1, if so, acquiring the cubic object profile with the three arms of the seed quadrangle as the upper surface and the arms of the effective quadrangle except the overlapping arms as the sides of the side vertical plane, acquiring all the cubic object profiles, and recording the cubic object profiles as a cubic object profile set;

c10, acquiring all point coordinates of the cubic object outline according to the end points of each edge of the cubic object outline, wherein the cubic object outline takes a seed quadrangle as the upper surface;

c11, traversing the cubic object contour set, calculating the y coordinate P.y of the center point of each cubic object contour and the y coordinate t.y of the center point of the upper surface of each cubic object contour, calculating the area P.area of each cubic object contour and the area t.area of the upper surface of each cubic object contour, and if t.y > P.y or t.area > P.area, rejecting the cubic object contour to obtain the final cubic object contour set.

9. The method for identifying a cubic object based on a depth image according to claim 8, wherein the left and right vertical planes corresponding to each quadrilateral on the upper surface are obtained from the segmented image of the left vertical plane and the segmented image of the right vertical plane, and the cubic object is positioned by: traversing all the cubic object contour sets, acquiring all the contours of the left vertical surface and corresponding rotating rectangles and external rectangles according to the segmented images of the left vertical surface, and judging whether the y coordinate of the center point of the rotating rectangle is larger than or equal to the y coordinate of the center point of the upper surface quadrilateral; if the former condition is met, judging whether the ratio of the cubic object outline area corresponding to the upper surface quadrangle and the overlapping area of the circumscribed rectangle to the cubic object outline area is larger than or equal to a given area ratio threshold value or not; if the area ratio is larger than or equal to the given area ratio threshold, finding the contour with the largest area in all the left vertical plane contours meeting the conditions, namely the left vertical plane contour of the current cubic object;

and acquiring all the outlines of the right vertical plane and corresponding rotating rectangles and circumscribed rectangles of the right vertical plane according to the segmentation image of the right vertical plane, wherein the processing mode of all the outlines of the segmentation image of the right vertical plane is consistent with the processing mode of all the outlines of the left vertical plane, so as to acquire the left vertical plane and the right vertical plane corresponding to the upper surface of each cubic object, and each group of the upper surface, the left vertical plane and the right vertical plane forms one cubic object, so that the detection of the cubic objects is realized.

10. The method for identifying a cubic object based on a depth image as claimed in claim 8, wherein the specific way of determining whether the final seed quadrilateral of the current contour shares a corner point with the effective quadrilateral in C8 is as follows: and calculating the distances from two corner points of the final seed quadrangle to two corner points of the effective quadrangle, if the four distances are all larger than a given threshold condition, indicating that no shared corner point exists between the seed quadrangle and the effective quadrangle, and if the four distances are not all larger than the given threshold condition, indicating that the shared corner point exists between the seed quadrangle and the effective quadrangle.