WO2018152710A1 - Image correction method and device - Google Patents

Abstract

Provided is an image correction method, acquiring a first aspect ratio of a quadrangle of an image to be corrected and a geometric feature of the quadrangle; acquiring a compensation factor of the quadrangle according to the geometric feature of the quadrangle; calculating a second aspect ratio of the quadrangle according to the first aspect ratio of the quadrangle and the compensation factor of the quadrangle; calculating a transformation matrix according to the second aspect ratio of the quadrangle; and correcting the image to be corrected according to the transformation matrix. By means of the solution provided in the present application, an estimated value closer to the original aspect ratio of a captured subject can be obtained, thereby obtaining a corrected image that is substantially consistent with the original aspect ratio of the captured subject.

Description

Image correction method and device Technical field

The present application relates to the field of image processing, and in particular, to a method and apparatus for image correction.

Background technique

With the rapid spread of smart terminals, the use of smart terminals to obtain data has become the choice of more and more users. The camera hardware used by smart terminals is constantly upgraded, and the software functions are becoming more and more abundant. Image correction is one of the featured functions, which can meet the needs of users to obtain various types of images.

When the smart terminal acquires an image of the subject from the front side, an image that substantially matches the ratio of the subject can be obtained, thereby allowing the user to view or perform Optical Character Recognition (OCR). However, when the smart terminal acquires an image of the subject from the side, the image may have a perspective distortion, which will affect the review or recognition, so it is necessary to correct the distortion image.

Summary of the invention

The present application describes a method and apparatus for image correction that can be obtained by increasing the accuracy of the original image aspect ratio estimation.

In a first aspect, an image correction method is provided, which acquires a first aspect ratio of a quadrilateral of an image to be corrected and a geometric feature of the quadrilateral; and acquires a compensation factor of the quadrilateral according to the geometric feature of the quadrilateral; Computing a first aspect ratio of the quadrilateral and a compensation factor of the quadrilateral, calculating a second aspect ratio of the quadrilateral; calculating a transformation matrix according to a second aspect ratio of the quadrilateral; correcting the method according to the transformation matrix The image to be corrected. By compensating for the first aspect ratio of the quadrilateral, the accuracy of the aspect ratio of the rectangle in the corrected image can be improved as a whole.

In one possible design, the step of acquiring the quadrilateral compensation factor according to the geometric features of the quadrilateral comprises obtaining a compensation factor of the quadrilateral according to the geometric features of the quadrilateral and a compensation formula. The determining method of the compensation formula includes: acquiring N images of a rectangle of a known aspect ratio at N different angles, wherein N is an integer not less than 2; from each of the N images, Obtaining a first aspect ratio of the rectangle and a geometric feature of the rectangle, obtaining corresponding data of the compensation factor of the N sets of the rectangle and the geometric feature of the rectangle; and fitting a compensation formula according to the N sets of corresponding data. By obtaining an image of a rectangle with a known aspect ratio at multiple angles, the compensation formula is fitted by statistical methods, so that the compensation result is more accurate.

In a possible design, the step of fitting a compensation formula according to the N sets of corresponding data includes dividing the N sets of corresponding data into a plurality of types corresponding to different geometric features, according to each of the plurality of types of data Fit the compensation formula separately. In this way, multiple shooting scenes can be combined to refine the scene classification, and each scene uses a compensation formula to make the compensation result more accurate.

In one possible design, the compensation formula is predetermined before the image is corrected. Thereby, the process of correcting the image can be simplified.

In one possible design, the geometric feature comprises at least one of a distance feature or an angular feature. Different image acquisition scenarios can be determined by different types of geometric features.

In one possible design, the angular feature comprises: a difference between two sets of opposite sides of the quadrilateral; or The trigonometric function value of the difference between the two sets of the sides of the quadrilateral; or the difference between the two sets of the trigonometric angles of the quadrilateral. In this way, a set of features can be obtained that effectively represent the degree of tilt of the quadrilateral.

In one possible design, the geometric features include a first geometric feature and a second geometric feature. The first geometric feature includes at least one of the following: a ratio of a length of a longest side of the quadrilateral to a side length of the image to be corrected, or a diagonal length of the quadrilateral and the to-be-corrected The ratio of the diagonal length of the image, or the angle between the two sets of opposite sides of the quadrilateral, or the trigonometric value of the set of opposite sides of the quadrilateral. The second geometric feature includes: a difference between two sets of opposite sides of the quadrilateral, or a trigonometric function of a difference between two sets of opposite sides of the quadrilateral, or two sets of opposite sides of the quadrilateral The difference between the trigonometric value of the included angle, or the absolute value of the difference between the two sets of opposite sides of the quadrilateral, or the trigonometric value of the absolute value of the difference between the two sets of opposite sides of the quadrilateral, Or the absolute value of the difference between the values of the trigonometric functions of the two sets of sides of the quadrilateral. In this way, a set of features can be obtained that effectively represent the relative size of the quadrilateral, the relative distance from the lens, and the degree of quadrilateral tilt. Different types of geometric features can be used to determine different shooting scenes and to obtain compensation formulas suitable for different shooting scenes.

In one possible design, the first aspect ratio of the quadrilateral or rectangle is a ratio of a width estimate to a height estimate, and the width estimate is a median line length that is smaller than an angle in the horizontal direction, The height estimate is the length of the median line that is less angled to the vertical. In this way, a preliminary estimate of the aspect ratio close to the original aspect ratio and regularly ruled can be obtained.

In one possible design, the first aspect ratio of the quadrilateral or rectangle is a ratio of a width estimate to a height estimate, the width estimate being a length of a pair of opposite sides that are smaller than the horizontal direction. The length of the edge is the length of the long side of a pair of opposite sides that is smaller than the vertical direction. In this way, a preliminary estimate of the aspect ratio close to the original aspect ratio and regularly ruled can be obtained.

In one possible design, the functional form of the compensation formula includes a first function, a quadratic function, or a higher order function form. In this way, the general form of the compensation formula can be determined, and the compensation formula can be obtained by calculating the parameters of the function.

In a second aspect, an apparatus is provided, including: a first acquiring module, configured to acquire a first aspect ratio of a quadrilateral in an image to be corrected and a geometric feature of the quadrilateral; and a second acquiring module, configured to Calculating a compensation factor of the quadrilateral according to a geometric feature of the quadrilateral; a first calculating module, configured to calculate a second aspect ratio of the quadrilateral according to a first aspect ratio of the quadrilateral and a compensation factor of the quadrilateral; a calculation module, configured to calculate a transformation matrix according to a second aspect ratio of the quadrilateral; and a correction module, configured to correct the image to be corrected according to the transformation matrix. By compensating for the first aspect ratio of the quadrilateral by the compensation factor, the accuracy of the aspect ratio of the rectangle in the corrected image can be improved as a whole.

In a possible design, the second obtaining module is configured to obtain a compensation factor of the quadrilateral according to the geometric features and the compensation formula of the quadrilateral. The second obtaining module includes: a first acquiring unit, configured to acquire N images of a rectangle of a known aspect ratio at N different angles, where N is an integer not less than 2; and the second acquiring unit obtains The aspect ratio of the rectangle and the geometric feature of the rectangle obtain corresponding data of the compensation factor of the N sets of the rectangle and the geometric feature of the rectangle; the fitting unit is configured to fit according to the N sets of corresponding data Compensation formula. By obtaining an image of a rectangle with a known aspect ratio at multiple angles, the compensation formula is fitted by statistical means. The result of the compensation is more accurate.

In a possible design, the fitting unit further includes: a classification sub-module for dividing the N sets of corresponding data into a plurality of types; and a fitting sub-module for each of the plurality of types A type of data is fitted to the compensation formula. In this way, multiple shooting scenes can be combined to refine the scene classification, and each type uses a compensation formula to make the compensation result more accurate.

In one possible design, the compensation formula is predetermined before the image is corrected. Thereby, the process of correcting the image can be simplified.

In one possible design, the geometric feature comprises at least one of a distance feature or an angular feature. Different image acquisition scenarios can be determined by different types of geometric features.

In one possible design, the angular feature comprises: a difference between two sets of opposite sides of the quadrilateral; or a trigonometric value of a difference between two sets of opposite sides of the quadrilateral; or the quadrilateral The difference between the two sets of trigonometric values of the angle between the two sides. In this way, a set of features can be obtained that effectively represent the degree of tilt of the quadrilateral.

In one possible design, the geometric features include a first geometric feature and a second geometric feature. The first geometric feature includes at least one of the following: a ratio of a length of a longest side of the quadrilateral to a side length of the image to be corrected, or a diagonal length of the quadrilateral and the to-be-corrected The ratio of the diagonal length of the image, or the angle between the two sets of opposite sides of the quadrilateral, or the trigonometric value of the set of opposite sides of the quadrilateral. The second geometric feature includes: a difference between two sets of opposite sides of the quadrilateral, or a trigonometric function of a difference between two sets of opposite sides of the quadrilateral, or two sets of opposite sides of the quadrilateral The difference between the trigonometric value of the included angle, or the absolute value of the difference between the two sets of opposite sides of the quadrilateral, or the trigonometric value of the absolute value of the difference between the two sets of opposite sides of the quadrilateral, Or the absolute value of the difference between the values of the trigonometric functions of the two sets of sides of the quadrilateral. In this way, a set of features can be obtained that effectively represent the relative size of the quadrilateral, the relative distance from the lens, and the degree of quadrilateral tilt. Different types of geometric features can be used to determine different shooting scenes and to obtain compensation formulas suitable for different shooting scenes.

In one possible design, the first aspect ratio of the quadrilateral or rectangle is a ratio of a width estimate to a height estimate, and the width estimate is a median line length that is smaller than an angle in the horizontal direction, The height estimate is the length of the median line that is less angled to the vertical. In this way, a preliminary estimate of the aspect ratio close to the original aspect ratio and regularly ruled can be obtained.

In one possible design, the first aspect ratio of the quadrilateral or rectangle is a ratio of a width estimate to a height estimate, the width estimate being a length of a pair of opposite sides that are smaller than the horizontal direction. The length of the edge is the length of the long side of a pair of opposite sides that is smaller than the vertical direction. In this way, a preliminary estimate of the aspect ratio close to the original aspect ratio and regularly ruled can be obtained.

In one possible design, the functional form of the compensation formula includes a first function, a quadratic function, or a higher order function form. In this way, the general form of the compensation formula can be determined, and the compensation formula can be obtained by calculating the parameters of the function.

In a third aspect, an apparatus is provided, comprising: one or more processors, a memory, and one or more programs, the one or more programs being stored in a memory and configured to be configured by one or more Executed by the processor, the one or more programs include instructions for performing the method of the first aspect.

In a fourth aspect, a computer program product comprising instructions for causing a computer to perform the method of the first aspect when the instructions are run on a computer.

In a fifth aspect, a computer readable storage medium is provided having stored therein instructions that, when executed on a computer, cause the computer to perform the method of the first aspect.

DRAWINGS

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly described below. Obviously, only some of the embodiments of the present invention are reflected in the following drawings, and other embodiments may be obtained by those skilled in the art without departing from the drawings. All such embodiments or implementations are within the scope of the present application.

FIG. 1 is a schematic diagram of an image correction scenario according to an embodiment of the present invention; FIG.

2 is a flowchart of an image correction method according to an embodiment of the present invention;

3 is a schematic diagram of a first aspect ratio according to an embodiment of the present invention;

4 is a flowchart of a method for determining a compensation formula according to an embodiment of the present invention;

FIG. 5 is a schematic diagram of acquiring N images of a rectangle having a known aspect ratio according to an embodiment of the present invention; FIG.

FIG. 6 is a schematic diagram of classification according to an image tilt posture according to an embodiment of the present invention; FIG.

FIG. 7 is a schematic diagram of classification according to image distance according to an embodiment of the present invention; FIG.

FIG. 8 is a schematic diagram of classification according to an image tilt posture and distance according to an embodiment of the present invention; FIG.

FIG. 9 is a flowchart of another method for determining a compensation formula according to an embodiment of the present invention; FIG.

FIG. 10 is a schematic structural diagram of a device according to an embodiment of the present invention; FIG.

FIG. 11 is a schematic structural diagram of a second acquiring module according to an embodiment of the present invention;

FIG. 12 is a schematic structural diagram of another second acquiring module according to an embodiment of the present invention;

FIG. 13 is a schematic structural diagram of another device according to an embodiment of the present invention.

detailed description

The technical solutions in the embodiments of the present invention are further described below in conjunction with the accompanying drawings in the embodiments of the present invention.

FIG. 1 is a schematic diagram of an image acquisition scenario according to an embodiment of the present invention. The subject 102 is placed on the support structure 103, and the device 101 takes an image of the subject 102. The subject 102 can be any object containing text and/or images, such as documents, pictures, business cards, documents, books, slides, whiteboards, street signs, and advertising signs. The device 101 can be any device capable of acquiring an image or processing an image, for example, a mobile phone (or "mobile phone"), a tablet (Tablet Personal Computer, TPC), a computer, a digital camera, a wearable device (Wearable Device) ), virtual reality devices, digital broadcast terminals, messaging devices, game consoles, medical devices, fitness equipment, personal digital assistants (PDAs), e-book readers, and scanners . The device 101 can acquire images through optical components such as cameras, cameras or optical sensors, which can be built into the device 101 or externally connected to the device 101. For the sake of clarity and convenience, the optical components of the present application are uniformly described using a camera. In part (A) of Fig. 1, when the device 101 is photographed from the front side, an image 104 substantially in proportion to the subject 102 can be obtained.

However, as shown in part (B) of Fig. 1, when the apparatus 101 photographs the subject 102 from the side, the see-through effect causes the acquired image 104' to be distorted, so the distortion image 104' needs to be corrected. During the image correction process, First, the image 104' is input, then the quadrilateral in the image is detected, then the aspect ratio of the object 102 is estimated according to the quadrilateral, and the transformation matrix is calculated according to the estimated aspect ratio, and finally the distortion image 104' is corrected by using the transformation matrix. It is a rectangular image 104". The above image correction process uses a complex formula for estimating the aspect ratio, which is based on the assumed lens focal length parameter. In practical applications, since the lens focal length is different from the assumed lens focal length, the pair is shot. The aspect ratio estimation error of the object is large, especially when the tilt angle of the device 101 is large, the deviation between the estimated value of the aspect ratio and the actual value is larger.

An image correction method according to an embodiment of the present invention will be described below with reference to FIGS. 2 through 6. Figure 2 shows a flow chart of the method, which may be performed by device 101, the method comprising:

Step 201: Acquire a first aspect ratio R ₀ of a quadrilateral of the image to be corrected and a geometric feature of the quadrilateral;

Step 202, obtaining a quadrilateral compensation factor k according to the geometric feature of the quadrilateral;

Step 203, calculating a second aspect ratio R _{1 of the} quadrilateral according to the first aspect ratio R _{0 of the} quadrilateral and the compensation factor k of the quadrilateral;

Step 204, calculating a transformation matrix K according to a second aspect ratio R _{1 of the} quadrilateral;

Step 205, correcting the image to be corrected according to the transformation matrix K.

The steps 201 to 203 are explained in detail below; the steps 204 and 205 can adopt the existing methods, and details are not described herein again.

In step 201, first, an image to be corrected of the subject is acquired. The image to be corrected may be an image acquired by the camera in real time, such as a frame for capturing a subject or capturing a preview video of the subject, or an existing image, such as an image stored on a memory. The object may be presented as a rectangle as a whole, such as a document, a picture, a business card, a document, a book, a slide, a whiteboard, a street sign, and an advertisement sign mentioned above, or may be a partial area including a rectangle, for example, in a subject. Contains a rectangular image or text block.

Then, the quadrilateral in the image to be corrected is extracted. In one example, device 101 detects a quadrilateral edge to extract a quadrilateral in the image. The detection algorithm of the quadrilateral edge can adopt a known method, and will not be described here.

In another example, the user can manually select a quadrilateral in the image, for example, the user drags the four corners of the quadrilateral box in the image, selects the quadrilateral in the image, and extracts the quadrilateral in the image.

Finally, after the quadrilateral extraction is completed, the four corner coordinates of the quadrilateral can be obtained according to the extracted quadrilateral, thereby calculating the first aspect ratio R ₀ of the quadrilateral and the geometric features of the quadrilateral.

The first aspect ratio R ₀ of the quadrilateral is a preliminary estimate of the original aspect ratio of the quadrilateral, and can be expressed by the ratio of the width estimate W _{e of the} quadrilateral to the height estimate H _e , that is, R ₀ =W _e /H _e . In one example, the first aspect ratio R ₀ may be a ratio of the lengths of the adjacent sides of the quadrilateral. As shown in part (A) of Fig. 3, from the two sets of opposite sides of the quadrilateral, the horizontal opposite side and the vertical opposite side are respectively determined, and the length l _CD of the longer side of the horizontal opposite side is taken as the width estimation value W. _e , that is, W _e = l _CD , the length l _AD of the longer side AD of the vertical opposite side is taken as the height estimation value H _e , that is, H _e = l _AD , therefore, the first aspect ratio R ₀ =W _e /H _e =l _CD /l _AD . The step of determining the horizontal opposite side and the vertical opposite side comprises: comparing the angle between any two adjacent sides of the quadrilateral and the horizontal direction, the side with the smaller angle and the opposite side being the horizontal opposite side, and the remaining A set of opposite sides is a vertical opposite side. Optionally, the angle between any two adjacent edges and the vertical direction may be compared, and the smaller side and the opposite side are vertical opposite sides, and the remaining set of opposite sides are horizontal opposite sides.

In another example, the first aspect ratio R ₀ may be the ratio of the lengths of the two median lines of the quadrilateral. As shown in (B), in two quadrangular bit line, the smaller the angle between the bit line and BD 3 _BD horizontal length L as the estimation value width W _e, i.e., W _e = l _BD , the length l _AC of the median line AC having a small angle with the vertical direction is taken as the height estimation value H _e , that is, H _e = l _AC , therefore, the first aspect ratio R ₀ =W _e /H _e = l _BD /l _AC .

The geometric features of the quadrilateral include angular features. In one example, the angular feature may include two sets of opposite sides of the quadrilateral (ie, the angle between the horizontally opposite sides θ _H and the vertical opposite sides θ _V ), and the absolute value of the difference between the two sets of opposite sides δ (ie δ=|θ _H -θ _V |). The smaller angle of the opposite side of the two sets of opposite sides can be used to determine the degree of inclination of the opposite side with less distortion of the quadrilateral: the larger the angle of the opposite side, the greater the inclination; the opposite side The smaller the angle, the smaller the tilt. The absolute value δ of the difference between the two sets of opposite sides can be used to determine the degree of deformation of the image as a whole: the larger the absolute value δ, the greater the degree of deformation; the smaller the absolute value δ, the smaller the degree of deformation.

In another example, the angular feature may include a trigonometric function value of two sets of opposite sides of the quadrilateral, and a trigonometric function value of the absolute value of the difference between the two sets of angles of the opposite sides or a trigonometric function value of the two sets of opposite sides The absolute value of the difference. The trigonometric value includes a sine, a tangent, a cosine, or a cotangent. The trigonometric function values of the two sets of angles of the opposite sides can be used to determine the degree of inclination of the opposite sides of the quadrilateral distortion, the trigonometric function values of the absolute values of the difference between the two sets of sides, or the two sets of opposite side clips The absolute value of the difference between the angular trigonometric values can be used to determine the degree of deformation of the image as a whole.

For the sake of clarity and convenience, when referring to the geometric features of the quadrilateral, the embodiment of the present invention refers to the absolute angle between the two sets of opposite sides of the quadrilateral θ _H and θ _V and the difference between the two sets of opposite sides. The value δ. When the method according to the embodiment of the present invention is applied to an embodiment in which the geometric feature includes a trigonometric function value, the two sets of opposite angles may be replaced by two sets of trigonometric values of the opposite sides, and two pairs of pairs may be used. The absolute value of the difference between the absolute value of the difference between the edges and the value of the trigonometric function of the two sets of angles replaces the absolute value of the difference between the two sets of opposite sides.

It should be noted that the length of each side of the quadrilateral, the length of the median line, the angle of the opposite side, and the value of the trigonometric function can be obtained by calculating the coordinates of the four corner points of the quadrilateral, which belongs to the conventional geometric knowledge in the field, and is not here. Let me repeat.

In step 202, according to the geometric features of the quadrilateral and the compensation formula, the geometric features of the quadrilateral are substituted into the compensation formula, and the compensation factor of the quadrilateral can be obtained. The compensation formula reflects the relationship between the geometric feature and the compensation factor k, and its general form is k=f(x), where x represents a geometric feature and k represents a compensation factor. The functional form of the compensation formula can be a one-time function, a quadratic function, or a higher-order function form. The compensation formula may be determined in advance before the image correction method of the embodiment of the present invention is implemented, or may be determined in the process of implementing the image correction method of the embodiment of the present invention. Since the tilt posture of the quadrilateral is different in different image acquisition scenes, the quadrilateral can be classified according to the characteristics of the quadrilateral tilt posture, and a compensation formula is determined for each type of quadrilateral. A flowchart of a method for determining the compensation formula is shown in FIG. 4, and the method may be performed by the device 101, including:

Step 2021: Acquire N images of rectangles of known aspect ratio at N different angles;

Step 2022: Obtain a first aspect ratio R _Q and a geometric feature of the rectangle from each of the N images, and obtain corresponding data of the compensation factor k _{Q of the} N sets of rectangles and the geometric feature;

Step 2023, the N sets of corresponding data are divided into multiple types;

In step 2024, the compensation formula is respectively fitted according to each type of data of multiple types.

In step 2021, N images of rectangles of known aspect ratio at N different angles may be acquired by the camera of device 110. Acquiring N images may capture the rectangle through the camera, or may capture a frame of the preview video of the rectangle. As shown in part (A) of Fig. 5, the original aspect ratio of the rectangle may be any known value, for example, the original aspect ratio R = 1 (i.e., square) of the rectangle. For convenience, this paper uses the moment of R=1 Unified description.

In order to obtain N images at N different angles, in one example, as shown in part (B) of Figure 5, in a spatial three-dimensional Cartesian coordinate system, the plane of the rectangle coincides with the XOY plane, when the device is in space When the P point in the middle acquires a rectangular image, the angle can be the angle between the projection P' of the P point on the XOY plane and the positive direction of the X axis α=∠P'OZ, and the angle between the P point and the positive direction of the Z axis. A combination of β = ∠ P'OZ is expressed, wherein the numerical ranges of α and β are α ∈ [0°, 360°), β ∈ [0°, 90°), respectively. When the device acquires a rectangular image from a point on the Z-axis (ie, taken from the front), the angle α = β = 0°. The number N of angles is an integer not less than 2, which is used to satisfy the basic condition for obtaining the compensation formula. Since the accuracy of the compensation formula increases as the number N increases, more images can be acquired at more angles in order to obtain a more accurate compensation formula. As an example, one direction is selected every 45° (i.e., α is taken as 0°, 45°, 90°, ..., 315°) in the range of α ∈ [0°, 360°), and in these directions. On the other hand, an image is taken every 4° in the range of β∈ (0°, 90°) (that is, β is taken as 4°, 8°, ..., 88°, respectively). It should be noted that the above angular range and angular interval are only examples, and other angular ranges and angular intervals may be selected to obtain more or less images.

In another example, as shown in part (C) of FIG. 5, the rectangle is displayed as a quadrilateral Q in the image, by selecting different horizontal-to-edge angles θ _{QH of the} quadrilateral Q and the angle between the vertical opposite sides θ _QV (reference numeral not shown), thereby acquiring N images. As an example, an image can be acquired every 1° in the range of the horizontal-to-edge angle θ _QH ∈ [0°, 50°]; at the same time, the angle θ _QV竖 [0°, 50° at the vertical opposite side In the range of ], an image is acquired every 1°. As another example, an image can also be acquired every 2°. It should be noted that the above angular range and angular interval are only examples, and other angular ranges and angular intervals may also be selected.

In 2022, the first aspect ratio rectangular quadrilateral R _Q R ₀ corresponding to the first aspect ratio, when the aspect ratio of a quadrangle R ₀ using a first adjacent side length ratio, the width of the rectangular first step The high ratio R _Q also uses the ratio of the corresponding adjacent side lengths. When the first aspect ratio R _{0 of the} quadrilateral uses the ratio of the lengths of the two median lines, the first aspect ratio R _{Q of the} rectangle also adopts the ratio of the lengths of the corresponding two median lines. For the specific method of obtaining the first aspect ratio R _Q of the rectangle, refer to the description of step 201, and details are not described herein again.

The geometrical features of the rectangle correspond to the geometric features of the quadrilateral. When the geometric features of the quadrilateral include the two sets of opposite angles θ _H and θ _{V of the} quadrilateral and the absolute value δ of the difference between the two sets of opposite sides, the geometrical features of the rectangle are also The absolute values δ _Q =|θ _QH -θ _QV | of the difference between the two sets of opposite angles θ _QH and θ _QV and the angle between the two sets of opposite sides are included.

The compensation factor k _Q of the rectangle corresponds to the compensation factor k of the quadrilateral, and the compensation factor k _Q of the rectangle reflects the relationship between the original aspect ratio R of the rectangle and the first aspect ratio R _Q . In one example, the compensation factor k _{Q of the} rectangle may be represented by the ratio of the original aspect ratio R of the rectangle to the first aspect ratio R _Q , ie k _Q =R/R _Q . In another example, the compensation factor for the rectangle may be expressed by the ratio of the first aspect ratio R _Q of the rectangle to the original aspect ratio R of the rectangle, ie k _Q =R _Q /R. Since the original aspect ratio R of the rectangle and the first aspect ratio R _Q are both known values, a corresponding set of rectangular compensation factors k _Q and geometric features can be obtained, that is, the compensation factor k _Q , two pairs of pairs The angle θ _QH and θ _QV of the side and the absolute value δ _Q of the difference between the angles of the two sets of opposite sides. By performing the above operations on the N images, corresponding data of the N sets of rectangular compensation factors k _Q and the geometric features of the rectangle can be obtained.

In step 2023, the geometric features of the rectangle are compared to a classification threshold to classify the N sets of corresponding data into a type corresponding to the quadrilateral type. In one example, as shown in FIG. 6, the quadrilateral can be classified into the following four categories: horizontal unilateral inclination, horizontal bilateral inclination, vertical unilateral inclination, and vertical bilateral inclination. The two sets of opposite side angles θ _QH and θ _{QV of the} rectangle are compared with the angle classification threshold θ _T , thereby dividing the N sets of corresponding data into four types corresponding to the quadrilateral type. Wherein, the horizontal-to-edge angle θ _QH of the horizontal unilateral tilt type shown in part (A) of FIG. 6 is smaller than the vertical-to-edge angle θ _QV , and the angle classification threshold θ _{T is} between the two sets of opposite sides The horizontal-to-edge angle θ _QH of the horizontal double-sided tilt type shown in part (B) of FIG. 6 is smaller than the vertical-to-edge angle θ _QV , and the angle classification threshold θ _{T is} smaller than the horizontal-to-edge angle θ _QH ; The vertical unilateral tilt type θ _QV of the vertical unilateral tilt type shown in part (C) is smaller than the horizontal opposite side angle θ _QH , and the angle classification threshold θ _{T is} between the two sets of opposite side angles; The vertical-to-edge angle θ _QV of the vertical double-sided tilt type shown in the (D) portion is smaller than the horizontal-to-edge angle θ _QH , and the angle classification threshold θ _{T is} smaller than the vertical-to-edge angle θ _QV .

The angle classification threshold θ _T can be used to classify the N sets of corresponding data into corresponding quadrilateral types according to a certain ratio or uniformly, and classify the corrected quadrilaterals to select a corresponding compensation formula. In one example, the angle classification threshold θ _T may be a horizontal-to-edge angle θ _{QH of} a rectangle or a statistical quantile of a vertical-to-edge angle θ _QV . The quantile includes quartiles or other proportional quantiles. For example, the angle classification threshold θ _T may be the median of the selection angle-to-edge angle θ _QH (also referred to as the second quartile), that is, all the horizontal-to-edge angles θ _QH are arranged from small to large and then ranked 50th. The value of %) can also be the median of the angle θ _QV between the vertical and the opposite sides, or the mean of the median of the angle between the horizontal and the opposite sides and the median of the angle between the vertical and the opposite sides.

In another example, the angle classification threshold θ _T may also be the mean of the horizontal-to-edge angle θ _QH of the rectangle or the vertical-to-edge angle θ _QV .

In another example, the angle classification threshold θ _T may also be valued empirically, for example, from a range of [2°, 5°].

In step 2024, in each of the N sets of corresponding data, a compensation formula is fitted according to the absolute value δ _Q of the difference between the two sets of opposite sides of the rectangle and the compensation factor k _Q . In one example, Table 1 shows the corresponding data of the absolute value δ _Q of the difference between the two sets of opposite sides of the rectangle in the horizontal unilateral tilt type and the compensation factor k _Q , thereby fitting the horizontal unilateral tilt The type of compensation formula. The form of the compensation formula may be any one of the following, where x represents a geometric feature and k represents a compensation factor:

A linear function (also called a linear function), namely: k = a * x + b, where parameters a and b are determined by data fitting in Table 1, where a = 0.0095 and b = 0.9409.

The quadratic function, ie: k = a * x ² + b * x + c, where the parameters a, b and c are determined by the data fit in Table 1, where a = 0.0003, b = 0.0013, c = 0.9858 .

The cubic function, ie: k = a * x ³ + b * x ² + c * x + d, where the parameters a, b, c and d are determined by the data fit in Table 1, where a = -5E- 6, b = 0.0005, c = -0.0018, d = 0.9945.

Quadratic function, ie: k = a * x ⁴ + b * x ³ + c * x ² + d * x + e, where the parameters a, b, c, d and e are determined by the data fit in Table 1. Where a = 3E-7, b = -2E-5, c = 0.0009, d = -0.0046, e = 0.9991.

The quintic function, ie: k=a*x ⁵ +b*x ⁴ +c*x ³ +d*x ² +e*x+f, where the parameters a, b, c, d, e and f pass the table The data fit in 1 was determined, wherein a=3E-9, b=-7E-8, c=-8E-6, d=0.0006, e=-0.0026, f=0.9961.

Those skilled in the art will appreciate that as long as the result of the accuracy requirement can be obtained by the fitting algorithm, it is also possible to compensate the template for a higher order function of five or more times. The above-mentioned fitting algorithm can adopt a known algorithm and will not be described here.

Table 1 Correspondence data table of absolute value δ _Q and compensation factor k _Q

After the compensation formula is determined, the two sets of opposite side angles θ _H and θ _{V of the} quadrilateral to be corrected are compared with the angle classification threshold θ _T , so that the quadrilateral to be corrected is divided into corresponding types, and the compensation formula under the type is selected. The absolute value δ of the difference between the two sets of angles is substituted into the compensation formula to calculate the compensation factor k of the quadrilateral. For the classification of the corrected quadrilateral, refer to the description of step 2023, and details are not described herein again.

In step 203, the second aspect ratio R ₁ of the quadrilateral is a result of compensating for the first aspect ratio R ₀ and is used as a final estimate of the quadrilateral aspect ratio. Since the compensation factor k rectangular quadrilateral _Q compensation factor k corresponding Therefore, when the rectangular original aspect ratio compensation factor k _Q R _Q R and the aspect ratio of the first ratio, the aspect ratio of the second quadrangular R ₁ can be expressed by the product of the compensation factor k and the first aspect ratio R ₀ , that is, R ₁ =k*R ₀ . When the compensation factor k _Q of the rectangle is the ratio of the first aspect ratio R _Q to the original aspect ratio R, the second aspect ratio R _{1 of the} quadrilateral can be represented by the ratio of the first aspect ratio R ₀ to the compensation factor k , that is, R ₁ =R ₀ /k. By substituting the values of the first aspect ratio R ₀ and the compensation factor k obtained in the foregoing steps into the above expression, the value of the second aspect ratio R ₁ can be obtained.

In the embodiment of the present invention, the image acquisition scenes of different angles are classified, and the compensation formula is obtained by using a statistical method for each type of image acquisition scene, and the preliminary estimation of the quadrilateral aspect ratio is compensated, so that the corrected image is obtained. The aspect ratio is closer to the original aspect ratio of the subject, which improves the image correction effect.

Another embodiment of the present invention provides an image correction method. The method will be described below with reference to FIGS. 2, 4, 7, and 8. The flowchart of the image correction method is as shown in FIG. 2, wherein steps 203 to 205 of the method are the same as steps 203 to 205 described above, and details are not described herein again. Hereinafter, the differences between steps 201 and 202 and steps 201 and 202 described above will be described.

In step 201, the geometric features of the quadrilateral include a first geometric feature and a second geometric feature. The first geometric feature is a combination of a length feature and an angle feature for classifying the quadrilateral. The length feature may include a side length ratio D of the quadrilateral. The aspect ratio D can be used to determine the relative distance between the subject and the camera. The larger the side length ratio, the closer the distance between the subject and the camera; the smaller the side length ratio, the farther the subject is from the camera. In one example, the side length ratio D may be the length h of the longest side of the quadrilateral and the side length of the image to be corrected (eg, For example, the ratio of the height H or the width W) of the image, that is, D=h/H or D=h/W.

In another example, the side length ratio D is the ratio of the diagonal length l of the quadrilateral to the diagonal length L of the image to be corrected, ie, D=l/L.

The angular feature may include two sets of opposite angles θ _H and θ _V , and may also include trigonometric values of two sets of opposite sides of the quadrilateral, for example, sine, tangent, cosine or Cotangent value. The above values can be used to determine the degree of tilt of the opposite sides with less quadrilateral distortion.

The second geometric feature includes an angular feature of the quadrilateral for calculating the compensation factor k. The angular feature of the quadrilateral may include the absolute value δ of the difference between the two sets of opposite sides (ie, δ=|θ _H -θ _V |), and may also include the difference between the two sets of opposite sides, or two sets of opposite side clips. The value of the trigonometric function of the difference between the angles, or the difference between the two sets of trigonometric values of the angles of the edges, or the trigonometric function of the absolute difference between the two sets of angles between the two sides, or the trigonometric function of the two sets of angles between the opposite sides The absolute value of the difference in values. The above values can be used to determine the degree of deformation of the entire image.

For the sake of clarity and convenience, when referring to the geometric features of the quadrilateral, the embodiment of the present invention refers to the ratio of the side length of the quadrilateral D=h/H, the angle between the two sets of opposite sides θ _H and θ _{V ,} and the like. The absolute value δ of the difference between the two groups. When the method according to the embodiment of the present invention is applied to an embodiment in which the geometric feature includes a trigonometric function value, the two sets of opposite angles may be replaced by two sets of trigonometric values of the opposite sides, and two pairs of pairs may be used. The absolute value of the difference between the absolute value of the difference between the edges and the value of the trigonometric function of the two sets of angles replaces the absolute value of the difference between the two sets of opposite sides.

In step 202, a compensation factor is obtained based on the geometric features of the quadrilateral and the compensation formula. Wherein, the second geometric feature of the quadrilateral is substituted into the compensation formula, and the compensation factor of the quadrilateral can be calculated. The compensation formula may reflect the relationship of the second geometric feature to the compensation factor k in the same form as described in step 202 above. In different image acquisition scenes, except that the shooting angle is different, the tilting posture of the quadrilateral Q is different, and the shooting distance is different, which causes the quadrilateral Q to have different sizes. Therefore, the quadrilateral can be classified according to the characteristics of the distance and the tilting posture, Each type of quadrilateral determines a compensation formula. A flowchart of the method for determining the compensation formula is shown in FIG. 4 . The steps 2021 and 2024 of the method are the same as the steps 2021 and 2024 described above, and are not described herein again. Hereinafter, the differences between steps 2022 and 2023 and steps 2022 and 2023 described above will be described.

In step 2022, since the geometric feature of the rectangle corresponds to the geometric feature of the quadrilateral to be corrected, when the geometric feature of the quadrilateral to be corrected includes the first geometric feature and the second geometric feature, the geometric feature of the rectangle also includes the corresponding first geometric feature. And the second geometric feature. As an example, when the first geometric feature of the quadrilateral includes the side length ratio D, the two sets of opposite side angles θ _H and θ _V , and the second geometric feature includes the absolute value δ of the difference between the two sets of opposite sides, the rectangle The first geometric feature correspondingly includes a side length ratio D _Q , two sets of opposite side angles θ _QH and θ _QV , and the second geometric feature includes an absolute value of the difference between the two sets of opposite sides δ _Q =| θ _QH - θ _QV |, thereby obtaining corresponding data of the N sets of rectangular compensation factors k _Q , the first geometric features and the second geometric features.

In step 2023, the first geometric feature of the rectangle is compared to a classification threshold to classify the N sets of corresponding data into a type corresponding to the quadrilateral type. In one example, quadrilaterals can be divided into the following three categories based on distance: near, medium, and far. In other examples, the quadrilateral can be divided into two types: near and far, or divided into five categories: near, medium, medium, medium, and long distance. It should be noted that the above classification should not be a limitation on the present application as long as the classification can describe the distance between the object and the device.

In the process of classifying the N sets of corresponding data, as an example, the side length ratio D _Q and the distance classification threshold T of the rectangle may be compared first, and the N sets of corresponding data are classified into a category corresponding to the quadrilateral distance type. The data is then compared with the two sets of opposite side angles θ _QH and θ _{QV of the} rectangle and the angle classification threshold θ _T , thereby further classifying each large class of data into small class data corresponding to the quadrilateral tilt type. As another example, it is also possible to compare the two sets of opposite side angles θ _QH and θ _{QV of the} rectangle with the angle classification threshold θ _T , and divide the N sets of corresponding data into the large class data corresponding to the quadrilateral tilt type; and then compare The side length of the rectangle occupies the ratio D _Q and the distance classification threshold T, and further divides each large class of data into small class data corresponding to the quadrilateral distance type.

In the above process, the determination of the angle classification threshold θ _T and the classification according to the tilt posture can refer to the description of the step 2023 described above, and details are not described herein again. The distance classification threshold T can be used to classify the N sets of corresponding data into corresponding quadrilateral distance types according to a certain ratio or substantially uniformly, and classify the corrected quadrilaterals to select a corresponding compensation formula. In one example, the distance classification threshold T may be a statistical quantile of the square side length ratio D _Q . For example, the distance classification threshold T may be a 1/3 quantile of the side length ratio D _Q (ie, all values are determined by The value of the 1/3 position after the small to large arrangement and the 2/3 quantile (that is, the value of all the values from the small to the large and the 2/3 position).

In another example, the distance classification threshold T may also be the mean of the rectangular side length ratio D _Q .

In another example, the distance classification threshold T may also be based on empirical values, such as 0.5 and 0.8.

As an example, as shown in FIG. 7, when the distance classification threshold T is a set of numbers 0.5 and 0.8, the quadrilateral can be classified into three types: far, medium, and close. Wherein, the side length ratio of the close-up quadrilateral shown in part (A) of FIG. 7 is h/H>0.8; the side length of the medium-distance quadrilateral shown in part (B) of FIG. 7 is 0.5<h/ H<0.8; the side length ratio of the distance-like quadrilateral shown in part (C) of Fig. 7 is h/H < 0.5. The N sets of corresponding data are divided into three categories of data corresponding to the three kinds of distances, and then each of the large types of data is divided into four sub-classes of data corresponding to the four tilting postures, as shown in FIG. data.

In the embodiment of the present invention, by classifying scenes with different distances and angles, the types of image acquisition scenes are further refined, so that the compensation formula corresponding to each type of image acquisition scene can more accurately reflect the geometric features and compensation of the quadrilateral. The relationship of the factors, so the corrected image aspect ratio is closer to the actual aspect ratio of the subject, thereby improving the effect of image correction in each type of scene.

Another embodiment of the present invention provides an image correction method. The method will be described below with reference to FIGS. 2 and 9. The flowchart of the image correction method is as shown in FIG. 2, wherein steps 203 to 205 of the method are the same as steps 203 to 205 described above, and details are not described herein again. Hereinafter, the differences between steps 201 and 202 and steps 201 and 202 described above will be described.

In step 201, the geometric feature of the quadrilateral may be the difference between the two sets of opposite sides of the quadrilateral. In one example, the angle difference between the two sides of the edge may be a horizontal angle θ _H of the vertical sides of the angle difference δ _HV of θ _V = θ _H -θ _V. The magnitude of the difference δ _HV can be used to determine the degree of tilt of the image as a whole, and the sign of the difference δ _HV is used to indicate the tilt direction (ie, horizontal or vertical tilt). When δ _HV >0, the image is tilted as a whole. When δ _HV <0, the image as a whole is vertically inclined.

In another example, the difference between the two sets of opposite sides may also be the difference δ _VH = θ _{V -} θ _{H between the} angle between the vertical and the opposite sides. When δ _VH > 0, the rectangular image is vertically inclined as a whole, and when δ _VH < 0, the rectangular image as a whole is horizontally inclined.

In other examples, the geometric feature of the quadrilateral may also be a trigonometric function value of the difference between the two sets of angles of the opposite sides, or a difference between the two sets of trigonometric values of the angles of the opposite sides. These geometric features can indicate the degree of tilt and tilt of the image as a whole. Oblique direction.

For the sake of clarity and convenience, when referring to the geometric features of the quadrilateral, the embodiment of the present invention refers to the difference δ _HV between the two sets of opposite sides of the quadrilateral unless otherwise stated. When the method according to the embodiment of the present invention is applied to an embodiment in which the geometric feature includes a trigonometric function value, the difference between the two sets of trigonometric values of the difference between the sides of the edges or the difference between the two sets of trigonometric values of the opposite sides can be used. The difference between the corresponding two sets of opposite sides.

In step 202, the geometric feature of the quadrilateral is substituted into the compensation formula, and the compensation factor k of the quadrilateral can be calculated. The compensation formula may reflect the relationship of the geometric feature to the compensation factor k in the same form as described in step 202 above. Since the difference between the two sets of opposite sides of the quadrilateral can indicate the degree of tilt and the tilt direction of the image as a whole, the quadrilateral can be classified without using a compensation formula to represent the relationship between the geometrical features of the quadrilateral and the compensation factor k.

Another method for determining a compensation formula according to an embodiment of the present invention is described below with reference to FIG. 9, which may be performed by the device 101, including:

Step 2021', obtaining N images of rectangles of known aspect ratio at N different angles.

Step 2022': Obtain a first aspect ratio R _Q and a geometric feature of the rectangle from each of the N images, and obtain corresponding data of the compensation factor k _{Q of the} N sets of rectangles and the geometric features.

Step 2023', fitting the compensation formula according to the N sets of compensation factors k _Q and the corresponding data of the geometric features.

Since step 2021' is similar to the previous step 2021, it will not be described again here. Step 2022' and step 2023' will be specifically described below.

In step 2022', since the geometric features of the rectangle correspond to the geometric features of the quadrilateral, when the geometric features of the quadrilateral include the difference between the two sets of opposite sides, the geometrical features of the rectangle also include the difference between the two sets of opposite sides. As an example, when the geometric feature of the quadrilateral includes the difference δ _HV between the two sets of opposite angles θ _H and θ _V , the geometrical features of the rectangle also include the difference δ _{QHV between the} two sets of opposite angles θ _QH and θ _QV . =θ _QH -θ _QV . Thereby, corresponding data of the compensation factor k _{Q of the} N sets of rectangles and the geometrical feature δ _HV are obtained.

In step 2023', since the quadrilateral is unclassified, the N sets of corresponding data may not be classified, and the compensation formula may be directly fitted according to the difference δ _QHV between the two sets of opposite sides of the _rectangle and the compensation factor k _Q . In one example, Table 2 shows the corresponding data of the difference δ _QHV between the two sets of opposite sides of the rectangle and the compensation factor k _Q , whereby the compensation formula can be fitted. As an example, the compensation formula is a quadratic function, ie: k=a*x ² +b*x+c, where x represents a geometric feature, k represents a compensation factor, and parameters a, b, and c pass through Table 2. Data fitting was determined, where a = 0.0001, b = 0.0165, and c = 1.0.

Those skilled in the art will appreciate that it is also possible to select one, three or higher order functions for the compensation template as long as the results that meet the accuracy requirements can be obtained by the fitting algorithm. The above-mentioned fitting algorithm can adopt a known algorithm and will not be described here.

Table 2 Corresponding data table of the difference δ _QHV and compensation factor k _Q between the two groups

In the embodiment of the present invention, by selecting geometric features capable of representing the overall tilt of the quadrilateral, the type of the image acquisition scene can be reduced, and the compensation formula can still accurately reflect the relationship between the geometric features of the quadrilateral and the compensation factor, thereby simplifying the image. The step of correcting improves the processing speed of image correction.

FIG. 10 is a schematic structural diagram of a device according to an embodiment of the present invention. As shown in FIG. 10, the device 1000 includes: The first obtaining module 1001, the second obtaining module 1002, the first calculating module 1003, and the second calculating module 1004 and the correcting module 1005. The first obtaining module 1001 is configured to acquire a first aspect ratio of a quadrilateral in the image to be corrected and a geometric feature of the quadrilateral. The second obtaining module 1002 is configured to obtain a compensation factor of the quadrilateral according to the geometric feature of the quadrilateral. The first calculating module 1003 is configured to calculate a second aspect ratio of the quadrilateral according to a first aspect ratio of the quadrilateral and a compensation factor of the quadrilateral. The second calculating module 1004 is configured to calculate a transformation matrix according to the second aspect ratio of the quadrilateral. The correction module 1005 is configured to correct the image to be corrected according to the transformation matrix.

FIG. 11 is a schematic structural diagram of a second acquiring module according to an embodiment of the present invention. As shown in FIG. 11 , the second acquiring module 1002 further includes: a first acquiring unit 10021, a second acquiring unit 10022, a classifying unit 10023, and Fitting unit 10024. The first obtaining unit 10021 is configured to acquire N images of a rectangle of a known aspect ratio at N different angles, where N is an integer not less than 2; and the second obtaining unit 10022 is configured to acquire the rectangle The aspect ratio and the geometric features of the rectangle obtain corresponding data of the compensation factor of the N sets of the rectangle and the geometric feature of the rectangle; the classifying unit 10023 is configured to divide the N sets of corresponding data into multiple types; The merging unit 10024 is configured to respectively fit the compensation formula according to each of the plurality of types of data.

FIG. 12 is a schematic structural diagram of another second acquiring module according to an embodiment of the present invention. As shown in FIG. 12, the second acquiring module 1002 further includes: a first acquiring unit 10021', a second acquiring unit 10022', and a Unit 10023'. The first obtaining unit 10021 ′ is configured to acquire N images of a rectangle of a known aspect ratio at N different angles, where N is an integer not less than 2; and the second obtaining unit 10022 is configured to acquire the rectangle. The aspect ratio and the geometric features of the rectangle, obtaining corresponding data of the compensation factor of the N sets of the rectangle and the geometric feature of the rectangle; the fitting unit 10023' is configured to fit the compensation according to the N sets of corresponding data formula.

The device provided by the embodiment of the present invention is used to implement the foregoing method in the embodiment shown in FIG. 1 to FIG. 9. For the specific process or principle, refer to the foregoing method embodiment.

FIG. 13 is a structural diagram of another apparatus according to an embodiment of the present invention. As shown in FIG. 13, the apparatus 1300 includes: a processor 1301, a camera 1302, a user input device 1303, a display 1304, a memory 1305, and a Or multiple programs. For the convenience of description, FIG. 13 only shows a part related to the embodiment of the present invention. If the specific technical details are not disclosed, please refer to the above method embodiment and other parts of the application file of the present application. Those skilled in the art should know that the number of hardware such as the processor 1301, the camera 1302, the user input device 1303, the display 1304 and the memory 1305 may be one or more, depending on the type of the device 1300.

The processor 1301 is coupled to the camera 1302, the user input device 1303, the display 1304, and the memory 1305 via one or more buses. The processor 1301 receives the image from the camera 1302 and the user command from the user input device 1303, calls the execution instruction stored in the memory 1305 for processing, and sends it to the display 1304 for presentation. The processor 1301 may be a central processing unit (CPU), a general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), and a field programmable gate array ( Field Programmable Gate Array (FPGA) or other programmable logic device, transistor logic device, hardware component, or any combination thereof. The processor 1301 can implement or perform various exemplary logical blocks, modules, and circuits described in connection with the present disclosure. The processor 1301 may also be a combination of computing functions, such as one or more microprocessor combinations, a combination of a DSP and a microprocessor, and the like. As an example, the processor 1301 is configured to support the device 1300 to perform step 202 in FIG. 2 . Up to 204, steps 2022 through 2024 in Fig. 4, and steps 2022' and 2023' in Fig. 9.

The camera 1302 transmits the acquired image to the processor 1301. The camera 1302 can also be a digital camera (also known as a digital camera, digital camera), a scanner, or other optical device capable of acquiring images. The acquired image is distinguished from the perspective of the content, and may be an image, a document, or a mixed image containing both the image and the document. As an example, the camera 1302 is used to support the device to perform step 201 in Fig. 2, step 2021 in Fig. 4, and step 2021' in Fig. 9.

The user input device 1302 receives the user's operation instructions and transmits them to the processor 1301 for processing. The user input device 1302 includes a touch screen, a button, or a microphone.

Display 1304 presents a preview image, a corrected image, and a human-computer interaction interface. Display 1304 can be a display built into device 1300 or other external display device externally connected to device 1300.

The memory 1305 stores images, preset models, and one or more programs. Memory 1305 can be any computer readable storage medium. The one or more programs are stored in a memory and configured to be executed by one or more processors, including both execution instructions of the device operating system and execution instructions of the device application, such as correcting images in embodiments of the present invention. Execution instructions.

In the above embodiments of the present invention, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, it may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in accordance with embodiments of the present invention are generated in whole or in part. The computer can be a general purpose computer, a special purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer readable storage medium or transferred from one computer readable medium to another computer readable medium, for example, the computer instructions can be wired from a website site, computer, server or data center (for example, coaxial cable, fiber, Digital Subscriber Line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) to another website, computer, server or data center. The computer readable storage medium can be any available media that can be accessed by a computer or a data storage device such as a server, data center, or the like that includes one or more available media. The usable medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a DVD), or a semiconductor medium (eg, a Solid State Disk (SSD)) or the like.

Those skilled in the art will appreciate that in one or more examples described above, the functions described herein can be implemented in hardware, software, firmware, or any combination thereof. When implemented in software, the functions may be stored in a computer readable medium or transmitted as one or more instructions or code on a computer readable medium. Computer readable media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one location to another. A storage medium may be any available media that can be accessed by a general purpose or special purpose computer.

The specific embodiments of the present invention have been described in detail with reference to the preferred embodiments of the present invention. The scope of protection of the invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and scope of the present invention are intended to be included within the scope of the present invention.

Claims (23)

1. An image correction method, the method comprising:

   Obtaining a first aspect ratio of a quadrilateral of the image to be corrected and a geometric feature of the quadrilateral;

   Acquiring a compensation factor of the quadrilateral according to geometric features of the quadrilateral;

   Calculating a second aspect ratio of the quadrilateral according to a first aspect ratio of the quadrilateral and a compensation factor of the quadrilateral;

   Calculating a transformation matrix according to a second aspect ratio of the quadrilateral;

   The image to be corrected is corrected according to the transformation matrix.
2. The image correction method according to claim 1, wherein

   Obtaining the quadrilateral compensation factor according to the geometric feature of the quadrilateral comprises: acquiring a compensation factor of the quadrilateral according to the geometric feature of the quadrilateral and a compensation formula;

   The method for determining the compensation formula includes:

   Obtaining N images of rectangles of known aspect ratio at N different angles, where N is an integer not less than 2;

   Obtaining, from each of the N images, a first aspect ratio of the rectangle and a geometric feature of the rectangle, to obtain corresponding data of the compensation factor of the N sets of the rectangle and the geometric feature of the rectangle;

   A compensation formula is fitted according to the N sets of corresponding data.
3. The image correction method according to claim 2, wherein the fitting the compensation formula according to the N sets of corresponding data comprises: dividing the N sets of corresponding data into a plurality of types; according to the plurality of types Each type of data is fitted to the compensation formula.
4. The image correction method according to claim 2 or 3, wherein the compensation formula is predetermined before the image is corrected.
5. The image correction method according to any one of claims 1 to 4, wherein the geometric feature comprises at least one of a distance feature or an angle feature.
6. The image correction method according to claim 5, wherein the angle feature comprises:

   a difference between the two sets of opposite sides of the quadrilateral;

   Or a trigonometric function value of the difference between the two sets of sides of the quadrilateral;

   Alternatively, the difference between the values of the trigonometric functions of the two sets of sides of the quadrilateral.
7. The image correction method according to any one of claims 1 to 4, wherein the geometric feature comprises a first geometric feature and a second geometric feature.

   The first geometric feature includes at least one of the following features:

   a ratio of a length of a longest side of the quadrilateral to a side length of the image to be corrected;

   Or the ratio of the diagonal length of the quadrilateral to the diagonal length of the image to be corrected;

   Or the size of the angle between the two sets of sides of the quadrilateral;

   Or a trigonometric function value of the two sets of opposite sides of the quadrilateral;

   The second geometric feature includes:

   a difference between the two sets of opposite sides of the quadrilateral;

   Or a trigonometric function value of the difference between the two sets of sides of the quadrilateral;

   Or the difference between the values of the trigonometric functions of the two sets of opposite sides of the quadrilateral;

   Or the absolute value of the difference between the two sets of sides of the quadrilateral;

   Or a trigonometric function value of the absolute value of the difference between the two sets of sides of the quadrilateral;

   Alternatively, the absolute value of the difference between the values of the trigonometric functions of the two sets of sides of the quadrilateral.
8. The image correction method according to any one of claims 1 to 7, wherein the first aspect ratio of the quadrilateral or the rectangle is a ratio of a width estimation value to a height estimation value, and the width estimation value is a level The length of the median line with a small angle of direction is the length of the median line that is smaller than the angle between the vertical directions.
9. The image correction method according to any one of claims 1 to 7, wherein the first aspect ratio of the quadrilateral or the rectangle is a ratio of a width estimation value to a height estimation value, and the width estimation value is a level The length of the long side of a pair of opposite sides having a smaller angle of the direction, the height estimate being the length of the long side of a pair of opposite sides having a smaller angle with the vertical direction.
10. The image correction method according to any one of claims 1 to 9, characterized in that the function form of the compensation formula is a one-time function, a quadratic function or a higher-order function form.
11. A device, characterized in that the device comprises:

    a first acquiring module, configured to acquire a first aspect ratio of a quadrilateral in the image to be corrected and a geometric feature of the quadrilateral;

    a second acquiring module, configured to acquire a compensation factor of the quadrilateral according to the geometric feature of the quadrilateral;

    a first calculating module, configured to calculate a second aspect ratio of the quadrilateral according to a first aspect ratio of the quadrilateral and a compensation factor of the quadrilateral;

    a second calculating module, configured to calculate a transformation matrix according to a second aspect ratio of the quadrilateral;

    And a correction module, configured to correct the image to be corrected according to the transformation matrix.
12. The apparatus according to claim 11, wherein the second obtaining module is configured to obtain a compensation factor of the quadrilateral according to a geometric feature of the quadrilateral and a compensation formula;

    The second obtaining module includes:

    a first acquiring unit, configured to acquire N images of a rectangle of a known aspect ratio at N different angles, where N is an integer not less than 2;

    a second acquiring unit, configured to acquire an aspect ratio of the rectangle and a geometric feature of the rectangle, to obtain corresponding data of a compensation factor of the N sets of the rectangle and a geometric feature of the rectangle;

    And a fitting unit, configured to fit a compensation formula according to the N sets of corresponding data.
13. The apparatus according to claim 12, wherein the fitting unit further comprises:

    a classification sub-module, configured to divide the N sets of corresponding data into multiple types;

    A fitting submodule is configured to respectively fit the compensation formula according to each of the plurality of types of data.
14. Apparatus according to claim 12 or claim 13 wherein the compensation formula is predetermined prior to correcting the image.
15. Apparatus according to any of claims 11-14, wherein the geometric feature comprises at least one of a distance feature or an angular feature.
16. The apparatus of claim 15 wherein said angular features comprise:

    a difference between the two sets of opposite sides of the quadrilateral;

    Or a trigonometric function value of the difference between the two sets of sides of the quadrilateral;

    Alternatively, the difference between the values of the trigonometric functions of the two sets of sides of the quadrilateral.
17. Apparatus according to any of claims 11-14, wherein the geometric features comprise a first geometric feature and a second geometric feature;

    The first geometric feature includes at least one of the following features:

    a ratio of a length of a longest side of the quadrilateral to a side length of the image to be corrected;

    Or the ratio of the diagonal length of the quadrilateral to the diagonal length of the image to be corrected;

    Or the size of the angle between the two sets of sides of the quadrilateral;

    Or a trigonometric function value of the two sets of opposite sides of the quadrilateral;

    The second geometric feature includes:

    a difference between the two sets of opposite sides of the quadrilateral;

    Or a trigonometric function value of the difference between the two sets of sides of the quadrilateral;

    Or the difference between the values of the trigonometric functions of the two sets of opposite sides of the quadrilateral;

    Or the absolute value of the difference between the two sets of sides of the quadrilateral;

    Or a trigonometric function value of the absolute value of the difference between the two sets of sides of the quadrilateral;

    Alternatively, the absolute value of the difference between the values of the trigonometric functions of the two sets of sides of the quadrilateral.
18. The apparatus according to any one of claims 11-17, wherein the first aspect ratio of the quadrilateral or rectangle is a ratio of a width estimate to a height estimate, the width estimate being clipped with a horizontal direction The length of the median line having a smaller angle, which is the length of the median line that is smaller than the angle in the vertical direction.
19. The apparatus according to any one of claims 11-17, wherein the first aspect ratio of the quadrilateral or rectangle is a ratio of a width estimate to a height estimate, the width estimate being clipped with a horizontal direction The length of the long side of a pair of opposite sides of a smaller angle, the height estimate being the length of the long side of a set of opposite sides that are smaller than the vertical.
20. Apparatus according to any one of claims 11-19, characterized in that the functional form of the compensation formula is in the form of a linear function, a quadratic function or a higher order function.
21. An apparatus, comprising: a display, one or more processors, a memory, and one or more programs, the one or more programs being stored in the memory and configured to be configured by one or more The processor executes, the one or more programs comprising instructions for performing the method of any of claims 1-10.
22. A computer readable storage medium comprising instructions which, when run on a computer, cause the computer to perform the method of any of claims 1-10.
23. A computer program product comprising instructions which, when run on a computer, cause the computer to perform the method of any of claims 1-10.

Images