[go: up one dir, main page]

WO2018152710A1 - Procédé et dispositif de correction d'image - Google Patents

Procédé et dispositif de correction d'image Download PDF

Info

Publication number
WO2018152710A1
WO2018152710A1 PCT/CN2017/074432 CN2017074432W WO2018152710A1 WO 2018152710 A1 WO2018152710 A1 WO 2018152710A1 CN 2017074432 W CN2017074432 W CN 2017074432W WO 2018152710 A1 WO2018152710 A1 WO 2018152710A1
Authority
WO
WIPO (PCT)
Prior art keywords
quadrilateral
sets
aspect ratio
image
difference
Prior art date
Application number
PCT/CN2017/074432
Other languages
English (en)
Chinese (zh)
Inventor
秦超
郜文美
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Priority to PCT/CN2017/074432 priority Critical patent/WO2018152710A1/fr
Priority to CN201780005563.1A priority patent/CN108780572B/zh
Publication of WO2018152710A1 publication Critical patent/WO2018152710A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/80—Geometric correction
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis

Definitions

  • the present application relates to the field of image processing, and in particular, to a method and apparatus for image correction.
  • the smart terminal When the smart terminal acquires an image of the subject from the front side, an image that substantially matches the ratio of the subject can be obtained, thereby allowing the user to view or perform Optical Character Recognition (OCR).
  • OCR Optical Character Recognition
  • the image may have a perspective distortion, which will affect the review or recognition, so it is necessary to correct the distortion image.
  • the present application describes a method and apparatus for image correction that can be obtained by increasing the accuracy of the original image aspect ratio estimation.
  • an image correction method which acquires a first aspect ratio of a quadrilateral of an image to be corrected and a geometric feature of the quadrilateral; and acquires a compensation factor of the quadrilateral according to the geometric feature of the quadrilateral; Computing a first aspect ratio of the quadrilateral and a compensation factor of the quadrilateral, calculating a second aspect ratio of the quadrilateral; calculating a transformation matrix according to a second aspect ratio of the quadrilateral; correcting the method according to the transformation matrix The image to be corrected.
  • the accuracy of the aspect ratio of the rectangle in the corrected image can be improved as a whole.
  • the step of acquiring the quadrilateral compensation factor according to the geometric features of the quadrilateral comprises obtaining a compensation factor of the quadrilateral according to the geometric features of the quadrilateral and a compensation formula.
  • the determining method of the compensation formula includes: acquiring N images of a rectangle of a known aspect ratio at N different angles, wherein N is an integer not less than 2; from each of the N images, Obtaining a first aspect ratio of the rectangle and a geometric feature of the rectangle, obtaining corresponding data of the compensation factor of the N sets of the rectangle and the geometric feature of the rectangle; and fitting a compensation formula according to the N sets of corresponding data.
  • the step of fitting a compensation formula according to the N sets of corresponding data includes dividing the N sets of corresponding data into a plurality of types corresponding to different geometric features, according to each of the plurality of types of data Fit the compensation formula separately. In this way, multiple shooting scenes can be combined to refine the scene classification, and each scene uses a compensation formula to make the compensation result more accurate.
  • the compensation formula is predetermined before the image is corrected. Thereby, the process of correcting the image can be simplified.
  • the geometric feature comprises at least one of a distance feature or an angular feature.
  • Different image acquisition scenarios can be determined by different types of geometric features.
  • the angular feature comprises: a difference between two sets of opposite sides of the quadrilateral; or The trigonometric function value of the difference between the two sets of the sides of the quadrilateral; or the difference between the two sets of the trigonometric angles of the quadrilateral.
  • the geometric features include a first geometric feature and a second geometric feature.
  • the first geometric feature includes at least one of the following: a ratio of a length of a longest side of the quadrilateral to a side length of the image to be corrected, or a diagonal length of the quadrilateral and the to-be-corrected The ratio of the diagonal length of the image, or the angle between the two sets of opposite sides of the quadrilateral, or the trigonometric value of the set of opposite sides of the quadrilateral.
  • the second geometric feature includes: a difference between two sets of opposite sides of the quadrilateral, or a trigonometric function of a difference between two sets of opposite sides of the quadrilateral, or two sets of opposite sides of the quadrilateral.
  • a set of features can be obtained that effectively represent the relative size of the quadrilateral, the relative distance from the lens, and the degree of quadrilateral tilt.
  • Different types of geometric features can be used to determine different shooting scenes and to obtain compensation formulas suitable for different shooting scenes.
  • the first aspect ratio of the quadrilateral or rectangle is a ratio of a width estimate to a height estimate
  • the width estimate is a median line length that is smaller than an angle in the horizontal direction
  • the height estimate is the length of the median line that is less angled to the vertical.
  • the first aspect ratio of the quadrilateral or rectangle is a ratio of a width estimate to a height estimate, the width estimate being a length of a pair of opposite sides that are smaller than the horizontal direction.
  • the length of the edge is the length of the long side of a pair of opposite sides that is smaller than the vertical direction.
  • the functional form of the compensation formula includes a first function, a quadratic function, or a higher order function form.
  • the general form of the compensation formula can be determined, and the compensation formula can be obtained by calculating the parameters of the function.
  • an apparatus including: a first acquiring module, configured to acquire a first aspect ratio of a quadrilateral in an image to be corrected and a geometric feature of the quadrilateral; and a second acquiring module, configured to Calculating a compensation factor of the quadrilateral according to a geometric feature of the quadrilateral; a first calculating module, configured to calculate a second aspect ratio of the quadrilateral according to a first aspect ratio of the quadrilateral and a compensation factor of the quadrilateral; a calculation module, configured to calculate a transformation matrix according to a second aspect ratio of the quadrilateral; and a correction module, configured to correct the image to be corrected according to the transformation matrix.
  • the second obtaining module is configured to obtain a compensation factor of the quadrilateral according to the geometric features and the compensation formula of the quadrilateral.
  • the second obtaining module includes: a first acquiring unit, configured to acquire N images of a rectangle of a known aspect ratio at N different angles, where N is an integer not less than 2; and the second acquiring unit obtains The aspect ratio of the rectangle and the geometric feature of the rectangle obtain corresponding data of the compensation factor of the N sets of the rectangle and the geometric feature of the rectangle; the fitting unit is configured to fit according to the N sets of corresponding data Compensation formula.
  • the fitting unit further includes: a classification sub-module for dividing the N sets of corresponding data into a plurality of types; and a fitting sub-module for each of the plurality of types A type of data is fitted to the compensation formula.
  • the compensation formula is predetermined before the image is corrected. Thereby, the process of correcting the image can be simplified.
  • the geometric feature comprises at least one of a distance feature or an angular feature.
  • Different image acquisition scenarios can be determined by different types of geometric features.
  • the angular feature comprises: a difference between two sets of opposite sides of the quadrilateral; or a trigonometric value of a difference between two sets of opposite sides of the quadrilateral; or the quadrilateral The difference between the two sets of trigonometric values of the angle between the two sides. In this way, a set of features can be obtained that effectively represent the degree of tilt of the quadrilateral.
  • the geometric features include a first geometric feature and a second geometric feature.
  • the first geometric feature includes at least one of the following: a ratio of a length of a longest side of the quadrilateral to a side length of the image to be corrected, or a diagonal length of the quadrilateral and the to-be-corrected The ratio of the diagonal length of the image, or the angle between the two sets of opposite sides of the quadrilateral, or the trigonometric value of the set of opposite sides of the quadrilateral.
  • the second geometric feature includes: a difference between two sets of opposite sides of the quadrilateral, or a trigonometric function of a difference between two sets of opposite sides of the quadrilateral, or two sets of opposite sides of the quadrilateral.
  • a set of features can be obtained that effectively represent the relative size of the quadrilateral, the relative distance from the lens, and the degree of quadrilateral tilt.
  • Different types of geometric features can be used to determine different shooting scenes and to obtain compensation formulas suitable for different shooting scenes.
  • the first aspect ratio of the quadrilateral or rectangle is a ratio of a width estimate to a height estimate
  • the width estimate is a median line length that is smaller than an angle in the horizontal direction
  • the height estimate is the length of the median line that is less angled to the vertical.
  • the first aspect ratio of the quadrilateral or rectangle is a ratio of a width estimate to a height estimate, the width estimate being a length of a pair of opposite sides that are smaller than the horizontal direction.
  • the length of the edge is the length of the long side of a pair of opposite sides that is smaller than the vertical direction.
  • the functional form of the compensation formula includes a first function, a quadratic function, or a higher order function form.
  • the general form of the compensation formula can be determined, and the compensation formula can be obtained by calculating the parameters of the function.
  • an apparatus comprising: one or more processors, a memory, and one or more programs, the one or more programs being stored in a memory and configured to be configured by one or more Executed by the processor, the one or more programs include instructions for performing the method of the first aspect.
  • a computer program product comprising instructions for causing a computer to perform the method of the first aspect when the instructions are run on a computer.
  • a computer readable storage medium having stored therein instructions that, when executed on a computer, cause the computer to perform the method of the first aspect.
  • FIG. 1 is a schematic diagram of an image correction scenario according to an embodiment of the present invention
  • FIG. 2 is a flowchart of an image correction method according to an embodiment of the present invention.
  • FIG. 3 is a schematic diagram of a first aspect ratio according to an embodiment of the present invention.
  • FIG. 4 is a flowchart of a method for determining a compensation formula according to an embodiment of the present invention
  • FIG. 5 is a schematic diagram of acquiring N images of a rectangle having a known aspect ratio according to an embodiment of the present invention
  • FIG. 6 is a schematic diagram of classification according to an image tilt posture according to an embodiment of the present invention.
  • FIG. 7 is a schematic diagram of classification according to image distance according to an embodiment of the present invention.
  • FIG. 8 is a schematic diagram of classification according to an image tilt posture and distance according to an embodiment of the present invention.
  • FIG. 9 is a flowchart of another method for determining a compensation formula according to an embodiment of the present invention.
  • FIG. 10 is a schematic structural diagram of a device according to an embodiment of the present invention.
  • FIG. 11 is a schematic structural diagram of a second acquiring module according to an embodiment of the present invention.
  • FIG. 12 is a schematic structural diagram of another second acquiring module according to an embodiment of the present invention.
  • FIG. 13 is a schematic structural diagram of another device according to an embodiment of the present invention.
  • FIG. 1 is a schematic diagram of an image acquisition scenario according to an embodiment of the present invention.
  • the subject 102 is placed on the support structure 103, and the device 101 takes an image of the subject 102.
  • the subject 102 can be any object containing text and/or images, such as documents, pictures, business cards, documents, books, slides, whiteboards, street signs, and advertising signs.
  • the device 101 can be any device capable of acquiring an image or processing an image, for example, a mobile phone (or "mobile phone"), a tablet (Tablet Personal Computer, TPC), a computer, a digital camera, a wearable device (Wearable Device) ), virtual reality devices, digital broadcast terminals, messaging devices, game consoles, medical devices, fitness equipment, personal digital assistants (PDAs), e-book readers, and scanners .
  • the device 101 can acquire images through optical components such as cameras, cameras or optical sensors, which can be built into the device 101 or externally connected to the device 101. For the sake of clarity and convenience, the optical components of the present application are uniformly described using a camera.
  • part (A) of Fig. 1 when the device 101 is photographed from the front side, an image 104 substantially in proportion to the subject 102 can be obtained.
  • the see-through effect causes the acquired image 104' to be distorted, so the distortion image 104' needs to be corrected.
  • the image correction process First, the image 104' is input, then the quadrilateral in the image is detected, then the aspect ratio of the object 102 is estimated according to the quadrilateral, and the transformation matrix is calculated according to the estimated aspect ratio, and finally the distortion image 104' is corrected by using the transformation matrix. It is a rectangular image 104".
  • the above image correction process uses a complex formula for estimating the aspect ratio, which is based on the assumed lens focal length parameter. In practical applications, since the lens focal length is different from the assumed lens focal length, the pair is shot.
  • the aspect ratio estimation error of the object is large, especially when the tilt angle of the device 101 is large, the deviation between the estimated value of the aspect ratio and the actual value is larger.
  • Figure 2 shows a flow chart of the method, which may be performed by device 101, the method comprising:
  • Step 201 Acquire a first aspect ratio R 0 of a quadrilateral of the image to be corrected and a geometric feature of the quadrilateral;
  • Step 202 obtaining a quadrilateral compensation factor k according to the geometric feature of the quadrilateral
  • Step 203 calculating a second aspect ratio R 1 of the quadrilateral according to the first aspect ratio R 0 of the quadrilateral and the compensation factor k of the quadrilateral;
  • Step 204 calculating a transformation matrix K according to a second aspect ratio R 1 of the quadrilateral
  • Step 205 correcting the image to be corrected according to the transformation matrix K.
  • an image to be corrected of the subject is acquired.
  • the image to be corrected may be an image acquired by the camera in real time, such as a frame for capturing a subject or capturing a preview video of the subject, or an existing image, such as an image stored on a memory.
  • the object may be presented as a rectangle as a whole, such as a document, a picture, a business card, a document, a book, a slide, a whiteboard, a street sign, and an advertisement sign mentioned above, or may be a partial area including a rectangle, for example, in a subject. Contains a rectangular image or text block.
  • device 101 detects a quadrilateral edge to extract a quadrilateral in the image.
  • the detection algorithm of the quadrilateral edge can adopt a known method, and will not be described here.
  • the user can manually select a quadrilateral in the image, for example, the user drags the four corners of the quadrilateral box in the image, selects the quadrilateral in the image, and extracts the quadrilateral in the image.
  • the four corner coordinates of the quadrilateral can be obtained according to the extracted quadrilateral, thereby calculating the first aspect ratio R 0 of the quadrilateral and the geometric features of the quadrilateral.
  • the first aspect ratio R 0 may be a ratio of the lengths of the adjacent sides of the quadrilateral. As shown in part (A) of Fig. 3, from the two sets of opposite sides of the quadrilateral, the horizontal opposite side and the vertical opposite side are respectively determined, and the length l CD of the longer side of the horizontal opposite side is taken as the width estimation value W.
  • the step of determining the horizontal opposite side and the vertical opposite side comprises: comparing the angle between any two adjacent sides of the quadrilateral and the horizontal direction, the side with the smaller angle and the opposite side being the horizontal opposite side, and the remaining A set of opposite sides is a vertical opposite side.
  • the angle between any two adjacent edges and the vertical direction may be compared, and the smaller side and the opposite side are vertical opposite sides, and the remaining set of opposite sides are horizontal opposite sides.
  • the geometric features of the quadrilateral include angular features.
  • the smaller angle of the opposite side of the two sets of opposite sides can be used to determine the degree of inclination of the opposite side with less distortion of the quadrilateral: the larger the angle of the opposite side, the greater the inclination; the opposite side The smaller the angle, the smaller the tilt.
  • the absolute value ⁇ of the difference between the two sets of opposite sides can be used to determine the degree of deformation of the image as a whole: the larger the absolute value ⁇ , the greater the degree of deformation; the smaller the absolute value ⁇ , the smaller the degree of deformation.
  • the angular feature may include a trigonometric function value of two sets of opposite sides of the quadrilateral, and a trigonometric function value of the absolute value of the difference between the two sets of angles of the opposite sides or a trigonometric function value of the two sets of opposite sides The absolute value of the difference.
  • the trigonometric value includes a sine, a tangent, a cosine, or a cotangent.
  • the trigonometric function values of the two sets of angles of the opposite sides can be used to determine the degree of inclination of the opposite sides of the quadrilateral distortion, the trigonometric function values of the absolute values of the difference between the two sets of sides, or the two sets of opposite side clips
  • the absolute value of the difference between the angular trigonometric values can be used to determine the degree of deformation of the image as a whole.
  • the embodiment of the present invention refers to the absolute angle between the two sets of opposite sides of the quadrilateral ⁇ H and ⁇ V and the difference between the two sets of opposite sides.
  • the value ⁇ When the method according to the embodiment of the present invention is applied to an embodiment in which the geometric feature includes a trigonometric function value, the two sets of opposite angles may be replaced by two sets of trigonometric values of the opposite sides, and two pairs of pairs may be used.
  • the absolute value of the difference between the absolute value of the difference between the edges and the value of the trigonometric function of the two sets of angles replaces the absolute value of the difference between the two sets of opposite sides.
  • the length of each side of the quadrilateral, the length of the median line, the angle of the opposite side, and the value of the trigonometric function can be obtained by calculating the coordinates of the four corner points of the quadrilateral, which belongs to the conventional geometric knowledge in the field, and is not here. Let me repeat.
  • step 202 according to the geometric features of the quadrilateral and the compensation formula, the geometric features of the quadrilateral are substituted into the compensation formula, and the compensation factor of the quadrilateral can be obtained.
  • the functional form of the compensation formula can be a one-time function, a quadratic function, or a higher-order function form.
  • the compensation formula may be determined in advance before the image correction method of the embodiment of the present invention is implemented, or may be determined in the process of implementing the image correction method of the embodiment of the present invention.
  • the quadrilateral can be classified according to the characteristics of the quadrilateral tilt posture, and a compensation formula is determined for each type of quadrilateral.
  • a flowchart of a method for determining the compensation formula is shown in FIG. 4, and the method may be performed by the device 101, including:
  • Step 2021 Acquire N images of rectangles of known aspect ratio at N different angles
  • Step 2022 Obtain a first aspect ratio R Q and a geometric feature of the rectangle from each of the N images, and obtain corresponding data of the compensation factor k Q of the N sets of rectangles and the geometric feature;
  • Step 2023 the N sets of corresponding data are divided into multiple types
  • step 2024 the compensation formula is respectively fitted according to each type of data of multiple types.
  • N images of rectangles of known aspect ratio at N different angles may be acquired by the camera of device 110. Acquiring N images may capture the rectangle through the camera, or may capture a frame of the preview video of the rectangle.
  • the plane of the rectangle coincides with the XOY plane, when the device is in space
  • the number N of angles is an integer not less than 2, which is used to satisfy the basic condition for obtaining the compensation formula. Since the accuracy of the compensation formula increases as the number N increases, more images can be acquired at more angles in order to obtain a more accurate compensation formula.
  • one direction is selected every 45° (i.e., ⁇ is taken as 0°, 45°, 90°, ..., 315°) in the range of ⁇ ⁇ [0°, 360°), and in these directions.
  • an image is taken every 4° in the range of ⁇ (0°, 90°) (that is, ⁇ is taken as 4°, 8°, ..., 88°, respectively). It should be noted that the above angular range and angular interval are only examples, and other angular ranges and angular intervals may be selected to obtain more or less images.
  • the rectangle is displayed as a quadrilateral Q in the image, by selecting different horizontal-to-edge angles ⁇ QH of the quadrilateral Q and the angle between the vertical opposite sides ⁇ QV (reference numeral not shown), thereby acquiring N images.
  • an image can be acquired every 1° in the range of the horizontal-to-edge angle ⁇ QH ⁇ [0°, 50°]; at the same time, the angle ⁇ QV ⁇ [0°, 50° at the vertical opposite side In the range of ], an image is acquired every 1°.
  • an image can also be acquired every 2°. It should be noted that the above angular range and angular interval are only examples, and other angular ranges and angular intervals may also be selected.
  • the first aspect ratio rectangular quadrilateral R Q R 0 corresponding to the first aspect ratio, when the aspect ratio of a quadrangle R 0 using a first adjacent side length ratio, the width of the rectangular first step
  • the high ratio R Q also uses the ratio of the corresponding adjacent side lengths.
  • the first aspect ratio R 0 of the quadrilateral uses the ratio of the lengths of the two median lines
  • the first aspect ratio R Q of the rectangle also adopts the ratio of the lengths of the corresponding two median lines.
  • the geometrical features of the rectangle correspond to the geometric features of the quadrilateral.
  • the geometric features of the quadrilateral include the two sets of opposite angles ⁇ H and ⁇ V of the quadrilateral and the absolute value ⁇ of the difference between the two sets of opposite sides
  • the compensation factor k Q of the rectangle corresponds to the compensation factor k of the quadrilateral, and the compensation factor k Q of the rectangle reflects the relationship between the original aspect ratio R of the rectangle and the first aspect ratio R Q .
  • a corresponding set of rectangular compensation factors k Q and geometric features can be obtained, that is, the compensation factor k Q , two pairs of pairs The angle ⁇ QH and ⁇ QV of the side and the absolute value ⁇ Q of the difference between the angles of the two sets of opposite sides.
  • the geometric features of the rectangle are compared to a classification threshold to classify the N sets of corresponding data into a type corresponding to the quadrilateral type.
  • the quadrilateral can be classified into the following four categories: horizontal unilateral inclination, horizontal bilateral inclination, vertical unilateral inclination, and vertical bilateral inclination.
  • the two sets of opposite side angles ⁇ QH and ⁇ QV of the rectangle are compared with the angle classification threshold ⁇ T , thereby dividing the N sets of corresponding data into four types corresponding to the quadrilateral type.
  • the vertical-to-edge angle ⁇ QV of the horizontal double-sided tilt type shown in part (B) of FIG. 6 is smaller than the vertical-to-edge angle ⁇ QV , and the angle classification threshold ⁇ T is smaller than the horizontal-to-edge angle ⁇ QH ;
  • the vertical unilateral tilt type ⁇ QV of the vertical unilateral tilt type shown in part (C) is smaller than the horizontal opposite side angle ⁇ QH , and the angle classification threshold ⁇ T is between the two sets of opposite side angles;
  • the vertical-to-edge angle ⁇ QV of the vertical double-sided tilt type shown in the (D) portion is smaller than the horizontal-to-edge angle ⁇ QH , and the angle classification threshold ⁇ T is smaller than the vertical-to-edge angle ⁇ QV .
  • the angle classification threshold ⁇ T can be used to classify the N sets of corresponding data into corresponding quadrilateral types according to a certain ratio or uniformly, and classify the corrected quadrilaterals to select a corresponding compensation formula.
  • the angle classification threshold ⁇ T may be a horizontal-to-edge angle ⁇ QH of a rectangle or a statistical quantile of a vertical-to-edge angle ⁇ QV .
  • the quantile includes quartiles or other proportional quantiles.
  • the angle classification threshold ⁇ T may be the median of the selection angle-to-edge angle ⁇ QH (also referred to as the second quartile), that is, all the horizontal-to-edge angles ⁇ QH are arranged from small to large and then ranked 50th.
  • the value of %) can also be the median of the angle ⁇ QV between the vertical and the opposite sides, or the mean of the median of the angle between the horizontal and the opposite sides and the median of the angle between the vertical and the opposite sides.
  • the angle classification threshold ⁇ T may also be the mean of the horizontal-to-edge angle ⁇ QH of the rectangle or the vertical-to-edge angle ⁇ QV .
  • the angle classification threshold ⁇ T may also be valued empirically, for example, from a range of [2°, 5°].
  • step 2024 in each of the N sets of corresponding data, a compensation formula is fitted according to the absolute value ⁇ Q of the difference between the two sets of opposite sides of the rectangle and the compensation factor k Q .
  • Table 1 shows the corresponding data of the absolute value ⁇ Q of the difference between the two sets of opposite sides of the rectangle in the horizontal unilateral tilt type and the compensation factor k Q , thereby fitting the horizontal unilateral tilt
  • the type of compensation formula may be any one of the following, where x represents a geometric feature and k represents a compensation factor:
  • the two sets of opposite side angles ⁇ H and ⁇ V of the quadrilateral to be corrected are compared with the angle classification threshold ⁇ T , so that the quadrilateral to be corrected is divided into corresponding types, and the compensation formula under the type is selected.
  • the absolute value ⁇ of the difference between the two sets of angles is substituted into the compensation formula to calculate the compensation factor k of the quadrilateral.
  • the image acquisition scenes of different angles are classified, and the compensation formula is obtained by using a statistical method for each type of image acquisition scene, and the preliminary estimation of the quadrilateral aspect ratio is compensated, so that the corrected image is obtained.
  • the aspect ratio is closer to the original aspect ratio of the subject, which improves the image correction effect.
  • FIG. 2 Another embodiment of the present invention provides an image correction method.
  • the method will be described below with reference to FIGS. 2, 4, 7, and 8.
  • the flowchart of the image correction method is as shown in FIG. 2, wherein steps 203 to 205 of the method are the same as steps 203 to 205 described above, and details are not described herein again.
  • steps 201 and 202 and steps 201 and 202 described above will be described.
  • the geometric features of the quadrilateral include a first geometric feature and a second geometric feature.
  • the first geometric feature is a combination of a length feature and an angle feature for classifying the quadrilateral.
  • the length feature may include a side length ratio D of the quadrilateral.
  • the aspect ratio D can be used to determine the relative distance between the subject and the camera. The larger the side length ratio, the closer the distance between the subject and the camera; the smaller the side length ratio, the farther the subject is from the camera.
  • the angular feature may include two sets of opposite angles ⁇ H and ⁇ V , and may also include trigonometric values of two sets of opposite sides of the quadrilateral, for example, sine, tangent, cosine or Cotangent value.
  • the above values can be used to determine the degree of tilt of the opposite sides with less quadrilateral distortion.
  • the second geometric feature includes an angular feature of the quadrilateral for calculating the compensation factor k.
  • the value of the trigonometric function of the difference between the angles, or the difference between the two sets of trigonometric values of the angles of the edges, or the trigonometric function of the absolute difference between the two sets of angles between the two sides, or the trigonometric function of the two sets of angles between the opposite sides The absolute value of the difference in values. The above values can be used to determine the degree of deformation of the entire image.
  • the absolute value ⁇ of the difference between the two groups When the method according to the embodiment of the present invention is applied to an embodiment in which the geometric feature includes a trigonometric function value, the two sets of opposite angles may be replaced by two sets of trigonometric values of the opposite sides, and two pairs of pairs may be used.
  • the absolute value of the difference between the absolute value of the difference between the edges and the value of the trigonometric function of the two sets of angles replaces the absolute value of the difference between the two sets of opposite sides.
  • a compensation factor is obtained based on the geometric features of the quadrilateral and the compensation formula.
  • the second geometric feature of the quadrilateral is substituted into the compensation formula, and the compensation factor of the quadrilateral can be calculated.
  • the compensation formula may reflect the relationship of the second geometric feature to the compensation factor k in the same form as described in step 202 above.
  • the quadrilateral can be classified according to the characteristics of the distance and the tilting posture, Each type of quadrilateral determines a compensation formula.
  • FIG. 4 A flowchart of the method for determining the compensation formula is shown in FIG. 4 .
  • the steps 2021 and 2024 of the method are the same as the steps 2021 and 2024 described above, and are not described herein again.
  • the differences between steps 2022 and 2023 and steps 2022 and 2023 described above will be described.
  • step 2022 since the geometric feature of the rectangle corresponds to the geometric feature of the quadrilateral to be corrected, when the geometric feature of the quadrilateral to be corrected includes the first geometric feature and the second geometric feature, the geometric feature of the rectangle also includes the corresponding first geometric feature. And the second geometric feature.
  • the first geometric feature of the rectangle is compared to a classification threshold to classify the N sets of corresponding data into a type corresponding to the quadrilateral type.
  • quadrilaterals can be divided into the following three categories based on distance: near, medium, and far.
  • the quadrilateral can be divided into two types: near and far, or divided into five categories: near, medium, medium, medium, and long distance. It should be noted that the above classification should not be a limitation on the present application as long as the classification can describe the distance between the object and the device.
  • the side length ratio D Q and the distance classification threshold T of the rectangle may be compared first, and the N sets of corresponding data are classified into a category corresponding to the quadrilateral distance type.
  • the data is then compared with the two sets of opposite side angles ⁇ QH and ⁇ QV of the rectangle and the angle classification threshold ⁇ T , thereby further classifying each large class of data into small class data corresponding to the quadrilateral tilt type.
  • the distance classification threshold T can be used to classify the N sets of corresponding data into corresponding quadrilateral distance types according to a certain ratio or substantially uniformly, and classify the corrected quadrilaterals to select a corresponding compensation formula.
  • the distance classification threshold T may be a statistical quantile of the square side length ratio D Q .
  • the distance classification threshold T may be a 1/3 quantile of the side length ratio D Q (ie, all values are determined by The value of the 1/3 position after the small to large arrangement and the 2/3 quantile (that is, the value of all the values from the small to the large and the 2/3 position).
  • the distance classification threshold T may also be the mean of the rectangular side length ratio D Q .
  • the distance classification threshold T may also be based on empirical values, such as 0.5 and 0.8.
  • the quadrilateral can be classified into three types: far, medium, and close.
  • the side length ratio of the close-up quadrilateral shown in part (A) of FIG. 7 is h/H>0.8
  • the side length of the medium-distance quadrilateral shown in part (B) of FIG. 7 is 0.5 ⁇ h/ H ⁇ 0.8
  • the side length ratio of the distance-like quadrilateral shown in part (C) of Fig. 7 is h/H ⁇ 0.5.
  • the N sets of corresponding data are divided into three categories of data corresponding to the three kinds of distances, and then each of the large types of data is divided into four sub-classes of data corresponding to the four tilting postures, as shown in FIG. data.
  • the types of image acquisition scenes are further refined, so that the compensation formula corresponding to each type of image acquisition scene can more accurately reflect the geometric features and compensation of the quadrilateral.
  • the relationship of the factors, so the corrected image aspect ratio is closer to the actual aspect ratio of the subject, thereby improving the effect of image correction in each type of scene.
  • FIG. 2 Another embodiment of the present invention provides an image correction method.
  • the method will be described below with reference to FIGS. 2 and 9.
  • the flowchart of the image correction method is as shown in FIG. 2, wherein steps 203 to 205 of the method are the same as steps 203 to 205 described above, and details are not described herein again.
  • steps 201 and 202 and steps 201 and 202 described above will be described.
  • the geometric feature of the quadrilateral may be the difference between the two sets of opposite sides of the quadrilateral.
  • the magnitude of the difference ⁇ HV can be used to determine the degree of tilt of the image as a whole, and the sign of the difference ⁇ HV is used to indicate the tilt direction (ie, horizontal or vertical tilt).
  • ⁇ HV >0 the image is tilted as a whole.
  • ⁇ HV ⁇ 0 the image as a whole is vertically inclined.
  • ⁇ VH >0 the rectangular image is vertically inclined as a whole
  • ⁇ VH ⁇ 0, the rectangular image as a whole is horizontally inclined.
  • the geometric feature of the quadrilateral may also be a trigonometric function value of the difference between the two sets of angles of the opposite sides, or a difference between the two sets of trigonometric values of the angles of the opposite sides.
  • These geometric features can indicate the degree of tilt and tilt of the image as a whole. Oblique direction.
  • the embodiment of the present invention refers to the difference ⁇ HV between the two sets of opposite sides of the quadrilateral unless otherwise stated.
  • the geometric feature includes a trigonometric function value
  • the difference between the two sets of trigonometric values of the difference between the sides of the edges or the difference between the two sets of trigonometric values of the opposite sides can be used.
  • the difference between the corresponding two sets of opposite sides can be used.
  • step 202 the geometric feature of the quadrilateral is substituted into the compensation formula, and the compensation factor k of the quadrilateral can be calculated.
  • the compensation formula may reflect the relationship of the geometric feature to the compensation factor k in the same form as described in step 202 above. Since the difference between the two sets of opposite sides of the quadrilateral can indicate the degree of tilt and the tilt direction of the image as a whole, the quadrilateral can be classified without using a compensation formula to represent the relationship between the geometrical features of the quadrilateral and the compensation factor k.
  • FIG. 9 Another method for determining a compensation formula according to an embodiment of the present invention is described below with reference to FIG. 9, which may be performed by the device 101, including:
  • Step 2021' obtaining N images of rectangles of known aspect ratio at N different angles.
  • Step 2022' Obtain a first aspect ratio R Q and a geometric feature of the rectangle from each of the N images, and obtain corresponding data of the compensation factor k Q of the N sets of rectangles and the geometric features.
  • Step 2023' fitting the compensation formula according to the N sets of compensation factors k Q and the corresponding data of the geometric features.
  • step 2021' is similar to the previous step 2021, it will not be described again here. Step 2022' and step 2023' will be specifically described below.
  • step 2022' since the geometric features of the rectangle correspond to the geometric features of the quadrilateral, when the geometric features of the quadrilateral include the difference between the two sets of opposite sides, the geometrical features of the rectangle also include the difference between the two sets of opposite sides.
  • step 2023' since the quadrilateral is unclassified, the N sets of corresponding data may not be classified, and the compensation formula may be directly fitted according to the difference ⁇ QHV between the two sets of opposite sides of the rectangle and the compensation factor k Q .
  • Table 2 shows the corresponding data of the difference ⁇ QHV between the two sets of opposite sides of the rectangle and the compensation factor k Q , whereby the compensation formula can be fitted.
  • the type of the image acquisition scene can be reduced, and the compensation formula can still accurately reflect the relationship between the geometric features of the quadrilateral and the compensation factor, thereby simplifying the image.
  • the step of correcting improves the processing speed of image correction.
  • FIG. 10 is a schematic structural diagram of a device according to an embodiment of the present invention.
  • the device 1000 includes: The first obtaining module 1001, the second obtaining module 1002, the first calculating module 1003, and the second calculating module 1004 and the correcting module 1005.
  • the first obtaining module 1001 is configured to acquire a first aspect ratio of a quadrilateral in the image to be corrected and a geometric feature of the quadrilateral.
  • the second obtaining module 1002 is configured to obtain a compensation factor of the quadrilateral according to the geometric feature of the quadrilateral.
  • the first calculating module 1003 is configured to calculate a second aspect ratio of the quadrilateral according to a first aspect ratio of the quadrilateral and a compensation factor of the quadrilateral.
  • the second calculating module 1004 is configured to calculate a transformation matrix according to the second aspect ratio of the quadrilateral.
  • the correction module 1005 is configured to correct the image to be corrected according to the transformation matrix.
  • FIG. 11 is a schematic structural diagram of a second acquiring module according to an embodiment of the present invention.
  • the second acquiring module 1002 further includes: a first acquiring unit 10021, a second acquiring unit 10022, a classifying unit 10023, and Fitting unit 10024.
  • the first obtaining unit 10021 is configured to acquire N images of a rectangle of a known aspect ratio at N different angles, where N is an integer not less than 2; and the second obtaining unit 10022 is configured to acquire the rectangle
  • the aspect ratio and the geometric features of the rectangle obtain corresponding data of the compensation factor of the N sets of the rectangle and the geometric feature of the rectangle;
  • the classifying unit 10023 is configured to divide the N sets of corresponding data into multiple types;
  • the merging unit 10024 is configured to respectively fit the compensation formula according to each of the plurality of types of data.
  • FIG. 12 is a schematic structural diagram of another second acquiring module according to an embodiment of the present invention.
  • the second acquiring module 1002 further includes: a first acquiring unit 10021', a second acquiring unit 10022', and a Unit 10023'.
  • the first obtaining unit 10021 ′ is configured to acquire N images of a rectangle of a known aspect ratio at N different angles, where N is an integer not less than 2; and the second obtaining unit 10022 is configured to acquire the rectangle.
  • the fitting unit 10023' is configured to fit the compensation according to the N sets of corresponding data formula.
  • the device provided by the embodiment of the present invention is used to implement the foregoing method in the embodiment shown in FIG. 1 to FIG. 9.
  • FIG. 13 is a structural diagram of another apparatus according to an embodiment of the present invention.
  • the apparatus 1300 includes: a processor 1301, a camera 1302, a user input device 1303, a display 1304, a memory 1305, and a Or multiple programs.
  • FIG. 13 only shows a part related to the embodiment of the present invention. If the specific technical details are not disclosed, please refer to the above method embodiment and other parts of the application file of the present application.
  • the number of hardware such as the processor 1301, the camera 1302, the user input device 1303, the display 1304 and the memory 1305 may be one or more, depending on the type of the device 1300.
  • the processor 1301 is coupled to the camera 1302, the user input device 1303, the display 1304, and the memory 1305 via one or more buses.
  • the processor 1301 receives the image from the camera 1302 and the user command from the user input device 1303, calls the execution instruction stored in the memory 1305 for processing, and sends it to the display 1304 for presentation.
  • the processor 1301 may be a central processing unit (CPU), a general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), and a field programmable gate array ( Field Programmable Gate Array (FPGA) or other programmable logic device, transistor logic device, hardware component, or any combination thereof.
  • the processor 1301 can implement or perform various exemplary logical blocks, modules, and circuits described in connection with the present disclosure.
  • the processor 1301 may also be a combination of computing functions, such as one or more microprocessor combinations, a combination of a DSP and a microprocessor, and the like. As an example, the processor 1301 is configured to support the device 1300 to perform step 202 in FIG. 2 . Up to 204, steps 2022 through 2024 in Fig. 4, and steps 2022' and 2023' in Fig. 9.
  • the camera 1302 transmits the acquired image to the processor 1301.
  • the camera 1302 can also be a digital camera (also known as a digital camera, digital camera), a scanner, or other optical device capable of acquiring images.
  • the acquired image is distinguished from the perspective of the content, and may be an image, a document, or a mixed image containing both the image and the document.
  • the camera 1302 is used to support the device to perform step 201 in Fig. 2, step 2021 in Fig. 4, and step 2021' in Fig. 9.
  • the user input device 1302 receives the user's operation instructions and transmits them to the processor 1301 for processing.
  • the user input device 1302 includes a touch screen, a button, or a microphone.
  • Display 1304 presents a preview image, a corrected image, and a human-computer interaction interface.
  • Display 1304 can be a display built into device 1300 or other external display device externally connected to device 1300.
  • the memory 1305 stores images, preset models, and one or more programs.
  • Memory 1305 can be any computer readable storage medium.
  • the one or more programs are stored in a memory and configured to be executed by one or more processors, including both execution instructions of the device operating system and execution instructions of the device application, such as correcting images in embodiments of the present invention. Execution instructions.
  • the present invention may be implemented in whole or in part by software, hardware, firmware, or any combination thereof.
  • software it may be implemented in whole or in part in the form of a computer program product.
  • the computer program product includes one or more computer instructions.
  • the computer program instructions When the computer program instructions are loaded and executed on a computer, the processes or functions described in accordance with embodiments of the present invention are generated in whole or in part.
  • the computer can be a general purpose computer, a special purpose computer, a computer network, or other programmable device.
  • the computer instructions can be stored in a computer readable storage medium or transferred from one computer readable medium to another computer readable medium, for example, the computer instructions can be wired from a website site, computer, server or data center (for example, coaxial cable, fiber, Digital Subscriber Line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) to another website, computer, server or data center.
  • the computer readable storage medium can be any available media that can be accessed by a computer or a data storage device such as a server, data center, or the like that includes one or more available media.
  • the usable medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a DVD), or a semiconductor medium (eg, a Solid State Disk (SSD)) or the like.
  • a magnetic medium eg, a floppy disk, a hard disk, a magnetic tape
  • an optical medium eg, a DVD
  • a semiconductor medium eg, a Solid State Disk (SSD)
  • the functions described herein can be implemented in hardware, software, firmware, or any combination thereof.
  • the functions may be stored in a computer readable medium or transmitted as one or more instructions or code on a computer readable medium.
  • Computer readable media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one location to another.
  • a storage medium may be any available media that can be accessed by a general purpose or special purpose computer.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

L'invention concerne un procédé de correction d'image, comprenant les étapes consistant : à acquérir un premier rapport d'aspect d'un quadrilatère d'une image à corriger et une caractéristique géométrique du quadrilatère; à acquérir un facteur de compensation du quadrilatère selon la caractéristique géométrique dudit quadrilatère; à calculer un second rapport d'aspect du quadrilatère en fonction du premier rapport d'aspect du quadrilatère et du facteur de compensation du quadrilatère; à calculer une matrice de transformation conformément au second rapport d'aspect du quadrilatère; et à corriger l'image à corriger selon la matrice de transformation. Au moyen de la solution apportée par la présente invention, une valeur estimée plus proche du rapport d'aspect original d'un sujet capturé peut être obtenue, ce qui permet d'obtenir une image corrigée sensiblement en accord avec le rapport d'aspect original du sujet capturé.
PCT/CN2017/074432 2017-02-22 2017-02-22 Procédé et dispositif de correction d'image WO2018152710A1 (fr)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/CN2017/074432 WO2018152710A1 (fr) 2017-02-22 2017-02-22 Procédé et dispositif de correction d'image
CN201780005563.1A CN108780572B (zh) 2017-02-22 2017-02-22 图像校正的方法及装置

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2017/074432 WO2018152710A1 (fr) 2017-02-22 2017-02-22 Procédé et dispositif de correction d'image

Publications (1)

Publication Number Publication Date
WO2018152710A1 true WO2018152710A1 (fr) 2018-08-30

Family

ID=63253488

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/074432 WO2018152710A1 (fr) 2017-02-22 2017-02-22 Procédé et dispositif de correction d'image

Country Status (2)

Country Link
CN (1) CN108780572B (fr)
WO (1) WO2018152710A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111652937A (zh) * 2019-03-04 2020-09-11 广州汽车集团股份有限公司 车载相机标定方法和装置

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110717492B (zh) * 2019-10-16 2022-06-21 电子科技大学 基于联合特征的图纸中字符串方向校正方法
CN112837394B (zh) * 2019-11-25 2024-06-18 珠海金山办公软件有限公司 一种多边形绘制方法、装置、电子设备及可读存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5027227A (en) * 1987-07-24 1991-06-25 Sharp Kabushiki Kaisha Image rotating device
EP1071274A2 (fr) * 1999-07-21 2001-01-24 Konica Corporation Appareil de lecture d'images et appareil de formation d'images
CN102254171A (zh) * 2011-07-13 2011-11-23 北京大学 一种基于文本边界的中文文档图像畸变校正方法
CN105450900A (zh) * 2014-06-24 2016-03-30 佳能株式会社 用于文档图像的畸变校正方法和设备
CN106296745A (zh) * 2015-05-26 2017-01-04 富士通株式会社 对文档图像进行校正的方法和装置

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2002101443A2 (fr) * 2001-06-12 2002-12-19 Silicon Optix Inc. Systeme et procede de correction d'une distorsion multiple de deplacement d'axe
CN1996389A (zh) * 2007-01-09 2007-07-11 北京航空航天大学 基于共线特征点的摄像机畸变快速校正方法
US20090046182A1 (en) * 2007-08-14 2009-02-19 Adams Jr James E Pixel aspect ratio correction using panchromatic pixels
JP4630936B1 (ja) * 2009-10-28 2011-02-09 シャープ株式会社 画像処理装置、画像処理方法、画像処理プログラム、画像処理プログラムを記録した記録媒体
CN102903092B (zh) * 2012-09-07 2016-05-04 珠海一多监测科技有限公司 一种基于四点变换的图像自适应校正方法
US9135720B2 (en) * 2013-03-27 2015-09-15 Stmicroelectronics Asia Pacific Pte. Ltd. Content-based aspect ratio detection
JP6128086B2 (ja) * 2014-09-12 2017-05-17 カシオ計算機株式会社 頁画像補正装置、頁画像補正方法及びプログラム
PL3308350T3 (pl) * 2015-06-12 2020-05-18 Moleskine S.R.L. Sposób korygowania uchwyconego obrazu, sposób wyboru rysunku naszkicowanego na stronie lub na dwóch sąsiednich stronach notatnika, powiązana aplikacja na smartfon, notatnik w twardej oprawie i terminarz w twardej oprawie

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5027227A (en) * 1987-07-24 1991-06-25 Sharp Kabushiki Kaisha Image rotating device
EP1071274A2 (fr) * 1999-07-21 2001-01-24 Konica Corporation Appareil de lecture d'images et appareil de formation d'images
CN102254171A (zh) * 2011-07-13 2011-11-23 北京大学 一种基于文本边界的中文文档图像畸变校正方法
CN105450900A (zh) * 2014-06-24 2016-03-30 佳能株式会社 用于文档图像的畸变校正方法和设备
CN106296745A (zh) * 2015-05-26 2017-01-04 富士通株式会社 对文档图像进行校正的方法和装置

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111652937A (zh) * 2019-03-04 2020-09-11 广州汽车集团股份有限公司 车载相机标定方法和装置
CN111652937B (zh) * 2019-03-04 2023-11-03 广州汽车集团股份有限公司 车载相机标定方法和装置

Also Published As

Publication number Publication date
CN108780572B (zh) 2021-04-20
CN108780572A (zh) 2018-11-09

Similar Documents

Publication Publication Date Title
US10915998B2 (en) Image processing method and device
US9275281B2 (en) Mobile image capture, processing, and electronic form generation
US9807263B2 (en) Mobile document capture assistance using augmented reality
WO2018214365A1 (fr) Procédé, appareil, dispositif et système de correction d'image, dispositif de prise de vues et dispositif d'affichage
WO2021233266A1 (fr) Procédé et appareil de détection de bord et dispositif électronique et support de stockage
US10909719B2 (en) Image processing method and apparatus
CN109120854B (zh) 图像处理方法、装置、电子设备及存储介质
WO2021147219A1 (fr) Procédé et appareil de reconnaissance de texte à base d'image, dispositif électronique et support de stockage
CN113627428A (zh) 文档图像矫正方法、装置、存储介质及智能终端设备
CN108965742A (zh) 异形屏显示方法、装置、电子设备及计算机可读存储介质
JP2017130929A (ja) 撮像装置により取得された文書画像の補正方法及び補正装置
WO2021135683A1 (fr) Procédé de réglage de terminal d'affichage et terminal d'affichage
CN107368829B (zh) 确定输入图像中的矩形目标区域的方法和设备
CN111325798B (zh) 相机模型纠正方法、装置、ar实现设备及可读存储介质
CN110431563B (zh) 图像校正的方法和装置
US20180253852A1 (en) Method and device for locating image edge in natural background
CN108230333A (zh) 图像处理方法、装置、计算机程序、存储介质和电子设备
CN108874187A (zh) 一种投影仪笔记系统
CN110119733A (zh) 书页识别方法及装置、终端设备、计算机可读存储介质
US20190355104A1 (en) Image Correction Method and Apparatus
WO2018152710A1 (fr) Procédé et dispositif de correction d'image
US10643095B2 (en) Information processing apparatus, program, and information processing method
US10586099B2 (en) Information processing apparatus for tracking processing
JP2018046337A (ja) 情報処理装置、プログラム及び制御方法
US10999513B2 (en) Information processing apparatus having camera function, display control method thereof, and storage medium

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17897882

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17897882

Country of ref document: EP

Kind code of ref document: A1