KR102149051B1 - System and method for analyzing document using self confidence based on ocr - Google Patents

Abstract

Disclosed is an OCR-based document analysis system and method using self-reliability information. The present invention can reduce the verification work time by the inspector by providing a confidence score for the recognized information based on OCR recognition.

Description

OCR-based document analysis system and method using self-reliability information {SYSTEM AND METHOD FOR ANALYZING DOCUMENT USING SELF CONFIDENCE BASED ON OCR}

The present invention relates to an OCR-based document analysis system and method using self-reliability information, and more specifically, a self-confirmation work time that can be reduced by an inspector by providing a confidence score for information recognized using OCR. It relates to an OCR-based document analysis system and method using reliability information.

Individuals and companies maintain and manage receipts collected during economic activities as post-mortem documents for accounting for expense reports, expenditure resolutions, or tax processing such as comprehensive income tax returns.

Since the receipts stored and managed in this way are made of paper, there is a risk of fundamentally damaging the original, there is a technical and economic burden to prevent exposure to contamination, loss, and corruption, and the receipt storage space in proportion to the increase in the amount of receipt storage There is a problem that needs to be increased.

In addition, the person in charge of an individual or company extracts and classifies the information necessary for the above accounting or tax processing from the conventional receipt by hand input method and writes it in the ledger or enters and stores it in a PC with an accounting management program installed. There is this inconvenient problem.

On the other hand, text (text) images included in documents can be converted through mechanical encoding. Characters converted through mechanical encoding can be electronically edited and searched, and converted characters are converted into a database in the form of a file, etc. You can also save it.

Such mechanical encoding may be performed mainly through optical character recognition (OCR), and image-based text documents may be automatically detected, identified, and encoded using a computer or the like.

In Korean Registered Patent Publication No. 10-1139801 (title of the invention: automatic information collection system and method through receipt reading), the purchased goods, purchase quantities, and usage amounts printed on conventional receipts are read and stored through OCR. By doing so, a configuration for automatically collecting and managing purchase information of the user of the receipt is disclosed.

However, the OCR according to the prior art has a problem in that the recognition accuracy of the OCR is deteriorated in the case of an image printed on a low-quality printer or fax machine, imaged in a low-resolution photographing means, or photographed in a wrinkled or inclined state.

In addition, the information collection system and method according to the prior art have a problem in that the inspector must check all the presence/absence of abnormalities in the information recognized by the inspector for the information collected through OCR.

Korean Registered Patent Publication No. 10-1139801 (Name of invention: Automatic information collection system and method through receipt reading)

In order to solve this problem, the present invention provides an OCR-based document analysis system and method using self-reliability information that can reduce the verification work time by the inspector by providing a confidence score for the recognized information based on OCR recognition. It aims to do.

In order to achieve the above object, an embodiment of the present invention is an OCR-based document analysis system using self-reliability information, and at least one of a form, letter, and number in an image to be recognized using an object detection model. It detects the location of an object, displays a square shape along the perimeter of the detected format, letter and number object to generate a square pixel position value, and uses the OCR model to retrieve the character and number information recognized in the square pixel. A document analysis device that recognizes, connects all adjacent square pixels based on the generated square pixel position value, and matches and displays letter and numeric information recognized through the OCR model with the connected square pixel position. Include.

In addition, the document analysis apparatus according to the embodiment is characterized in that the reliability score of the recognized letters and numbers is calculated based on the recognition rate using the OCR model, and the calculated confidence score is reflected in the display information to be visually displayed. .

In addition, the document analysis apparatus according to the embodiment is characterized in that the normal recognition area, the wrong area, and the error occurrence area including the correction area are displayed as visualization information of different colors according to the confidence score.

In addition, the confidence score according to the embodiment is characterized in that the reconstruction rate calculated by performing at least one of a form, a shape, and a location is additionally reflected using a correction model.

In addition, the document analysis apparatus according to the embodiment includes an input unit for receiving an image to be recognized; In the received recognition target image, by using an object detection model, the position of at least one of a type, a letter, and a number object is detected, a square shape is displayed around the detected type, letter and number object, and the displayed square pixel position An object detection modeling unit that generates a value; An OCR modeling unit that outputs letter and number information recognized in the square pixel by using an OCR model; As the recognized character is corrected with the character information of the item DB, the pixel position of a random rectangle with numeric information is moved in the left direction and the upper direction as the starting position, but when the character information is searched, all the rectangular pixels searched during movement A form configuration modeling unit that connects them and matches the corrected letter and number information through the OCR model to the connected square pixel positions and displays them; A reliability evaluation unit that calculates a confidence score of the recognized letters and numbers using a correction model, and reflects it on the display based on the calculated confidence score to be visually displayed; And a database storing the generated square pixel position value, recognized letters, number information, trust information, and a form of document data used by a specific institution.

In addition, it further includes an item DB generation unit for generating item DB information for preset characters from an arbitrary document with respect to the characters included in the recognition target image according to the embodiment.

In addition, the form configuration modeling unit according to the embodiment may match the detected text object with item DB information, and when the recognized text is corrected according to the matching result, the corrected text is reflected.

In addition, the form configuration modeling unit according to the embodiment is characterized in that the color of the rectangular box of letters and numbers is displayed in different colors between a normal recognition area and an error area including a wrong area and a correction area.

In addition, the OCR-based document analysis method using self-reliability information according to an embodiment of the present invention includes: a) at least one of an arbitrary format, letter, and numeric object by using an object detection model in a recognition target image received by the document analysis device. Detecting a position of and generating a square pixel position value by displaying a square shape around the detected type, letter and number object; b) outputting, by the document analysis device, character and number information recognized within the detected rectangular pixels using the OCR model; c) The document analysis device moves in the left direction and the upward direction using the generated square pixel position value and a random square pixel position having numerical information based on the recognized letter and number information as a starting position. , When the text information is searched, connecting all the square pixels retrieved while moving, and matching the text and number information recognized through the OCR model to the connected square pixel positions to be displayed; And d) calculating, by the document analysis device, a confidence score of the recognized letters and numbers using a correction model, and visually displaying it by reflecting on a display based on the calculated confidence score.

In addition, step c) according to the embodiment includes the steps of generating an item DB defining character (item) information used in an arbitrary document in order to compare the character information recognized in the recognition target image by the document analysis device; It characterized in that it further includes.

In addition, step c) according to the embodiment may further include the step of matching the detected text object with item DB information, and correcting the recognized text according to the matching result.

In addition, the correction model in step d) according to the above embodiment is trusted based on the reconstruction rate according to performing correction of at least one of form, shape, and position, and whether corrected characters are reflected according to the matching result with item DB information. It is characterized by calculating a score.

The present invention provides the normally recognized part and the wrong part or suspected part as visualization information of different colors based on the confidence score for the information recognized through OCR, thereby reducing the verification work time by the inspector. There is this.

In addition, the present invention has the advantage of providing accurate and reliable usage information for receipts in various formats used by institutions such as hospitals and insurance companies.

1 is a block diagram showing an OCR-based document analysis system using self-reliability information according to an embodiment of the present invention.
FIG. 2 is an exemplary diagram for explaining generation of an item DB in an OCR-based document analysis system using self-reliability information according to the embodiment of FIG. 1;
3 is an exemplary diagram showing an item DB of an OCR-based document analysis system using self-reliability information according to the embodiment of FIG. 1.
FIG. 4 is an exemplary diagram for explaining character position detection in an OCR-based document analysis system using self-reliability information according to the embodiment of FIG. 1;
5 is an exemplary view showing a result of character position detection according to the embodiment of FIG. 4.
6 is an exemplary view showing a process of recognizing a character object at a character position according to the embodiment of FIG. 4.
7 is an exemplary view showing a result of OCR recognition of an OCR-based document analysis system using self-reliability information according to the embodiment of FIG. 1.
FIG. 8 is an exemplary view for explaining a connection process of an object detection box of an OCR-based document analysis system using self-reliability information according to the embodiment of FIG. 1.
9 is an exemplary view for explaining a connection process according to FIG. 7.
10 is an exemplary view showing a reconstructed image of an OCR-based document analysis system using self-reliability information according to the embodiment of FIG. 1.
11 is a flowchart illustrating an OCR-based document analysis process using self-reliability information according to an embodiment of the present invention.

Hereinafter, the present invention will be described in detail with reference to a preferred embodiment of the present invention and the accompanying drawings, but it will be described on the premise that the same reference numerals refer to the same elements.

Prior to describing specific details for the implementation of the present invention, it should be noted that configurations that are not directly related to the technical gist of the present invention have been omitted within the scope not disturbing the technical gist of the present invention.

In addition, the terms or words used in the specification and claims are based on the principle that the inventor can define the concept of an appropriate term to describe his or her invention in the best way. Should be interpreted as.

In this specification, the expression that a certain part "includes" a certain component does not exclude other components, but means that other components may be further included.

In addition, terms such as "... unit", "... group", and "... module" mean units that process at least one function or operation, which can be classified into hardware, software, or a combination of the two.

In addition, the term “at least one” is defined as a term including the singular and plural, and even if the term “at least one” does not exist, each component may exist in the singular or plural, and may mean the singular or plural. Will say self-evident.

In addition, it will be said that each component may be provided in singular or plural, and may be changed according to embodiments.

Hereinafter, a preferred embodiment of an OCR-based document analysis system and method using self-reliability information according to an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram showing an OCR-based document analysis system using self-reliability information according to an embodiment of the present invention.

Referring to FIG. 1, an OCR-based document analysis system using self-reliability information according to an embodiment of the present invention is an OCR-based document analysis system using self-reliability information, and an arbitrary format in an image to be recognized using an object detection model. Detects the location of at least one object among (form), letters and numbers, displays a square shape along the perimeter of the detected format, letter and number object to generate a square pixel position value, and uses the OCR model Recognizes character and number information recognized within a rectangular pixel, connects all adjacent rectangular pixels based on the generated rectangular pixel position value, and recognizes characters and numbers recognized through the OCR model at the connected rectangular pixel positions It may be configured to include a document analysis device 100 that matches and displays numeric information.

In addition, the document analysis apparatus 100 may calculate the confidence score of the recognized letters and numbers according to the recognition rate, and reflect the calculated confidence score on the display to be visually displayed.

In addition, the document analysis apparatus 100 may cause the reconstruction form to display a normal recognition area and an error occurrence area including an incorrect area and a correction area as visualization information of different colors according to the confidence score.

In addition, the document analysis apparatus 100 allows the reconstruction rate calculated according to the correction of at least one of a form, a shape, and a position to be additionally reflected by using a correction model in which a confidence score is used.

To this end, the document analysis apparatus 100 may be configured to include an input unit 110 for receiving an image to be recognized.

The input unit 110 may be configured as a data communication means for receiving an image transmitted from an external terminal connected through a network or an image scanned through a scanner.

In addition, the document analysis apparatus 100 includes an item DB generation unit 120 that generates item DB (300, see FIG. 3) information for a character set in advance from an arbitrary document with respect to the character included in the recognition target image. Can be configured.

As shown in FIG. 2, the item DB generation unit 120 is fixedly included in a document 200 such as a hospital receipt, and analyzes information on the item 210 displayed in text such as billing details and medical treatment details. .

In addition, the item DB generation unit 120 generates the item DB 300 as shown in FIG. 3 with respect to the item analyzed in FIG. 2 to be stored in the database 170.

In the present invention, for convenience of explanation, the image to be recognized is described as an example of an image related to a hospital receipt, but is not limited thereto, and includes pharmacy receipts, tax invoices, estimates, bills, transaction statements, various bills and receipts, etc. I can.

In addition, the document analysis apparatus 100 detects the location of an arbitrary form, text, and numeric object included in the image using an object detection model in the recognition target image received through the input unit 110, and detects the It may be configured to include an object detection modeling unit 130 for generating square pixel position information by displaying a square shape along the perimeter of a format, letter, and number object.

That is, the object detection modeling unit 130 performs the recognition of the relative position of the type, letter, and number objects, and checks the arrangement according to the position of the type, letter, and number objects. A square shape is displayed along the perimeter, and a pixel position value (coordinate information) on the displayed square is generated.

Here, the object detection model uses a deep learning model based on PSENet (Progressive Scale Expansion Network) to detect format, letter, and numeric objects and their location from training data including document images, and performs learning to improve detection rate. can do.

In other words, based on the original image of the receipt, an image in which an arbitrary part of the document is folded into n equal parts, an image in which the position of the document is inclined at an arbitrary angle, an image with an arbitrary brightness adjusted, and the content displayed on the document is clear. Learning data can be learned based on images with broken connecting lines, images in which arbitrary parts of the document are curved, and images overlapping numbers and connecting lines.

In this way, images that can be photographed in various environments, such as lighting, shooting angle, shaking during shooting, shooting composition, photo resolution, etc., are compared with the original image to learn in advance the images with various changes. By doing so, it is possible to improve a detection rate or recognition rate for images received in an actual environment.

On the other hand, in the present invention, for convenience of explanation, the image to be recognized is described as an example of an image related to a hospital receipt, but is not limited thereto, and a pharmacy receipt, tax invoice, estimate, bill, transaction statement, various bills and receipts, etc. Can include.

In addition, the object detection model may find an optimal rule in performing detection based on images to which various changes have been made through auto augmentation.

In addition, the object detection model may set a square pixel for an object of a detected type, letter, and number, and generate a position value of the set pixel.

That is, as shown in FIG. 4, the text object 411 detected from the arbitrary detection area 410 in the input document image 400 is displayed as a rectangular box along the outer peripheral surface.

In addition, the object detection model displays information on the recognized object as an object detection result 420 as shown in FIG. 5, thereby enabling pattern recognition based on a square shape.

In the present embodiment, the text object is described for convenience of description, but it is not limited thereto, and it will be apparent to those skilled in the art that a form constituting the form of a number and a receipt may be included as an object.

In addition, based on a rectangular configuration (or arrangement) pattern displayed along the perimeter of the form, the form configuration modeling unit 150 to be described later may be compared with a receipt configuration of an institution (hospital) previously stored to distinguish which institution's receipt is.

Here, the format is one cell in the form constituting the document table, and may be formed in a rectangle.

In addition, the document analysis apparatus 100 may be configured to include an OCR modeling unit 140 for recognizing letters and numbers using an OCR model with respect to the type, text and number objects detected by the object detection modeling unit 130. have.

Here, the OCR modeling unit 140 automatically detects and recognizes an image-based text document, and may be configured using a known OCR model.

In addition, the OCR modeling unit 140 includes the prediction information 431 and the prediction information 431 recognized for the OCR recognition result 430 recognized as shown in FIG. 7 through the OCR model in a form configuration modeling unit to be described later. A confidence score based on the calculated confidence score 432 may be provided together.

Here, the prediction information 431 indicates letters and numbers to be included in the recognized object, and the confidence score 432 reflects the case that the content is not clear or the connected part is disconnected during the recognition process through OCR. It may be a recognition rate obtained by calculating the ratio of parts recognized in.

In addition, the document analysis apparatus 100 compares the recognized character information with the item DB 300, and corrects the recognized character with the character information of the item DB 300, thereby determining the position of an arbitrary square pixel having numeric information. As the starting position, it moves in the left direction and upward direction, but when text information is searched, all square pixels retrieved during movement are connected, and the corrected letter and number information through the OCR model are matched to the connected square pixel position. It may be configured to include a form configuration modeling unit 150 to be displayed.

The form configuration modeling unit 150 defines character (item) information used in arbitrary documents, for example, hospital receipts, pharmacy receipts, transaction statements, tax invoices, etc., in order to compare with the character information recognized in the recognition target image. Item DB 300 can be created.

In addition, as shown in FIG. 6, the form configuration modeling unit 150 is an NLP object 411a for each morpheme or segmental sound for a character detected through natural language processing (NLP) for the recognized character object 411. ) May be analyzed, and a confidence score for the recognized character may be calculated based on a comparison between the analyzed result and the item DB 300 information.

For example, if the recognized character is "MRI diagnosis-", after searching for information on the item stored in the item DB 300, the confidence score is compared with the item DB 300 information corresponding to the searched item. Yields

That is, even if a new item is recognized on the hospital receipt or a mistake or an unrecognized person occurs due to an OCR mistake, the form configuration modeling unit 150 can process it according to the situation using the NLP.

In addition, the confidence score for the letters calculated by the form configuration modeling unit 150 is predicted information 431 recognized in the OCR recognition result 430 and the prediction information ( A confidence score 432 calculated for 431) may be provided.

In addition, the form configuration modeling unit 150 creates a reconstructed form by connecting the object positions of all letters and numbers adjacent to the left and upward directions from a random starting position based on the detected position of the text and the object.

On the other hand, when a new item is recognized, the form configuration modeling unit 150 includes the horizontal and vertical lengths of the box size of the numeric object equal to the horizontal and vertical lengths of other adjacent boxes or in the horizontal and vertical lengths of other boxes. If you do, try to connect.

When described in more detail with reference to FIG. 8, the horizontal size 600 and the vertical size 610 of the starting object box 500 are determined based on the starting object box 500 recognized as a number. ) And the horizontal size (600a) and the vertical size (610a) of the upper object box 520, or the same, or to the horizontal and vertical size of the left object box 510 and the upper object box 520 If included, the left connection line 700 and the upper connection line 710 are connected and displayed.

In the present embodiment, for convenience of explanation, the number object located in the center part of the starting position is described as the starting object box 500, but the right object box 500a and the starting object box located to the right of the starting object box 500 are described. You can also set the lower object box located below 500) as the start position.

For example, if the right object box 500a is set as the starting position, the text object recognized (or detected) for "industrial burden" may become the upper object box 520a.

In addition, the form configuration modeling unit 150 repeats the above-described process to connect only the nearest left and upper sides, and after the connection, starts from a number and moves until a text, for example, a character object (item) appears on the left and upper sides. And connect through search.

That is, as shown in FIG. 9, in performing a connection with the next object through movement and search from the document image 800 to the left and the upper side of the reference object box 810, the left object box 820 located on the left is If it is a letter, it is connected through the left connection line 840.

In addition, if the upper object box 830 is a text object (item) through moving upward and searching, it is continuously connected through the upper connection line 841.

In addition, the form configuration modeling unit 150 determines that there are no typos or unrecognized characters when the item of the reconstruction form is composed of only corrected characters, that is, characters selected from the item DB 300, and the OCR modeling unit 140 Only the numbers recognized through the OCR model are recognized, and the recognized numbers and items can be matched.

The form configuration modeling unit 150 matches the detected text object with the item DB 300 information, and when the recognized text is corrected according to the matching result, the corrected text is reflected.

In addition, the form configuration modeling unit 150 not only finds text objects and numeric objects through rectangle recognition, but also corrects the document image in a horizontal state through reconstruction by vertices when the document image is an inclined image. have.

In addition, when the document image is a trapezoidal image according to the photographing angle, the form configuration modeling unit 150 may convert the document image into a rectangular document image through reconstruction through trapezoidal correction.

In addition, when the document image is photographed including a wrinkled or folded portion, the form configuration modeling unit 150 may convert the document image into a rectangular document image using a program for correcting the shape or form.

On the other hand, the form configuration modeling unit 150 performs correction to improve the recognition accuracy and operates to achieve faithful reconstruction of a rectangular shape and accurate recognition based on this, but the possibility of error occurrence through correction and reconstruction accordingly Can also increase with.

To this end, the document analysis apparatus 100 uses a correction model to quantify information on the risk of a judgment error due to correction and reconstruction, for example, information obtained by quantifying the type and number of correction and reconstruction, and the reliability of recognized letters and numbers. Consists of a reliability evaluation unit 160 that calculates a confidence score based on and reflects it on the display through the form configuration modeling unit 150 so that the user can check based on the calculated confidence score, so that it is visually displayed. Can be.

That is, the reliability evaluation unit 160 provides information on the risk of a judgment error due to correction and reconstruction, for example, a confidence score obtained by quantifying the type and number of correction and reconstruction to the form configuration modeling unit 150.

The form configuration modeling unit 150 displays the normal object area and the error-prone object area including the wrong object area and the correction area in different colors of the rectangular box of letters and numbers.

That is, the form configuration modeling unit 150 displays an object area having a confidence score equal to or greater than a preset reference value in blue in the reconstructed image 900 as shown in FIG. 10 based on the confidence score provided through the reliability evaluation unit 160. Thus, the normal object areas 910 and 911 are displayed so that they can be identified.

In addition, the form configuration modeling unit 150 displays the object area in which the confidence score provided through the reliability evaluation unit 160 is less than or equal to the reference value in red in the reconstructed image 900 to display the error-prone object areas 920, 921, 922, and 923. , 924) are marked so that they can be identified.

The user can quickly and accurately check through the display of different colors in the form configuration modeling unit 150.

In addition, the document analysis device 100 may be configured to include a database 170 that stores the generated square pixel position value, recognized letters, number information, trust information, and a form of document data used by a specific institution. I can.

Here, the specific institution may include hospitals, pharmacies, companies, etc., all places that issue receipts and accounting-related documents in an arbitrary form.

The following describes an OCR-based document analysis method using self-reliability information according to an embodiment of the present invention.

11 is a flowchart illustrating an OCR-based document analysis process using self-reliability information according to an embodiment of the present invention.

1 to 11, the document analysis apparatus 100 receives (S100) an image of a receipt to be recognized through an external terminal connected through a network or a fax.

In addition, the document analysis apparatus 100 detects the object and position of letters and numbers using an object detection model in the received image of the recognition target receipt, and detects an object forming a rectangular box in the detected text and number objects. Perform the step (S200).

In addition, in step S200, the object detection model uses a PSENet (Progressive Scale Expansion Network)-based deep learning model to detect format, letter and numeric objects, and their location from training data including document images, and improve detection rate. To be able to perform learning.

In addition, the object detection model includes an original image, an image in which an arbitrary part of the document is folded, an image in which the position of the document is inclined at an arbitrary angle, an image with an arbitrary light intensity, and the content displayed in the document is not clear and broken. Learning data can be learned based on an image with a connecting line, an image in which an arbitrary part of a document is curved, and an image in which numbers and connecting lines overlap.

Subsequently, the document analysis apparatus 100 performs an OCR recognition step (S300) of recognizing letter and number information using an OCR model within a square pixel of the detected format, letter and number object.

After performing step S300, the document analysis apparatus 100 generates text information fixed from an arbitrary document, that is, character information about an item, as item DB 300 information, and the generated item DB 300 information is It is stored in the database 160 (S400).

As the recognized character information is compared with the item DB 300, and the recognized character is corrected with the character information of the item DB 300, it is determined whether there is a typo or an unrecognized person to determine whether to correct the document analysis device ( 100) performs NLP-based calibration (S500).

That is, in step S500, the document analysis apparatus 100 analyzes a character detected through natural language processing (NLP).

In addition, in step S500, the document analysis apparatus 100 may calculate and output a confidence score for the analyzed character based on a comparison between the analyzed character and the item DB 300 information.

Subsequently, the document analysis apparatus 100 moves in a left direction and an upward direction with a position of an arbitrary rectangular pixel having numeric information as a starting position, but when text information is searched, all the rectangular pixels retrieved during movement are connected, The letter and number information corrected through the OCR model are matched to the connected square pixel location (S600).

That is, the document analysis apparatus 100 moves and connects the starting position in a left direction and an upward direction by using the starting position as a numeric object.

At this time, only the closest left and upper sides are connected from the starting position first, then all adjacent objects are connected, and numeric objects are searched and moved until a text object (item) appears.

In addition, when the search for an item including a text object is completed, the document analysis apparatus 100 matches the recognized letters and corrected letters and numbers according to each attribute.

In addition, the document analysis apparatus 100 performs correction to improve the recognition accuracy, and the information obtained by numerically quantifying the type and number of correction and reconstruction processed so that the faithful reconstruction of a rectangular shape and accurate recognition based on this can be achieved, A confidence score is calculated based on the reliability of the recognized letters and numbers, and reflected on the display so that the user can check it based on the calculated confidence score to be visually displayed (S700).

In addition, in step S700, if the document image is in an inclined state, the document analysis device 100 corrects the document image in a horizontal state through reconstruction by vertices, and if the document image is a trapezoidal image, the document analysis apparatus 100 performs reconstruction through trapezoidal correction. It can also be converted to a rectangular document image.

In addition, if the document image includes a wrinkled or folded portion, by converting the document image to a rectangular document image using a program for correcting the shape or form, the reconstruction rate according to the correction of at least one of form, shape, and position, Confidence score can be calculated.

In addition, in step S700, the document analysis apparatus 100 causes the normal object area and the error-prone object area including the wrong object area and correction area to be displayed in different colors according to the confidence score. .

That is, as shown in FIG. 10, an object region having a confidence score equal to or greater than a preset reference value is displayed in blue in the reconstructed image 900 so that the normal object regions 910 and 911 can be identified, and the confidence score is less than the reference value. The object area is displayed in red on the reconstructed image 900 so that the error-prone object areas 920, 921, 922, 923, and 924 can be identified.

The user can quickly and accurately check through the display of different colors in the form configuration modeling unit 150.

In addition, the recognized text, the corrected text, and the form reconstructed through correction and reconstruction are converted into a reconstructed image, and the final result is output (S800) and stored in the database 170.

Therefore, the user can quickly and accurately check the information recognized through OCR by providing the normally recognized part, the wrong part or the suspected part as visualization information of different colors based on the confidence score. There is an advantage that can reduce the working time.

In addition, accurate and reliable usage information can be provided for receipts in various formats used by institutions such as hospitals and insurance companies.

As described above, the present invention has been described with reference to preferred embodiments of the present invention, but those skilled in the art can variously modify and change the present invention within the scope not departing from the spirit and scope of the present invention described in the following claims. You will understand that you can do it.

In addition, reference numerals in the claims of the present invention are provided for clarity and convenience of description, and are not limited thereto. In the process of describing the embodiments, the thickness of the lines shown in the drawings, the size of components, etc. May be exaggerated for clarity and convenience of description.

In addition, the above-described terms are terms defined in consideration of functions in the present invention and may vary according to the intention or custom of users and operators, so interpretation of these terms should be made based on the contents throughout the present specification. .

In addition, even if not explicitly shown or described, a person having ordinary knowledge in the technical field to which the present invention pertains can make various modifications including the technical idea according to the present invention from the description of the present invention. It is obvious, and this still belongs to the scope of the present invention.

In addition, the above embodiments described with reference to the accompanying drawings are described for the purpose of describing the present invention, and the scope of the present invention is not limited to these embodiments.

100: document analysis device 110: input unit
120: Item DB generation unit 130: Object detection modeling unit
140: OCR modeling unit 150: Form configuration modeling unit
160: reliability evaluation unit 170: database
200: document 210: item
300: Item DB 400: Document image
410: detection area 411: text object
411a: NLP object 420: Object detection result
430: OCR recognition result 431: prediction information
432: confidence score 500: starting object box
500a: right object box 510: left object box
520: upper object box 520a: upper object box
600: transverse size 610, 610a: longitudinal size
700: left connecting line 710: upper connecting line
720: lower connecting line 800: document image
810: reference object box 810a: right object box
820: left object box 830: upper object box
840: left connecting line 841: upper connecting line
900: reconstructed image 910, 911: normal object area
920, 921, 922, 923, 924: object area with error

Claims (10)

The object detection model is used to detect the location of at least one object among arbitrary forms, letters, and numbers in the recognition target image, and displays a square shape along the perimeter of the detected type, letter, and number object. Generates a pixel position value, recognizes letter and number information recognized in the square pixel using an OCR model, connects all adjacent square pixels based on the generated square pixel position value, and connects the connected square Including a document analysis device 100 for matching and displaying character and number information recognized through the OCR model at a pixel position on the image,
The document analysis device 100 calculates a confidence score of recognized letters and numbers based on a recognition rate using an OCR model, and reflects the calculated confidence score to the display information to be visually displayed,
The confidence score is an OCR-based document analysis system using self-reliability information, characterized in that the reconstruction rate calculated by performing at least one of a form, a shape, and a location using a correction model is additionally reflected. The method of claim 1,
The document analysis device 100 displays a normal recognition area, an error-prone area including an incorrect area and a correction area as visualization information of different colors according to a confidence score, an OCR-based document using self-reliability information. Analysis system. delete An input unit 110 for receiving an image to be recognized;
In the received recognition target image, by using an object detection model, the position of at least one of a type, a letter, and a number object is detected, a square shape is displayed around the detected type, letter and number object, and the displayed square pixel position An object detection modeling unit 130 that generates a value;
An OCR modeling unit 140 for outputting letter and number information recognized within the rectangular pixel by using an OCR model;
As the recognized character is corrected with the character information of the item DB 300, a pixel position on an arbitrary rectangle having numeric information is used as a starting position and moves in the left direction and upward direction, but when the character information is searched, all searched during movement A form configuration modeling unit 150 that connects square pixels and matches the corrected letter and number information through the OCR model to the positions of the connected square pixels to be displayed;
A reliability evaluation unit 160 for calculating a confidence score of the recognized letters and numbers using a correction model, reflecting on the display based on the calculated confidence score to be visually displayed; And
OCR-based document analysis system using self-reliability information including a database 170 for storing the generated square pixel position value, recognized letter, number information, trust information, and a form of document data used by a specific institution . The method of claim 4,
An item DB generation unit 120 for generating item DB 300 information for a preset character from an arbitrary document with respect to the character included in the recognition target image; further includes,
The form configuration modeling unit 150 matches the detected text object with item DB 300 information, and when the recognized text is corrected according to the matching result, the corrected text is reflected. OCR-based document analysis system using reliability information. The method according to claim 4 or 5,
The form configuration modeling unit 150 uses self-reliability information, characterized in that the color of the rectangular box of letters and numbers is displayed in different colors between a normal recognition area and an error area including a wrong area and a correction area. OCR-based document analysis system. a) The document analysis device 100 detects the location of at least one of an arbitrary format, letter, and number object using an object detection model in the received recognition target image, and a rectangle around the detected format, letter and number object Displaying a shape to generate a square pixel position value;
b) outputting, by the document analysis device 100, character and number information recognized within the detected rectangular pixels using the OCR model;
c) The document analysis device 100 uses the generated square pixel position value and the position of a random square pixel having numeric information based on the recognized letter and number information as a start position, and the left direction and the upper direction Moving to, but if the text information is searched, connecting all the square pixels searched during the movement, and matching the text and number information recognized through the OCR model to the connected square pixel positions to be displayed; And
d) calculating, by the document analysis device 100, a confidence score of the recognized letters and numbers using a correction model, reflecting on a display based on the calculated confidence score, and visually displaying the confidence score including; OCR-based document analysis method using information. The method of claim 7,
In the step c), the document analyzing apparatus 100 generates an item DB 300 defining character (item) information used in an arbitrary document in order to compare it with the character information recognized in the recognition target image; OCR-based document analysis method using self-reliability information, characterized in that it further comprises. The method of claim 8,
The step c) further comprises the step of matching the detected text object with item DB 300 information, and correcting the recognized text according to the matching result. OCR-based document analysis method using. The method of claim 9,
The correction model in step d) calculates a confidence score based on the reconstruction rate according to the correction of at least one of form, shape, and location, and whether the corrected characters are reflected according to the matching result with the item DB 300 information. OCR-based document analysis method using self-reliability information, characterized in that.

Images