JPH07220023A - Table recognition method and apparatus thereof - Google Patents

Abstract

(57) [Summary] (Correction) [Purpose] The objective is to improve the method and apparatus for recognizing fields in a table from image data in the table. Further, there is provided a method and apparatus for recognizing characters in a field by correcting the position of the image data even if the field position of the image data fluctuates and properly recognizing the position. [Structure] Outer frame lines of a table are extracted from the image input in step 101. Step 1 based on this outline
In 03, the format data is sequentially read from the format information database 31, and the four corners of the format are associated with the four corners of the outer frame, whereby the ruled lines represented by the ruled line information 32 are coordinate-converted on the image and the matching degree with the image evaluate.
In step 106, it is determined from the evaluation results which one of the plurality of formats. In step 108, the position of the ruled line is corrected, and the ruled line is deleted from the image. In step 109, the field position is set from the ruled line information and the field information 33, the characters in the field are recognized in step 110, and the result is output to the printer 70.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a table recognition method and apparatus for recognizing a table field and characters from a table image data, for example, an optical reading apparatus such as a scanner for reading the table format data written on a paper. Relates to a table format data automatic input system for recognizing the table structure and the characters in the table by reading it as image data in a computer.

[0002]

2. Description of the Related Art A form OCR is generally used as a conventional technique for inputting data written on paper. This uses a sheet of paper in which an entry frame is shown by using a color that drops out when the scanner is reading. When reading with a scanner, the entry frame is dropped out, and only the data entered in the frame is taken out, and the data entry position is determined by the relative position from the mark written around the form.

Along with the widespread use of FAX in recent years, a system for inputting the above-mentioned forms using FAX has been developed. However, with FAX, it is often difficult to drop out only a predetermined color. In such a system, an unspecified number of FAX
Since various types of image data are transmitted from
It is not possible to use the dropout method only for a predetermined form.

Therefore, a method has been adopted in which it is assumed that the entry frame exists at a predetermined position in the input image, and the data is erased from the image and the data in the field is extracted.

In addition, there is a method of analyzing the format of an image and determining which format it is in when a plurality of formats are mixed and input. For example, it is a form recognition device disclosed in Japanese Patent Application No. 1-49800.

In these methods, elements of ruled lines or field entry frames are extracted from the image, assembled to estimate the format, and it is checked which of the specified formats is applicable.

However, many noises are present in the input image, and there are many cases in which ruled lines are discontinued. Therefore, it is not always possible to estimate the format satisfactorily, and it also takes processing time.

[0008]

At present, a problem with such a system is that the data writing position also changes non-linearly due to unevenness and distortion of the paper conveyance speed.

In order to solve this problem, it is necessary to correct the writing position of the data shown in the format while detecting the non-linear shift of the field. In addition, this problem is a problem that may occur when any image reading method is used when the image is optically read into the computer.

As a method of correcting such a position shift of the field, the position of the reference point of the format is detected, the approximate position of the field is set from this shift, and the shift of the field position is determined from the peripheral image of the field. There is a correction method.

However, in such a system, the number 1 is written near the ruled line of the field,
There is a problem that the field position is erroneously determined, which causes a reduction in recognition rate.

It is an object of the present invention to improve a method and apparatus for recognizing a table field from a table image data.
Further, the present invention corrects the position of the field even if the field position of the input image data of the table changes non-linearly due to, for example, unevenness or distortion of the paper conveyance speed. It is an object of the present invention to provide a table recognition method and apparatus capable of appropriately recognizing characters with high accuracy and recognizing characters in a field.

[0013]

According to the present invention, format information indicating the format of a table is prepared in advance, the ruled line indicated by the format information is converted into coordinates on the image data, and the coordinate-converted ruled line is obtained. The image data and the image data are used to detect the non-linear positional deviation of the image as described above. Then, the position of the ruled line is corrected based on the detected positional deviation. In the coordinate conversion of the format information, a reference line for setting a rough position of the format of the table is extracted from the image, and based on this, the coordinate is converted into coordinates on the image.

After applying the rough format on the image, the element of the frame line on the image is extracted and the format is associated with it. Further, by correcting the format according to the frame line and deleting the frame line, it is possible to prevent noise from being mixed in the character pattern.

The format is set by relative coordinates based on two reference lines each for vertical and horizontal,
It is possible to set format information that does not depend on the reading line density of the scanner even for the paper size.

Further, correction is performed not for each field but for each ruled line to prevent erroneous extraction of a field position due to characters in the field.

When a plurality of formats are identified by the above method, format information corresponding to the number of formats is required. Therefore, it may be necessary to prepare a large number of formats because one part of the formats is different. In that case, by increasing the number of formats by holding only the formats of different parts in another database.

[0018]

By using the present invention, it is possible to extract the data entry position of a form with high accuracy, and reduce erroneous recognition of field extraction (recognition of field position) caused by non-linear distortion of an image. it can.

Even when a plurality of formats are mixed,
High-speed and highly accurate format determination is possible.

Further, even if a part of the formats is different, it is not necessary to prepare a large number of formats, the processing amount for format matching is reduced, and the processing speed is increased.

[0021]

Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 is a system configuration of a table recognition system according to the first embodiment of the present invention. The system of this embodiment includes a CPU 10, a memory 20, a hard disk 30, a hard disk controller 40, a terminal 50, a terminal controller 51, a scanner 60, a disk device 61, an image input controller 62, a printer 70, a printer controller 71, and a system bus 80. I have it.

A table recognition processing program 21 and a data storage area 22 are provided in the memory 20. CP
The U10 manages and controls the entire system and executes the table recognition processing program 21. Hard disk 30
The format information of the table to be recognized is stored therein as the format information database 31. The hard disk controller 40 controls the hard disk 30, and the hard disk 30 is controlled by the CPU 10.
Input and output the data of.

From the terminal 50, a human gives a processing instruction and checks the display of the result. The terminal 50 is controlled by the terminal controller 51, whereby input from the keyboard and display on the display are performed. Scanner 6
An image is input from 0, the disk device 61, or the like through the image input controller 62. The input image data is stored in the data storage area 22 of the memory 20.

Data is sent to the printer 70 through the printer controller 71, and the recognition result is output.
The system bus 80 is for communicating various data.

FIG. 2 is a flowchart showing the procedure of table recognition processing. First, the overall processing flow will be described.

The image data of the table is read from the scanner 60 or the disk of the disk device 61 and stored in the data storage area 22 on the memory 20 (step 101).
Next, the read image data is searched to extract the outer frame line of the table (step 102). In this embodiment,
This outer frame is used as a reference line for format conversion. The process of step 102 will be described later in detail with reference to FIG.

Next, the matching degree between the format ruled line and the input image is evaluated (step 103). This is the following processing. First, format information database 3
One format information is read from 1. One piece of format information consists of ruled line information 32 and field information 33 representing the format of one table, which will be described later in detail with reference to FIGS. The ruled line obtained from the ruled line information 32 of the read format information is matched with the ruled line on the input image, the degree of coincidence is calculated based on the ratio of the corresponding number of ruled lines to the total number of ruled lines, and stored in the memory 20. . The process of step 103 is shown in FIG.
It will be described in detail later using 1.

Next, it is determined whether or not step 103 has been executed for all the format information stored in the database 31 (step 104). If step 103 has been executed for all the format information, go to step 106, otherwise go to step 10.
Returning to step 3, the next format information is read out and the evaluation of the degree of coincidence of the image is continued.

Next, from the evaluation results (evaluation values obtained by the same number as the number of formats), it is judged which of the plurality of formats the input image (table) is (step 106). The determination process of step 106 will be described later in detail with reference to FIG.

In step 106, it is judged whether or not there is a matching format (step 107). If there is no matching format, the processing is terminated without performing the recognition processing after step 108. If there is a matching format, the recognition processing from step 108 onward is performed using that format.

First, ruled line information 3 of the corresponding format
2 is read out, the ruled lines are matched with the ruled lines of the input image data, the position of the ruled lines is corrected, and the ruled line portion is deleted from the image data (step 108). The process of step 108 will be described later in detail with reference to FIG.

Next, a field position is set from the ruled line correction result and the field information 33 of the corresponding format (step 109). The process of step 109 will be described later in detail with reference to FIG.

Further, characters are recognized from the image in each field to be recognized, and the recognition result is output to the printer 70 (step 110). The process of step 110 will be described later in detail with reference to FIG.

The above is the outline of the table recognition process.

3 and 4 show examples of table format patterns recognized by the table recognition system of this embodiment.

These formats are separately stored in the format information database 31 as ruled line information 32 and field information 33 for each format. The ruled line information 32 is information on the positions of the ruled lines forming the table. The position information is represented by an orthogonal (xy) coordinate system in which the upper left of the table area is the origin (0,0) and the lower right is (10000,10000). Field information 33
Is composed of ruled lines at the top, bottom, left and right of a field which is an area surrounded by ruled lines, and information such as the type of the area.

FIG. 5 shows the contents of the ruled line information 32.

The ruled line information 32 is divided into vertical line information 32-a and horizontal line information 32-b and stored.

Both the vertical line information 32-a and the horizontal line information 32-b have a ruled line number 501-a, 5 for each ruled line.
01-b and the coordinates 502-a, 50 of the starting point of the ruled line
2-b and coordinates 503-a and 503-b of the end point. Further, the vertical line information 32-a has a horizontal line number 504-a that intersects at the start point of the vertical line and a horizontal line number 505-a that intersects at the end point of the vertical line. In the horizontal line information 32-b, a vertical line number 504-b intersecting at the starting point of the horizontal line,
It has a vertical line number 505-b that intersects at the end of the horizontal line.

The numbers of vertical lines and horizontal lines are stored in vertical line number information 506-a and horizontal line number information 506-b, respectively.

FIG. 6 shows the contents of the field information 33.

The field information 33 includes a field number 601, which is a number for specifying a field, a field upper line (horizontal line above the field) number 602, and a field underline (horizontal line below the field) number 60.
3, a field left line (vertical line on the left side of the field) number 604 and a field right line (vertical line on the right side of the field) number 605 are stored. These numbers 602 to 605 are represented by the ruled line numbers 501-a and 501-b of the ruled line information 32 in FIG. That is,
The position of the field is identified based on the ruled line information of four ruled lines surrounding the field.

Further, the field information 33 stores a field type 606, a field title 607, and a field number 608.

The field type 606 defines the contents of the field. The field type 606 is used to perform classification when storing the recognition result in the database,
You can define the attributes of the field (only numbers, only certain restricted characters, etc.). The field title 607 is a title used for displaying the result of recognition of the field so that a person can understand what the content of the field is.

FIG. 7 is a detailed flowchart of the outer frame extraction processing in step 102 of FIG. The contents of the process will be described below step by step.

Step 701: In order to extract the position of the vertical line from the image, the black pixels in the image data are projected in the vertical direction (y-axis direction). This projection is performed by taking the total number of black pixels in the y-axis direction for all x coordinates.

Step 702: All peak points (x coordinates) where the projection values obtained in step 701 are equal to or more than a predetermined threshold value are obtained. There will be vertical lines around the calculated peak point. Further, among the obtained peak points, the peak point having the smallest x coordinate is obtained. The left line of the outer frame exists around the peak point. Further, of the obtained peak points, the peak point with the maximum x coordinate is obtained. The right line of the outer frame exists around the peak point.

However, since noise may exist around the image, it is assumed that the peak point extraction range must be separated from both ends of the image by a certain distance or more. As this value, an appropriate value is set depending on the image input device, the size of the table in the input data, and the like.

Step 703: For each of the left and right peak points (x coordinates) corresponding to the outer frame lines obtained in step 702, a range of constant values in the left and right direction from the peak point is a vertical line, that is, the left and right outer frames. The range (value of x coordinate) where the vertical line exists.

Step 704: An image is searched for from the left side (smaller x-coordinate) within the range where the outer frame vertical line on the left side set in step 703 exists for each value of y-coordinates, and white is first searched. Find all the coordinates of the change points that change from pixels to black pixels. Similarly, for the right outer frame vertical line, an image is searched for from the right side (the one with the larger x coordinate) within the range where the right outer frame vertical line set in step 703 exists, and white pixels are first changed to black. Find all the coordinates of the change points that change to pixels.

Step 705: For each of the left outer frame vertical line and the right outer frame vertical line, approximate straight lines are obtained for all the coordinates of the change points obtained in step 704.

The coordinates of the change point are (x [i], y [i])
If (0 ≦ i <N (= number of change points)), the approximate straight line is obtained by the following mathematical formula 1.

[0054]

[Equation 1]

By the above steps 701 to 705, the left and right outer frame vertical lines in the table of the input image are obtained.

Step 706: In order to extract the position of the horizontal line from the image, the black pixels in the image data are projected in the horizontal direction (x-axis direction). This projection is performed by taking the total number of black pixels in the x-axis direction for all y coordinates.

Step 707: All the peak points (y coordinates) where the projection values obtained in step 706 are equal to or more than a predetermined threshold value are obtained. A horizontal line exists around the obtained peak point. Further, among the obtained peak points, the peak point having the smallest y coordinate is obtained. The outer line of the outer frame will exist around the peak point. Further, among the obtained peak points, the peak point having the largest y coordinate is obtained. The underline of the outer frame exists around the peak point.

However, since noise may exist around the image, it is assumed that the extraction range of the peak points must be apart from both ends of the image by a certain distance or more. As this value, an appropriate value is set depending on the image input device, the size of the table in the input data, and the like.

Step 708: For each of the upper and lower peak points (y-coordinates) corresponding to the outer frame line obtained in step 707, a horizontal line represents a range of constant values in the vertical direction from the peak point, that is, upper and lower outer frame horizontal lines. Is a range (value of y coordinate) in which exists.

Step 709: An image is searched from the upper side (smaller y-coordinate) within the range where the upper outer frame horizontal line set in step 708 exists for each x-coordinate value, and the white pixel is first searched. Find all the coordinates of the change point that changes from black pixel to black pixel. Similarly, for the lower outer frame horizontal line, an image is searched from the lower side (larger y coordinate) within the range where the lower outer frame horizontal line set in step 708 exists, and the white pixel is first searched. Find all the coordinates of the change points that change to black pixels.

Step 710: For each of the upper outer frame horizontal line and the lower outer frame horizontal line, approximate straight lines are obtained for all the coordinates of the change points obtained in step 709.

The coordinates of the change point are (x [i], y [i])
If (0 ≦ i <number of change points), the approximate straight line is obtained by the following mathematical formula 2.

[0063]

[Equation 2]

By the above steps 706 to 710, the upper and lower outer frame horizontal lines in the table of the input image are obtained.

8 and 9 show steps 701 to 701 of FIG.
7 illustrates a state in which outer frame vertical lines are extracted from an input image by the procedure of 705. Since the extraction of the horizontal line is also performed by the same procedure as the extraction of the vertical line, the description will be omitted.

FIG. 8 shows the result of obtaining the range where the vertical line of the outer frame exists from the image data.

Table data is stored in the image data 801. This image 801 is an orthogonal coordinate system with the upper left as the origin, the x axis in the right direction, and the y axis in the downward direction. 80
2 is the result of projecting the black pixels of this image in step 701 of FIG. 803 is a threshold value for determining the peak point in step 702. 804 is the leftmost peak position. The range from the dotted line 805 to the dotted line 806 is the range in which the outer frame left line obtained from the peak point 804 on the left and right exists. This range is set in step 703. A search is made during this period to find the left line in the table.

FIG. 9 shows how to find the left line of the outer frame.

For each line of pixels (for each y coordinate) in the range between the dotted lines 805 and 806 in FIG. 8, the pixels are traced rightward from the position of the dotted line 805. Then, the change point from the white pixel to the black pixel that is found first is set as the position of the outer frame. This position corresponds to step 704 in FIG.
Required by. By approximating the point found in each line with the obtained straight line by the above-mentioned mathematical formula 1, the left line 90
1 is obtained.

FIG. 10 shows the results obtained by obtaining the four lines above, below, left and right of the outer frame from the image 801 of FIG.

Since the upper, lower, left, and right lines of the outer frame of the table were obtained, the outer frame of the table is the intersection of the two horizontal lines (upper line, underline) and the two vertical lines (left line, right line). Express. The coordinates of the intersection 1001 on the upper left of the table are (xlu, ylu), and the intersection 10 on the upper right
The coordinate of 02 is (xru, yru), and the intersection 1003 at the lower left
Is (xld, yld), and the coordinate of the lower right intersection 1004 is (xrd, yrd).

FIG. 11 is a detailed flowchart of the format matching degree evaluation processing in step 103 of FIG.
For format matching, each ruled line that makes up the format table is sequentially matched with the input image to check if they exist in the image, and find the ratio of the number of existing ruled lines to all the ruled lines of the format. Get at. The contents of the process will be described below for each step.

Step 1101: Counter CRT and E
Clear RR to 0. The CRT is a counter for counting the number of ruled lines in the ruled line information 32 when they exist in the input image, and the ERR is a counter for ruled lines that did not exist in the input image.

Step 1102: One ruled line is read from the ruled line information 32 of the matching target format. As for the ruled line information 32, as shown in FIG. 5, the upper left coordinate of the table is (0, 0) and the lower right coordinate is (10000, 10000).
The coordinate system has the coordinates of the start and end points of the ruled line. On the other hand, in step 102 (FIG. 7) of FIG. 2, since the outer frame of the table is obtained in the coordinate system in which the upper left of the image 801 is (0, 0) as shown in FIG. 10, the ruled line is based on this. Information 32
The coordinate of the ruled line read from is converted.

When the coordinates of the start and end points of the ruled line indicated by the ruled line information 32 are converted into the coordinates in the image based on the coordinates of the four corner points in the table in the image, the following formula 3 is obtained.

[0076]

[Equation 3]

Step 1104: The coordinate-converted ruled line is
Matches the input image and obtains the evaluation value. Details of the matching processing will be described later with reference to FIG.

Step 1105: If the result of step 1104 is that a ruled line exists, step 11
Proceed to 06. Otherwise, it is determined that there is no ruled line and the process proceeds to step 1107.

Step 1106: Ruled line information 32 in the image
Since there is a ruled line, the counter CR for the number of ruled lines that existed
Add 1 to T.

Step 1107: Ruled line information 32 in the image
Since there is no such ruled line, 1 is added to the counter ERR for the number of ruled lines that does not exist.

Step 1108: It is determined whether or not matching has been completed for all ruled lines in the ruled line information 32. If not completed, the process returns to step 1102 and the next ruled line is similarly processed. When the matching is completed for all ruled lines, the process proceeds to step 1109.

Step 1109: The ratio of the number of ruled lines existing in the image to all the ruled lines of the format is set as the evaluation value for format matching.

FIG. 12 shows a detailed flowchart of the matching process between the ruled line and the image in step 1104 of FIG. The matching of ruled lines and images is performed by extracting a short black run near the ruled line that is perpendicular to the ruled line on the image, calculates the position of the ruled line on the image based on this black run, and determines the black run that forms this ruled line. Is determined by whether or not the number of The processing contents of each step will be described below.

Step 1201: It is determined whether the ruled line to be matched is a vertical ruled line. If it is a vertical ruled line, step 12
Go to 02. If not, the line is a horizontal ruled line, and the process proceeds to step 1212.

Below, in the case of vertical ruled lines, step 1
The contents of the processing from 202 to step 1211 are shown in FIG.
3 and FIG. 14 will be described. FIG. 13 shows an image 801.
The part around the ruled line is shown. Reference numerals 1304 and 1305 denote black pixel portions in the image, 1304 denotes a ruled line, and 1305 denotes a character. FIG. 14 shows the enlarged contents of the portion 1306 of FIG.

Step 1202: A ruled line in the format to be matched (a ruled line read from the ruled line information 32, which will be referred to as a format ruled line hereinafter) is set in the image. Reference numeral 1301 in FIG. 13 indicates a format ruled line set in the image. Then, with the format ruled line 1301 as the center, a region having a constant width sandwiched between the dotted lines 1302 and 1303 is set. Within this range, the ruled line in the image is searched.

Step 1203: Ruled line 1 of the image in FIG.
From the horizontal black runs that make up 304 and characters 1305,
Range set in step 1202 (dotted lines 1302 to 1
(Range of 303), a black run having a length equal to or less than the threshold value γ is extracted.

In FIG. 14, the black run filled in a black band is extracted as a black run having a length equal to or less than the threshold value γ.
The black runs 1304-2 and 1304-3 were not extracted because the length was equal to or greater than the threshold value γ.

Step 1204: The centroids (1402-1, 1402-2, etc.) of all the black runs (1304-1, 1305-1, etc.) extracted in step 1203 are obtained, and the centroid positions (as many as the number of black runs) are obtained. And the format ruled line 1301. The shift amount is 0 when the center of gravity is on the format ruled line 1301 and is the distance to the format ruled line 1301 in pixel units when the center of gravity is on the right of the format ruled line 1301.
If the position of the center of gravity is to the left of the format ruled line 1301, the distance to the format ruled line 1301 in units of pixels is -1.
It is the value multiplied by.

Step 1205: Regarding the deviation amount of the black run obtained in Step 1204, with respect to the deviation amount value,
A histogram 1403 of the number of black runs is created.

Step 1206: The amount of deviation 1404 that maximizes the value of the histogram created in step 1205.
Ask for.

Step 1207: A range 1407 having a constant width sandwiched by dotted lines 1405 and 1406 is set around the position where the format ruled line is displaced by the amount of shift obtained in step 1206.

Step 1208: Of the black runs having a length equal to or smaller than the threshold value γ extracted in step 1203, only the runs including the center of gravity within the range set in step 1207 are extracted.

Step 1209: If the number of black runs extracted in Step 1208 is larger than the constant δ (<1) times the length (the number of pixels) of the pixel of the format ruled line as a unit, go to Step 1210. move on. Otherwise, proceed to step 1211.

Step 1210: It is judged that there is a ruled line corresponding to the format ruled line in the image.

Step 1211: It is judged that there is no ruled line corresponding to the format ruled line in the image.

If the ruled line to be matched is a horizontal ruled line in step 1201, instead of steps 1202 to 1208, steps 1212 to 1
218 is executed. Step 1212 to Step 1
Since step 218 can be similarly realized by performing the processing from step 1202 to step 1208 in the coordinate system in which the x coordinate and the y coordinate are exchanged, the description thereof will be omitted.

FIG. 15 is a detailed flowchart of the format judgment processing in step 106 of FIG. The format having the largest evaluation value of the format matching degree is the format corresponding to the image. If the same evaluation value is obtained, the format with a large number of ruled lines is set as the format corresponding to the image. However, if the evaluation value is smaller than the threshold value β, it is determined that there is no applicable format and an error occurs. Each step will be described below.

Step 1501: The numbers of each format are sorted in descending order by the number of ruled lines.

Step 1502: Initialize corresponding format number fmt_no = 0 and comparison format number i = 1.

Step 1503: The evaluation value of the coincidence degree of the i-th format is compared with the evaluation value of the coincidence degree of the fmt_no-th format. The i-th evaluation value is
If it is larger than the fmt_no-th evaluation value, the process proceeds to step 1504. Otherwise, it proceeds to step 1505.

Step 1504: Set the corresponding format number fmt_no = i.

Step 1505: Add 1 to the value of i.

Step 1506: It is checked whether the value of i has become the same as the total number of formats. If the same,
Since the comparison has been made for all the formats, the process proceeds to step 1507. Otherwise, proceed to step 1503 to compare with the next format.

Step 1507: It is judged whether the evaluation value of the fmt_no-th format is equal to or larger than the threshold value β. If it is greater than or equal to the threshold β, the process proceeds to step 1508. Otherwise, proceed to step 1509.

Step 1508: Assuming that the corresponding format exists, the format number fmt_no is set.

Step 1509: Assuming that the corresponding format does not exist, a matching error, fmt_no
= -1 is set.

FIG. 16 is a detailed flowchart of the ruled line correction and ruled line erase processing from the image in step 108 of FIG. For each ruled line, the black run extracted in steps 1208 and 1218 of the flowchart shown in FIG. 12 is erased, and the approximate straight line of the center of gravity is obtained to correct the position of the ruled line in accordance with the image. The contents of each step will be described below.

Step 1601: Ruler number counter i
Is initialized to i = 0.

Step 1602: Match the i-th format ruled line with the image. This is performed by calling the format ruled line and image matching process of FIG. 12 as in step 1104 of FIG.

Step 1603: If there is a corresponding ruled line in the image as a result of matching, step 1604
Proceed to. Otherwise, proceed to step 1606.

Step 1604: FIG. 17 is a diagram showing how the position of the ruled line is corrected. In step 1604, as shown in FIG. 17, an approximate straight line 170 passing through the black run extracted by the matching between the format ruled line and the image.
Ask for 1. The approximate straight line 1701 represents the center of gravity of all the extracted black runs as (x [i], y [i]) (0 ≦ i <the number of black runs).
Then, in the case of a vertical line, the above Equation 1 is used, and in the case of a horizontal line, the above Equation 2 is used to obtain each.

Step 1605: The black run extracted by the matching between the ruled line and the image in step 1602 is erased from the image to erase the ruled line from the image.

Step 1606: Add 1 to the ruled line number i.

Step 1607: If the ruled line number i is smaller than the total number of ruled lines, the process proceeds to step 1602 to correct the next ruled line and erase the ruled line from the image. Otherwise, proceed to step 1608.

Step 1608: For all the ruled lines, the intersections with the ruled lines intersecting at the start and end points are obtained, and the coordinates of the start point and the end point are updated as the corrected ruled lines.

FIG. 18 is a detailed flowchart of the field position correction processing in step 109 of FIG. The position where the field exists is calculated from the ruled lines on the top, bottom, left and right of the field. Each step will be described below.

Step 1801: Field data is sequentially read from the field information 33 of the format corresponding to the image in the format information database 31.

Step 1802: Field information 33
Is the upper, lower, left, and right 4 surrounding the field as shown in FIG.
Because it is composed of ruled lines of a book, based on the number of ruled lines,
The information of the ruled line corrected in step 108 is read. Two horizontal lines, an upper line and an underline of the read field,
The intersection of two vertical lines, the left line and the right line, is obtained, and this is used as field position information.

Step 1803: Coordinates of four corners of the field obtained in Step 1802 (upper left, upper right, lower left,
(Lower right) is output as field position information 35.

FIG. 19 shows the format of the field position information. A field number for identifying the field is stored in 1901. In 1902, the upper left coordinate on the image of the field is stored. In 1903, the upper right coordinates of the field on the image are stored. 190
The lower left coordinate on the image of the field is stored in 4. In 1905, the lower right coordinates on the image of the field are stored.

Step 1804: If there is a field for which the field position has not yet been obtained, the process returns to step 1801 to process the next field. If not, the processing for setting the field position ends.

FIG. 20 is a detailed flowchart of the in-field character recognition processing in step 110 of FIG. The character pattern (image) included in the field obtained in step 109 is extracted, and the character is recognized. Each step will be described below.

Step 2001: A connected component of black pixels is extracted from the image. Various methods have been publicly known for extraction of connected components of black pixels, and therefore description thereof will be omitted.

Step 2002: The field number counter i is initialized to i = 0.

Step 2003: Field position information 3
From 5 (FIG. 19), the field position of the i-th field is read.

Step 2004: From the black pixel connected components extracted in step 2001, only the connected components included in the field are extracted. Furthermore, these connected components are integrated to create a circumscribing rectangle of a character, and a character pattern is extracted.

Step 2005: Character recognition is performed on the character pattern extracted in step 2004.

Step 2006: The recognition result is sent to the printer 7.
Output to 0.

Step 2007: Add 1 to the format number counter i.

Step 2008: If the value of the format number counter i is smaller than the total number of fields, the process returns to step 2003 for character recognition of the next field.
If not, the character recognition process ends.

Next, as a second embodiment, a table analysis method in the case where some fields of the analyzed table are further divided by ruled lines will be described.

FIG. 21 is a diagram showing the flow of processing when one field in the format is further divided by ruled lines.

Reference numeral 2101 is the same table format as the format example 1 in FIG. This format 2101
The field 2102 has a format (hereinafter, referred to as a field division format) for further dividing this field.

Fields 2103 and 2104 are field 210.
2 is the upper line and the lower line, and is the ruled line of the numbers 5 and 6 of the horizontal line information 32-b, respectively. 2105, 2
Similarly, 106 are left and right lines of the field 2102, and are ruled lines of numbers 8 and 9 of the vertical line information 32-a, respectively.

Reference numeral 2107 is a field division format corresponding to this field 2102. Each ruled line forming the field division format 2107 has a line number in this format, and field information is created based on this line number. This field division format 2107 is used for the field 21
02, the ruled line 2108 becomes the field 21
02 corresponding to the field top line 2103. Similarly, the ruled line 2109 is the field underline 21 of the field 2102.
04, the ruled line 2110 is the field left line 2105 of the field 2102, and the ruled line 2111 is the field 2102.
Corresponding to the field right line 2106 of each.

2112, 2113, and 2114 are added as new ruled lines. As a result, two horizontal lines 2116 and 2117 and one vertical line 2118 are added as ruled lines, respectively, as shown in the synthesis result. The fields increase with the addition of these ruled lines.

FIG. 22 shows a case where the above processing is performed.
This is a modification of the flowchart of FIG. Between the field position setting (step 109) and the in-field character recognition (step 110), step 2201 of determining the field division format and changing the format information is added.

In step 2201, the field division format database 36 is set for all formats.
To field split format data (eg
2107) of No. 1), the presence / absence of field division is checked, and addition / change processing of the field division format to the format information of the entire table is performed. As a result, the field for the divided format is also reflected in the field position information 35. Then in step 110,
Character recognition processing of each field is performed based on the format information after the addition and modification processing of the field division format to the format information of the entire table.

FIG. 23 is a detailed flowchart of the field division determination and field division format addition / modification processing to the format information of the entire table.

FIG. 24 shows an example of the field division format. Similar to the format information of the entire table, these field division formats have (0,0) at the upper left,
It is expressed as format information in the coordinate system whose lower right is (10000, 10000).

FIG. 25 shows the contents of these field division formats. The field division format information 37 stores a field type 37-1, which is a target of the field division format, ruled line information 37-2 of the field division format, and field information 37-3 of the field division format.

The field type 37-1 is represented by the field type 606 of the field information 33 in FIG. That is, for example, in the field information 33 of FIG. 6, if the field type 606 of a certain field is p,
If the field division format information 37 of FIG. 25 stores information in which the field type 37-1 is p, the field is field-divided based on the corresponding field division format information. The ruled line information 37-2 is the ruled line information 32 shown in FIG.
It has the same contents as. The field information 37-3 is shown in FIG.
It has the same contents as the field information 33 shown in FIG.

The processing contents will be described below with reference to FIG.

Step 2301: Since the field division formats are sequentially checked, the counter i designating the number of the field to be checked is initialized to i = 0.

Step 2302: The field type (606 of FIG. 6) of the field of the field number i is taken out, and the field division format information 37 having the field type 37-1 having the same value as the value of the field type 37
From the field division format database 36.

Step 2303: If the field division format corresponding to the field type of the field of the field number i to be processed exists in the database,
Proceed to step 2304. If it is not, it means that the field is not field-divided, so the field-division processing is not performed and step 23 is executed.
Go to 10.

Step 2304: When the corresponding format exists, the format judgment is performed for all the field division formats extracted in step 2303 by the same procedure as step 106 (FIG. 15).

Step 2305: As a result of the format judgment, if the applicable format exists, proceed to step 2306. Otherwise, the field division process is not performed and the process proceeds to step 2310.

Step 2306: Correct the position of the ruled line corresponding to the field division format in the field and erase the ruled line from the image by the same procedure as the process of correcting the position of the ruled line and deleting the ruled line from the image shown in the flowchart of FIG. To do.

Step 2307: The ruled line information of the field division format is added to the ruled line information 32 of the format information of the entire table.

FIG. 26 shows a state in which ruled line information in the field division format is added to the ruled line information 32-132-2. Here, it is assumed that the ruled line information of the field division format shown in FIG. 21 is added. Vertical ruled line information 32
New ruled line information 2601 is added to the end of -1. New additional ruled line information 2602 is added to the end of the horizontal ruled line information 32-2. As a result, the number 0 vertical line (21 in FIG. 21) on the field division format is
12) is No. 37 in the format of the entire table (Fig. 21).
2118). Also, the horizontal lines of 0 and 1 in the field division format (2112, 211 in FIG. 21).
3) is number 18 and number 19 (2116 and 2117 in FIG. 21) in the format of the entire table.

Step 2308: Add and change the field information of the field division format to the field information 33 of the entire table. That is, the field information of the field to be divided is replaced with the field information in the field division format. As a result, new field information 33 including the divided fields is created.

FIG. 27 shows how the field information is changed. The field 2702 to be divided in the field information 2701 of the entire table is replaced with the field information 2703 of the field division format. At this time, the ruled line numbers of the top, bottom, left and right in the field division format are converted into the ruled line numbers in the entire table shown in FIG. 26 (2704).

Step 2309: In the same procedure as the field position setting process of FIG. 18, based on the field information 33 newly created in step 2308, the field position information 35 is obtained from the intersection of the upper, lower, left and right ruled lines surrounding the field. Set with the coordinates of the four corners.

Step 2310: Add 1 to the field number counter i.

Step 2311: If the field number counter i is smaller than the number of fields, proceed to Step 2302 for processing the next field. If not, the field division process ends.

In both the first and second embodiments described above, all the ruled lines of the format are used for the format determination, but only the portions that differ from other formats are used for the determination. Therefore, the speed can be increased.

FIG. 28 is a flow chart of a method of creating format determination ruled line information used for such determination.

When there are three formats A, B, and C, first, for format A and format B,
All ruled lines in each format are compared with ruled lines of other formats, and only ruled lines that have no similar length or position are extracted. As a result, a ruled line that distinguishes A and B is obtained. By performing the above processing also for the combinations of A and C and B and C, the ruled lines necessary for judging the respective formats can be obtained. This is obtained as format determination ruled line information 34 for each format.

By replacing the ruled line information 34 for format determination with the ruled line information 32 at the time of format determination,
It is possible to speed up the format determination process.

The steps of FIG. 28 will be described below.

Step 2801: Create all combinations for selecting two formats from a plurality of formats.

Step 2802: The ruled line information in two formats is read from one of the format combinations created in step 2801.

Step 2803: All ruled lines of each format are compared and ruled lines having similar lengths and positions are excluded.

Step 2804: The ruled lines not removed in step 2803 are converted to the ruled line information 3 for format determination.
Store in 4. For example, when formats A and B are compared, the ruled line information of format A that does not correspond to format B is stored in 34-1. Also, 3
4-2, format A with the ruled line information of format B
The ruled lines that do not correspond to are stored.

Step 2805: Steps 2802 to 2804 are executed for combinations of other formats obtained in step 2801 as well, and format determination ruled lines generated in each combination are obtained.

By the above processing, format determination ruled line information is created.

[0169]

According to the present invention, since the non-linear shift of the image is corrected by using the ruled line surrounding the field, highly accurate field extraction can be performed. Further, even if a ruled line is partly broken in the image data, correct field extraction is possible.

By describing the format not as a set of fields but as a set of ruled lines forming a table and a field surrounded by the ruled lines, it is difficult to correct the position on a field-by-field basis. Exists ”
1) etc.) can be eliminated.

If a long ruled line extending over a plurality of fields is used as one ruled line in the ruled line information, it may not be possible to deal with the nonlinear distortion of the image. However, in the present invention, 1
It is possible to divide the ruled lines of a book in the middle and handle them as a plurality of ruled line information. Therefore, even a long ruled line can be sufficiently dealt with by partial linear approximation.

By determining the presence / absence of a line element corresponding to a format, it is possible to determine a plurality of formats without additional information such as ID. In addition, even for a partial variation of the format (for example, when the field has another structure), it is possible to recognize the table only by having the format of the varying portion.

[Brief description of drawings]

FIG. 1 is a system configuration diagram of a first embodiment of the present invention.

FIG. 2 is a schematic flowchart of the entire table recognition process.

FIG. 3 is a diagram showing an example (part 1) of format data.

FIG. 4 is a diagram showing an example (part 2) of format data.

FIG. 5 is a diagram showing the contents of ruled line information in the format information.

FIG. 6 is a diagram showing the contents of field information in format information.

FIG. 7 is a flowchart of an outer frame extraction process.

FIG. 8 is a diagram showing a state of a rough position determination of an outer frame in the outer frame extraction processing.

FIG. 9 is a diagram showing how the outer frame is corrected in the outer frame extraction processing.

FIG. 10 is a diagram showing the extraction result of the entire outer frame in the outer frame extraction processing.

FIG. 11 is a flowchart of a process of evaluating the degree of coincidence between the image and the format

FIG. 12 is a flowchart of matching processing of ruled lines and images.

FIG. 13 is a diagram showing how a search range for ruled lines is set in a ruled line / image matching process.

FIG. 14 is a diagram showing how a ruled line existing range is determined in a ruled line and image matching process.

FIG. 15 is a flowchart of identification processing of image data format.

FIG. 16 is a flowchart of ruled line position correction processing and ruled line removal processing in an image.

FIG. 17 is a diagram showing a state of ruled line position correction processing.

FIG. 18 is a flowchart of a recognition field position setting process.

FIG. 19 is a diagram showing the contents of field position information.

FIG. 20 is a flowchart of a character recognition process in a field.

FIG. 21 is a diagram showing an outline of field division processing.

22 is a diagram showing the addition of field division processing to the flowchart of FIG. 2;

FIG. 23 is a flowchart of field division determination and division processing.

FIG. 24 is a diagram showing an example of a field division format.

FIG. 25 is a diagram showing the contents of field division format information.

FIG. 26 is a diagram showing addition of ruled line information of a field division format to ruled line information of the entire table.

FIG. 27 is a diagram showing addition of a field division format to field information of the entire table.

FIG. 28 is a flowchart of a process of creating format determination ruled line information.

[Explanation of symbols]

10 CPU 20 Memory 30 Hard Disk 40 Hard Disk Controller 50 Input / Output Terminal 51 Terminal Controller 60 Image Scanner 61 Disk 62 Image Input Controller 70 Printer 71 Printer Controller 80 System Bus 31 Format Information Database 32 Format Ruled Line Information 33 Format Field Information 34 Format Judgment Line Minute information 35 Field position information 36 Partial format database

Claims (14)

[Claims] 1. A table recognition method for reading a tabular document composed of ruled lines of a predetermined format as image data using image input means and recognizing the table structure from the image data. A reference line extracting step for extracting the reference line of the table from the table, a coordinate converting step for converting the ruled line of the predetermined format into coordinates on the image data based on the extracted reference line, and the coordinate converted ruled line. And a position correction step of correcting the position of the ruled line on the basis of the detected positional deviation and detecting the positional deviation from the image data. 2. A table recognition method in which a tabular document composed of ruled lines of a predetermined format is read as image data using image input means and the table structure is recognized from the image data. A reference line extraction step of extracting the reference line of the table from the table, and reading ruled line information representing a ruled line of one format from a plurality of prepared format information, and obtaining the degree of coincidence between the read ruled line and the image data. A matching degree evaluation step, a step of performing the matching degree evaluation process on all of the plurality of format information, and obtaining a matching degree corresponding to each of the plurality of formats; and a format having the highest matching degree, Determining a corresponding format corresponding to the image data, ruled lines of the corresponding format and the image A step of detecting a positional deviation from the data and correcting the position of the ruled line based on the detected positional deviation; and a step of obtaining the position of the field surrounded by the ruled line based on the position of the corrected ruled line. A table recognition method characterized by being provided. 3. A table recognition method for reading a tabular document composed of ruled lines of a predetermined format as image data using image input means and recognizing the table structure from the image data. A reference line extraction step of extracting a reference line of the table from the table, and reading ruled line information representing a ruled line of one format from a plurality of prepared format information, and using the read ruled line as a reference with the extracted reference line. A coordinate conversion step of performing coordinate conversion into coordinates on the image data, matching the coordinate-converted ruled line with the image data, and checking whether there is a ruled line corresponding to the coordinate-converted ruled line in the image data. A ruled line search step for determining whether or not the coordinate conversion step and the ruled line search step are performed in the plurality of formats. For each format, the step of obtaining the ratio of the matched ruled line to the total number of ruled lines for each format and setting the ratio as the degree of matching between the format and the image data, and the format with the highest degree of matching, Determining a corresponding format corresponding to the image data, detecting a positional deviation between a ruled line of the corresponding format and the image data, and correcting the position of the ruled line based on the detected positional deviation; And a step of determining the position of the field surrounded by the ruled line based on the position of the subsequent ruled line. 4. The table recognition method according to claim 2, wherein when the highest degree of coincidence does not reach a predetermined threshold value, it is rejected and the subsequent table recognition process is not performed. 5. A ruled line forming a table is divided into partial units forming one field, and the position is corrected for each divided ruled line partial unit.
Table recognition method described in. 6. A ruled line forming a table is divided into unit parts forming one field, a plurality of continuous parts of the divided ruled line parts are combined, and the position is set in units of the combined part. 4. The table recognition method according to claim 1, wherein the correction is performed. 7. The table recognition method according to claim 1, wherein the position is corrected for each ruled line forming the table. 8. For a table having a format in which some of the fields in the table can be split into multiple fields,
The partial format in the field is prepared as field division format information different from the format information, and further, the field division format in the field is determined using the field division format information, The table recognition method according to claim 2 or 3, wherein the obtained field division format is added to the corresponding format, and the added field is recognized in the same manner as other fields. 9. A ruled line for format determination is prepared in advance by extracting characteristic ruled lines that do not appear in other formats from the ruled lines forming the format and preparing this as ruled line information for format determination. The table recognition method according to claim 2, wherein the format is determined by determining the degree of coincidence between the image and the image. 10. A table recognition device for reading a tabular document composed of ruled lines of a predetermined format as image data using image input means and recognizing the table structure from the image data. A reference line extracting means for extracting the reference line of the table from the table, a coordinate converting means for converting the ruled line of the predetermined format into the coordinates on the image data with reference to the extracted reference line, and the coordinate-converted ruled line. A table recognition device comprising: a position correction unit that detects a position shift from the image data and corrects the position of the ruled line based on the detected position shift. 11. A table recognition device for reading a document in a tabular format composed of ruled lines of a predetermined format as image data using image input means and recognizing the table structure from the image data. Storage means for storing format information that defines the format of the format, reference line extraction means for extracting the reference line of the table from the image data, and ruled line information representing a ruled line of one format from the plurality of format information of the storage means And a matching degree evaluation unit that obtains the matching degree between the read ruled line and the image data, and the matching degree evaluation process is performed on all of the plurality of format information to correspond to each of the plurality of formats. The means for obtaining the degree of coincidence and the format with the highest degree of coincidence are determined to be the corresponding format corresponding to the image data. Means for detecting the positional deviation between the ruled line of the corresponding format and the image data, and correcting the position of the ruled line based on the detected positional deviation, and the ruled line based on the position of the corrected ruled line. And a means for determining the position of the field surrounded by. 12. A table recognition device for reading a tabular document composed of ruled lines of a predetermined format as image data by using an image input means and recognizing the table structure from the image data. Storage means for storing format information that defines the format of the format, reference line extraction means for extracting the reference line of the table from the image data, and ruled line information representing a ruled line of one format from the plurality of format information of the storage means And a coordinate conversion unit for converting the read ruled line into coordinates on the image data with the extracted reference line as a reference, and matching the coordinate-converted ruled line with the image data, A ruled line search means for determining whether or not there is a ruled line corresponding to the coordinate-converted ruled line, and the coordinate conversion means and the ruled line The processing of the searching means is performed for all ruled lines of the plurality of formats, and the ratio of the matched ruled lines to the total number of ruled lines is calculated for each format, and the ratio is used as the degree of matching between the format and the image data. Means for determining the format having the highest degree of coincidence as a corresponding format corresponding to the image data, detecting a positional deviation between the ruled line of the corresponding format and the image data, and based on the detected positional deviation, the ruled line And a means for correcting the position of the field surrounded by the ruled line based on the corrected position of the ruled line. 13. A tabular document composed of ruled lines of a predetermined format and characters written in a field surrounded by the ruled lines is read as image data by using an image input means, A method for automatically reading characters for recognizing characters in a table structure and a field, comprising a reference line extracting step of extracting a reference line of a table from the image data, and a single format from a plurality of format information prepared in advance. The ruled line information representing the ruled lines of the image data is read, and the matching score evaluation step of obtaining the matching score between the read ruled lines and the image data, and the matching score evaluation process are performed for all of the plurality of format information. The step of obtaining the degree of coincidence corresponding to each format, and the format having the highest degree of coincidence A step of determining a corresponding format corresponding to the data, a step of detecting a positional deviation between the ruled line of the corresponding format and the image data, and a step of correcting the position of the ruled line based on the detected positional deviation; Characters characterized by including the step of obtaining the position of the field enclosed by the ruled line based on the position of the ruled line, and the step of performing character recognition by extracting the character pattern contained in the field after position correction Automatic reading method. 14. A tabular document composed of ruled lines of a predetermined format and characters described in a field surrounded by the ruled lines is read as image data by using image input means, An automatic reading device for recognizing characters in a table structure and a field, a storage means for storing format information defining a plurality of table formats, and a reference line extraction for extracting a reference line of the table from the image data. Means, read out ruled line information representing a ruled line of one format from the plurality of format information in the storage means, and obtain a match degree between the read ruled line and the image data; and a process of the match degree evaluation. For all of the plurality of format information, and obtaining a matching degree corresponding to each of the plurality of formats. Means for determining the format having the highest degree of coincidence as a corresponding format corresponding to the image data, detecting a positional deviation between the ruled line of the corresponding format and the image data, and based on the detected positional deviation, the ruled line Of the position of the field surrounded by the ruled line based on the position of the ruled line after the correction, and the character pattern included in the field after the position correction for character recognition. A device for automatically reading characters, characterized by comprising: