CN115965989A - Document identification method, device, electronic device and storage medium - Google Patents

Abstract

The embodiment of the application discloses a document identification method, a document identification device, electronic equipment and a storage medium, wherein the method comprises the following steps: when the camera works, acquiring a preview picture; detecting whether a first document rectangle exists in the preview picture according to a first document identification algorithm; detecting a second document rectangle in the shot image according to a second document identification algorithm when a shooting instruction is received and a first document rectangle exists in the preview picture; displaying the second document rectangle in the shot image in an overlapping manner; wherein the computation amount of the first document identification algorithm is smaller than that of the second document identification algorithm, and the accuracy of the first document identification algorithm is lower than that of the second document identification algorithm. By implementing the embodiment of the application, the performance occupied by document identification can be reduced.

Description

Document identification method, device, electronic device and storage medium

technical field

The present application relates to the technical field of image processing, and in particular to a document recognition method, device, electronic equipment and storage medium.

Background technique

Among the existing camera functions, the document recognition function is more important and used more frequently, especially for students, the document recognition function is more important. However, the existing document recognition function cannot perform document recognition when the camera is previewing, or requires a considerable amount of performance of the electronic device to be able to complete the document recognition on the preview screen in real time.

Contents of the invention

The embodiment of the present application discloses a document identification method, device, electronic equipment and storage medium, which can reduce the performance occupied by document identification.

The embodiment of the present application discloses a document identification method, the method includes:

When the camera is working, get a preview image;

According to the first document recognition algorithm, detecting whether there is a first document rectangle in the preview screen;

There is a first document rectangle in the preview screen and when a shooting instruction is received, according to a second document recognition algorithm, detecting a second document rectangle in the captured image;

superimposing and displaying the second document rectangle in the captured image;

Wherein, the calculation amount of the first document recognition algorithm is smaller than that of the second document recognition algorithm, and the accuracy of the first document recognition algorithm is lower than the accuracy of the second document recognition algorithm.

In one embodiment, the preview image includes a multi-frame preview image, and the detecting whether there is a first document rectangle in the preview image according to the first document recognition algorithm includes:

Determining document pixels whose grayscale values satisfy preset conditions in the target preview image; wherein, the target preview image is any frame preview image in the multi-frame preview images;

determining whether a first document rectangle exists in the target preview image according to the denseness of the document pixels in the target preview image;

In the case where it is determined that the first document rectangle exists in the target preview image, the method further includes:

The first document rectangle is superimposed and displayed in the target preview image.

In one embodiment, the determination of document pixels whose grayscale values satisfy preset conditions in the target preview image includes:

Calculate the average gray value of all pixels in the target preview image;

Subtracting the average gray value from the gray value of each pixel to obtain a calculation result corresponding to each pixel;

The pixels whose corresponding calculation results are greater than the preset threshold are taken as the document pixels whose gray value meets the preset condition.

In one embodiment, when the first document rectangle exists in the target preview image, according to the density of the document pixels in the target preview image, in the target preview image Determine whether a first document rectangle exists, including:

determining at least one document rectangle in the target preview image according to the denseness of the document pixels in the target preview image;

Determining the corresponding document pixel density, color contrast and confidence of each document rectangle;

According to the corresponding document pixel point density, color difference contrast and confidence of each said document rectangle, calculate the target result of each said document rectangle by weighting;

The document rectangle with the largest target result among the plurality of document rectangles is used as the first document rectangle.

In one embodiment, the determining the corresponding document pixel density, color contrast and confidence of each document rectangle includes:

Calculate the document pixel density corresponding to each of the document rectangles according to the number of document pixels in each of the document rectangles;

calculating the color difference contrast corresponding to each of the document rectangles according to the color values of the pixels in each of the document rectangles and the color values of the pixels outside of each of the document rectangles;

According to the text information and grayscale information of each of the document rectangles, the confidence corresponding to each of the document rectangles is obtained.

In one embodiment, the determining at least one document rectangle in the target preview image according to the density of the document pixels in the target preview image includes:

Analyzing the denseness of the document pixels in the target preview image by means of a density recognition model, identifying at least one model recognition region whose document pixel density is greater than a density threshold;

At least one document rectangle corresponding to the at least one model recognition area is determined in the target preview image.

In one embodiment, the determining at least one document rectangle corresponding to the at least one model recognition area in the target preview image includes:

According to the leftmost pixel in the target model recognition area, determine the left boundary of the document rectangle corresponding to the target model recognition area; wherein, the target model recognition area is any model in the at least one model recognition area identification area;

Determine the right boundary of the document rectangle corresponding to the target model recognition region according to the rightmost pixel in the target model recognition region;

Determine the upper boundary of the document rectangle corresponding to the target model recognition region according to the uppermost pixel in the target model recognition region;

Determine the lower boundary of the document rectangle corresponding to the target model recognition region according to the lowermost pixel in the target model recognition region;

A document rectangle corresponding to the target model recognition area is determined according to the left boundary, the right boundary, the upper boundary, and the lower boundary.

The embodiment of the present application discloses a document identification device, the device includes:

The obtaining module is used to obtain the preview image when the camera is working;

A first detection module, configured to detect whether a first document rectangle exists in the preview image according to a first document recognition algorithm;

The second detection module is used to detect the second document rectangle in the captured image according to the second document recognition algorithm when the first document rectangle exists in the preview image and the shooting instruction is received;

a display module, configured to superimpose and display the second document rectangle in the captured image;

Wherein, the calculation amount of the first document recognition algorithm is smaller than that of the second document recognition algorithm, and the accuracy of the first document recognition algorithm is lower than the accuracy of the second document recognition algorithm.

In one embodiment, the preview image includes multiple frames of preview images, and the first detection module is further configured to determine document pixels whose grayscale values in the target preview image meet preset conditions; wherein, the target preview image is the Any frame of the preview image in the plurality of frames of preview images; according to the density of the document pixels in the target preview image, determine whether there is a first document rectangle in the target preview image.

The display module is further configured to superimpose and display the first document rectangle in the target preview image.

In one embodiment, the first detection module is further configured to calculate the average gray value of all pixels in the target preview image; subtract the average gray value from the gray value of each pixel to obtain Calculation results corresponding to each of the pixels; pixels whose corresponding calculation results are greater than a preset threshold are used as document pixels whose grayscale value satisfies a preset condition.

In one embodiment, in the case that the first document rectangle exists in the target preview image, the first detection module is further configured to determine according to the denseness of the document pixels in the target preview image At least one document rectangle in the target preview image; determining the corresponding document pixel density, color difference contrast and confidence of each of the document rectangles; according to the corresponding document pixel density, color difference contrast and confidence of each of the document rectangles, Weighted calculation of the target result of each of the document rectangles; taking the document rectangle with the largest target result among the plurality of document rectangles as the first document rectangle.

In one embodiment, the first detection module is further configured to calculate the document pixel density corresponding to each document rectangle according to the number of document pixels in each document rectangle; according to the number of pixels in each document rectangle Calculate the color difference contrast corresponding to each of the document rectangles based on the color value and the color value of each pixel outside the document rectangle; obtain the confidence level corresponding to each of the document rectangles according to the text information and grayscale information of each of the document rectangles .

In one embodiment, the first detection module is further configured to analyze the denseness of the document pixels in the target preview image through a density identification model, and identify at least one model whose document pixel density is greater than a density threshold Identification area: determining at least one document rectangle corresponding to the at least one model identification area in the target preview image.

In one embodiment, the first detection module is further configured to determine the left boundary of the document rectangle corresponding to the target model recognition area according to the leftmost pixel in the target model recognition area; wherein, the target model The recognition area is any model recognition area in the at least one model recognition area; according to the rightmost pixel in the target model recognition area, determine the right boundary of the document rectangle corresponding to the target model recognition area; according to the The uppermost pixel in the target model recognition area, determine the upper boundary of the document rectangle corresponding to the target model recognition area; according to the lowermost pixel in the target model recognition area, determine the corresponding The lower boundary of the document rectangle; according to the left boundary, the right boundary, the upper boundary and the lower boundary, determine the document rectangle corresponding to the target model recognition area.

The embodiment of this application discloses a terminal device, including:

a memory storing executable program code;

a processor coupled to the memory;

The processor invokes the executable program code stored in the memory to execute the method described in any of the foregoing embodiments.

The embodiment of the present application discloses a computer-readable storage medium, where the computer-readable storage medium stores a computer program, wherein, when the computer program is executed by a processor, the processor executes any of the above-mentioned embodiments. described method.

Through the document recognition method, device, electronic device, and storage medium disclosed in the embodiments of the present application, the electronic device can obtain a preview screen when the camera is working, and detect whether there is a first document rectangle in the preview screen according to the first document recognition algorithm. , there is a first document rectangle in the preview screen and when a photographing instruction is received, the second document rectangle in the photographed image is detected according to the second document recognition algorithm, and the second document is superimposed and displayed in the photographed image Rectangular, the calculation amount of the first document recognition algorithm used by the electronic device in the preview image is less than the calculation amount of the second document recognition algorithm in the captured image, so that the electronic device can complete the preview image in real time with low performance For document recognition, the accuracy of the first document recognition algorithm used by the electronic device in the preview screen is lower than the accuracy of the second document recognition algorithm used in the captured image, and the electronic device only uses a more accurate method for the captured image captured by the user. Recognition by advanced algorithms not only improves the user experience, but also reduces the performance occupied by document recognition.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the accompanying drawings that need to be used in the embodiments will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present application. For Those of ordinary skill in the art can also obtain other drawings based on these drawings without making creative efforts.

FIG. 1 is a schematic diagram of an application scenario of a document recognition method disclosed in an embodiment of the present application;

FIG. 2 is a schematic flowchart of a document identification method disclosed in an embodiment of the present application;

Fig. 3 is a schematic flowchart of another document identification method disclosed in the embodiment of the present application;

Fig. 4 is a schematic flowchart of another document identification method disclosed in the embodiment of the present application;

Fig. 5 is a modular schematic diagram of a document identification device disclosed in the embodiment of the present application;

Fig. 6 is a structural block diagram of an electronic device disclosed in an embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of them. Based on the embodiments in this application, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the scope of protection of this application.

It should be noted that the terms "comprising" and "having" and any variations thereof in the embodiments of the present application are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or process that includes a series of steps or units. The apparatus is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to the process, method, product or apparatus.

It can be understood that the terms "first", "second" and the like used in this application may be used to describe various elements herein, but these elements are not limited by these terms. These terms are only used to distinguish one element from another element. For example, a first document recognition algorithm could be termed a second document recognition algorithm, and, similarly, a second document recognition algorithm could be termed a first document recognition algorithm, without departing from the scope of the present application. Both the first document recognition algorithm and the second document recognition algorithm are document recognition algorithms, but they are not the same document recognition algorithm.

The embodiment of the present application discloses a document identification method, device, electronic equipment and storage medium, which can reduce the performance occupied by document identification.

A detailed description will be given below in conjunction with the accompanying drawings.

As shown in FIG. 1, FIG. 1 is a schematic diagram of an application scenario of a document identification method disclosed in the embodiment of the present application. The application scenario may include an electronic device 10 and a document 20. The electronic device 10 may include but is not limited to a mobile phone, a tablet Computers, wearable devices, notebook computers, PCs (Personal Computers, personal computers), etc. In addition, the operating system of electronic device 10 may include but not limited to Android (Android) operating system, IOS operating system, Symbian (Symbian) operating system, Black Berry (Blackberry) operating system, Windows Phone8 operating system, etc., the embodiment of the present application Not limited. Wherein, the camera of the electronic device 10 can be aimed at the document 20 to work, so as to capture an image with the document 20 .

Wherein, when the camera is working, the electronic device 10 acquires a preview image, and then detects whether there is a first document rectangle in the preview image according to the first document recognition algorithm. When the first document rectangle exists in the preview image and a shooting instruction is received, The electronic device 10 detects the second document rectangle in the captured image according to the second document recognition algorithm, and superimposes and displays the second document rectangle in the captured image, wherein the calculation amount of the first document recognition algorithm is less than the calculation amount of the second document recognition algorithm , the accuracy of the first document recognition algorithm is lower than the accuracy of the second document recognition algorithm.

As shown in Figure 2, Figure 2 is a schematic flow chart of a document identification method disclosed in the embodiment of the present application. The document identification method can be applied to the electronic device in the above embodiment, and the document identification method can include the following steps:

Step 210, when the camera works, obtain a preview image.

When the camera of the electronic device is working, the preview image can be acquired and displayed in real time on the display screen of the electronic device. Specifically, the electronic device may receive a control command for calling the camera, and control the camera to work according to the control command, obtain a real-time preview image captured by the camera, and control the display screen to display the real-time preview image captured by the camera in a corresponding area.

Step 220, according to the first document recognition algorithm, detect whether there is a first document rectangle in the preview image.

The electronic device may detect whether a first document rectangle exists in the preview screen according to the first document identification algorithm, and the first document rectangle may be a rectangle including a document area identified by the first document identification algorithm. The first document recognition algorithm is used to identify whether there is a first document rectangle in the multi-frame preview images included in the preview screen. The type of the first document recognition algorithm is not limited, and the first document recognition algorithm can be manually preset.

As an optional implementation manner, the electronic device may determine the first document recognition algorithm among multiple document recognition algorithms according to the performance parameters of the electronic device and the frame rate of the preview image. Wherein, the electronic device may include multiple hardware performance parameters, such as CPU (central processing unit, central processing unit) performance parameters, GPU (graphics processing unit, graphics processing unit) performance parameters, RAM (Random Access Memory, random access memory) performance parameters, etc., the frame rate of the preview image refers to the number of image frames included in each preview image per second.

Wherein, the electronic device may score the performance parameters of the electronic device according to the performance parameters of each hardware to obtain the performance score of the electronic device. According to the performance score, the electronic device can determine the document recognition speed corresponding to each document recognition algorithm. The document recognition speed can be the number of image frames recognized per second when performing document recognition, and then according to the document recognition speed corresponding to each document recognition algorithm As well as the frame rate of the preview image, the first document recognition algorithm is determined from multiple document recognition algorithms. For example, there are document recognition algorithm A, document recognition algorithm B, and document recognition algorithm C in the electronic device. The document recognition speed corresponding to document recognition algorithm A can be 15 frames per second, and the document recognition speed corresponding to document recognition algorithm B can be 15 frames per second. 25 frames, the document recognition speed corresponding to the document recognition algorithm C can be 35 frames per second, and the frame rate of the preview screen is 30 frames per second, then the document recognition algorithm C can be selected as the first document recognition algorithm.

Implementing this embodiment, the electronic device can select the first document recognition algorithm adapted to the preview screen for recognition, so that the performance of the electronic device is sufficient to detect whether there is a first document rectangle in the preview screen, and the image processing efficiency of the electronic device is improved.

Step 230 , when the first document rectangle exists in the preview image and the shooting instruction is received, the second document rectangle in the captured image is detected according to the second document recognition algorithm.

The electronic device has the first document rectangle in the preview screen and detects the second document rectangle in the captured image according to the second document recognition algorithm when receiving the photographing instruction. Wherein, when the electronic device receives the shooting instruction, it may acquire the captured image currently captured by the camera. It should be noted that the processing algorithms of the image captured by the electronic device and the preview image are usually different, but since the preview image is captured in real time, there may also be a frame of image in the preview image corresponding to the shooting instruction when a shooting instruction is received. Therefore, when the electronic device has the first document rectangle in the preview screen and receives a shooting instruction, that is, the electronic device may call the second Document recognition algorithm for detection. And when there is no first document rectangle in the preview screen, that is, the electronic device determines that there is no document in the preview screen according to the first document recognition algorithm, even if the shooting instruction is received, the electronic device only needs to obtain the captured image currently captured by the camera, and does not The second document recognition algorithm needs to be invoked for detection, which reduces the computation load of the electronic device.

Wherein, the calculation amount of the first document recognition algorithm is smaller than that of the second document recognition algorithm, and the accuracy of the first document recognition algorithm is lower than that of the second document recognition algorithm. Specifically, the first document recognition algorithm is used to process the multi-frame preview images of the preview screen, and the second document recognition algorithm is used to process the captured images corresponding to the shooting instructions, and the number of captured images corresponding to the shooting instructions is usually far less than that of the preview images. The multi-frame preview image of the screen, therefore, the electronic device can use the first document recognition algorithm with a small amount of calculation to identify the preview screen, so that the performance of the electronic device can meet the document recognition of the preview screen, and then use the higher accuracy of the first document recognition algorithm Second, the document recognition algorithm recognizes the captured image, so that the document rectangle recognized in the captured image captured by the electronic device is more accurate. In a specific embodiment, the first document recognition algorithm may determine the first document rectangle according to the gray value of the pixel, and the second document recognition algorithm may determine the second document rectangle by identifying feature points of the image.

Step 240, superimposing and displaying the second document rectangle in the captured image.

The electronic device may superimpose and display the second document rectangle in the captured image. Optionally, after acquiring the captured image, the electronic device may display the captured image on the display screen, and superimpose and display the second document rectangle in the captured image. Alternatively, after receiving the display instruction of the captured image, the electronic device may display the captured image on the display screen, and superimpose and display the second document rectangle in the captured image.

In this embodiment of the application, the electronic device can obtain a preview image when the camera is working, and detect whether there is a first document rectangle in the preview image according to the first document recognition algorithm, and the first document rectangle exists in the preview image and receives During the shooting instruction, according to the second document recognition algorithm, the second document rectangle in the captured image is detected, and the second document rectangle is superimposed and displayed in the captured image, and the calculation amount of the first document recognition algorithm used by the electronic device in the preview screen is less than The calculation amount of the second document recognition algorithm in the captured image enables the electronic device to complete the document recognition on the preview screen in real time with low performance, and the accuracy of the first document recognition algorithm used by the electronic device in the preview screen is lower than that in the shooting image. The accuracy of the second document recognition algorithm used in the image, the electronic device only recognizes the captured image taken by the user with a more accurate algorithm, which not only improves the user experience, but also reduces the performance occupied by document recognition.

As shown in Figure 3, Figure 3 is a schematic flow chart of another document identification method disclosed in the embodiment of the present application. The document identification method can be applied to the electronic device in the above embodiment, and the document identification method can include the following steps:

Step 310, when the camera is working, obtain a preview image.

The method in step 310 is the same as the method in step 210 in the above embodiment, and will not be repeated here.

Step 320, determining document pixels whose grayscale values satisfy a preset condition in the target preview image.

The preview image acquired by the electronic device may include a multi-frame preview image, and the target preview image may be any frame preview image in the multiple multi-frame preview images, and the electronic device may determine the document pixels whose grayscale values in the target preview image meet the preset conditions point. Wherein, the preview condition may be manually preset, or determined by the electronic device according to the pixels of the target preview image. The electronic device can calculate the gray value of each pixel in the target preview image. Optionally, the electronic device may calculate the gray value of each pixel according to the RGB (red, green, blue) value of each pixel, and the embodiment of the present application does not limit the calculation formula from the RGB value to the gray value.

In one embodiment, the electronic device may calculate the average gray value of all pixels in the target preview image, and then subtract the average gray value from the gray value of each pixel to obtain a calculation result corresponding to each pixel, and A pixel whose corresponding calculation result is greater than the preset threshold is regarded as a document pixel whose gray value satisfies the preset condition. Among them, the electronic device can first determine the gray value of each pixel in the target preview image, and then calculate the average gray value of all pixels. The preset threshold can be manually preset, or it can be determined according to the average gray value. For example, the preset threshold may be 5% of the average gray value. The electronic device subtracts the average gray value from the gray value of each pixel to obtain a calculation result corresponding to each pixel. The calculation result can be a positive value or a negative value. For example, the target preview image includes multiple pixels. The average gray value is 100, and the default threshold is 15. The gray values of 3 pixels are 50, 80 and 120 respectively, and the calculation results are -50, -30 and 20 respectively, and the gray value is 120. Points can be used as document pixels. This embodiment is implemented, according to the gray value of the pixel in the target preview image.

Step 330: Determine whether the first document rectangle exists in the target preview image according to the density of document pixels in the target preview image.

The electronic device may determine whether the first document rectangle exists in the target preview image according to the density of document pixels in the target preview image. The density can be determined by the electronic device according to the distribution of document pixels in the target preview image, or can be determined according to the distance between each document pixel, or can be determined according to the density recognition model obtained through deep learning training , which is not limited in this embodiment of the present application.

As an optional implementation method, the electronic device may divide the target preview image into multiple regions, and determine the ratio of document pixels in each region to all pixels in each region, and use the ratio corresponding to each region as The density of each area, if there is an area whose proportion is greater than the proportion threshold, it is determined that the first document rectangle exists in the target preview image. By implementing this embodiment, the electronic device can determine whether the first document rectangle exists in the target preview image according to the pre-divided area, thereby reducing the complexity of the budget.

Step 340, if it is determined that the first document rectangle exists in the target preview image, superimpose and display the first document rectangle in the target preview image.

If it is determined that the first document rectangle exists in the target preview image, the electronic device may determine the position of the first document rectangle, and superimpose and display the first document rectangle in the target preview image, that is, the first document rectangle may be displayed in real time on the preview screen. If it is determined that the first document rectangle does not exist in the target preview image, nor does the first document rectangle exist in the preview screen of the electronic device, the user may be prompted whether a document is detected in the current preview screen by whether to display the first document rectangle. Wherein, the line type of the first document rectangle may include straight line, wavy line, dashed line, etc., which is not limited, and the line color of the first document rectangle is also not limited. Optionally, the electronic device may acquire the RGB values of the pixels in the edge area of the first document rectangle, and determine the line color of the first document rectangle according to the RGB values of the pixels in the edge area.

Step 350, when the first document rectangle exists in the preview image and the shooting instruction is received, the second document rectangle in the captured image is detected according to the second document recognition algorithm.

Step 360, superimposing and displaying the second document rectangle in the captured image.

The methods in steps 350-360 are the same as the methods in steps 230-240 in the above embodiment, and will not be repeated here.

In this embodiment of the application, the electronic device can also determine the document pixels whose grayscale values meet the preset conditions in the target preview image, and determine whether there is The first document rectangle, the calculation amount of the document recognition algorithm is small, which reduces the performance of the electronic equipment occupied by the document recognition algorithm, so that the performance of the electronic equipment can meet the real-time recognition of the multi-frame preview image of the preview screen, in the target preview When it is determined that the first document rectangle exists in the image, the electronic device may also superimpose and display the first document rectangle in the target preview image, which further improves user experience.

As shown in FIG. 4 , it is a schematic flowchart of another document recognition method disclosed in the embodiment of the present application. The document recognition method can be executed when there is a first document rectangle in the target preview image, and can be applied to the above-mentioned embodiments. An electronic device, the document identification method may include the following steps:

Step 410, when the camera works, obtain a preview image.

Step 420, determine the pixel points of the document whose grayscale value satisfies the preset condition in the target preview image.

Step 430: Determine at least one document rectangle in the target preview image according to the density of document pixels in the target preview image.

The electronic device may determine at least one document rectangle in the target preview image according to the density of document pixels in the target preview image. Wherein, the electronic device may determine the target area whose density reaches the density requirement in the target preview image. If there is a first document rectangle in the target preview image, there may be one or more target areas, and each target area may correspond to A document rectangle. Optionally, if the distance between the two target areas is smaller than the distance threshold, a document rectangle may also be determined according to the two target areas, and there is no limitation on this.

In one embodiment, the electronic device can analyze the denseness of document pixels in the target preview image through a density recognition model, identify at least one model recognition region with a document pixel density greater than a density threshold, and then identify the document in the target preview image At least one document rectangle corresponding to at least one model recognition area is determined.

Wherein, the density recognition model is a pre-trained model according to the training data set, and the training data set may be multiple labeled training images, which will not be described in this embodiment of the present application. The electronic device analyzes the denseness of the document pixel in the target preview image, and if there is a first document rectangle in the target preview image, at least one model recognition area can be obtained, and the shape of the model recognition area can include but not limited to a circle Shape, rhombus, other regular shapes and irregular shapes are not limited, and the electronic device can determine the document rectangle corresponding to each model recognition region according to the document pixels in each model recognition region.

As an optional implementation manner, the step of determining at least one document rectangle corresponding to at least one model recognition area in the target preview image may include: determining the target model according to the leftmost pixel in the target model recognition area The left boundary of the document rectangle corresponding to the recognition area; determine the right boundary of the document rectangle corresponding to the target model recognition area according to the rightmost pixel point in the target model recognition area; determine the target according to the uppermost pixel point in the target model recognition area The upper boundary of the document rectangle corresponding to the model recognition area; determine the lower boundary of the document rectangle corresponding to the target model recognition area according to the lowermost pixel in the target model recognition area; according to the left boundary, right boundary, upper boundary and lower boundary, Determine the document rectangle corresponding to the target model recognition area.

Wherein, the target model recognition area is any model recognition area in at least one model recognition area. The electronic device may establish a pixel coordinate system according to the target preview image, and the pixel coordinate system may establish a Cartesian coordinate system with a pixel as a unit with the upper left corner of the target preview image as an origin. The electronic device determines the leftmost pixel according to the abscissa of each pixel in the target model recognition area, and determines the left boundary according to the abscissa of the leftmost pixel. The method for the electronic device to determine the right boundary, the upper boundary and the lower boundary is similar to the method for determining the left boundary, which will not be repeated here. The electronic device may use the intersection point of the left boundary and the upper boundary as the upper-left vertex of the document rectangle, the intersection point of the left boundary and the lower boundary as the lower-left vertex of the document rectangle, and the intersection point of the right boundary and the upper boundary as the upper-right vertex of the document rectangle , taking the intersection of the right border and the bottom border as the bottom right vertex of the document rectangle. The electronic device may determine the document rectangle according to the upper-left vertex, the lower-left vertex, the upper-right vertex, and the lower-right vertex.

Step 440, determine the corresponding document pixel density, color difference contrast and confidence of each document rectangle.

In one embodiment, the electronic device may calculate the document pixel density corresponding to each document rectangle according to the number of document pixels in each document rectangle, and calculate the pixel density of each document rectangle according to the color value of each pixel within each document rectangle and the color value of each pixel outside each document rectangle. Color value, calculate the color difference contrast corresponding to each document rectangle, and then obtain the confidence corresponding to each document rectangle according to the text information and grayscale information of each document rectangle.

The electronic device can obtain the total number of pixels of each document rectangle, and divide the number of document pixels by the number of pixels to obtain the pixel density corresponding to the document rectangle. The color value of the pixel can be the RGB value of the pixel. The pixel in the document rectangle can refer to the pixel in the edge area of the document rectangle, or it can refer to all the pixels in the document rectangle. The pixel outside the document rectangle can be Refers to the pixels in the surrounding area outside the document rectangle, and can also refer to all the pixels in the target preview image. Wherein, the inner edge area of the document rectangle may include pixels within the document rectangle whose distance from any boundary does not exceed the first preset number of pixels, for example, pixels within the document rectangle that are no more than 3 pixels away from any boundary, and the document rectangle The outer surrounding area may include pixels outside the document rectangle whose distance from any border does not exceed a second preset number of pixels. The electronic device can also detect the text information in the document rectangle, the text information can include but not limited to the number of text, text type, text size, etc., the grayscale information can refer to the grayscale values of all pixels in the document rectangle, according to the Text information and grayscale information can be used to obtain the confidence corresponding to the document rectangle.

Step 450 , according to the corresponding document pixel density, color difference contrast and confidence of each document rectangle, calculate the target result of each document rectangle by weighting.

Among them, the document pixel density, color contrast and confidence can be respectively corresponding to weights, and the weights corresponding to the document pixel density, color contrast and confidence respectively can be preset. Optionally, the weight corresponding to the document pixel density can be Greater than the weights corresponding to the color difference contrast and the confidence respectively, the electronic device can use the document pixel density as the main influencing factor of the target result.

Step 460, taking the document rectangle with the largest target result among the plurality of document rectangles as the first document rectangle.

The electronic device may use the document rectangle with the largest target result among the plurality of document rectangles as the first document rectangle.

Step 470, superimposing and displaying the first document rectangle in the target preview image.

Step 480 : There is a first document rectangle in the preview image and when a photographing instruction is received, detect a second document rectangle in the photographed image according to a second document recognition algorithm.

Step 490, superimposing and displaying the second document rectangle in the captured image.

The methods in steps 470-490 are the same as the methods in steps 340-360 in the above embodiment, and will not be repeated here.

In the embodiment of the present application, the electronic device determines at least one document rectangle in the target preview image according to the denseness of document pixels in the target preview image, and determines the corresponding document pixel density, color difference contrast and confidence of each document rectangle. Degree, calculate the target result of each document rectangle by weighting, and then use the document rectangle with the largest target result among multiple document rectangles as the first document rectangle, which improves the accuracy of the first document rectangle, and the second document rectangle is more accurate. In some cases, the first document rectangle is closer to the second document rectangle, thereby improving the consistency of user experience.

As shown in FIG. 5 , FIG. 5 is a modular schematic diagram of a document recognition device disclosed in an embodiment of the present application. The document recognition device 500 includes an acquisition module 510 , a first detection module 520 , a second detection module 530 and a display module 540 ,in:

Obtaining module 510, used for obtaining the preview image when the camera works;

The first detection module 520 is configured to detect whether there is a first document rectangle in the preview screen according to the first document recognition algorithm;

The second detection module 530 is configured to detect the second document rectangle in the captured image according to the second document recognition algorithm when the first document rectangle exists in the preview screen and the shooting instruction is received;

A display module 540, configured to superimpose and display the second document rectangle in the captured image;

Wherein, the calculation amount of the first document recognition algorithm is smaller than that of the second document recognition algorithm, and the accuracy of the first document recognition algorithm is lower than that of the second document recognition algorithm.

In one embodiment, the preview image includes a multi-frame preview image, and the first detection module 520 is also used to determine document pixels whose grayscale values in the target preview image meet preset conditions; wherein, the target preview image is a multi-frame preview image any frame of the preview image; according to the density of document pixels in the target preview image, determine whether there is a first document rectangle in the target preview image.

The display module 540 is further configured to superimpose and display the first document rectangle in the target preview image.

In one embodiment, the first detection module 520 is also used to calculate the average gray value of all pixels in the target preview image; the gray value of each pixel is subtracted from the average gray value to obtain the gray value corresponding to each pixel Calculation results; pixels whose corresponding calculation results are greater than the preset threshold are used as document pixels whose gray value meets the preset condition.

In one embodiment, in the case that the first document rectangle exists in the target preview image, the first detection module 520 is further configured to determine at least one document rectangle in the target preview image according to the density of document pixels in the target preview image Document rectangle; determine the corresponding document pixel density, color difference contrast and confidence of each document rectangle; according to the corresponding document pixel density, color difference contrast and confidence of each document rectangle, calculate the target result of each document rectangle by weighting; Among document rectangles, the document rectangle with the largest target result is used as the first document rectangle.

In one embodiment, the first detection module 520 is further configured to calculate the document pixel density corresponding to each document rectangle according to the number of document pixels in each document rectangle; The color value of the pixels outside the rectangle is used to calculate the color difference contrast corresponding to each document rectangle; according to the text information and grayscale information of each document rectangle, the confidence corresponding to each document rectangle is obtained.

In one embodiment, the first detection module 520 is further configured to analyze the denseness of document pixels in the target preview image through a density identification model, and identify at least one model identification area whose document pixel density is greater than a density threshold; At least one document rectangle corresponding to at least one model recognition area is determined in the target preview image.

In one embodiment, the first detection module 520 is further configured to determine the left boundary of the document rectangle corresponding to the target model recognition region according to the leftmost pixel in the target model recognition region; wherein, the target model recognition region is at least one Any model recognition area in the model recognition area; according to the rightmost pixel in the target model recognition area, determine the right boundary of the document rectangle corresponding to the target model recognition area; determine the target according to the uppermost pixel in the target model recognition area The upper boundary of the document rectangle corresponding to the model recognition area; determine the lower boundary of the document rectangle corresponding to the target model recognition area according to the lowermost pixel in the target model recognition area; according to the left boundary, right boundary, upper boundary and lower boundary, Determine the document rectangle corresponding to the target model recognition area.

In this embodiment of the application, the electronic device can obtain a preview image when the camera is working, and detect whether there is a first document rectangle in the preview image according to the first document recognition algorithm, and the first document rectangle exists in the preview image and receives During the shooting instruction, according to the second document recognition algorithm, the second document rectangle in the captured image is detected, and the second document rectangle is superimposed and displayed in the captured image, and the calculation amount of the first document recognition algorithm used by the electronic device in the preview screen is less than The calculation amount of the second document recognition algorithm in the captured image enables the electronic device to complete the document recognition on the preview screen in real time with low performance, and the accuracy of the first document recognition algorithm used by the electronic device in the preview screen is lower than that in the shooting image. The accuracy of the second document recognition algorithm used in the image, the electronic device only recognizes the captured image taken by the user with a more accurate algorithm, which not only improves the user experience, but also reduces the performance occupied by document recognition.

As shown in FIG. 6, in one embodiment, an electronic device is provided, and the electronic device may include:

a memory 610 storing executable program code;

a processor 620 coupled to the memory 610;

The processor 620 invokes the executable program code stored in the memory 810 to implement the document identification methods provided in the above-mentioned embodiments.

The memory 1010 may include a random access memory (Random Access Memory, RAM), and may also include a read-only memory (Read-Only Memory, ROM). Memory 610 may be used to store instructions, programs, codes, sets of codes, or sets of instructions. The memory 610 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.) , instructions for implementing the foregoing method embodiments, and the like. The storage data area can also store data and the like created by the electronic device during use.

Processor 620 may include one or more processing cores. The processor 620 uses various interfaces and lines to connect various parts of the entire electronic device, and executes electronic functions by running or executing instructions, programs, code sets or instruction sets stored in the memory 610, and calling data stored in the memory 610. Various functions and processing data of the device. Optionally, the processor 620 may adopt at least one of Digital Signal Processing (Digital Signal Processing, DSP), Field Programmable Gate Array (Field-Programmable Gate Array, FPGA), and Programmable Logic Array (Programmable LogicArray, PLA). implemented in the form of hardware. The processor 620 may integrate one or a combination of a central processing unit (Central Processing Unit, CPU), an image processor (Graphics Processing Unit, GPU), a modem, and the like. Among them, the CPU mainly handles the operating system, user interface and application programs, etc.; the GPU is used to render and draw the displayed content; the modem is used to handle wireless communication. It can be understood that the above modem may also not be integrated into the processor 620, but implemented by a communication chip alone.

It can be understood that the electronic device may include more or less structural elements than those in the above structural block diagram, for example, including a power module, physical buttons, WiFi (Wireless Fidelity, wireless fidelity) module, speaker, Bluetooth module, sensor, etc., It can also not be limited here.

The embodiment of the present application discloses a computer-readable storage medium, which stores a computer program, where the computer program causes a computer to execute the methods described in the foregoing embodiments.

In addition, the embodiment of the present application further discloses a computer program product. When the computer program product is run on a computer, the computer can execute all or part of the steps in any one of the document identification methods described in the above embodiments.

Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above-mentioned embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, and the storage medium includes read-only Memory (Read-Only Memory, ROM), Random Access Memory (Random Access Memory, RAM), Programmable Read-Only Memory (Programmable Read-only Memory, PROM), Erasable Programmable Read-Only Memory (Erasable Programmable Read Only Memory, EPROM ), One-time Programmable Read-Only Memory (OTPROM), Electronically Erasable Programmable Read-Only Memory (EEPROM), Compact Disc Read-Only Memory, CD-ROM) or other optical disk storage, magnetic disk storage, tape storage, or any other computer-readable medium that can be used to carry or store data.

A document recognition method, device, electronic device and storage medium disclosed in the embodiments of the present application have been described above in detail. In this paper, specific examples are used to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used To help understand the method of the present application and its core idea; at the same time, for those of ordinary skill in the art, according to the idea of the present application, there will be changes in the specific implementation and application scope. In summary, the content of this specification does not It should be understood as a limitation on the present application.

Claims (10)

1. A method of document identification, the method comprising:

when the camera works, acquiring a preview picture;

detecting whether a first document rectangle exists in the preview picture according to a first document identification algorithm;

detecting a second document rectangle in the shot image according to a second document identification algorithm when a shooting instruction is received and a first document rectangle exists in the preview picture;

displaying the second document rectangle in the shot image in an overlapping manner;

wherein the computation amount of the first document identification algorithm is smaller than that of the second document identification algorithm, and the accuracy of the first document identification algorithm is lower than that of the second document identification algorithm.

2. The method of claim 1, wherein the preview screen comprises a plurality of frames of preview images, and wherein detecting whether a first document rectangle is present in the preview screen according to a first document identification algorithm comprises:

determining document pixel points of which the gray values meet preset conditions in the target preview image; the target preview image is any one frame preview image in the multi-frame preview images;

determining whether a first document rectangle exists in the target preview image according to the density degree of the document pixel points in the target preview image;

in a case where it is determined that the first document rectangle exists in the target preview image, the method further includes:

and displaying the first document rectangle in the target preview image in an overlapping manner.

3. The method of claim 2, wherein the determining the document pixel points with gray scale values satisfying the preset condition in the target preview image comprises:

calculating the average gray value of all pixel points in the target preview image;

subtracting the average gray value from the gray value of each pixel point to obtain a calculation result corresponding to each pixel point;

and taking the pixel points of which the corresponding calculation results are larger than a preset threshold value as document pixel points of which the gray values meet preset conditions.

4. The method of claim 2, wherein, in the case that the first document rectangle exists in the target preview image, the determining whether the first document rectangle exists in the target preview image according to the density of the document pixel points in the target preview image comprises:

determining at least one document rectangle in the target preview image according to the density degree of the document pixel points in the target preview image;

determining the density, the color difference contrast and the confidence coefficient of the corresponding document pixel point of each document rectangle;

weighting and calculating a target result of each document rectangle according to the corresponding document pixel density, color difference contrast and confidence coefficient of each document rectangle;

and taking the document rectangle with the maximum target result in the plurality of document rectangles as a first document rectangle.

5. The method of claim 4, wherein said determining a corresponding document pixel density, color difference contrast, and confidence for each of said document rectangles comprises:

calculating the density of document pixel points corresponding to each document rectangle according to the number of the document pixel points in each document rectangle;

calculating the color difference contrast corresponding to each document rectangle according to the color value of the pixel point in each document rectangle and the color value of the pixel point outside each document rectangle;

and obtaining the confidence corresponding to each document rectangle according to the character information and the gray information of each document rectangle.

6. The method of claim 4, wherein determining at least one document rectangle in the target preview image according to the density of the document pixel points in the target preview image comprises:

analyzing the density degree of the document pixel points in the target preview image through a density identification model, and identifying at least one model identification area with the density of the document pixel points being greater than a density threshold value;

and determining at least one document rectangle in one-to-one correspondence with the at least one model identification area in the target preview image.

7. The method of claim 6, wherein determining at least one document rectangle in the target preview image in one-to-one correspondence with the at least one model identification area comprises:

determining the left boundary of a document rectangle corresponding to the target model identification region according to the leftmost pixel point in the target model identification region; wherein the target model identification region is any one of the at least one model identification region;

determining the right boundary of a document rectangle corresponding to the target model identification region according to the rightmost pixel point in the target model identification region;

determining the upper boundary of a document rectangle corresponding to the target model identification region according to the uppermost pixel point in the target model identification region;

determining the lower boundary of a document rectangle corresponding to the target model identification area according to the lowest pixel point in the target model identification area;

and determining a document rectangle corresponding to the target model identification area according to the left boundary, the right boundary, the upper boundary and the lower boundary.

8. An apparatus for identifying a document, the apparatus comprising:

the acquisition module is used for acquiring a preview picture when the camera works;

the first detection module is used for detecting whether a first document rectangle exists in the preview picture according to a first document identification algorithm;

the second detection module is used for detecting a second document rectangle in the shot image according to a second document recognition algorithm when the first document rectangle exists in the preview picture and a shooting instruction is received;

the display module is used for displaying the second document rectangle in an overlapping mode in the shot image;

wherein the computation of the first document identification algorithm is less than the computation of the second document identification algorithm, and the accuracy of the first document identification algorithm is less than the accuracy of the second document identification algorithm.

9. An electronic device, comprising:

a memory storing executable program code;

a processor coupled with the memory;

the processor calls the executable program code stored in the memory to perform the method of any of claims 1 to 7.

10. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, causes the processor to perform the method of any of claims 1 to 7.

Images