JP2020129298A - Computation system - Google Patents

Abstract

To appropriately extract an objective area from an image.SOLUTION: Disclosed is a computation system that trains a model for extracting an objective target area from an input image. The computation system includes one or more storage devices for storing programs and training data, and one or more processors to operate in accordance with the programs. The one or more processors perform: obtaining a colored image; generating a background separation image in which an objective area and a background area are separated from the colored image based on color information of the colored image; generating a grayscale image obtained by converting the colored image to its grayscale; embracing a set of the grayscale image and the background separation image in the training data; and training the model to which the grayscale image is inputted and from which the background separation image is outputted, using the training data.SELECTED DRAWING: Figure 2

Description

The present invention relates to training a machine learning model for extracting a target area from an input image.

For example, the use of biometrics technology is expanding as a safe and convenient personal authentication means that can be replaced with an ID/password. In financial services, the use of biometric authentication is being realized in account opening from mobile terminals, balance inquiry, account transfer, personal authentication when using ATM, and the like. Until now, the use of biometric authentication was limited to smartphones and other devices that have a dedicated device such as a fingerprint sensor, but it does not require a dedicated device for biometric authentication or a specific light source, and is used as a standard feature on smartphones and tablets. Personal authentication using a camera is also being realized.

As a subject of biometric authentication using a general-purpose camera, there is a high degree of freedom in the posture of holding a hand and the illumination environment, and the extracted biometric feature is not stable. In Patent Document 1, as a subject of the finger vein authentication technology using a general-purpose camera, even if the shooting conditions such as the illumination environment and the imaging device are different between the registration and the authentication, the living body is captured at the time of the registration and the authentication. By minimizing the color difference between the biometric images obtained, the technology for authenticating with high accuracy is disclosed by reducing the influence of illumination variation and individual difference between devices.

JP, 2018-180660, A

However, the method disclosed in Patent Document 1 uses the color system of the living body region and the background region when the background of the image captured by the camera has a similar color to the skin-colored biological information image or when capturing under an incandescent lamp. Since there is no difference, it becomes difficult to select the representative color of the biometric region at the time of registration or authentication.

At the time of registration or authentication, even if there is an object of a skin-colored similar color in the background or under an incandescent lamp, even if it is difficult to separate the finger and the background area due to color information, by correctly extracting the finger area, A technique capable of reducing the influence of the conversion of the shooting environment and performing authentication with high accuracy is desired. Further, even in a field different from personal authentication with fingers, a technique capable of accurately extracting a target area (target image) from an image is desired.

One aspect of the present invention is a computer system that trains a model for extracting a target region from an input image, and includes one or more storage devices that store programs and training data, and one or more processors that operate according to the programs. The one or more processors obtain a color image, generate a background separation image in which a target region and a background region are separated from the color image based on color information of the color image, The image is grayscaled to generate a grayscale image, the set of the grayscale image and the background separation image is included in the training data, and a model in which the grayscale image is input and the background separation image is output is defined as the training data. To train using.

According to one aspect of the present invention, a target area can be appropriately extracted from an image.

It is a figure which shows the structural example of the biometrics authentication system of 1st Embodiment. It is a figure which shows the process outline of the biometrics authentication system of 1st Embodiment. It is an example of a processing flow of the biometric authentication system of the first embodiment. It is an example of successful image processing of the first embodiment. It is an example of the failed image processing of the first embodiment. The software configuration example of the biometrics authentication system of 2nd Embodiment is shown. It is a figure which shows the process outline of the biometrics authentication system of 2nd Embodiment. It is an example of a processing flow in the learning phase of the second embodiment. It is a block diagram which shows the outline of a structure of the biometrics authentication system of 3rd Embodiment. The software configuration example of the biometrics authentication system of 4th Embodiment is shown. It is a figure which shows the process outline of the biometrics authentication system of 4th Embodiment. The example which compares the position variation of the background separation image between continuous input image frames in 4th Embodiment is shown.

Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. It should be noted that the present embodiment is merely an example for realizing the present invention and does not limit the technical scope of the present invention.

[First Embodiment]
FIG. 1 is a diagram illustrating a configuration example of the biometric authentication system according to the first embodiment. The biometric authentication system 10 is, for example, a user terminal (computer). The biometric authentication system 10 includes, for example, an input device 11, a display device 12, a CPU (Central Processing Unit) 13, an imaging device 14, a main storage device 15, and an auxiliary storage device 16.

The input device 11 is a device that receives an input such as an instruction from a user, and may be, for example, a keyboard, a mouse, a touch panel, or the like. For example, the input device 11 receives a process of selecting a corresponding process from a plurality of boxes (buttons) displayed on the display device 12.

The display device 12 is a device that outputs various characters and information such as an image of a region cutout result to the user, and may be an image display device such as a liquid crystal display. The input device 11 and the display device 12 may be integrated. The imaging device 14 is, for example, a camera, and acquires biometric information for identifying an individual as an image.

The main storage device 15 stores a program executed by the CPU 13. As the main storage device 15, a high-speed and volatile storage device such as a DRAM (Dynamic Random Access Memory) is typically adopted. In the present embodiment, the programs stored in the main memory 15 are an OS (Operating System) 151, a color image area division program 152, a grayscale image area division model 153, a grayscale conversion program 154, and an area division model training. The program 155 and the authentication program 156 are included.

At least a part of each program and data stored in the auxiliary storage device 16 may be temporarily copied to the main storage device 15 as needed when the CPU 13 executes various processes. The program and its reference data may be stored. Further, the main storage device 15 may store the result of the process executed by the CPU 13.

The auxiliary storage device 16 stores information that the CPU 13 refers to in order to execute various processes based on each program. The auxiliary storage device 16 of the present embodiment stores training data 161 used for training (learning) the grayscale image area division model 153 and authentication reference information 162 referred to for personal authentication. As the auxiliary storage device 16, a large-capacity and non-volatile storage device such as an HDD (Hard Disk Drive) or a flash memory is typically adopted. The main storage device 15, the auxiliary storage device 16, and a combination thereof are storage devices each including a non-transitory storage medium.

The CPU 13 is a processor, and executes various processes according to the instruction code of the program stored in the main storage device 15. The CPU 13 realizes various functions by executing programs. For example, the CPU 13 follows the color image area division program 152, the grayscale image area division model 153, the grayscale conversion program 154, the area division model training program 155, and the authentication program 156 according to the color image area division unit and the grayscale image area. It operates as a division model unit, an image conversion unit, a region division model training unit, and an authentication unit.

The CPU 13 (processor) can be configured by a single processing unit or a plurality of processing units, and can include a single or a plurality of arithmetic units or a plurality of processing cores. The CPU 13 manipulates signals based on one or more central processing units, microprocessors, microcomputers, microcontrollers, digital signal processors, state machines, logic circuits, graphics processing units, chip-on systems, and/or control instructions. Can be implemented as a device.

As described above, various processes of the biometric authentication system 10 are executed by the CPU 13 executing the program. Therefore, in the following, the processing executed by the functional unit or the program is the processing by the CPU 13 or the biometric authentication system 10.

The color image area dividing program 152 generates teacher data of the training data 161. The teacher data generated from one input image is a background separated image. The background separated image is composed of a biometric information area extracted from the input image and a background image separated from the biometric information area. For example, the background separated image is a binary image, and the biometric information area and the background area are binary separated. For example, 1 is given to each pixel in the biometric information area, and 0 is given to each pixel in the background area. The training data 161 is a set of a grayscale input image and teacher data generated by the color image area dividing program 152.

The color image area dividing program 152 separates the biometric information area and the background area in the input color image based on the color information. For example, the color image area dividing program 152 separates the biometric information area and the background area based on the value range of the color space information such as RGB and YUV. In this example, the color image area dividing program 152 is a rule-based program, but it may be a machine learning model.

The grayscale image area division model 153 is a machine learning model, and is trained using the training data 161 in the learning phase. In the authentication phase of performing biometric authentication of an individual, the grayscale image area division model 153 divides the biometric information area and the background area of the grayscaled input image to generate a background separated image. The background area is separated from the input image and the biometric information area is extracted from the input image.

The grayscale conversion program 154 converts the input color image into a grayscale image for the processing by the grayscale image area division model 153 in the authentication phase. The region segmentation model training program 155 uses the training data 161 to train the grayscale image region segmentation model 153 in the learning phase.

The authentication program 156 identifies the biometric information area in the input image based on the background separated image generated by the area division model training program 155, and performs personal authentication based on the information of the biometric information area and the authentication reference information 162.

The number of components of the hardware and software shown in FIG. 1 is arbitrary, and some components may be omitted. At least a part of the functions of the biometric authentication system 10 may be configured by a logic circuit different from the processor. In the example of FIG. 1, the biometric authentication system 10 is composed of one computer, but the biometric authentication system 10 communicates via a network, and a plurality of devices each including one or more processors and one or more storage devices. (Calculator) may be included.

The program may be installed in a different computer. For example, the grayscale image area division model 153 after training and the grayscale program 154 are installed in the user terminal, and the grayscale image area division model 153 before and after training and other programs (including the grayscale program 154) are stored in the server. May be implemented in.

FIG. 2 is a diagram showing a processing outline of the biometric authentication system 10 according to the first embodiment. The processing of the biometric authentication system 10 includes a learning phase (training phase) of the grayscale image area division model 153 and an authentication phase of executing authentication processing by the grayscale image area division model 153 after learning (after training).

The learning phase includes generation of training data 161 and learning of the grayscale image area division model 153 by the training data 161. The input color image 200 is data for generating the training data 161. The input color image 200 is a visible light image in which an object having biometric information to be personally authenticated is captured. In the example described below, the target having biometric information is one or more fingers.

The input color image 200 is divided by the color image area division program 152 into a biometric information area which is a target area and a background area (202), and one or a plurality of background separated images 203 are generated. The background separated image is composed of, for example, a separated biometric information area (for example, a finger) and a background area, each of which is given one of binary values. For example, one background separated image 203 of a plurality of fingers, a background separated image 203 of one finger, or a plurality of background separated images 203 each showing one finger is generated. The color image area dividing program 152 divides the biometric information area and the background area in the input color image 200 based on the value range of the color space information such as RGB and YUV, and the generated background separated image 203 is used as the training data 161. Include in.

The input color image 200 is further grayscaled (201) by the grayscale conversion program 154. The grayscaled input image is included in the training data 161 together with the background separated image 203. The grayscaled input image is input data for learning, and one or a plurality of background separated images 203 is teacher data. In this way, the data 204 for the training data 161 can be mechanically (automatically) collected from a plurality of input images in which the biometric information area and the background area can be separated.

The grayscale image region segmentation model (training model) 153 is trained (205) by the region segmentation model training program 155 using the training data 161. The grayscale image area division model 153 outputs one or a plurality of background separated images from the input grayscale image. The area division model training program 155 updates the parameters of the grayscale image area division model 153 based on the comparison result between the output of the grayscale image area division model 153 and the training data of the training data 161. The grayscale image area division model 153 is, for example, a multi-layer neural network. The update parameters in this example include the connection weights between the adjacencies of the input layer, multiple hidden layers and the output layer.

Next, the authentication phase for personal authentication will be described. The input color image 207 is a visible light image in which an object having biometric information of a person who performs individual authentication is captured. The input color image 207 is grayscaled (208) by the grayscale conversion program 154 to generate a grayscale image 209. The trained grayscale image segmentation model 153 segmentes the grayscale image 209 (210). Specifically, the grayscale image area division model 153 generates a background separated image 211, which is a binary image of the separated biometric information area and background area, from the input grayscale image 209, and stores it in the main storage device 15. Store in the storage area.

The authentication program 156 (not shown in FIG. 2) specifies the biometric information area in the input color image 207 based on the background separated image 211. The biometric information is, for example, a vein or fingerprint pattern. The authentication program 156 analyzes the biometric information area in the input color image 207 and acquires biometric information. The authentication program 156 compares the biometric information with the biometric information of the person stored in the authentication reference information 162, and authenticates the person concerned.

The grayscale image area division model 153 used in the authentication phase is limited in the number of layers and the number of learning parameters so that the grayscale image area division model 153 operates at a practical speed of about 30 to 60 fps in a smartphone or a tablet, for example. The learning phase process may be executed in a server having a large memory or disk capacity. Therefore, when the color image area division program 152 is configured by a neural network, the number of layers and the number of parameters thereof may be larger than those of the grayscale image area division model 153.

<Flowchart>
FIG. 3 is an example of a processing flow of the biometric authentication system 10 according to the first embodiment. In the learning phase, the biometric information area and the background area of the video frame image are divided based on the range of values of color space information such as RGB and YUV, and training data for use in supervised learning is generated and training is performed. Train (generate) a machine learning model using the data. In the authentication phase, the biometric information area is extracted from the moving image frame image using the generated machine learning model.

First, the learning phase will be explained. In step S301, the biometric authentication system 10 activates the general-purpose camera and acquires a frame image of a moving image. Next, in step S302, the biometric authentication system 10 cuts out, from the frame image, a partial area of a certain size in which the area of the hand to be extracted is shown. The position of the cut-out area is set in advance, and the biometric authentication system 10 displays, for example, a guide indicating the position where the hand is placed on the screen.

In step S<b>303, the biometric authentication system 10 (grayscale conversion program 154) converts the cut out image into grayscale and stores it in the storage area of the auxiliary storage device as a part of the training data 161. In step S304, the biometric authentication system 10 (color image area division program 152) determines a hand area (biological target area) and a background area from the cut image based on color space information (for example, skin color information) such as RGB and YUV. Is separated and the living body target area is extracted.

In step S305, the biometric authentication system 10 (color image area division program 152) traces the contour of the extracted hand area, and in step S306, extracts the key points of the fingertip and finger crotch. In step S307, the biometric authentication system 10 (color image area division program 152) extracts the ROI area for each finger based on the extracted key points and contour information. The extracted image group (background separated image group) of the ROI area for each finger is associated with the grayscaled image and is stored in the storage area of the auxiliary storage device 16 as a part of the training data 314.

When the number of samples of the training data reaches a predetermined number in step S316, the biometric authentication system 10 (region division model training program 155) inputs the grayscale image including the biometric information using the training data 161 in step S308. The grayscale image area division model 153 that outputs the finger ROI area image (background separated image) as is trained.

Next, the authentication phase will be described. In the authentication phase, the generated (trained) grayscale image area division model 153 is used to extract the biometric information area from the moving image frame image. First, in step S309, the biometric authentication system 10 activates the general-purpose camera and acquires a frame image of a moving image. In step S310, as in step S302, the biometric authentication system 10 cuts out, from the frame image, a partial area of a certain size in which the area of the hand to be extracted appears.

Next, in step S311, the biometric authentication system 10 (grayscale conversion program 154) converts the clipped image into grayscale. In step S312, the biometric authentication system 10 (grayscale image area division model 153) extracts a hand area. For example, a binary image in which the biometric information area is 1 and the background information area is 0 is output.

The input image 401 in FIG. 4 is an example of an image including biometric information cut out in steps S302 and S310. The image 402 is an example of the hand region image obtained in step S304. The image group 403 is an example of a finger ROI region image group obtained in step S307.

On the other hand, an image 501 in FIG. 5 shows an example in which a face having a color similar to that of the hand is reflected behind the hand region, and an image 503 is similar in color to a hand such as a cardboard or a table in the background. An example in which an object with is reflected is shown. In these examples, since the skin color area and the biometric information area do not match in step S304, the correct finger ROI area cannot be extracted in step S307. Specifically, as illustrated in the image 502 of FIG. 5 and the image 504, each finger region cannot be independently extracted. Note that such images may be excluded so that they are not added to the training data 1651. An example of how to determine the proper separation is described in the second embodiment.

On the other hand, step S312 using the grayscale image area division model 153 that outputs the finger area ROI with the grayscaled image as an input can reduce the influence of color in the input image, and thus the image of FIG. Similar to 401, each finger region ROI can be appropriately extracted.

With the above configuration, the supervised data (training data) necessary for learning can be mechanically generated, so that efficient training data generation is realized. In addition, since the machine learning model performs background separation using a grayscale image as an input, it is possible to realize stable separation of finger areas even under a skin color similar background or an incandescent lamp.

In the present embodiment, the biometric information area for personal authentication is separated in the input image, but the image separation techniques of the present embodiment and other embodiments are different from the biometric authentication in other target images, such as a specific image. It can be used to separate objects and human bodies from the input image.

[Second Embodiment]
FIG. 6A shows a software configuration example of the biometric authentication system 10 according to the second embodiment. In addition to the program of the first embodiment, the biometric authentication system 10 includes a color conversion program 157.

FIG. 6B is a diagram showing a processing outline of the biometric authentication system 10 according to the second embodiment. The input color image 600 is a visible light image in which biometric information to be personally authenticated is captured. The input color image 600 is separated by the color image area dividing program 152 into a biometric information area and a background area based on a range of values of color space information such as RGB and YUV. The color image area dividing program 152 further determines whether or not the biometric information area is appropriately separated (602).

In this example, the color image area dividing program 152 determines whether or not the biometric information area is appropriately separated based on the shape of the biometric information area. The color image area dividing program 152 executes the processing of steps S304 to S307 of FIG. 3 and determines whether or not a certain number or more of finger ROI areas having widths and lengths within a certain range have been extracted.

When it is determined that the finger ROI area has been correctly extracted, the color image area dividing program 152 stores the generated background separated image 606 in the storage area of the auxiliary storage device 16 as the teacher data of the training data 161. On the other hand, when the finger ROI area cannot be properly extracted, the color conversion program 157 does not include the generated background separation image in the training data 161, and the input color image is changed from an image under an incandescent lamp to a fluorescent light. It is converted into an image (603) and a color converted image 604 is generated. The color image area dividing program 152 extracts the biometric information area from the color converted image 604 (605) and includes it as the background separated image 606 in the training data 161.

The grayscale program 154 grayscales (601) the input color image 600 and includes it in the training data 161, along with the corresponding background separation image 606. Thereby, the training data 161 can be mechanically generated not only from the input color image in which the biometric information area and the background information can be separated but also from the input color image in which the biometric information area cannot be separated before color conversion.

The region segmentation model training program 155 trains the grayscale image region segmentation model 153 using the training data 161 (608). The grayscale image area division model 153 separates and outputs the input gray image as a binary image of the biometric information area and the background area (for example, the biometric information area is 1, the background information area is 0). The grayscale image area division model 153 is, for example, a multi-layer neural network.

As the image color conversion 603, for example, an image conversion system (Unpaired Image-to-Image Translation using using domain conversion, such as converting a winter landscape into a summer landscape, converting an image under incandescent light into an image under fluorescent light, etc. Cycle-Consistent Adversarial Networks, Jun-Yan Zhu Taesung Park Phillip Isola Alexei A. Efros, In ICCV 2017) can be used.

In general, the domain conversion method has a large model size and is difficult to implement on low-resource terminals such as smartphones and tablets. Therefore, the generation time of the training data 161 can be shortened by executing the generation of the training data 161 not by the user terminal but by the server.

<Flowchart>
FIG. 7 is an example of a processing flow in the learning phase of the second embodiment. In the learning phase, the biometric information area and the background area of the video frame image are separated based on the range of values of color space information such as RGB and YUV, and training data for use in supervised learning is generated. Train (generate) a learning model.

First, in step S701, the biometric authentication system 10 activates the general-purpose camera and acquires a frame image of a moving image. Next, in step S702, the biometric authentication system 10 cuts out, from the frame image, a partial area of a certain size in which the area of the hand to be extracted is shown. The position to be cut out is set in advance, and the biometric authentication system 10 displays, for example, a guide indicating the position to hold the hand on the screen of.

In step S<b>709, the biometric authentication system 10 (grayscale conversion program 154) converts the cut out image into a grayscale image and includes it in the training data 161. In step S703, the biometric authentication system 10 (color image area division program 152) extracts a hand area from the cut image based on color space information (for example, skin color information) such as RGB and YUV. In step S704, the biometric authentication system 10 (color image area dividing program 152) tracks the contour of the extracted hand area, and in step S705, extracts the key points of the fingertip and finger crotch.

In step S706, the biometric authentication system 10 (color image area division program 152) extracts the ROI area for each finger based on the extracted key points and contour information. In step S710, the biometric authentication system 10 (color image area dividing program 152) determines whether the finger ROI area has been correctly separated as the background. The determination as to whether it is correct may be based on, for example, whether or not a certain number or more of finger ROI regions whose width and length are within a certain range have been extracted. If determined to be correct, the extracted ROI region image for each finger is stored in the storage region of the auxiliary storage device 16 as a part of the training data 161, together with the grayscaled image.

If determined to be incorrect, in the image conversion step S707, the biometric authentication system 10 (color conversion program 157) assumes that the input color image is an image under an incandescent lamp, and the input color image is an image under a fluorescent lamp. It is converted by a function preset to. After that, the biometric authentication system 10 repeats the steps from step S703 again.

If it is determined in step S710 that the correct finger ROI area has been acquired, the background separated image is stored in the storage area of the auxiliary storage device 16 as a part of the training data 161, together with the grayscaled image.

When the number of training data samples reaches a predetermined number in step S713, in step S708, the biometric authentication system 10 (region division model training program 155) trains the grayscale image region division model 153 using the training data 161. .. The grayscale image area division model 153 inputs a grayscale image including biometric information and outputs a finger ROI area.

As described above, the biometric authentication system 10 determines whether or not the target area is appropriately separated based on the shape of the target area in the background separated image generated from the color image. When it is determined that the target area is not properly separated, color conversion of the color image is performed, and a new background separated image is generated.

With the configuration of the present embodiment, teacher data can be generated even from an image in which it is normally difficult to mechanically generate training data, such as an incandescent lamp or a skin color background image. As a result, it is possible to realize stable separation of the finger area under a skin-colored background or an incandescent lamp.

[Third Embodiment]
FIG. 8 is a block diagram showing a schematic configuration of the biometric authentication system 10 according to the third embodiment. The color image area division program 152 and the grayscale image area division model 153 each receive a moving image frame image as an input, separate the biometric information area and the background area, and at the same time determine whether or not the hand is held at the correct position. With the function to do. The color image area division program 152 and the grayscale image area division model 153 each include a common feature extraction unit 802 that extracts the features of the input image, a region division unit 803, and a shift direction estimation unit 804. Note that only one of the color image area division program 152 and the grayscale image area division model 153 may have the function.

The input image 801 for the color image area division program 152 is a color image in which the biometric information of the personal authentication target is captured, and the input image 801 for the grayscale image area division model 153 is a grayscale image. The common feature extraction unit 802 extracts, from the input image 801, a common feature amount that can be used by both the area division unit 803 and the shift direction estimation unit 804. For example, a convolutional neural network used to determine the type of object such as a car or a person in image processing can be used.

In the present embodiment, the common feature extraction unit 802 downsamples the size of the image and extracts the feature amount used for estimation of the three classes (X, Y, Z directions) required by the displacement direction estimation unit. To do. For example, the X direction and the Y direction are directions parallel to the screen and perpendicular to each other, and the Z direction is the direction perpendicular to the screen.

The area dividing unit 803 learns a binary classification problem of whether or not each pixel of the input image 801 is included in the finger ROI area. Therefore, the area dividing unit 803 upsamples the common feature amount that has been downsampled to the image size, and performs the two-class determination for the number of pixels. Therefore, generally, the processing time of the area dividing unit 803 becomes longer than that of the shift direction estimating unit 804 (class classification) processing.

The displacement direction estimation unit 804 uses the output result of the common feature extraction unit 802 to estimate whether or not the input image is at a position suitable for extraction of the finger ROI region. For example, when the part of five fingers is not imaged or partly missing, the region division is not performed correctly, and the extraction of the finger ROI region continues to fail while holding the hand.

The displacement direction estimation unit 804 estimates how much the position of the hand deviates from the preset reference position in the left-right direction or the depth direction (X, Y, Z directions). The biometric authentication system 10 executes a process for correcting the estimated shift.

Specifically, the shift direction estimation unit 804 notifies the camera control unit 805. The camera control unit 805 transmits a position shift correction control signal for zooming in, zooming out, or the like to the imaging device 14 according to the notified position shift (shift direction and shift amount) of the hand.

Furthermore, the guide display unit 806 may display an arrow in the direction opposite to the shift direction notified from the shift direction estimation unit 804. The guide display unit 806 displays a message such as "Please move closer" or "Keep away". The positional deviation of the input image may be corrected by instructing with an image or voice on the display device 12.

The color image area division program 152 or the grayscale image area division model 153 acquires a new input image whose positional deviation has been corrected by the camera control unit 805 or the guide display unit 806. Note that one of the camera control unit 805 and the guide display unit 806 may be omitted. Further, the deviation may be corrected by another method.

As described above, the biometric authentication system 10 estimates the displacement of the position of the target area in the background separation image generated from the color image or the grayscale image, and performs a new process after correcting the displacement. Get a color image.

As described above, by correcting the position of the input image by the displacement direction estimating unit 804 that operates at high speed, the number of frame images in which the processing of the region dividing unit 803 that takes a long processing time is failed is reduced, so that the moving image frame It is possible to realize high-speed area division of an image.

[Fourth Embodiment]
FIG. 9A shows a software configuration example of the biometric authentication system 10 of the fourth exemplary embodiment. In addition to the program of the first embodiment, the biometric authentication system 10 includes a stability determination program 158.

FIG. 9B is a diagram showing a processing outline of the biometric authentication system 10 according to the fourth embodiment. One or both of the color image area division program 152 in the learning phase and the grayscale image area division model 153 in the authentication phase execute the processing described below. Thereby, it is possible to generate or authenticate appropriate training data. The processing in the authentication phase will be described below.

The input frame images 901 to 903 represent grayscale moving image frame images generated from continuous color moving image frame images. First, when the nth frame image 901 is input, the grayscale image region division model 153 outputs the background separated image 907 as a separated image candidate (904). Next, also for the (n+1)th input frame image 902, the grayscale image area division model 153 outputs the background separated image 908 (905). The stability determination program 158 compares the coordinates of the background separated image 907 and the background separated image 908 in the original image, and determines the magnitude of the position variation (910). The position variation can be represented by, for example, a coefficient indicating the overlap between two images.

When the position variation is large, the stability determination program 158 interrupts the process because the authentication process is executed for an image that is not appropriate in the subsequent personal authentication such as the image blurring. The area division model 153 moves to the processing of the (n+2)th frame.

The grayscale image area division model 153 outputs the background separated image 909 in the same manner for the (n+2)th input frame image 903 (906). The stability determination program 158 compares the coordinates of the background separated image 908 and the background separated image 909 in the original image (911), determines that the position variation is available when the magnitude of the position variation is equal to or less than a certain threshold, and separates the background. The image 908 or 909 is passed to biometric feature extraction or authentication processing.

With the above configuration, in a device that performs biometrics extraction using continuous video frames to perform personal authentication, by excluding images that are not suitable for authentication, high-speed personal authentication can be realized. FIG. 10 shows an example of comparing the positional fluctuation of the background separated image between consecutive input image frames. The area division program 152/153 outputs the background separated images 1004, 1005, 1006 for each finger from the input image 1001.

Similarly, the area dividing program 152/153 outputs, from the input image 1002, background separated images 1007, 1008, and 1009 for each finger. The stability determination program 158 changes the position of the pixel regions of the background separated images 1004 and 1007 (for example, the degree of overlap of white pixel regions), the position change of the background separated images 1005 and 1008, and the background separated images 1006 and 1009 for each finger. The position variation of is determined. The stability determination program 158, for example, prohibits the authentication process when the index for stationary determination is a certain number or less.

In the learning phase, the color image area dividing program 152 performs the processing shown in FIG. 9B. The input frame images 901 to 903 represent continuous color moving image frame images. When the magnitude of the position variation is larger than the fixed threshold value, the background separated images 908 and 909 are discarded without being used as the training data 161.

As described above, as described above, the biometric authentication system 10 determines the position between the target area of the background separated image generated from the frame image and the target area of the background separated image generated from another frame image. A change is determined, and whether the background separated image is used is determined based on the position change.

With the configuration of the present embodiment, it is possible to prevent an inappropriate image from being passed on to the processing after the generation of the background separated image.

It should be noted that the present invention is not limited to the above-described embodiment, and various modifications are included. For example, the above-described embodiments have been described in detail in order to explain the present invention in an easy-to-understand manner, and are not necessarily limited to those having all the configurations described. Further, a part of the configuration of one embodiment can be replaced with the configuration of another embodiment, and the configuration of another embodiment can be added to the configuration of one embodiment. Further, it is possible to add/delete/replace other configurations with respect to a part of the configurations of the respective embodiments.

Further, the above-described respective configurations, functions, processing units, etc. may be realized by hardware by partially or entirely designing them with an integrated circuit, for example. Further, the above-described respective configurations, functions and the like may be realized by software by a processor interpreting and executing a program that realizes each function. Information such as a program, a table, and a file that realizes each function can be placed in a memory, a hard disk, a recording device such as an SSD (Solid State Drive), or a recording medium such as an IC card or an SD card. Further, the control lines and information lines shown are those that are considered necessary for explanation, and not all the control lines and information lines in the product are necessarily shown. In reality, it may be considered that almost all configurations are connected to each other.

10: biometric authentication system 11: input device 12: display device 13: CPU
14: Imaging device 15: Main storage device 16: Auxiliary storage device

Claims (9)

A computer system for training a model for extracting a target area from an input image,
One or more storage devices for storing programs and training data;
One or more processors operating according to the program;
Including,
The one or more processors are
Get a color image,
From the color image, based on the color information of the color image, to generate a background separation image in which the target area and the background area are separated,
Grayscale the color image to generate a grayscale image,
Including the set of the grayscale image and the background separation image in the training data,
A computer system that uses the training data to train a model that receives a grayscale image and outputs a background separated image. The computer system according to claim 1, wherein
The one or more processors are
Based on the shape of the first target area in the first background separation image generated from the first color image, it is determined whether the first target area is appropriately separated,
When it is determined that the first target area is not properly separated, color conversion of the first color image is performed,
A computer system for generating a second background separated image from the color-converted first color image. The computer system according to claim 1, wherein
Further comprising an imaging device,
The one or more processors are
In the first background separated image generated from the first color image picked up by the image pickup apparatus, the position shift of the first target area is estimated,
A computer system that acquires a new color image captured by the image capturing apparatus after performing the process for correcting the displacement. The computer system according to claim 1, wherein
The one or more processors are
Acquiring a first color image and a second color image of each of the first frame and the second frame in the video,
Between the first target area of the first background separated image generated from the first color image and the second target area of the second background separated image generated from the second color image , Determine the position variation,
A computer system that determines whether to include the first background separated image or the second background separated image in the training data based on the position variation. The computer system according to claim 1, wherein
Further comprising an imaging device,
The one or more processors are
Acquiring a first grayscale image in which the first color image captured by the imaging device is grayscaled,
A computer system for generating a first background separation image from the first grayscale image by the model. The computer system according to claim 5,
A first target area of the first background separated image is a biometric information area,
A computer system in which the one or more processors perform biometric authentication based on the biometric information area. The computer system according to claim 5,
The one or more processors are
Acquiring a second grayscale image that is a grayscale version of the second color image,
In the second background separated image generated from the second grayscale image, estimating the positional shift of the second target area,
A computer system that acquires a new color image captured by the image capturing apparatus after performing the process for correcting the displacement. The computer system according to claim 5,
The one or more processors are
Obtaining a second grayscale image obtained by grayscale-converting the second color image of one frame in the video,
Obtaining a third grayscale image obtained by grayscale-converting the third color image of the other frame in the video,
A second target area of the second background separated image generated from the second grayscale image and a third target area of the third background separated image generated from the third grayscale image. Between the positions,
A computer system that determines whether to use the second background separated image or the third background separated image based on the position variation. A computer system is a method for training a model for extracting a target region from an input image,
The computer system acquires a color image,
The computer system, from the color image, based on the color information of the color image, to generate a background separated image in which the target area and the background area are separated,
The computer system grayscales the color image to produce a grayscale image,
The computer system includes a set of the grayscale image and the background separated image in the training data,
The method, wherein the computer system trains a model that receives a grayscale image and outputs a background separated image using the training data.

Images