CN110363891A - An intelligent visitor system suitable for multiple scenarios - Google Patents

Abstract

The invention discloses an intelligent visitor system applicable to multiple scenes, comprising a face detection module, a video display module, a database management module, a face recognition module, a visitor information display module, a log printing module and a system display interface. Each module can implement the corresponding software module by python language, so as to realize the operation on the cloud server or terminal equipment, and then realize the detection, identification and information management of visitors. The present invention can greatly reduce the requirements for identity authentication information of visitors, effectively prevent user identity information from being fraudulently used or embezzled, improve visitor reception efficiency and visitor experience, simplify the management and viewing process of visitor information by administrators, and further improve office efficiency .

Description

An intelligent visitor system suitable for multiple scenarios

technical field

The invention relates to the technical field of pattern recognition and intelligent systems, in particular to an intelligent visitor system applicable to multiple scenarios.

Background technique

At present, in order to facilitate the management of visitor information, improve the accuracy of reception time and office efficiency, many enterprises, universities and government units will install visitor systems or arrange duty personnel at the door, duty room or security office to identify the identity of visitors and record them. Visitor information. On the one hand, arranging on-duty personnel increases the labor cost of the unit, and the use of traditional paper-based visitor registration information by the on-duty personnel leads to low work efficiency, and it may also be difficult to identify the information of visitors; on the other hand, some units currently use visitor registration information. The system still identifies visitors by swiping cards, scanning mobile phone QR codes, fingerprints, and manually entering passwords to provide identification information. Although these methods have improved the reception efficiency of visitors to a certain extent, there will still be fraudulent use by others. , The disadvantages of requiring users to provide corresponding identification information.

Faced with such problems, most units use visitor systems to replace on-duty personnel, but this type of visitor system still requires visitors to provide identification information. If the visitor forgets to carry the identification information, the identity cannot be authenticated. Visitors are not friendly enough. At the same time, the traditional identification information authentication method is easy to be fraudulently used or embezzled by others, and there are great security risks. In addition, this type of visitor system is generally for a single scenario, that is, different identification information needs to be provided to visit different units. For example, if you enter a university laboratory, you need to fingerprint and enter a password, and you need to swipe your student card to enter a university library. This puts forward higher identity authentication requirements for visitors, and also reduces the efficiency of visitors passing.

Based on the above problems, for enterprises, universities and government units to manage visitor information, it is necessary to propose a visitor system that can automatically identify visitor information applicable to multiple scenarios. The visitor system can accurately and quickly identify the identity of visitors and respond , and should also minimize the system's requirements for the identification information of visitors, that is, use a unified and friendly identity authentication method.

Contents of the invention

The purpose of the present invention is to overcome the shortcomings and deficiencies of the prior art, and propose an intelligent visitor system suitable for multiple scenarios, which solves the problem that the existing visitor system requires high and unfriendly identity authentication information for visitors, and eliminates the need for visitors Identity authentication information is easy to be fraudulently used or stolen by others. It solves the problem that the visitor system has a single usage scenario and requires visitors to provide multiple identity authentication information. It improves the user experience of visitors, simplifies the process of managing visitor information by unit managers, and improves the efficiency of visitor reception.

In order to achieve the above purpose, the technical solution provided by the present invention is: an intelligent visitor system applicable to multiple scenarios, including:

The human face detection module is used to realize the automatic detection of the human face in the input video stream, and the human face position in the output video image;

The video display module is used to realize the visualization of the output result of the face detection module, and output the face image cut out from the original image; wherein, the module can display the detection results of up to four cameras, including USB cameras and remote networks cameras, and can maximize the display area of the detection results of any one of the cameras;

The database management module is used to realize the registration, deletion, modification and display viewing operations of user data in the database;

The face recognition module is used to realize the identification of the identity of the input face image in the database, and its identity information includes: user ID and user name; output the similarity between this person and the identified identity in the database, and the identity of the person to the time of the visit and the identity of the person in the database;

The visitor information display module is used to display the identity information output by the face recognition module, the similarity and the visiting time of the visitor;

The log printing module is used to record the information of all visitors identified by the face recognition module, including identity information, similarity and visit time, and print out after archiving by date;

The system display interface is used to realize the information interaction between the visitor system and the user, and the interface includes a video display area, a LOGO display area, a visitor information display area, a user information input column and an operation area.

Further, the face detection module includes a camera or a video stream reading module and a face detector module, wherein:

The camera or video stream reading module is responsible for sequentially reading images from the video stream produced by USB camera or remote network camera or video file, and then preprocessing the image data. The preprocessing steps include image format conversion, filtering and denoising and Histogram equalization, and finally the preprocessed image is sent to the face detector module;

Described human face detector module is responsible for carrying out human face detection to the image of input, and detection result is sent into video display module; Detection result comprises: detected human face number, detects the confidence degree and detection of each human face to the position information of each face; wherein, the position information of the face is the parameter of the rectangular frame containing the face, that is, the coordinates (x value, y value) and The length and width of the rectangle.

Further, the video display module includes a video processing module and a display adjustment module, wherein:

The video processing module is responsible for carrying out the following processing to the original image according to the detection result of face detection:

a. If no face is detected, the original image will not be processed, and will be directly sent to the display adjustment module;

b. If one or more faces are detected, draw the face with the highest confidence probability in the original image with a green rectangle frame, cut it out and output it to the face recognition module; frame the rest of the detected faces with a blue rectangle frame in the original image draw out;

The display adjustment module is responsible for displaying the processing result of the original image by the video processing module, and displays the processing result to the corresponding position of the video display area of the system display interface according to the camera number to which the original image belongs, and can maximize the image according to the user's instruction. Or minimize the video display area of any camera.

Further, the database management module includes a database operation module, which has four functions of registering, modifying, deleting, and displaying the current user, wherein:

The database operation module is used to respond to the corresponding instructions of the administrator, including registering new users, modifying user information, deleting selected users and displaying a list of currently registered user data;

The registration function is to register a new user. This function needs to provide a face image of a new user. The user's face image can be collected by a camera on the spot, and can also be uploaded by the new user; The user name; then, the system will send this face image to the feature extractor of the face recognition module to generate a 128-dimensional feature code; the system will automatically assign a unique user ID to the user; finally, by The database operation module stores the relevant information of this user in the database, wherein the relevant information includes user ID, user name, feature code and user face image;

The modification function, that is, modifying user information, except that user ID cannot be modified, user name, feature code and user face image can be modified accordingly;

The deletion function is to delete a selected piece of user data;

The function of displaying the current user is to display all registered user information in the current database on the database management interface.

Further, the face recognition module includes a feature extractor, a database module and a visitor information generation module, wherein:

The feature extractor is responsible for converting the input face image into a 128-dimensional feature code;

The database module is responsible for storing user information on the one hand, and is responsible for retrieving the user database on the other hand, finds the user data most similar to a given feature code, and calculates the cosine similarity of two feature codes respectively, one is a given feature code , the other is a certain feature code in the database, and the similarity of the two feature codes corresponding to the face or identity is obtained, so as to determine the face or identity corresponding to the given feature code; if the similarity is greater than the preset threshold, Then take the one with the highest similarity as the recognition result. If there is no user data with a similarity greater than the threshold in the database, the recognition result is that the user is not registered;

The visitor information generating module is responsible for sending the identified user identity information and the current time to the visitor information display module and the log printing module.

Further, the visitor information display module is responsible for displaying visitor information to the visitor information display area of the system display interface, if the user has not visited today, then the user ID, user name, similarity, visit time, detection The user's face image and the user's face image in the database are added to the visitor information display area together; if the visitor has visited today, it will not be added.

Further, the log printing module is responsible for recording each piece of detected and identified user data, including user ID, user name, similarity and visit time, and printing all user visit data today into a log file, and finally Put it under the folder named today's date.

Further, the video display area is responsible for displaying the content output by the video display module on the system interface; the LOGO display area is responsible for displaying the name of the visitor system on the system interface; the visitor information display area is responsible for displaying the name of the visitor on the system interface. The content output by the information display module; the user information input column is responsible for receiving the user name input by the user and sending it to the database management module; the operation area is responsible for interacting with the administrator to complete the user's operation on the database management module.

Compared with the prior art, the present invention has the following advantages and beneficial effects:

1. The identity of the visitor is authenticated by face detection and recognition, which greatly reduces the requirements for the identity authentication information of the visitor; the non-contact identity authentication method is used, which greatly improves the friendliness of the visitor.

2. Using biometrics as the basis of identity authentication can effectively prevent others from fraudulently using or stealing identity information.

3. Visitors can use their own face as a pass. By using the visitor system described in the present invention, they can conveniently and efficiently visit various application scenarios. "passage" non-stop passage, greatly improving the reception efficiency of visitors.

4. Through the video display module, the administrator can see the traffic situation of multiple entrances in real time, which is convenient for the administrator to receive and manage the visitors.

5. Through the database management module, administrators can easily add, modify, delete, and view information in the database.

6. Through the visitor information display module and the log printing module, administrators can easily understand and view today's visitor information.

Description of drawings

Figure 1 is a schematic diagram of the relationship between each module of the system.

Figure 2 shows the system display interface.

Figure 3 is a schematic diagram of the system display interface.

Detailed ways

The present invention will be further described below in conjunction with specific examples.

The intelligent visitor system applicable to multiple scenarios provided by this embodiment is a visitor system developed on a cloud server or a terminal device (such as NVIDIA Jetson TX2) developed using Pycharm software and Python language. As shown in Figure 1, the system is connected to The camera and internal modules realize the face detection and identity recognition of visitors, so as to realize the display, recording and management of visitor information. It includes:

The face detection module mainly realizes the automatic detection of the face in the video stream in the external camera, and outputs the position of the face in the video image and the confidence of the face. The position of the face includes the coordinates (x coordinate and y coordinate) of the upper left corner of the detected face rectangle frame relative to the upper left corner of the original image and the length and width of the rectangle frame; the confidence of the face indicates that the face frame output by the system contains a human face The probability;

The video display module mainly realizes the visualization of the output result of the face detection module, and outputs the face image cut out from the original image; among them, this module can display the detection of up to four cameras (including USB cameras and remote network cameras) As a result, it can maximize the display area of the detection results of any one of the cameras, which is convenient for the administrator to view the overall visitor information and the visitor information of a certain camera;

The database management module mainly realizes the registration, deletion, modification and display viewing operations of user data in the database;

The face recognition module mainly realizes the identification of the identity of the input face image in the database, and its identity information includes: user ID and user name. Output the similarity between this person and the identified identity in the database, as well as the person's visit time and the person's identity information in the database;

The visitor information display module mainly realizes the display of visitor information to the visitor information display area of the system display interface. The visitor information includes the identity information output by the face recognition module, the similarity and the visitor's visit time;

The log printing module mainly records the information of all visitors identified by the face recognition module, including identity information, similarity and visit time; archives by date and prints out;

The system display interface, as shown in Figure 2 and Figure 3, mainly realizes the information interaction between the visitor system and the user. The interface includes five parts: video display area, LOGO display area, visitor information display area, user information input column, and operation area ,details as follows:

The video display area is responsible for displaying the content output by the video display module on the system interface;

The LOGO display area is responsible for displaying the name of the visitor system on the system interface;

The visitor information display area is responsible for displaying the output content of the visitor information display module on the system interface;

The user information input column is responsible for receiving the user name input by the user and sending it to the database management module;

The operation area is responsible for interacting with the administrator to complete the user's operation on the database management module.

The face detection module includes a camera or a video stream reading module and a face detector module, wherein:

The camera or video stream reading module is responsible for reading images frame by frame from the video stream of the camera, and then preprocessing each frame of image data. The preprocessing steps are: first, convert the data format of the image into a specified data type (such as float32) and color channel order (RGB); then perform Gaussian filtering on the image to remove the noise contained in it; then perform a histogram on the image Equalization eliminates the influence of light factors to a certain extent. Finally, the preprocessed image is sent to the face detector module;

The human face detector module is responsible for carrying out human face detection to the input image, and sending the detection result into the video display module; using Multi-task convolutional neural network (MTCNN) as the human face detector. The detection results include: the number of detected faces, the confidence level of each detected face, and the position information of each detected face; where the position information of a face is a rectangular frame containing the face Parameters: the coordinates (x value, y value) of the upper left corner of the rectangular frame relative to the upper left corner of the original image and the length and width of the rectangular frame.

The video display module includes a video processing module and a display adjustment module, wherein:

The video processing module is responsible for carrying out the following processing to the original image according to the detection result of face detection:

a. If no face is detected, the original image will not be processed, and will be directly sent to the display adjustment module;

b. If one or more faces are detected, draw the face with the highest confidence probability in the original image with a green rectangle frame, cut it out and output it to the face recognition module; frame the rest of the detected faces with a blue rectangle frame in the original image Picture drawn.

The display adjustment module is responsible for displaying the processing result of the original image by the video processing module. It displays the processing result on the corresponding position of the video display area of the system display interface according to the camera number to which the original image belongs (that is, the source of the input video stream). When the user clicks a certain video display area, if the video display area is minimized, the video display area is maximized; if the video display area is maximized, the video display area is minimized.

The database management module includes a database operation module, which has four functions of registering, modifying, deleting and displaying current users, wherein:

The database operation module is used to respond to the corresponding instructions of the administrator, including registering new users, modifying user information, deleting selected users and displaying a list of currently registered user data;

The registration function is to add a new user in the database. This function needs to provide a face image of a new user. This user face image can be collected by a camera on the spot, or can be uploaded by the new user by himself; The user name that the user wants to register; then, the system will send the face image to the feature extractor of the face recognition module to generate a 128-dimensional feature code; the system will automatically assign a unique user ID to the user ; At last, the user's relevant information (comprising user ID, user name, feature code, user face image) is stored in the database by the database operation module;

The modification function, i.e. modifying the user information, except that the user ID cannot be modified, the user name, feature code and user face image can be modified accordingly;

The deletion function is to delete a selected piece of user data;

The function of displaying the current user is to display all registered user information in the current database on the database management interface, and the registered user information includes user ID, user name and user face image saved during registration.

The face recognition module includes a feature extractor, a database module and a visitor information generation module, wherein:

The feature extractor is responsible for converting the input face image into a 128-dimensional feature code, using FaceNet: AUnified Embedding for Face Recognition and Clustering (FaceNet) as a feature extractor;

The database module uses mySQL as the database model, and its functions are responsible for storing user information on the one hand, and retrieving the user database on the other hand to find the user data most similar to a given feature code. By calculating the cosine similarity of two feature codes (one is a given feature code and the other is a certain feature code in the database) respectively, the similarity of the two feature codes corresponding to the face or identity is obtained, so as to determine the given Determine the face or identity corresponding to the feature code; if the degree of similarity is greater than a preset threshold (such as 0.85), then take the one with the largest degree of similarity as the recognition result. If there is no user data with similarity greater than the threshold in the database, the identification result is that the user is not registered;

The visitor information generating module is responsible for sending the identified user identity information and the current time to the visitor information display module and the log printing module.

The visitor information display module is responsible for displaying visitor information in the visitor information display area of the system display interface. If the user has not visited today, then add the user's user ID, user name, similarity, visit time, detected user face image and user face image in the database to the visitor information display area; Do not add if already visited today.

The log printing module is responsible for recording each detected and identified user data, including user ID, user name, similarity, and visit time, and prints all user visit data today into a log file, and finally puts Under the folder named today's date.

The above-described embodiments are only preferred embodiments of the present invention, and are not intended to limit the scope of the present invention. Therefore, all changes made according to the shape and principles of the present invention should be covered within the protection scope of the present invention.

Claims (8)

1. a kind of intelligent visitor system suitable for more scenes characterized by comprising

Face detection module, the face position for realizing the automatic detection to face in input video stream, in output video image It sets；

Video display module for realizing the visualization to face detection module output result, and is exported and is cut out from original image Facial image；Wherein, which can at most show the testing result of four-way CCD camera, including USB camera and telenet Network camera, and the display area of any of them camera detection result all the way can be maximized；

Database management module checks operation for realizing registration, deletion, modification and the display to user data in database；

Face recognition module, for realizing to input facial image in the database belonging to identity identification, identity information packet It includes: User ID and user name；Output this person and the similarity and this person's visiting time of identity in the database recognized and this The identity information of people in the database；

Visitor information display module, for realizing the identity information of display face recognition module output, similarity and visiting personnel Visiting time；

Log print module, for realizing the information of all visitors that record face recognition module recognizes, including identity information, Similarity and visiting time print out after being filed by date；

System display interface, for realizing the information exchange of the guest system and user, which includes video viewing area, LOGO Viewing area, visitor information viewing area, user information input field and operating space.

2. a kind of intelligent visitor system suitable for more scenes according to claim 1, it is characterised in that: the face inspection Surveying module includes camera or video flowing read module and human-face detector module, in which:

What the camera or video flowing read module were responsible for generating from USB camera or telecommunication network camera or video file It is successively read image in video flowing, then image data is pre-processed, pre-treatment step includes picture format conversion, filtering Pretreated image is finally sent into human-face detector module by denoising and histogram equalization；

The human-face detector module is responsible for carrying out Face datection to the image of input, and will test result and be sent into video display mould Block；Testing result includes: the face number detected, detects the confidence level of each face and detect each face Location information；Wherein, the location information of face is to contain the parameter of the rectangle frame of this face, i.e., the rectangle frame upper left corner relative to The length and width of coordinate (x value, the y value) and rectangle frame in the original image upper left corner.

3. a kind of intelligent visitor system suitable for more scenes according to claim 1, it is characterised in that: the video is aobvious Show that module includes video processing module and display adjustment module, in which:

The video processing module is responsible for carrying out the following processing original image according to the testing result of Face datection:

If face a, is not detected, original image is not processed, and is sent directly into display adjustment module；

If b, detecting one or more faces, the maximum face of fiducial probability is drawn with green rectangle frame in original image, It cuts and is output to face recognition module；The face that remaining is detected is drawn with blue rectangle frame in original image；

The display adjustment module is responsible for showing processing result of the video processing module to original image, the camera shooting according to belonging to original image Processing result, is shown to the corresponding position of the video viewing area of system display interface, and can be according to the finger of user by head number Enable the video viewing area for maximizing or minimizing any camera all the way.

4. a kind of intelligent visitor system suitable for more scenes according to claim 1, it is characterised in that: the database Management module includes database operating modules, has registration, modification, deletes, display four Xiang Gongneng of active user, in which:

The database operating modules are used for the command adapted thereto of response management person, including register new user, modification user information, delete Except the user data list for choosing user current registered with display；

The registering functional registers new user, this function needs to provide the face image of a new user, user face figure As camera acquisition can be carried out at the scene, can also be uploaded by new user self-help；Also to input the user that user wants registration Name；Then, this facial image can be sent into the feature extractor of face recognition module by system, generate the feature of one 128 dimension Code；System automatically can distribute a unique User ID for the user；Finally, by database operating modules by the phase of this user Information storage is closed into database, wherein the relevant information includes User ID, user name, condition code and user's face image；

The modification function, i.e. modification user information, in addition to User ID, user name, condition code and user face figure cannot be modified As that can be modified accordingly；

The deletion function deletes certain user data chosen；

Display active user's function, i.e., registered all user's letters in data base administration interface display current database Breath.

5. a kind of intelligent visitor system suitable for more scenes according to claim 1, it is characterised in that: the face is known Other module, including feature extractor, database module and visitor information generation module, in which:

The feature extractor is responsible for converting the facial image of input to the condition code of 128 dimensions；

On the one hand the database module is responsible for storage user information, be on the other hand responsible for retrieval customer data base, find and to Determine the most like user data of condition code, by calculating separately the cosine similarity of two condition codes, one is given feature Code, the other is some condition code in database, obtains the similarity degree that the two condition codes correspond to face or identity, from And determine the given corresponding face of condition code or identity；If similarity degree is greater than preset threshold value, similarity is taken most Big is used as recognition result, if the user data for not having similarity degree to be greater than threshold value in database, recognition result is the use Family is unregistered；

The visitor information generation module is responsible for the subscriber identity information that will be recognized and the current time sends jointly to visitor Information display module and log print module.

6. a kind of intelligent visitor system suitable for more scenes according to claim 1, it is characterised in that: visitor's letter Breath display module is responsible for for visitor information being shown to the visitor information viewing area of system display interface, if the user does not arrive today It visits, then it will be in the User ID of the user, user name, similarity, visiting time, the user's face image and database that detect User's face image is added to visitor information viewing area together；If today has visited, do not add.

7. a kind of intelligent visitor system suitable for more scenes according to claim 1, it is characterised in that: the log is beaten Impression block is responsible for recording the user data that each is detected and identified, including when User ID, user name, similarity and visiting Between, and all users of today are visited into data-printing into a log document, finally it is placed on the file of date today name Under folder.

8. a kind of intelligent visitor system suitable for more scenes according to claim 1, it is characterised in that: the video is aobvious Show that area is responsible for showing the content of video display module output on system interface；It is responsible on system interface the viewing area LOGO Show the title of the guest system；It is responsible for showing that visitor information display module is defeated on system interface in the visitor information viewing area Content out；The user information input field is responsible for receiving the user name of user's input, and is sent to database management module；Institute It states operating space to be responsible for interacting with administrator, completes operation of the user to database management module.