CN109508638B - Face emotion recognition method and device, computer equipment and storage medium - Google Patents
Face emotion recognition method and device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN109508638B CN109508638B CN201811183763.9A CN201811183763A CN109508638B CN 109508638 B CN109508638 B CN 109508638B CN 201811183763 A CN201811183763 A CN 201811183763A CN 109508638 B CN109508638 B CN 109508638B
- Authority
- CN
- China
- Prior art keywords
- face
- detected
- video data
- face image
- emotion
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/174—Facial expression recognition
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Biophysics (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Multimedia (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Human Computer Interaction (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a face emotion recognition method, a device, computer equipment and a storage medium, wherein video data to be detected are firstly obtained, and at least one face image to be detected is extracted from the video data to be detected; detecting a face image to be detected by adopting a face detection algorithm to obtain N target face image areas of video data to be detected, wherein N is a positive integer; video interception is carried out on the video data to be detected according to the N target face image areas, and N face video data to be identified are obtained; and inputting the N pieces of face video data to be recognized into a face emotion recognition model to recognize, and obtaining N pieces of face emotion information in the video to be detected, wherein the face emotion recognition model is obtained by training a convolutional neural network-long and short-time recurrent neural network model. The method directly identifies the video data, avoids the shortage of information represented by a single frame of picture, and ensures the accuracy of facial emotion identification.
Description
Technical Field
The present invention relates to the field of microexpressive recognition, and in particular, to a method and apparatus for recognizing facial emotion, a computer device, and a storage medium.
Background
With the continuous development of artificial intelligence technology, micro-expression recognition technology is adopted as an auxiliary tool or auxiliary means in many fields. However, at present, when detecting the micro-expression of the face, the face emotion is mostly detected through a single image or a single frame picture in a video, and the face emotion has a change process, and the condition of the face emotion in the video can not be accurately reflected through the single frame picture, so that the recognition accuracy of the face emotion recognition is poor.
Disclosure of Invention
The embodiment of the invention provides a face emotion recognition method, a device, computer equipment and a storage medium, which are used for solving the problem of low recognition accuracy in the face emotion recognition process.
A face emotion recognition method, comprising:
Acquiring video data to be detected, and extracting at least one face image to be detected from the video data to be detected;
Detecting the face image to be detected by adopting a face detection algorithm to obtain N target face image areas of the video data to be detected, wherein N is a positive integer;
carrying out video interception on the video data to be detected according to the N target face image areas to obtain N face video data to be identified;
And inputting the N face video data to be identified into a face emotion recognition model to be identified, and obtaining N face emotion information in the video data to be detected, wherein the face emotion recognition model is obtained by training a convolutional neural network-long and short time recurrent neural network model.
A facial emotion recognition device, comprising:
A to-be-detected video data acquisition module for acquiring to-be-detected video data, extracting at least one face image to be detected from the video data to be detected;
The target face image area acquisition module is used for detecting the face image to be detected by adopting a face detection algorithm to obtain N target face image areas of the video data to be detected, wherein N is a positive integer;
The face video data interception module to be identified is used for intercepting the video of the video data to be detected according to N target face image areas to obtain N face video data to be identified;
the face emotion information acquisition module is used for inputting the N face video data to be identified into a face emotion identification model for identification to obtain N face emotion information in the video data to be detected, wherein the face emotion identification model is obtained by training a convolutional neural network-long-short-time recurrent neural network model.
A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, the processor implementing the steps of the facial emotion recognition method described above when the computer program is executed.
A computer readable storage medium storing a computer program which, when executed by a processor, implements the steps of the facial emotion recognition method described above.
In the face emotion recognition method, the face emotion recognition device, the computer equipment and the storage medium, firstly, video data to be detected are obtained, and face images to be detected are extracted from the video data to be detected; detecting a face image to be detected by adopting a face detection algorithm to obtain N target face image areas, wherein N is a positive integer; video interception is carried out on the video data to be detected according to the N target face image areas, and N face video data to be identified are obtained; and inputting the N pieces of face video data to be recognized into a face emotion recognition model to recognize, and obtaining N pieces of face emotion information in the video to be detected, wherein the face emotion recognition model is obtained by training a convolutional neural network-long and short-time recurrent neural network model. By intercepting the face video data to be detected according to the target face image area, the accuracy of intercepting each face video data to be identified is ensured, the face emotion recognition model for subsequent recognition is obtained by training a convolutional neural network-long-short-time recurrent neural network model, and the method directly recognizes video data, so that the defect of insufficient information represented by a single frame of picture is avoided, and the accuracy of face emotion recognition is ensured.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the description of the embodiments of the present invention will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic view of an application environment of a face emotion recognition method according to an embodiment of the present invention;
FIG. 2 is a diagram illustrating an example of a face emotion recognition method according to an embodiment of the present invention;
FIG. 3 is a diagram showing another example of a face emotion recognition method according to an embodiment of the present invention;
FIG. 4 is a diagram showing another example of a face emotion recognition method according to an embodiment of the present invention;
FIG. 5 is a diagram showing another example of a face emotion recognition method according to an embodiment of the present invention;
FIG. 6 is a diagram showing another example of a face emotion recognition method according to an embodiment of the present invention;
FIG. 7 is a schematic block diagram of a facial emotion recognition device according to an embodiment of the present invention;
FIG. 8 is another schematic block diagram of a facial emotion recognition device in an embodiment of the present invention;
FIG. 9 is another schematic block diagram of a facial emotion recognition device in an embodiment of the present invention;
FIG. 10 is a schematic diagram of a computer device in accordance with an embodiment of the invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are some, but not all embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The facial emotion recognition method provided by the embodiment of the invention can be applied to an application environment as shown in fig. 1, wherein a client (computer equipment) communicates with a server through a network. The client sends the video data to be detected to the server, and the server detects the face image to be detected by adopting a face detection algorithm to obtain N target face image areas; video interception is carried out on the video data to be detected according to the N target face image areas, and N face video data to be identified are obtained; and finally, inputting the N pieces of face video data to be recognized into a face emotion recognition model for recognition to obtain N pieces of face emotion information in the video to be detected. Among other things, clients (computer devices) may be, but are not limited to, various personal computers, notebook computers, smartphones, tablet computers, and portable wearable devices. The server may be implemented by a stand-alone server or a server cluster formed by a plurality of servers.
In an embodiment, as shown in fig. 2, a face emotion recognition method is provided, and the method is applied to the server in fig. 1 for illustration, and includes the following steps:
S10: and acquiring video data to be detected, and extracting at least one face image to be detected from the video data to be detected.
The video data to be detected is original video data, and optionally, the video data to be detected may be video data containing a face image of a person, which is acquired by a client in real time by using a video acquisition tool of the video data to be detected, or may be video data acquired and stored in advance by the client, or video data directly uploaded locally or transmitted to the client. The client sends the video data to be detected to the server, and the server acquires the video data to be detected.
The face image to be detected is image data containing a face image of a person extracted from video data to be detected. Optionally, the number of the face images to be detected is at least one, and an image which can most represent the face position of the person in the video data to be detected is extracted from the video data to be detected as the face image to be detected. Alternatively, a plurality of face images may be extracted from the video data to be detected, and then a face image which is the most representative may be selected as the face image to be detected. Specifically, an image in which the face images are all frontally oriented may be selected from the plurality of face images as the face image to be detected. The face orientation detection can be realized by training the corresponding neural network model in advance to obtain a face orientation recognition model. The face orientation recognition model is obtained by marking a large amount of image data representing different face orientations and inputting the marked image data into a neural network model for training.
In a specific embodiment, the number of face images to be detected is at least two, and the accuracy of subsequent emotion recognition is improved by extracting a plurality of face images from the video data to be detected as the face images to be detected. Preferably, any two stages from the initial stage, the intermediate stage and the stage in the video data to be detected can be selected to extract one face image as the face image to be detected. It can be understood that the number of face images to be detected may be greater, that is, the greater the number of face images to be detected, the higher the accuracy of emotion recognition on the video data to be detected later, however, the higher the computational complexity of the server side may be, and the specific number may be set according to different application scene requirements. If the recognition accuracy is focused, the number of face images to be detected can be increased, and if the recognition efficiency is focused, the number of face images to be detected can be properly reduced.
Specifically, the server side can extract the face image to be detected from the video data to be detected in a screen capturing mode. The process of acquiring the face image to be detected can be realized through OpenCV, and the OpenCV provides a simple and easy-to-use framework for extracting the image frames in the video file. Illustratively, videoCapture classes are employed for video read and write operations. Firstly, a cap=cv2.videocapture () function in VideoCapture classes is adopted to display corresponding video data to be detected, and then the video data to be detected is read according to a preset frame rate through a cap.read () function in VideoCapture classes, wherein two return values of the cap.read () function are provided: ret and frame. And if the video data to be detected is read to the end, the return value is False, and whether the video data to be detected is read completely can be judged through the return value of a cap.read () function. The frame is the currently truncated image, which may be a three-dimensional matrix.
S20: and detecting the face image to be detected by adopting a face detection algorithm to obtain N target face image areas of the video data to be detected, wherein N is a positive integer.
The face detection algorithm is a detection algorithm for selecting a face region from image data. Specifically, the face detection algorithm may be a face detection algorithm based on feature extraction, a face detection algorithm based on a neural network, or a face detection algorithm based on deep learning. Detecting a face image to be detected through a preset face detection algorithm, and detecting a face region in the face image to be detected, so as to obtain a target face region. N is the number of target face areas included in the face image to be detected, and N is a positive integer. It will be appreciated that the target face region may be of various types, for example: the shape of the circle, rectangle, square, etc. can be specifically set according to actual needs. Preferably, the target face image area is rectangular.
S30: and carrying out video interception on the video data to be detected according to the N target face image areas to obtain N face video data to be identified.
After N target face areas are obtained, video interception is carried out in the video data to be detected according to each target face area, and N face video data to be identified are obtained. Specifically, if a target face area is a rectangular frame a, capturing video data at a position corresponding to the rectangular frame a in the video data to be detected, and obtaining face video data to be identified. And circularly intercepting the video data to obtain N faces to be recognized.
In one embodiment, a filter function in FFmpeg may be employed to effect video interception of the video data to be detected. FFmpeg is a set of open source computer programs that can be used to record, convert digital audio, video, and convert it to streams. And implementing video interception of the video data to be detected by adopting a loop function in the filter. Specifically, video interception of video data to be detected according to the target face image area is achieved through crop=width: height: x: y. Where width and height represent the dimensions after clipping and x: y represents the upper left corner coordinates of the clipping region. The video interception area is determined according to the size of the face image area and the left upper corner coordinate, so that video interception of the video data to be detected is realized according to the N target face image areas.
S40: and inputting the N pieces of face video data to be recognized into a face emotion recognition model to recognize, and obtaining N pieces of face emotion information in the video data to be detected, wherein the face emotion recognition model is obtained by training a convolutional neural network-long-short-time recurrent neural network model.
The face emotion recognition model is a network model which is obtained through training in advance and is used for recognizing the face emotion in the face video data to be recognized, and a recognition result, namely face emotion information, is output. Illustratively, the facial emotion information is happy, sad, fear, angry, surprise, aversion, or contempt. The face emotion recognition model can judge probability values of faces corresponding to preset multiple emotions in the input video data, and if the probability value of a certain emotion exceeds a corresponding preset threshold value, the emotion corresponding to the input video data is obtained to be face emotion information. For example, in the present embodiment, the emotion in the microexpressive recognition model can be set to be happy, sad, fear, angry, surprise, aversion, and contempt. Specifically, a large amount of video data representing the 7 emotions can be collected in advance for labeling to form a video data set, and then a corresponding neural network model or classifier is selected for training, so that a face emotion recognition model is finally obtained.
Specifically, the face emotion recognition model is obtained by training a convolutional neural network-long and short recurrent neural network model. The convolutional neural network-long and short-term recurrent neural network model is a model obtained by combining the convolutional neural network model and the long and short-term recurrent neural network model. It is understood that the convolutional neural network-long and short recurrent neural network model is equivalent to a model formed by connecting a convolutional neural network with a long and short recurrent neural network model.
Convolutional neural network (Convolutional Neural Network, CNN)) is a locally connected network. The biggest features of the network are local connectivity and weight sharing compared with the fully connected network. For a certain pixel p in an image, the closer the pixel p is to it, the more affected (local connectivity) is. In addition, according to the statistical characteristics of the natural image, the weight of one region can be used for another region, namely weight sharing. Weight sharing can be understood as convolution kernel sharing, in a Convolutional Neural Network (CNN), one kind of image feature can be extracted by performing convolution operation on one convolution kernel and a given image, and different convolution kernels can extract different image features. The complexity of the model is reduced due to the local connectivity of the convolutional neural network, so that the model training efficiency can be improved; in addition, due to the weight sharing property of the convolutional neural network, the convolutional neural network can learn in parallel, and the model training efficiency is further improved.
A long-short-term recurrent neural network (LSTM) model is a model of a time recurrent neural network, which is suitable for processing and predicting important events having a time sequence with relatively long time sequence intervals and delays. The LSTM model has a time memory function, and the long-short-time recurrent neural network model is adopted to train the extracted features so as to embody the long-term memory capacity of the data and improve the accuracy of model identification.
In the step, N pieces of face video data to be recognized are respectively input into a face emotion recognition model for recognition, and N pieces of face emotion information in the video to be detected are obtained.
In this embodiment, first, video data to be detected is obtained, and at least one face image to be detected is extracted from the video data to be detected; detecting a face image to be detected by adopting a face detection algorithm to obtain N target face image areas of video data to be detected, wherein N is a positive integer; video interception is carried out on the video data to be detected according to the N target face image areas, and N face video data to be identified are obtained; and inputting the N pieces of face video data to be recognized into a face emotion recognition model to recognize, and obtaining N pieces of face emotion information in the video to be detected, wherein the face emotion recognition model is obtained by training a convolutional neural network-long and short-time recurrent neural network model. The method has the advantages that the video data to be detected are intercepted according to the target face image area, so that the accuracy of intercepting each face video data to be identified is guaranteed, the face emotion recognition model for subsequent recognition is obtained by training a convolutional neural network-long-short-time recurrent neural network model, the video data are directly recognized, the defect of information represented by a single frame picture is avoided, and the accuracy of face emotion recognition is further guaranteed.
In an embodiment, the number of face images to be detected is at least two. As shown in fig. 3, a face detection algorithm is adopted to detect a face image to be detected, so as to obtain N target face image areas of video data to be detected, and the method specifically includes the following steps:
S21: and detecting each face image to be detected by adopting a face detection algorithm to obtain N initial face image areas of each face image to be detected.
In the step, a face detection algorithm is adopted to detect each face image to be detected, and N initial face image areas of each face image to be detected are obtained.
S22: and recognizing N initial face image areas of each face image to be detected by adopting a face recognition algorithm to obtain the user identification of each initial face image area.
The user identifier is a unique identifier for distinguishing different users, specifically, the user identifier may be a mobile phone number, an identification card number or a unique identifier allocated by a system to different users, and optionally, the unique identifier may be composed of at least one of a number, a letter, a text or a symbol. The face recognition algorithm is used for comparing different face images and obtaining a comparison result. Specifically, the face recognition algorithm may be implemented by using a geometric feature-based recognition algorithm, a local feature analysis-based recognition algorithm, a feature face-based recognition algorithm, a neural network-based recognition algorithm, or an elastic model-based recognition algorithm. Optionally, a face image library may be preset, corresponding user identifications and reference face images are stored, then N initial face image areas of each face image to be detected are respectively identified with each reference face image in the face image library by adopting a face recognition algorithm, and the user identification corresponding to the reference face image with the highest matching degree is used as the user identification corresponding to the initial face image area.
In a specific embodiment, a corresponding user identifier is allocated to each initial face image area of the first face image to be detected, and then each initial face image area and the corresponding user identifier in the first face image to be detected are used as a face image library. Specifically, the first face image to be detected may be determined according to a time sequence of the images in the video data to be detected, for example, the face image to be detected with the forefront time sequence is selected as the first face image to be detected. And respectively identifying each initial face image area of the subsequent face image to be detected with the face image library, and taking the user identification with the highest matching degree in the face image library as the user identification corresponding to the initial face image area. The identification process of one image can be reduced, and the identification efficiency is improved.
S23: and carrying out image region integration on the initial face image regions corresponding to the same user identification in the face image to be detected, and obtaining N target face image regions of the video data to be detected.
In the step, the face image areas of the same user identification in different face images to be detected are integrated by taking the user identification as a unit, so that a target face image area is obtained. And after the image area integration is carried out on the face image areas corresponding to the N user identifications, N target face image areas are obtained. The image region integration refers to a process of summarizing and calculating a plurality of initial face image regions, so that the obtained target face image region can better reflect face information of a corresponding user, and further, the accuracy of subsequent recognition is guaranteed. Specifically, the face image areas of the same user identification may be subjected to an averaging process. For example, if the face image area is a rectangular area, four vertex coordinates of each face image area are obtained, the vertex coordinates at the same position are summed and then averaged to obtain four averaged vertex coordinates, and the target face image area is formed according to the four averaged vertex coordinates.
In one embodiment, the face image areas of the same user identification are maximized. For example, if the face image area is a rectangular area, four vertex coordinates of each face image area are obtained, and the vertex coordinates at the same position are subjected to a limiting value. And selecting all vertex coordinates to ensure that the acquired target face image area is maximum. For example, if the lower left corner of the image is taken as the origin position, it can be understood that the upper left corner vertex coordinates of each face image area, that is, the upper left corner vertex coordinates in which the abscissa is the smallest and the ordinate is the largest, are taken as the upper left corner vertex coordinates of the target face image area. And taking the maximum value of the abscissa and the maximum value of the ordinate as the upper right corner vertex coordinates of the target face image area for the upper right corner vertex coordinates of each face image area. And taking the minimum value of the abscissa and the minimum value of the ordinate as the left lower corner vertex coordinate of the target face image area for the left lower corner vertex coordinate of each face image area. And taking the value with the maximum abscissa and the value with the minimum ordinate as the left lower corner vertex coordinate of the target face image area for the right lower corner vertex coordinate of each face image area. The target face image area formed by the method can better ensure the integrity of face information and ensure the accuracy of subsequent identification.
In this embodiment, a face detection algorithm is first adopted to detect each face image to be detected, so as to obtain N initial face image areas of each face image to be detected; then adopting a face recognition algorithm to recognize N initial face image areas of each face image to be detected, and obtaining a user identification of each initial face image area; and finally, respectively integrating the initial face image areas corresponding to the same user identification in the face image to be detected into image areas to obtain N target face image areas. The target face area determined by the method can better ensure the accuracy of the subsequent face emotion recognition.
In an embodiment, as shown in fig. 4, after the step of inputting N pieces of face video data to be recognized into a face emotion recognition model to perform recognition, to obtain N pieces of face emotion information in the video to be detected, the face emotion recognition method further includes the following steps:
S50: and counting the N pieces of face emotion information in the video to be detected to obtain emotion statistic information.
The emotion statistical information refers to statistical information reflecting emotion types corresponding to different faces in the video to be detected. Alternatively, the emotion statistics may reflect the number of all different emotions, or reflect the proportion of a particular emotion. Specifically, N pieces of face emotion information in video data to be detected are obtained, statistics is performed on each piece of face emotion information in the N pieces of face emotion information, and the number of each piece of face emotion information is obtained, namely emotion statistics information is formed.
S60: and acquiring a preset emotion detection rule, and checking the emotion statistical information according to the preset emotion detection rule to obtain emotion recognition information of the video data to be detected.
The preset emotion detection rule is a preset detection rule, and can be specifically set according to different application scenes in a user-defined manner. The preset emotion detection rule can be obtained from a client or a database in a server. For example, if it is required to detect whether the happy emotion in the video data to be detected exceeds 80%, the preset emotion detection rule may be to detect whether the number of happy face emotion information in the emotion statistics information is more than 80% of all face emotion information. And outputting emotion recognition information according to the test result, wherein the emotion recognition information can be a result of reaching or not reaching the index, can also be a corresponding number of specific emotions, and can be set according to actual needs.
In the embodiment, the emotion statistical information is obtained by counting N pieces of face emotion information in the video data to be detected; and acquiring a preset emotion detection rule, and checking the emotion statistical information according to the preset emotion detection rule to obtain emotion recognition information of the video data to be detected. And verifying the emotion statistical information according to a preset emotion detection rule to obtain emotion recognition information of the video data to be detected so as to more intuitively and clearly reflect the emotion recognition result of the video data to be detected.
In an embodiment, as shown in fig. 5, before the step of inputting N pieces of face video data to be recognized into the face emotion recognition model to perform detection to obtain N pieces of face emotion information in the video data to be detected, the face emotion recognition method further includes the following steps:
S41: original video sample data are obtained, wherein each original video sample data is subjected to sample labeling.
The original video sample data is open source video data acquired from a data set disclosed by the internet or a third party mechanism/platform, and the open source video data comprises original video sample data corresponding to various facial emotions, and each original video sample data is marked with a sample, for example, the original video sample data A is marked with "happy" corresponding to the mark data.
S42: and carrying out video framing and face detection processing on each original video sample data to obtain a training face picture.
The training face picture is a picture containing facial features of a person, which is obtained by carrying out video framing and face detection on original video sample data. Because the facial emotion recognition model in the embodiment is trained based on the micro-expression features, video framing and face detection processing are required to be performed on original video sample data, and the obtained picture containing the facial features of the person is the training face picture, so that model training is performed by adopting the training face picture, the facial emotion recognition model can extract the micro-expression features based on the training face picture, deep learning is performed, and the recognition accuracy of the facial emotion recognition model is improved.
S43: and grouping the training face pictures according to the preset quantity to obtain target training data, wherein each target training data comprises training face pictures of continuous M frames.
The method comprises the steps of grouping according to a preset quantity, obtaining at least one group of target training data, and enabling each group of target training data to contain training face pictures of continuous M frames so as to obtain micro-expression characteristic changes of faces from the training face pictures of the continuous M frames, so that the training face pictures have time sequence, and the accuracy of a face emotion recognition model is improved.
In this embodiment, the preset number of ranges may be set to [50,200], which is because if training face images smaller than 50 frames are used as a set of training data in the training set, the face emotion recognition model is not high in recognition accuracy due to the fact that the training face images are too few and cannot show the change process of facial features of a person lying. If the training face picture with the frame more than 200 is used as a group of training data in the training set, the model training time is too long, and the model training efficiency is reduced. In this embodiment, model training is performed according to each hundred frames of training face pictures as a set of training data, so as to improve training efficiency of the model and recognition accuracy of the face emotion recognition model obtained by training.
S44: and inputting target training data into a convolutional neural network-long and short recurrent neural network model for training to obtain a face emotion recognition model.
After the target training data are obtained, the target training data are input into a convolutional neural network-long-short-time recurrent neural network model for training, and then the face emotion recognition model is obtained.
In this embodiment, since training is performed on training face images of continuous M frames, which is the target training data, feature extraction is performed on the training face images, and the convolutional neural network model is a neural network commonly used for image feature extraction, and the weight sharing and local connectivity of the convolutional neural network greatly increase the model training efficiency. In this embodiment, features of each frame of training face picture are closely related to features of the training face pictures of the front frame and the rear frame, so that long-short-time recurrent neural network models are adopted to train the extracted face features, long-term memory capacity of data is reflected, and accuracy of the models is improved. Because the weight sharing property and the local connectivity of the convolutional neural network and the long-time recurrent neural network model can embody the advantage of the long-time memory capacity of the data, the training efficiency of the face emotion recognition model and the accuracy of the face emotion recognition model obtained by training the convolutional neural network-long-time recurrent neural network model are greatly increased.
In one embodiment, as shown in fig. 6, the target training data is input into a convolutional neural network-long and short recurrent neural network model for training, and a face emotion recognition model is obtained, which specifically includes the following steps:
s441: model parameters of a convolutional neural network-long and short recurrent neural network model are initialized.
The initialization of the convolutional neural network-long and short-term recurrent neural network model refers to initializing the model parameters (namely, convolution kernel and offset) of the convolutional neural network model in advance and initializing the model parameters (namely, connection weights among layers) in the LSTM model. The convolution kernel refers to the weight of the convolution neural network, when training data is input, the weight is multiplied, namely the convolution kernel, and then the output of the neuron is obtained, which reflects the importance degree of the training data. The bias is a linear component used to alter the range of the weight-by-input. Based on the determined convolution kernel, offset and connection weights among layers in the LSTM model, the model training process can be completed.
S442: and carrying out feature extraction on the target training data by adopting a convolutional neural network to obtain the face features.
The face features are facial features obtained by extracting features of training face pictures of continuous N frames, which are target training data in a training set, by adopting a convolutional neural network. Specifically, a convolutional neural network is adopted to extract characteristics of target training data in a training set. Specifically, the calculation formula of the convolution operation includesWherein, represents convolution operation; x j represents the j-th input feature map; y j represents the j-th output feature map; w ij is the convolution kernel (weight) between the ith input feature map and the jth output feature map; b j represents the bias term for the j-th output profile. Then the convolved feature map is subjected to downsampling operation by adopting maximum pooling downsampling to realize dimension reduction of the feature map, wherein the calculation formula is as followsWherein y j represents the ith output spectrum (i.e. the feature map after downsampling) in the downsampling process, and each neuron in the downsampling process is obtained by locally sampling the ith input spectrum (the feature map after convolution) by using a downsampling frame of s×s, i.e. performing S times downsampling on the input spectrum, wherein a specific value of S can be set according to actual sampling requirements; m and n represent the step sizes of the downsampling frame movements, respectively.
S443: and inputting the facial features into the long-short-time recurrent neural network model for training to obtain a facial emotion recognition model.
Specifically, the LSTM model is one of neural network models having long-term memory capability, and has a three-layer network structure of an input layer, a hidden layer, and an output layer. The input layer is the first layer of the LSTM model and is used for receiving external signals, namely, face features carrying time sequence states. In this embodiment, since the training face image has time sequence, the face features obtained after the training face image is processed in step S442 also have time sequence, so that the training face image can be applied to the LSTM model, and the LSTM model can obtain the face features carrying the time sequence state. The output layer is the last layer of the LSTM model and is used for outputting signals, namely, the output layer is responsible for outputting the calculation result of the LSTM model. The hidden layer is each layer except the input layer and the output layer in the LSTM model and is used for processing the input face features to obtain the calculation result of the LSTM model. The original wind control model is obtained by carrying out multiple iterations on the face features carrying the time sequence state by adopting an LSTM model until convergence. Understandably, model training of the extracted face features by using the LSTM model enhances the time sequence of the obtained original wind control model, thereby improving the accuracy of the face emotion recognition model.
In this embodiment, the output layer of the LSTM model performs regression processing by using Softmax (regression model) for classifying the output weight matrix. The Softmax (regression model) is a classification function commonly used for a neural network, maps the output of a plurality of neurons into a [0,1] interval, can be understood as probability, and is simple and convenient to calculate, so that multi-classification output is performed, and the output result is more accurate.
In this embodiment, the convolutional neural network-long and short time recurrent neural network model is initialized first, so that the target training data in the training set is trained based on the convolutional neural network model, the face features are obtained, then the obtained face features are input into the LSTM model for training, the process does not need to manually extract the features, only the training face picture is directly input into the convolutional neural network-long and short time recurrent neural network model, the features can be automatically extracted by the model, and the training efficiency of the model is improved.
It should be understood that the sequence number of each step in the foregoing embodiment does not mean that the execution sequence of each process should be determined by the function and the internal logic, and should not limit the implementation process of the embodiment of the present invention.
In an embodiment, a face emotion recognition device is provided, and the face emotion recognition device corresponds to the face emotion recognition method in the embodiment one by one. As shown in fig. 7, the facial emotion recognition device includes a to-be-detected video data acquisition module 10, a target facial image area acquisition module 20, a to-be-recognized facial video data interception module 30, and a facial emotion information acquisition module 40. The functional modules are described in detail as follows:
the to-be-detected video data acquisition module 10 is used for acquiring to-be-detected video data and extracting at least one to-be-detected face image from the to-be-detected video data;
The target face image area obtaining module 20 is configured to detect the face image to be detected by using a face detection algorithm to obtain N target face image areas, where N is a positive integer;
The face video data interception module 30 to be identified is configured to intercept video of the video data to be detected according to N target face image areas, so as to obtain N face video data to be identified of the video data to be detected;
The facial emotion information acquisition module 40 is configured to input N pieces of facial video data to be identified into a facial emotion recognition model for recognition, so as to obtain N pieces of facial emotion information in the video data to be detected, where the facial emotion recognition model is obtained by training with a convolutional neural network-long-short-time recurrent neural network model.
Preferably, the number of the face images to be detected is at least two; as shown in fig. 8, the target face image area acquisition module 20 includes an initial face image area acquisition unit 21, an initial face image area recognition unit 22, and a target face image area acquisition unit 23.
An initial face image area obtaining unit 21, configured to detect each of the face images to be detected by using a face detection algorithm, so as to obtain N initial face image areas of each of the face images to be detected;
An initial face image area recognition unit 22, configured to recognize N initial face image areas of each face image to be detected by using a face recognition algorithm, so as to obtain a user identifier of each initial face image area;
And the target face image area obtaining unit 23 is configured to integrate the initial face image areas corresponding to the same user identifier in the face image to be detected into an image area, so as to obtain N target face image areas of the video data to be detected.
Preferably, as shown in fig. 9, the facial emotion recognition device further includes an emotion statistics information acquisition module 50 and an emotion recognition information acquisition module 60.
The emotion statistical information acquisition module 50 is configured to perform statistics on N face emotion information in the video data to be detected, so as to obtain emotion statistical information;
And the emotion recognition information acquisition module 60 is configured to acquire a preset emotion detection rule, and verify the emotion statistics information according to the preset emotion detection rule to obtain emotion recognition information of the video data to be detected.
Preferably, the facial emotion recognition device further comprises an original video sample data acquisition module, a training facial picture acquisition module, a target training data acquisition module and a facial emotion recognition model acquisition module.
The system comprises an original video sample data acquisition module, a video processing module and a video processing module, wherein the original video sample data acquisition module is used for acquiring original video sample data, and each original video sample data is subjected to sample labeling.
The training face picture acquisition module is used for carrying out video framing and face detection processing on each original video sample data to obtain a training face picture.
The target training data acquisition module is used for grouping training face pictures according to the preset quantity to acquire target training data, and each target training data comprises training face pictures of continuous M frames.
The face emotion recognition model acquisition module is used for inputting target training data into the convolutional neural network-long-short-time recurrent neural network model for training to acquire a face emotion recognition model.
Preferably, the facial emotion recognition model acquisition module comprises a model parameter initialization unit, a facial feature acquisition unit and a facial emotion recognition model acquisition unit.
And the model parameter initializing unit is used for initializing model parameters of the convolutional neural network-long and short recurrent neural network model.
The face feature acquisition unit is used for carrying out feature extraction on the target training data by adopting the convolutional neural network to acquire face features.
The facial emotion recognition model acquisition unit is used for inputting facial features into the long-short-time recurrent neural network model for training to acquire a facial emotion recognition model.
For specific limitations of the facial emotion recognition device, reference may be made to the above limitations of the facial emotion recognition method, and no further description is given here. The modules in the facial emotion recognition device can be realized in whole or in part by software, hardware and a combination thereof. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.
In one embodiment, a computer device is provided, which may be a server, and the internal structure of which may be as shown in fig. 10. The computer device includes a processor, a memory, a network interface, and a database connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database of the computer device is used for storing the data used in the face emotion recognition method. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a face emotion recognition method.
In one embodiment, a computer device is provided comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing the steps of when executing the computer program:
Acquiring video data to be detected, and extracting at least one face image to be detected from the video data to be detected;
Detecting the face image to be detected by adopting a face detection algorithm to obtain N target face image areas of the video data to be detected, wherein N is a positive integer;
carrying out video interception on the video data to be detected according to the N target face image areas to obtain N face video data to be identified;
And inputting the N face video data to be identified into a face emotion recognition model to be identified, and obtaining N face emotion information in the video data to be detected, wherein the face emotion recognition model is obtained by training a convolutional neural network-long and short time recurrent neural network model.
In one embodiment, a computer readable storage medium is provided having a computer program stored thereon, which when executed by a processor, performs the steps of:
Acquiring video data to be detected, and extracting at least one face image to be detected from the video data to be detected;
Detecting the face image to be detected by adopting a face detection algorithm to obtain N target face image areas of the video data to be detected, wherein N is a positive integer;
carrying out video interception on the video data to be detected according to the N target face image areas to obtain N face video data to be identified;
And inputting the N face video data to be identified into a face emotion recognition model to be identified, and obtaining N face emotion information in the video data to be detected, wherein the face emotion recognition model is obtained by training a convolutional neural network-long and short time recurrent neural network model.
Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in embodiments provided herein may include non-volatile and/or volatile memory. The nonvolatile memory can include Read Only Memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (SYNCHLINK) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), among others.
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-described division of the functional units and modules is illustrated, and in practical application, the above-described functional distribution may be performed by different functional units and modules according to needs, i.e. the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-described functions.
The above embodiments are only for illustrating the technical solution of the present invention, and not for limiting the same; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention, and are intended to be included in the scope of the present invention.
Claims (10)
1. A method for identifying a facial emotion, comprising:
Acquiring video data to be detected, and extracting at least one face image to be detected from the video data to be detected;
Detecting the face image to be detected by adopting a face detection algorithm to obtain N target face image areas of the video data to be detected, wherein N is a positive integer;
carrying out video interception on the video data to be detected according to the N target face image areas to obtain N face video data to be identified;
And inputting the N face video data to be identified into a face emotion recognition model to be identified, and obtaining N face emotion information in the video data to be detected, wherein the face emotion recognition model is obtained by training a convolutional neural network-long and short time recurrent neural network model.
2. The face emotion recognition method of claim 1, wherein the number of face images to be detected is at least two;
the face detection algorithm is adopted to detect the face image to be detected, and N target face image areas of the video data to be detected are obtained, and the method specifically comprises the following steps:
Detecting each face image to be detected by adopting a face detection algorithm to obtain N initial face image areas of each face image to be detected;
Recognizing N initial face image areas of each face image to be detected by adopting a face recognition algorithm to obtain a user identification of each initial face image area;
And integrating the initial face image areas corresponding to the same user identification in the face image to be detected into an image area to obtain N target face image areas of the video data to be detected.
3. The face emotion recognition method as set forth in claim 1, wherein after the step of inputting N pieces of the face video data to be recognized into a face emotion recognition model for recognition to obtain N pieces of face emotion information in a video to be detected, the face emotion recognition method further comprises the steps of:
Counting the N pieces of face emotion information in the video data to be detected to obtain emotion statistic information;
and acquiring a preset emotion detection rule, and checking the emotion statistical information according to the preset emotion detection rule to obtain emotion recognition information of the video data to be detected.
4. The face emotion recognition method as set forth in claim 1, wherein before the step of inputting N pieces of the face video data to be recognized into a face emotion recognition model for detection to obtain N pieces of face emotion information in the video data to be detected, the face emotion recognition method further comprises the steps of:
obtaining original video sample data, wherein each original video sample data is subjected to sample marking;
carrying out video framing and face detection processing on each piece of original video sample data to obtain a training face picture;
grouping the training face pictures according to a preset quantity to obtain target training data, wherein each target training data comprises training face pictures of continuous M frames;
and inputting the target training data into a convolutional neural network-long and short recurrent neural network model for training, and obtaining a face emotion recognition model.
5. The method for recognizing human face emotion according to claim 4, wherein said target training data is inputted into a convolutional neural network-long and short recurrent neural network model for training, and a human face emotion recognition model is obtained, comprising the steps of:
initializing model parameters of a convolutional neural network-long-short-time recurrent neural network model;
Performing feature extraction on the target training data by adopting a convolutional neural network to obtain face features;
and inputting the facial features into a long-short-time recurrent neural network model for training, and obtaining the facial emotion recognition model.
6. A facial emotion recognition device, comprising:
A to-be-detected video data acquisition module for acquiring to-be-detected video data, extracting at least one face image to be detected from the video data to be detected;
The target face image area acquisition module is used for detecting the face image to be detected by adopting a face detection algorithm to obtain N target face image areas of the video data to be detected, wherein N is a positive integer;
The face video data interception module to be identified is used for intercepting the video of the video data to be detected according to N target face image areas to obtain N face video data to be identified;
the face emotion information acquisition module is used for inputting the N face video data to be identified into a face emotion identification model for identification to obtain N face emotion information in the video data to be detected, wherein the face emotion identification model is obtained by training a convolutional neural network-long-short-time recurrent neural network model.
7. The facial emotion recognition device as recited in claim 6, wherein said number of facial images to be detected is at least two; the target face image area acquisition module comprises:
the initial face image area acquisition unit is used for detecting each face image to be detected by adopting a face detection algorithm to obtain N initial face image areas of each face image to be detected;
The initial face image area identification unit is used for identifying N initial face image areas of each face image to be detected by adopting a face identification algorithm to obtain a user identification of each initial face image area;
And the target face image area acquisition unit is used for integrating the initial face image areas corresponding to the same user identification in the face image to be detected into image areas to obtain N target face image areas of the video data to be detected.
8. The facial emotion recognition device as recited in claim 6, said facial emotion recognition device further comprising:
The emotion statistical information acquisition module is used for counting the N pieces of face emotion information in the video data to be detected to obtain emotion statistical information;
and the emotion recognition information acquisition module is used for acquiring a preset emotion detection rule, and checking the emotion statistical information according to the preset emotion detection rule to obtain the emotion recognition information of the video data to be detected.
9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the steps of the facial emotion recognition method of any of claims 1 to 5 when the computer program is executed.
10. A computer readable storage medium storing a computer program, characterized in that the computer program when executed by a processor implements the steps of the facial emotion recognition method of any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811183763.9A CN109508638B (en) | 2018-10-11 | 2018-10-11 | Face emotion recognition method and device, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811183763.9A CN109508638B (en) | 2018-10-11 | 2018-10-11 | Face emotion recognition method and device, computer equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109508638A CN109508638A (en) | 2019-03-22 |
| CN109508638B true CN109508638B (en) | 2024-09-17 |
Family
ID=65746530
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811183763.9A Active CN109508638B (en) | 2018-10-11 | 2018-10-11 | Face emotion recognition method and device, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109508638B (en) |
Families Citing this family (34)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110175505A (en) * | 2019-04-08 | 2019-08-27 | 北京网众共创科技有限公司 | Determination method, apparatus, storage medium and the electronic device of micro- expression type |
| CN110310169A (en) * | 2019-05-22 | 2019-10-08 | 深圳壹账通智能科技有限公司 | Information-pushing method, device, equipment and medium based on interest value |
| CN110245291A (en) * | 2019-05-23 | 2019-09-17 | 广州至真信息科技有限公司 | A kind of display methods of business datum, device, computer equipment and storage medium |
| CN110321825A (en) * | 2019-06-25 | 2019-10-11 | 深圳市壹鸽科技有限公司 | A kind of video feeling analysis method |
| CN110751016B (en) * | 2019-09-02 | 2023-04-11 | 合肥工业大学 | Facial movement unit double-flow feature extraction method for emotional state monitoring |
| CN110650306B (en) * | 2019-09-03 | 2022-04-15 | 平安科技（深圳）有限公司 | Method and device for adding expression in video chat, computer equipment and storage medium |
| CN110689056A (en) * | 2019-09-10 | 2020-01-14 | Oppo广东移动通信有限公司 | A classification method and device, equipment and storage medium |
| CN110765860B (en) * | 2019-09-16 | 2023-06-23 | 平安科技（深圳）有限公司 | Tumble judging method, tumble judging device, computer equipment and storage medium |
| CN110865753B (en) * | 2019-11-07 | 2021-01-22 | 支付宝(杭州)信息技术有限公司 | Application message notification method and device |
| CN112949359A (en) * | 2019-12-10 | 2021-06-11 | 清华大学 | Convolutional neural network-based abnormal behavior identification method and device |
| CN111178294A (en) * | 2019-12-31 | 2020-05-19 | 北京市商汤科技开发有限公司 | State recognition method, device, equipment and storage medium |
| CN111428641A (en) * | 2020-03-24 | 2020-07-17 | 深圳供电局有限公司 | Secure dressing detection method and device, computer equipment and readable storage medium |
| CN111488813B (en) * | 2020-04-02 | 2023-09-08 | 咪咕文化科技有限公司 | Video emotion marking method and device, electronic equipment and storage medium |
| CN111626123B (en) * | 2020-04-24 | 2024-08-20 | 平安国际智慧城市科技股份有限公司 | Video data processing method, device, computer equipment and storage medium |
| CN111666890B (en) * | 2020-06-08 | 2023-06-30 | 平安科技（深圳）有限公司 | Spine deformation crowd identification method and device, computer equipment and storage medium |
| CN112036242B (en) * | 2020-07-28 | 2023-07-21 | 重庆锐云科技有限公司 | Face picture acquisition method and device, computer equipment and storage medium |
| CN111950447B (en) * | 2020-08-11 | 2023-08-22 | 合肥工业大学 | Emotion recognition method and system based on walking posture, storage medium |
| CN112153343B (en) * | 2020-09-25 | 2022-09-02 | 北京百度网讯科技有限公司 | Elevator safety monitoring method and device, monitoring camera and storage medium |
| CN112364736A (en) * | 2020-10-30 | 2021-02-12 | 深圳点猫科技有限公司 | Dynamic facial expression recognition method, device and equipment |
| CN112699774B (en) * | 2020-12-28 | 2024-05-24 | 深延科技(北京)有限公司 | Emotion recognition method and device for characters in video, computer equipment and medium |
| CN112686156A (en) * | 2020-12-30 | 2021-04-20 | 平安普惠企业管理有限公司 | Emotion monitoring method and device, computer equipment and readable storage medium |
| CN112651363B (en) * | 2020-12-31 | 2025-02-14 | 沈阳康泰电子科技股份有限公司 | Micro-expression fitting method and system based on multiple feature points |
| CN114973209A (en) * | 2021-02-20 | 2022-08-30 | 宝马股份公司 | Method, device, equipment, medium and vehicle for recognizing emotion of driver |
| CN113313096B (en) * | 2021-07-30 | 2021-11-05 | 平安科技（深圳）有限公司 | Multi-dimensional data generation method based on micro-expression and facies and related equipment thereof |
| CN113822164A (en) * | 2021-08-25 | 2021-12-21 | 深圳市安视宝科技有限公司 | Dynamic emotion recognition method and device, computer equipment and storage medium |
| CN114238593A (en) * | 2021-11-25 | 2022-03-25 | 蚂蚁财富(上海)金融信息服务有限公司 | Data processing method, device, electronic device and computer storage medium |
| CN113949773B (en) * | 2021-12-20 | 2022-04-15 | 上海金仕达软件科技有限公司 | Real-time queuing method and system for multi-person videos |
| CN114913573A (en) * | 2022-04-19 | 2022-08-16 | 深圳市彼岸心智科技有限公司 | Expression recognition method, device, storage medium and device |
| CN114821742A (en) * | 2022-05-19 | 2022-07-29 | 北京市西城区和平门幼儿园 | Method and device for identifying facial expressions of children or teenagers in real time |
| CN114842399B (en) * | 2022-05-23 | 2023-07-25 | 马上消费金融股份有限公司 | Video detection method, training method and device for video detection model |
| CN115797966A (en) * | 2022-10-27 | 2023-03-14 | 杭州智诺科技股份有限公司 | Method, system, device and medium for collecting and identifying emotion data |
| CN116343314B (en) * | 2023-05-30 | 2023-08-25 | 之江实验室 | Expression recognition method and device, storage medium and electronic equipment |
| CN117079324B (en) * | 2023-08-17 | 2024-03-12 | 厚德明心(北京)科技有限公司 | Face emotion recognition method and device, electronic equipment and storage medium |
| CN119048965B (en) * | 2024-10-17 | 2025-05-27 | 北京妙音数科股份有限公司 | A personalized video special effects matching method, device and computer-readable storage medium based on artificial intelligence |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106934382A (en) * | 2017-03-20 | 2017-07-07 | 许彐琼 | Method and apparatus based on video identification terror suspect |
| CN107636684A (en) * | 2015-03-18 | 2018-01-26 | 阿凡达合并第二附属有限责任公司 | Emotion identification in video conference |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20160083566A (en) * | 2014-12-31 | 2016-07-12 | 한국전자통신연구원 | Emotional information provision method and device in video conferencing participants |
| CN108563978A (en) * | 2017-12-18 | 2018-09-21 | 深圳英飞拓科技股份有限公司 | A kind of mood detection method and device |
| CN108491764B (en) * | 2018-03-05 | 2020-03-17 | 竹间智能科技（上海）有限公司 | Video face emotion recognition method, medium and device |
-
2018
- 2018-10-11 CN CN201811183763.9A patent/CN109508638B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107636684A (en) * | 2015-03-18 | 2018-01-26 | 阿凡达合并第二附属有限责任公司 | Emotion identification in video conference |
| CN106934382A (en) * | 2017-03-20 | 2017-07-07 | 许彐琼 | Method and apparatus based on video identification terror suspect |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109508638A (en) | 2019-03-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109508638B (en) | Face emotion recognition method and device, computer equipment and storage medium | |
| CN109492643B (en) | Certificate identification method and device based on OCR, computer equipment and storage medium | |
| CN108710847B (en) | Scene recognition method, device and electronic device | |
| CN109697416B (en) | Video data processing method and related device | |
| CN107679448B (en) | Eyeball action-analysing method, device and storage medium | |
| US20190095701A1 (en) | Living-body detection method, device and storage medium | |
| CN111191568B (en) | Method, device, equipment and medium for identifying flip image | |
| CN110197146B (en) | Face image analysis method based on deep learning, electronic device and storage medium | |
| Fourati et al. | Anti-spoofing in face recognition-based biometric authentication using image quality assessment | |
| WO2020252917A1 (en) | Fuzzy face image recognition method and apparatus, terminal device, and medium | |
| CN111275685B (en) | Method, device, equipment and medium for identifying flip image of identity document | |
| CN112215180A (en) | Living body detection method and device | |
| CN110222787A (en) | Multiscale target detection method, device, computer equipment and storage medium | |
| CN108875542B (en) | Face recognition method, device and system and computer storage medium | |
| CN110969045B (en) | Behavior detection method and device, electronic equipment and storage medium | |
| CN115115552A (en) | Image correction model training method, image correction device and computer equipment | |
| Vazquez-Fernandez et al. | Built-in face recognition for smart photo sharing in mobile devices | |
| Anand et al. | An improved local binary patterns histograms techniques for face recognition for real time application | |
| CN110197107B (en) | Micro-expression recognition method, micro-expression recognition device, computer equipment and storage medium | |
| CN113298158B (en) | Data detection method, device, equipment and storage medium | |
| US11087121B2 (en) | High accuracy and volume facial recognition on mobile platforms | |
| CN111144285B (en) | Fat and thin degree identification method, device, equipment and medium | |
| CN114627534B (en) | Living body discriminating method, electronic apparatus, and storage medium | |
| CN110363103A (en) | Insect pest identification method and device, computer equipment and storage medium | |
| CN111191521A (en) | Face living body detection method and device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |