[go: up one dir, main page]

WO2018126275A1 - Modélisation et apprentissage de traits de caractère et de condition médicale sur la base de caractéristiques faciales 3d - Google Patents

Modélisation et apprentissage de traits de caractère et de condition médicale sur la base de caractéristiques faciales 3d Download PDF

Info

Publication number
WO2018126275A1
WO2018126275A1 PCT/US2018/012092 US2018012092W WO2018126275A1 WO 2018126275 A1 WO2018126275 A1 WO 2018126275A1 US 2018012092 W US2018012092 W US 2018012092W WO 2018126275 A1 WO2018126275 A1 WO 2018126275A1
Authority
WO
WIPO (PCT)
Prior art keywords
convolutional
interest
convolutional neural
image data
regions
Prior art date
Application number
PCT/US2018/012092
Other languages
English (en)
Inventor
Dirk Schneemann
Original Assignee
Dirk Schneemann, LLC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Dirk Schneemann, LLC filed Critical Dirk Schneemann, LLC
Publication of WO2018126275A1 publication Critical patent/WO2018126275A1/fr

Links

Classifications

    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H30/00—ICT specially adapted for the handling or processing of medical images
    • G16H30/40—ICT specially adapted for the handling or processing of medical images for processing medical images, e.g. editing
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/0059—Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence
    • A61B5/0075—Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence by spectroscopy, i.e. measuring spectra, e.g. Raman spectroscopy, infrared absorption spectroscopy
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/0059—Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence
    • A61B5/0077—Devices for viewing the surface of the body, e.g. camera, magnifying lens
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/103—Measuring devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/107—Measuring physical dimensions, e.g. size of the entire body or parts thereof
    • A61B5/1077—Measuring of profiles
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/103—Measuring devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/11—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor or mobility of a limb
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/16—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state
    • A61B5/167—Personality evaluation
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7235—Details of waveform analysis
    • A61B5/7264—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
    • A61B5/7267—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems involving training the classification device
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7271—Specific aspects of physiological measurement analysis
    • A61B5/7275—Determining trends in physiological measurement data; Predicting development of a medical condition based on physiological measurements, e.g. determining a risk factor
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7271—Specific aspects of physiological measurement analysis
    • A61B5/7282—Event detection, e.g. detecting unique waveforms indicative of a medical condition
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/74—Details of notification to user or communication with user or patient; User input means
    • A61B5/7475—User input or interface means, e.g. keyboard, pointing device, joystick
    • A61B5/748—Selection of a region of interest, e.g. using a graphics tablet
    • A61B5/7485—Automatic selection of region of interest
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2413—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/082—Learning methods modifying the architecture, e.g. adding, deleting or silencing nodes or connections
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/09—Supervised learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G06T7/0014—Biomedical image inspection using an image reference approach
    • G06T7/0016—Biomedical image inspection using an image reference approach involving temporal comparison
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G06V10/443—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components by matching or filtering
    • G06V10/449—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters
    • G06V10/451—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters with interaction between the filter responses, e.g. cortical complex cells
    • G06V10/454—Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/64—Three-dimensional objects
    • G06V20/653—Three-dimensional objects by matching three-dimensional models, e.g. conformal mapping of Riemann surfaces
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/161—Detection; Localisation; Normalisation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/168—Feature extraction; Face representation
    • G06V40/169—Holistic features and representations, i.e. based on the facial image taken as a whole
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/168—Feature extraction; Face representation
    • G06V40/171—Local features and components; Facial parts ; Occluding parts, e.g. glasses; Geometrical relationships
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/174—Facial expression recognition
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H20/00—ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance
    • G16H20/70—ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance relating to mental therapies, e.g. psychological therapy or autogenous training
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/20—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for computer-aided diagnosis, e.g. based on medical expert systems
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/70—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for mining of medical data, e.g. analysing previous cases of other patients
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B2576/00—Medical imaging apparatus involving image processing or analysis
    • A61B2576/02—Medical imaging apparatus involving image processing or analysis specially adapted for a particular organ or body part
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/0002—Remote monitoring of patients using telemetry, e.g. transmission of vital signals via a communication network
    • A61B5/0015—Remote monitoring of patients using telemetry, e.g. transmission of vital signals via a communication network characterised by features of the telemetry system
    • A61B5/0022—Monitoring a patient using a global network, e.g. telephone networks, internet
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/103—Measuring devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/107—Measuring physical dimensions, e.g. size of the entire body or parts thereof
    • A61B5/1079—Measuring physical dimensions, e.g. size of the entire body or parts thereof using optical or photographic means
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/103—Measuring devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/11—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor or mobility of a limb
    • A61B5/1103—Detecting muscular movement of the eye, e.g. eyelid movement
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/16—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state
    • A61B5/163—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state by tracking eye movement, gaze, or pupil change
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
    • A61B6/02—Arrangements for diagnosis sequentially in different planes; Stereoscopic radiation diagnosis
    • A61B6/03—Computed tomography [CT]
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
    • A61B6/02—Arrangements for diagnosis sequentially in different planes; Stereoscopic radiation diagnosis
    • A61B6/03—Computed tomography [CT]
    • A61B6/032—Transmission computed tomography [CT]
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
    • A61B6/52—Devices using data or image processing specially adapted for radiation diagnosis
    • A61B6/5211—Devices using data or image processing specially adapted for radiation diagnosis involving processing of medical diagnostic data
    • A61B6/5217—Devices using data or image processing specially adapted for radiation diagnosis involving processing of medical diagnostic data extracting a diagnostic or physiological parameter from medical diagnostic data
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
    • A61B6/52—Devices using data or image processing specially adapted for radiation diagnosis
    • A61B6/5211—Devices using data or image processing specially adapted for radiation diagnosis involving processing of medical diagnostic data
    • A61B6/5229—Devices using data or image processing specially adapted for radiation diagnosis involving processing of medical diagnostic data combining image data of a patient, e.g. combining a functional image with an anatomical image
    • A61B6/5247—Devices using data or image processing specially adapted for radiation diagnosis involving processing of medical diagnostic data combining image data of a patient, e.g. combining a functional image with an anatomical image combining images from an ionising-radiation diagnostic technique and a non-ionising radiation diagnostic technique, e.g. X-ray and ultrasound
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B8/00—Diagnosis using ultrasonic, sonic or infrasonic waves
    • A61B8/08—Clinical applications
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B8/00—Diagnosis using ultrasonic, sonic or infrasonic waves
    • A61B8/52—Devices using data or image processing specially adapted for diagnosis using ultrasonic, sonic or infrasonic waves
    • A61B8/5215—Devices using data or image processing specially adapted for diagnosis using ultrasonic, sonic or infrasonic waves involving processing of medical diagnostic data
    • A61B8/5223—Devices using data or image processing specially adapted for diagnosis using ultrasonic, sonic or infrasonic waves involving processing of medical diagnostic data for extracting a diagnostic or physiological parameter from medical diagnostic data
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/40—Software arrangements specially adapted for pattern recognition, e.g. user interfaces or toolboxes therefor
    • G06F18/41—Interactive pattern learning with a human teacher
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N7/00—Computing arrangements based on specific mathematical models
    • G06N7/01—Probabilistic graphical models, e.g. probabilistic networks

Definitions

  • the disclosed technology relates generally to applications for identifying character traits and medical condition of a target subject, and more particularly, some embodiments relate to systems and methods modeling and learning character traits based on 3D facial features and expressions.
  • Facial recognition technology has become more widely applications other than simple identification of a target subject.
  • analysis of facial features may be used to determine personality traits for an individual.
  • studies have focused on determining personality traits using analysis of facial and body expressions, and "body language,” including gestures and gesticulations.
  • body language including gestures and gesticulations.
  • some research suggests that the shape of the nasal root supplies statements about the life zone with the expression of spiritual impulses in the interaction with other people, the energy use becomes apparent at the temples, the forehead regions express spiritual activity, the upper forehead allows recognition of goodwill and affection and the chin and lower jaw provide information on motivation and assertiveness.
  • SVM Support Vector Machine
  • HMM Hidden Markov Model
  • SVM or HMM based techniques may be applied to small datasets, for example, to compare captured data from a target subject against predetermined or hardcoded reference datasets
  • the SVM and HMM algorithms do not scale up with larger data sets, e.g., comprising thousands of images.
  • available personality trait recognition systems and methods tend to be limited, not only to smaller data sets, but also to small and discrete result sets that may only include a few (e.g., tens) of personality traits.
  • systems and methods for modeling and learning character traits based on 3D facial features may include applying a convolutional neural network learning algorithm to an image data set to identify a correlation to one or more character traits or medical conditions.
  • a convolutional neural network learning model By applying the convolutional neural network learning model to multiple regions of interest within the image data set, a more granular analysis may be achieved across a large number of possible character traits with higher specificity than is possible with previous SVM and HMM based models.
  • Another feature of the convolutional neural network model is its ability to learn through tuning by evaluating different sets of regions of interest available in the image data set (e.g., different specific features of interest on a target subject's face), and then adjusting the model based on comparison with historical data, data acquired by other diagnostic tools, or user input. Patterns may be detected across groups of regions of interest, wherein each region of interest group may be applied as a convolutional feature layer within the convolutional neural network model. Patterns detected by the convolutional neural network model may then be correlated with specific character traits or medical conditions, and the results may be tuned via supervised learning using user feedback.
  • Figure 1A illustrates an example system for modeling and learning character traits based on 3D facial features, consistent with embodiments disclosed herein.
  • Figure IB illustrates an example image data set for modeling and learning character traits of a target subject, consistent with embodiments disclosed herein.
  • Figure 1C is a flow chart illustrating an example method for acquiring and processing image data sets for modeling and learning character traits, consistent with embodiments disclosed herein.
  • Figure 2 is a flow chart illustrating an example method for acquiring and processing image data sets for modeling character traits, consistent with embodiments disclosed herein.
  • Figure 3 illustrates an example method of processing and learning from image data sets using a convolutional neural networks, consistent with embodiments disclosed herein.
  • Figure 4 is a flow chart illustrating an example method for processing and learning from image data sets using convolutional neural networks, consistent with embodiments disclosed herein.
  • Figure 5 is a flow chart illustrating an example method for acquiring and processing 3D image data sets for to identify medical or health related information about a target subject, consistent with embodiments disclosed herein.
  • Figure 6 illustrates an example system for identifying medical or health related information about a target subject, consistent with embodiments disclosed herein.
  • Figure 7 illustrates an example system for identifying character traits about a target subject using feedback data from a remote data source, consistent with embodiments disclosed herein.
  • Figure 8 illustrates an example system for identifying character traits about a target subject using a mobile acquisition device and feedback data from other sources such as other users with mobile devices, consistent with embodiments disclosed herein.
  • Figure 9 illustrates an example computing engine that may be used in implementing various features of embodiments of the disclosed technology.
  • the technology disclosed herein is directed toward a system and method for identifying character traits using facial and expression recognition to analyze image data sets.
  • Embodiments disclosed herein incorporate the use of a convolutional neural network algorithm and a learning feedback loop to correlate the image data sets to a database of character traits, inclusive of medical conditions, and to learn based on historical data or user feedback.
  • examples of the disclosed technology include acquiring image data of a target subject from one or more image data sources, rendering or acquiring a 3D image data set, comparing a plurality of regions of interest within the 3D image set to historical image data to determine the presence of features within each of the plurality of regions of interest, grouping subsets of the regions of interest into one or more convolutional feature layers, wherein each convolutional feature layer probabilistically maps to a pre- identified character trait, and applying a convolutional neural network algorithm to identify whether the target subject possesses the pre-identified character trait.
  • Some embodiments further include training the convolutional neural network using feedback input through a user interface.
  • the character traits may include medical conditions.
  • the regions of interest may relate to features detected on a target subject's head or face, and may further include expressions detected using video or time- sequenced image data.
  • Figure 1A illustrates an example system for modeling and learning character traits based on 3D facial features.
  • system for modeling and learning character traits based on 3D facial features and/or texture data 100 includes an image data source 110.
  • Image data source 110 may be a still, video, a standard definition, a high definition, ultrahigh definition, infrared, 3D point cloud data, or other digital or analog camera as known in the art.
  • Image data source 110 may also include laser scanner, CAT scanner, MRI scanner, ultrasound scanner, or other detection devices capable of imaging anatomical features or objects either by texture or 3D shape.
  • image data source 110 may be a mobile phone camera.
  • Image data source 110 may also include an image data store such as a picture archive or historical database.
  • image data source 110 may include multiple imaging devices, such that imaging data from different sources may be combined.
  • imaging data from a high-definition or ultrahigh definition video camera may be combined with imaging data from a still camera, laser scanner, or medical imaging device such as a CAT scanner, ultrasound scanner, or MRI scanner.
  • Image data source 110 is configured to acquire imaging data from a target subject.
  • the target subject or subjects may include a human face, a human body, an animal face, or an animal body.
  • Image data source 110 may be communicatively coupled to characteristic recognition server (CRS) 130.
  • CRS 130 may be direct attached to image data source 110.
  • image data source 110 may communicate with CRS 130 using wireless, local area network, or wide area network technologies.
  • image data source 110 may be configured to store data locally on a removable data storage device, and data from the mobile data storage device may then be transferred or uploaded to CRS 130.
  • CRS 130 may include one or more processors and one or more non-transitory computer readable media with software embedded thereon, where the software is configured to perform various characteristic recognition functions as disclosed herein.
  • CRS 130 may include feature recognition engine 122.
  • Feature recognition engine 122 may be configured to receive imaging data from image data source 110, and render 3D models of the target subject.
  • Feature recognition engine 122 may further be configured to identify spatial patterns specific to the target subject.
  • feature recognition engine 122 may be configured to examine one or more regions of interest on the target subject, compare the image data and/or 3D render data from those regions of interest with spatial data stored in data store 120, to determine if known patterns stored data store 120 match patterns identified in the examined regions of interest from the acquired image data set.
  • CRS 130 may also include a saliency recognition engine 124.
  • Saliency recognition engine 124 may be configured to receive video image data, 3D point clouds, or still frame time sequence data from image data source 110. Similar to feature recognition engine 122, saliency recognition engine 124 may be configured to examine one or more regions of interest on the target subject, and identify specific movement patterns within the image data set. For example, saliency recognition engine 124 may be configured to identify twitches, expressions, eye blinks, brow raises, or other types of movement patterns which may be specific to a target subject.
  • Historical data sets of both still frame image data and saliency data may be stored in data store 120. Data store 120 may be direct attached to CRS 130. Alternatively, data store 120 may be network attached or located in a storage area network, in the cloud-based, or otherwise communicatively coupled to CRS 130, and/or image data source 110.
  • CRS 130 may also include a prediction and learning engine 126.
  • Prediction and learning engine 126 may be configured to predict characteristics specific to the target subject based on patterns identified by feature recognition engine 122 and/or saliency recognition engine 124 using prediction algorithms as disclosed herein.
  • the prediction algorithms may include Bayesian algorithms to determine the probability that a specific character trait is associated with a region of interest, or pattern of multiple regions of interest within image data taken of a target subject and which correlate to a particular character trait.
  • Prediction and learning engine 126 may be configured to adapt and learn. For example, a first prediction of a first character trait may be identified to be associated with the target subject.
  • a user using user interface device 140, may evaluate the accuracy of the first prediction, and determined that the prediction was incorrect.
  • prediction and learning engine 126 may identify a second character trait that is likely associated with the target subject. U pon confirmation that the second prediction is accurate, prediction and learning engine 126 may update a historical database of predictions and associated feature and/or saliency patterns identified within one or more regions of interest in the image data set, as stored in data store 120.
  • Figure IB illustrates an example image data set for modeling and learning character traits of a target subject.
  • Example image data set 155 may be a rendered 3D representation of the target subject combined with texture/color information (not shown in the figure).
  • regions of interest e.g., regions of interest 165 and 175
  • regions of interest may be predefined in data store 120 or using user interface 140, and/or learned by prediction and learning engine 126. Regions of interest may be selected based on the propensity for reflecting specific behavioral, personality, or other character traits of the target subject.
  • a region of interest may be examined by either feature recognition engine 122 or saliency recognition engine 124.
  • Prediction and learning engine 126 may analyze pattern matches identified by feature recognition engine 122 and saliency recognition is 124 across multiple regions of interest to identify specific patterns which correlate to known character traits or medical condition. For example, the target subject may display a specific raise of the brow, sigh, and squint of the eye, all at the same time, which may match a pattern which correlates to a character trait (e.g., an introverted or extroverted personality, stress, personality disorder, medical condition, etc.). The system may also identify correlations between two static areas of interest without any movement considerations. Static areas can be described by color and 3D shape. For example, static 3D shapes can be the geometry of facial landmarks such as nose, ears, chin, cheeks. Static color areas can be the coloring and texture from pimples, dents, bumps, folds, and wrinkles in the face.
  • static 3D shapes can be the geometry of facial landmarks such as nose, ears, chin, cheeks.
  • Static color areas can be the coloring and texture from pi
  • Figure 1C is a flow chart illustrating an example method for acquiring and processing image data sets for modeling and learning character traits.
  • the example method illustrated in Figure 1C may assure that a sufficient amount of high resolution image data is acquired to generate a dense 3D texture map sufficient to evaluate features within one or more desired regions of interest, while not acquiring too much image data as to overburden the system and data storage.
  • the example method may include: 1) associating sparse image data with a rough 3D model; and 2) ensuring all regions are sufficiently covered with high resolution data.
  • an example method for acquiring and processing image data sets may include a sparse acquisition process 1010, intense acquisition process 1020, and a 3D modeling process 1030.
  • sparse acquisition process 1010 and dense acquisition process 1020 may be performed by image data source 110 and feature recognition engine 122.
  • 3D modeling process 1030 may be performed by feature recognition engine 122.
  • Sparse acquisition process 1010 may include acquiring single images from different perspectives, computing online 3D model shape matching (using feature recognition engine 122), determining whether matching was successful. If matching is unsuccessful, the process may include acquiring more images from the same or different perspectives.
  • the method may further include dense acquisition process 1020.
  • Dense acquisition process 1020 may include acquiring high-resolution video while moving the camera, or alternatively, while the target subject moves or turns his/her head.
  • Dense acquisition process 1020 may further include matching the acquired data with a model stored in data store 120 using saliency recognition engine 124.
  • User may visualize the data coverage on the 3D model via user interface 140 to determine if the rendered image data sufficiently covers the model.
  • saliency prediction engine 124 may automatically evaluate whether the image data sufficiently covers the model using automated 3D rendering techniques as known in the art. If the image data coverage is insufficient, then more high-resolution video may be acquired.
  • the method may further include 3D modeling at step 1030.
  • 3D modeling may include computing a 3D detection model and storing the model in a database, for example, located on data store 120.
  • the dense 3D texture modeling may be performed by saliency recognition engine 124, or may be accomplished using an off-line 3D rendering system or a cloud-based rendering system.
  • FIG. 2 is a flow chart illustrating an example method for acquiring and processing image data sets for modeling and learning character traits.
  • a method for acquiring and processing image data sets may further include a model matching process 2010, an inference process 2020, and a comparison process with historical data at step 2030.
  • Model matching process 2010 may include receiving dense 3D data, for example from dense acquisition process 1020, extraction of texture and shape descriptors, and alignment to a 3D region mask using spatial pattern matching techniques as known in the art.
  • a user may assist in the alignment of the dense 3D data set to the 3D region mask process through user interface 140.
  • Inference process 2020 may include extraction of inference relevant regions of interest, computation of region activations, and a probabilistic inference, e.g., using prediction and learning engine 126.
  • prediction and learning engine 126 may use a Bayesian reasoning algorithm.
  • the region activations may reflect specific modeled 3D image data within identified regions of interest which match historic 3D image data from data store 120 for the same regions of interest which correlate to previously identified character traits.
  • multiple regions of interest will be activated creating a pattern of region activations.
  • the probabilistic inference may be a weighted value identifying a likely correlation between the pattern of region activations and specific character traits.
  • the probabilistic inference may be initially seeded by a user through user interface 140 (e.g., using expert knowledge or historical data), or by a predetermined or historical weighting.
  • Figure 3 illustrates an example method of acquiring processing image data sets using a convolutional neural networks.
  • a preprocessing algorithm 3010 may be applied to imaging data acquired from image data source 110 prior to identifying region activations, for example, an inference process 2020 referenced in Figure 2.
  • Preprocessing algorithm 3010 may include extracting a depth image based on shadowing or detection of structures from motion, as detected in the image data set to identify features in all three spatial dimensions.
  • Preprocessing algorithm 3010 may also include extracting texture image data, for example, to identify hair, whiskers, eyebrows, pock marks, rough skin, wrinkles, or other textural elements present on a target subject's face.
  • the method may further include convolution and subsampling process 3020.
  • convolution and subsampling process 3020 includes identifying one or more convolutional layers.
  • a convolutional layer may include a set of regions of interest which, if activated by matching them to data acquired from the target subject, may be correlated with a specific character trait.
  • mouth movement, brow movement, and eye lid movement may together comprise an example convolutional feature layer which may be activated if a target subject sighs, raises an eyebrow, and closes his/her eyes at the same time. Detection and identification of static features and dynamic features may be incorporated in the same convolutional feature layer or network.
  • Static features detected by the network may be, for example, color, texture, spatial geometry and size of facial landmarks such as nose, mouth, cheeks, forehead regions, ears, yaw.
  • Color and texture based static features detected by the network can be, for example, wrinkles, bumps, dents and folds.
  • Multiple convolutional layers may be analyzed across a single image data set in a manner consistent with convolutional neural network analysis.
  • depth image data and/or texture image data from the image data model may be applied to an L-l convolutional feature map.
  • Data from the L-l convolutional feature map may then be subsampled and applied to an L-2 convolutional feature map, and the process may be repeated through N layers.
  • facial features are composed by combining several feature maps from lower levels.
  • the facial feature of strong cheek bones may be composed of several low level features such as specific color combinations and combinations of geometrical primitives and 3D surface arrangements.
  • Each final feature map in the L-N layer may be associated with one or more facial feature such as spatial geometry and texture of facial regions and landmarks.
  • combinations of feature maps of the L-N layer may be associated with one or more character traits, such as personality, behavior, and medical condition. For example, a particular personality trait or medical condition may be detected only when a combination of underlying dependent convolutional layers are activated.
  • the activation of a convolutional layer may correspond to all of the regions of interest within that convolutional layer being activated.
  • a region of interest may also be associated with more than one convolutional layer, and convolutional layers may themselves be evaluated and sub sampled in different orders.
  • Inclusion or exclusion of a particular region of interest within any one of the convolutional layers may be determined through a supervised learning process by comparing output from the convolutional neural network process, e.g., at step 3030, with historical data stored in data store 140.
  • a user may adjust the convolutional neural network process by tuning which regions of interest should be applied in which convolutional layers, in the order in which the convolutional layers themselves should be applied.
  • FIG. 4 is a flow chart illustrating an example method for building a classifier model from a trained convolutional neural network.
  • the method includes extracting new features from learned convolutional neural networks comprising hierarchal arrangements of convolutional feature layers that are relevant to a classification of the underlying image data set to one or more character traits.
  • features may be extracted and added to region masks (2030), which are then applied to a probabilistic inference (2020).
  • the method may include: (1) augmenting an existing model (region mask and rules); and (2) creating a new model (region mask and rules) if no historical data is available.
  • trained convolutional neural network data from step 3030 may be processed by extracting 3D shape-based features at step 4010, extracting texture-based features at step 4020, and/or extracting feature correlations (e.g., relationships between two or more regions of interest as correlated with a particular character trait) at step 4030.
  • the extracted data may then be applied to a 3D model augmentation process at step 4040 or a three model construction process at step 4050 as known in the art of 3D rendering.
  • a user may interact with the 3D rendering process using user interface 140.
  • the verification of the region mask and probabilistic inference data at step 4060 may include determining the activation of specific convolutional layers within the applied convolutional neural network and weighting the correlation of that data to possible sets of character traits using probabilistic coefficients, then tuning the convolutional neural network (e.g., by adding or removing regions of interest from convolutional layers, or changing the order that the convolutional layers are applied) to determine which convolutional neural networks highly correlate with which character traits or medical condition.
  • FIG. 5 is a flow chart illustrating an example method for acquiring and processing textured 3-D image data sets to identify medical or health related information about a target subject.
  • 3-D modeling process 5010 and model matching process 5020 are similar to steps 2010 and 2020 referred to in Figure 2.
  • extraction of a small subset of diagnosis relevant regions of interest may be useful.
  • the diagnosis relevant regions of interest may be used.
  • Diagnosis regions of interest may be based on historical data, e.g., as stored in a database, or expert knowledge.
  • Inference step 5030 that includes computation of region activation and application of a probabilistic inference, using the convolutional neural network process referred to in Figure 3.
  • Correlation of activated convolutional layers across, for example, eight or more diagnosis relevant regions of interest may then be correlated with a set of medical conditions.
  • the resulting data may be correlated with historical diagnosis data taken using other methods, for example, evaluation by a medical professional using medical diagnostic equipment.
  • the convolutional neural network for medical diagnosis may then be tuned as described above with reference to Figure 3 to correlate the activated medical diagnosis relevant convolutional layers to individual medical conditions.
  • the trained convolutional neural network may be applied to an image data set from a target subject to assist in identification of the target subject's individual medical conditions.
  • Figure 6 illustrates an example system for training the convolutional neural network referenced in Figure 3 to identify medical or health related information about a target subject.
  • a feedback loop 6020 may be applied to data output from 3D texture modeling process 5010 and model matching process 5020 to tune the convolutional neural network.
  • output data from the convolutional neural network algorithm 3020 may be used to generated a predicted diagnosis. If the prediction is not accurate as compared with desired output from a medical database, for example, as stored on data store 120, then an error signal may be generated and used to fine tune the convolutional neural network process.
  • FIG. 7 illustrates an example system for training the convolutional neural network referenced in Figure 3 to identify character traits of a target subject using feedback data from a remote data source, similar to the learning process described above with respect to Figure 6.
  • regions of interest, or zones may be applied to one or more convolutional layers in the convolutional neural network in order to identify
  • Figure 8 illustrates an example system for identifying character traits about a target subject using a mobile acquisition device and feedback data from a remote data source.
  • image data source 110 may be a mobile device, such as a smart phone.
  • a user may acquire image data using the smart phone camera and upload the data, using a wireless or cellular network, to either a local or a remote CRS 130.
  • model matching step 2010 and inference step 2020, as well as application of the convolutional neural network algorithm 3020 may be accomplished using a cloud-based server.
  • Results may be stored on a user database, for example in data store 120, which may also be located in the cloud.
  • Identified character traits may be made available for evaluation through user interface 140, which for example, may be another mobile device, such as a family member's or friend's smart phone.
  • a mobile device based app may then be used to evaluate the data and apply feedback to the convolutional neural network in order to train the convolutional neural network.
  • Historical data across individual target subjects, as well as compilations of multiple target subjects, may be stored in the user database and used to tune the overall accuracy of the convolutional neural network in order to more accurately identify character traits based on uploaded image data sets.
  • an alert may be sent back to the source mobile device via an app to alert the user to acquire additional image data sets.
  • an engine might describe a given unit of functionality that can be performed in accordance with one or more embodiments of the technology disclosed herein.
  • an engine might be implemented utilizing any form of hardware, software, or a combination thereof.
  • processors for example, one or more processors, controllers, ASICs, PLAs, PALs, CPLDs, FPGAs, logical components, software routines or other mechanisms might be implemented to make up a engine.
  • the various engines described herein might be implemented as discrete engines or the functions and features described can be shared in part or in total among one or more engines.
  • the various features and functionality described herein may be implemented in any given application and can be implemented in one or more separate or shared engines in various combinations and permutations.
  • computing engine 900 may represent, for example, computing or processing capabilities found within desktop, laptop and notebook computers; hand-held computing devices (PDA's, smart phones, cell phones, palmtops, etc.); mainframes, supercomputers, workstations or servers; or any other type of special-purpose or general- purpose computing devices as may be desirable or appropriate for a given application or environment.
  • Computing engine 900 might also represent computing capabilities embedded within or otherwise available to a given device.
  • a computing engine might be found in other electronic devices such as, for example, digital cameras, navigation systems, cellular telephones, portable computing devices, modems, routers, WAPs, terminals and other electronic devices that might include some form of processing capability.
  • Computing engine 900 might include, for example, one or more processors, controllers, control engines, or other processing devices, such as a processor 904.
  • Processor 904 might be implemented using a general-purpose or special-purpose processing engine such as, for example, a microprocessor, controller, or other control logic.
  • processor 904 is connected to a bus 902, although any communication medium can be used to facilitate interaction with other components of computing engine 900 or to communicate externally.
  • Computing engine 900 might also include one or more memory engines, simply referred to herein as main memory 908. For example, preferably random access memory (RAM) or other dynamic memory, might be used for storing information and instructions to be executed by processor 904. Main memory 908 might also be used for storing temporary variables or other intermediate information during execution of instructions to be executed by processor 904. Computing engine 900 might likewise include a read only memory (“ROM”) or other static storage device coupled to bus 902 for storing static information and instructions for processor 904.
  • ROM read only memory
  • the computing engine 900 might also include one or more various forms of information storage mechanism 910, which might include, for example, a media drive 912 and a storage unit interface 920.
  • the media drive 912 might include a drive or other mechanism to support fixed or removable storage media 914.
  • a hard disk drive, a floppy disk drive, a magnetic tape drive, an optical disk drive, a CD or DVD drive (R or RW), or other removable or fixed media drive might be provided.
  • storage media 914 might include, for example, a hard disk, a floppy disk, magnetic tape, cartridge, optical disk, a CD or DVD, or other fixed or removable medium that is read by, written to or accessed by media drive 912.
  • the storage media 914 can include a computer usable storage medium having stored therein computer software or data.
  • information storage mechanism 190 might include other similar instrumentalities for allowing computer programs or other instructions or data to be loaded into computing engine 900.
  • Such instrumentalities might include, for example, a fixed or removable storage unit 922 and an interface 920.
  • Examples of such storage units 922 and interfaces 920 can include a program cartridge and cartridge interface, a removable memory (for example, a flash memory or other removable memory engine) and memory slot, a PCMCIA slot and card, and other fixed or removable storage units 922 and interfaces 920 that allow software and data to be transferred from the storage unit 922 to computing engine 900.
  • Computing engine 900 might also include a communications interface 924.
  • Communications interface 924 might be used to allow software and data to be transferred between computing engine 900 and external devices.
  • Examples of communications interface 924 might include a modem or softmodem, a network interface (such as an Ethernet, network interface card, WiMedia, IEEE 802.XX or other interface), a communications port (such as for example, a USB port, IR port, RS232 port Bluetooth ® interface, or other port), or other communications interface.
  • Software and data transferred via communications interface 924 might typically be carried on signals, which can be electronic, electromagnetic (which includes optical) or other signals capable of being exchanged by a given communications interface 924. These signals might be provided to communications interface 924 via a channel 928.
  • This channel 928 might carry signals and might be implemented using a wired or wireless communication medium.
  • Some examples of a channel might include a phone line, a cellular link, an RF link, an optical link, a network interface, a local or wide area network, and other wired or wireless communications channels.
  • computer program medium and “computer usable medium” are used to generally refer to media such as, for example, memory 908, storage unit 920, media 914, and channel 928.
  • These and other various forms of computer program media or computer usable media may be involved in carrying one or more sequences of one or more instructions to a processing device for execution.
  • Such instructions embodied on the medium are generally referred to as “computer program code” or a “computer program product” (which may be grouped in the form of computer programs or other groupings). When executed, such instructions might enable the computing engine 900 to perform features or functions of the disclosed technology as discussed herein.
  • the term “including” should be read as meaning “including, without limitation” or the like; the term “example” is used to provide exemplary instances of the item in discussion, not an exhaustive or limiting list thereof; the terms “a” or “an” should be read as meaning “at least one,” “one or more” or the like; and adjectives such as “conventional,” “traditional,” “normal,” “standard,” “known” and terms of similar meaning should not be construed as limiting the item described to a given time period or to an item available as of a given time, but instead should be read to encompass conventional, traditional, normal, or standard technologies that may be available or known now or at any time in the future. Likewise, where this document refers to technologies that would be apparent or known to one of ordinary skill in the art, such technologies encompass those apparent or known to the skilled artisan now or at any time in the future.

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Theoretical Computer Science (AREA)
  • Medical Informatics (AREA)
  • Biomedical Technology (AREA)
  • Public Health (AREA)
  • Molecular Biology (AREA)
  • General Physics & Mathematics (AREA)
  • Biophysics (AREA)
  • Pathology (AREA)
  • Artificial Intelligence (AREA)
  • Surgery (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Animal Behavior & Ethology (AREA)
  • Veterinary Medicine (AREA)
  • Evolutionary Computation (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Multimedia (AREA)
  • Data Mining & Analysis (AREA)
  • Software Systems (AREA)
  • Psychiatry (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • General Engineering & Computer Science (AREA)
  • Epidemiology (AREA)
  • Primary Health Care (AREA)
  • Physiology (AREA)
  • Databases & Information Systems (AREA)
  • Human Computer Interaction (AREA)
  • Computational Linguistics (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Psychology (AREA)
  • Radiology & Medical Imaging (AREA)
  • Signal Processing (AREA)
  • Dentistry (AREA)
  • Developmental Disabilities (AREA)

Abstract

L'invention concerne un procédé mis en œuvre par ordinateur pour identifier des traits de caractère associés à un sujet cible qui consiste à acquérir des données d'image d'un sujet cible à partir d'une source de données d'image, le rendu d'un ensemble de données d'image 3D, la comparaison de chacune d'une pluralité de régions d'intérêt à l'intérieur de l'ensemble d'images 3D à un ensemble de données d'image historique pour identifier des régions d'intérêt actives, regrouper des sous-ensembles des régions d'intérêt en une ou plusieurs couches de caractéristique de convolution, chaque couche de caractéristique de convolution correspondant de manière probabiliste à un trait de caractère pré-identifié, et l'application d'un modèle de réseau neuronal convolutionnel aux couches de caractéristique de convolution pour identifier un motif de régions actives d'intérêt à l'intérieur de chaque couche de caractéristique de convolution pour prédire si un sujet cible possède le trait de caractère pré-identifié.
PCT/US2018/012092 2016-12-30 2018-01-02 Modélisation et apprentissage de traits de caractère et de condition médicale sur la base de caractéristiques faciales 3d WO2018126275A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US201662440574P 2016-12-30 2016-12-30
US62/440,574 2016-12-30

Publications (1)

Publication Number Publication Date
WO2018126275A1 true WO2018126275A1 (fr) 2018-07-05

Family

ID=62709145

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2018/012092 WO2018126275A1 (fr) 2016-12-30 2018-01-02 Modélisation et apprentissage de traits de caractère et de condition médicale sur la base de caractéristiques faciales 3d

Country Status (2)

Country Link
US (2) US20180190377A1 (fr)
WO (1) WO2018126275A1 (fr)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110245613A (zh) * 2019-06-17 2019-09-17 珠海华园信息技术有限公司 基于深度学习特征对比的船牌识别方法
WO2020080685A1 (fr) * 2018-10-16 2020-04-23 주식회사 파코웨어 Procédé et système de génération de carte de profondeur de bloc de lecture à l'aide d'une seule image et d'un réseau de profondeur
CN111899281A (zh) * 2020-07-15 2020-11-06 北京思方技术开发有限公司 一种基于行为树的辅助监控系统控制策略实现方法及系统
CN113191171A (zh) * 2020-01-14 2021-07-30 四川大学 一种基于特征融合的疼痛强度评估方法
CN116503627A (zh) * 2023-06-25 2023-07-28 贵州省交通科学研究院股份有限公司 基于多源数据的遥感管理系统及方法

Families Citing this family (34)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108304755B (zh) * 2017-03-08 2021-05-18 腾讯科技（深圳）有限公司 用于图像处理的神经网络模型的训练方法和装置
US10943176B2 (en) * 2017-03-22 2021-03-09 Ebay Inc. Visual aspect localization presentation
US11151992B2 (en) 2017-04-06 2021-10-19 AIBrain Corporation Context aware interactive robot
US10963493B1 (en) * 2017-04-06 2021-03-30 AIBrain Corporation Interactive game with robot system
US10929759B2 (en) 2017-04-06 2021-02-23 AIBrain Corporation Intelligent robot software platform
EP3438869A1 (fr) * 2017-08-02 2019-02-06 Koninklijke Philips N.V. Anonymisation de données
LU100465B1 (en) * 2017-10-05 2019-04-09 Applications Mobiles Overview Inc System and method for object recognition
US11074453B2 (en) * 2018-01-31 2021-07-27 Hewlett Packard Enterprise Development Lp Video active region batching
US11068141B1 (en) * 2018-02-02 2021-07-20 Snap Inc. Device-based image modification of depicted objects
CN109117898A (zh) * 2018-08-16 2019-01-01 新智数字科技有限公司 一种手写字符识别方法及系统
CN110930350A (zh) * 2018-09-19 2020-03-27 鸿富锦精密电子(天津)有限公司 机器学习方法及应用机器学习方法的自动光学检测设备
CN109710272A (zh) * 2018-11-09 2019-05-03 深圳壹账通智能科技有限公司 更新文件的封装方法及装置
CN110166826B (zh) * 2018-11-21 2021-10-08 腾讯科技（深圳）有限公司 视频的场景识别方法、装置、存储介质及计算机设备
CN109886304B (zh) * 2019-01-22 2023-09-29 江苏大学 一种基于hmm-svm双层改进模型的复杂路况下周边车辆行为识别方法
TWI761671B (zh) * 2019-04-02 2022-04-21 緯創資通股份有限公司 活體偵測方法與活體偵測系統
US10650564B1 (en) * 2019-04-21 2020-05-12 XRSpace CO., LTD. Method of generating 3D facial model for an avatar and related device
CN110245573B (zh) * 2019-05-21 2023-05-26 平安科技（深圳）有限公司 一种基于人脸识别的签到方法、装置及终端设备
CN110348344B (zh) * 2019-06-28 2021-07-27 浙江大学 一种基于二维和三维融合的特殊人脸表情识别的方法
CN113841158B (zh) * 2019-07-03 2025-06-20 华为技术有限公司 数据处理系统与方法、编码单元、处理单元与存储介质
CN110321872B (zh) * 2019-07-11 2021-03-16 京东方科技集团股份有限公司 人脸表情识别方法及装置、计算机设备、可读存储介质
CN110555892B (zh) * 2019-08-09 2023-04-25 北京字节跳动网络技术有限公司 多角度图像生成方法、装置及电子设备
JP7497145B2 (ja) * 2019-08-30 2024-06-10 キヤノン株式会社 機械学習装置、機械学習方法及びプログラム、情報処理装置、放射線撮影システム
US11602331B2 (en) * 2019-09-11 2023-03-14 GE Precision Healthcare LLC Delivery of therapeutic neuromodulation
CN110993102A (zh) * 2019-11-18 2020-04-10 温州医科大学 一种基于校园大数据的学生行为与心理检测结果的精准分析方法及系统
JP7379120B2 (ja) * 2019-11-28 2023-11-14 キヤノン株式会社 超音波診断装置、医用画像撮影装置、学習装置、超音波画像表示方法及びプログラム
CN111178602A (zh) * 2019-12-18 2020-05-19 中国大唐集团科学技术研究院有限公司火力发电技术研究院 基于支持向量机和神经网络的循环水损失预测方法
CN111311695B (zh) * 2020-02-12 2022-11-25 东南大学 一种基于卷积神经网络的清水混凝土表面色差分析方法
CN111557672B (zh) * 2020-05-15 2023-06-27 上海市精神卫生中心（上海市心理咨询培训中心） 一种烟酸皮肤反应图像分析方法和设备
US11864860B2 (en) 2020-09-14 2024-01-09 Christian Heislop Biometric imaging and biotelemetry system
US11222466B1 (en) * 2020-09-30 2022-01-11 Disney Enterprises, Inc. Three-dimensional geometry-based models for changing facial identities in video frames and images
US11848097B2 (en) 2020-12-17 2023-12-19 Evicore Healthcare MSI, LLC Machine learning models for automated request processing
US11971885B2 (en) * 2021-02-10 2024-04-30 Adobe Inc. Retrieval aware embedding
CN115661923B (zh) * 2022-10-19 2025-07-04 浙江大学 自适应建模域特征的域泛化行人重识别方法
CN117671284B (zh) * 2023-12-06 2024-04-30 广州科松医疗智能科技有限公司 侵入性胎盘植入影像特征ai智能提取系统

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110007174A1 (en) * 2009-05-20 2011-01-13 Fotonation Ireland Limited Identifying Facial Expressions in Acquired Digital Images
US20130300900A1 (en) * 2012-05-08 2013-11-14 Tomas Pfister Automated Recognition Algorithm For Detecting Facial Expressions
US20140219526A1 (en) * 2013-02-05 2014-08-07 Children's National Medical Center Device and method for classifying a condition based on image analysis
US20140243651A1 (en) * 2013-02-27 2014-08-28 Min Jun Kim Health diagnosis system using image information
US20150242707A1 (en) * 2012-11-02 2015-08-27 Itzhak Wilf Method and system for predicting personality traits, capabilities and suggested interactions from images of a person

Family Cites Families (32)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB2398976B (en) * 2003-02-28 2006-03-08 Samsung Electronics Co Ltd Neural network decoder
US20110161854A1 (en) * 2009-12-28 2011-06-30 Monica Harit Shukla Systems and methods for a seamless visual presentation of a patient's integrated health information
US10869626B2 (en) * 2010-06-07 2020-12-22 Affectiva, Inc. Image analysis for emotional metric evaluation
US8873813B2 (en) * 2012-09-17 2014-10-28 Z Advanced Computing, Inc. Application of Z-webs and Z-factors to analytics, search engine, learning, recognition, natural language, and other utilities
US8682049B2 (en) * 2012-02-14 2014-03-25 Terarecon, Inc. Cloud-based medical image processing system with access control
US8553965B2 (en) * 2012-02-14 2013-10-08 TerraRecon, Inc. Cloud-based medical image processing system with anonymous data upload and download
US9070218B2 (en) * 2013-07-01 2015-06-30 Xerox Corporation Reconstructing an image of a scene captured using a compressed sensing device
WO2015054290A1 (fr) * 2013-10-07 2015-04-16 Ckn Group, Inc. Systèmes et procédés pour la collecte interactive de données numériques
US9928601B2 (en) * 2014-12-01 2018-03-27 Modiface Inc. Automatic segmentation of hair in images
WO2016132113A1 (fr) * 2015-02-16 2016-08-25 University Of Surrey Modélisation tridimensionnelle
JP6754619B2 (ja) * 2015-06-24 2020-09-16 三星電子株式会社Ｓａｍｓｕｎｇ Ｅｌｅｃｔｒｏｎｉｃｓ Ｃｏ．，Ｌｔｄ． 顔認識方法及び装置
KR102477190B1 (ko) * 2015-08-10 2022-12-13 삼성전자주식회사 얼굴 인식 방법 및 장치
EP3373810A4 (fr) * 2016-01-14 2020-03-25 Bigfoot Biomedical, Inc. Système de prise en charge du diabète
US9773196B2 (en) * 2016-01-25 2017-09-26 Adobe Systems Incorporated Utilizing deep learning for automatic digital image segmentation and stylization
US10043058B2 (en) * 2016-03-09 2018-08-07 International Business Machines Corporation Face detection, representation, and recognition
US10839573B2 (en) * 2016-03-22 2020-11-17 Adobe Inc. Apparatus, systems, and methods for integrating digital media content into other digital media content
US10835167B2 (en) * 2016-05-06 2020-11-17 The Board Of Trustees Of The Leland Stanford Junior University Systems and methods for using mobile and wearable video capture and feedback plat-forms for therapy of mental disorders
EP3475920A4 (fr) * 2016-06-23 2020-01-15 Loomai, Inc. Systèmes et procédés pour générer des modèles d'animation adaptés à l'ordinateur d'une tête humaine à partir d'images de données capturées
US10127680B2 (en) * 2016-06-28 2018-11-13 Google Llc Eye gaze tracking using neural networks
US10902243B2 (en) * 2016-10-25 2021-01-26 Deep North, Inc. Vision based target tracking that distinguishes facial feature targets
US10216983B2 (en) * 2016-12-06 2019-02-26 General Electric Company Techniques for assessing group level cognitive states
US11026634B2 (en) * 2017-04-05 2021-06-08 doc.ai incorporated Image-based system and method for predicting physiological parameters
US20180293754A1 (en) * 2017-04-05 2018-10-11 International Business Machines Corporation Using dynamic facial landmarks for head gaze estimation
US10762335B2 (en) * 2017-05-16 2020-09-01 Apple Inc. Attention detection
WO2018222808A1 (fr) * 2017-05-31 2018-12-06 The Procter & Gamble Company Systèmes et procédés de détermination de l'âge apparent de la peau
US20190005195A1 (en) * 2017-06-28 2019-01-03 General Electric Company Methods and systems for improving care through post-operation feedback analysis
CA3071120A1 (fr) * 2017-07-31 2019-02-07 Cubic Corporation Rapport et reconnaissance de scenario automatisee a l'aide de reseaux neuronaux
US11003933B2 (en) * 2017-08-15 2021-05-11 Noblis, Inc. Multispectral anomaly detection
US11002958B2 (en) * 2017-08-24 2021-05-11 International Business Machines Corporation Dynamic control of parallax barrier configuration
CN107516090B (zh) * 2017-09-11 2021-09-17 北京百度网讯科技有限公司 一体化人脸识别方法和系统
US10796135B2 (en) * 2017-09-28 2020-10-06 Nec Corporation Long-tail large scale face recognition by non-linear feature level domain adaptation
US10740617B2 (en) * 2017-12-19 2020-08-11 Intel Corporation Protection and recovery of identities in surveillance camera environments

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110007174A1 (en) * 2009-05-20 2011-01-13 Fotonation Ireland Limited Identifying Facial Expressions in Acquired Digital Images
US20130300900A1 (en) * 2012-05-08 2013-11-14 Tomas Pfister Automated Recognition Algorithm For Detecting Facial Expressions
US20150242707A1 (en) * 2012-11-02 2015-08-27 Itzhak Wilf Method and system for predicting personality traits, capabilities and suggested interactions from images of a person
US20140219526A1 (en) * 2013-02-05 2014-08-07 Children's National Medical Center Device and method for classifying a condition based on image analysis
US20140243651A1 (en) * 2013-02-27 2014-08-28 Min Jun Kim Health diagnosis system using image information

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020080685A1 (fr) * 2018-10-16 2020-04-23 주식회사 파코웨어 Procédé et système de génération de carte de profondeur de bloc de lecture à l'aide d'une seule image et d'un réseau de profondeur
CN110245613A (zh) * 2019-06-17 2019-09-17 珠海华园信息技术有限公司 基于深度学习特征对比的船牌识别方法
CN113191171A (zh) * 2020-01-14 2021-07-30 四川大学 一种基于特征融合的疼痛强度评估方法
CN113191171B (zh) * 2020-01-14 2022-06-17 四川大学 一种基于特征融合的疼痛强度评估方法
CN111899281A (zh) * 2020-07-15 2020-11-06 北京思方技术开发有限公司 一种基于行为树的辅助监控系统控制策略实现方法及系统
CN111899281B (zh) * 2020-07-15 2023-10-31 北京和利时系统工程有限公司 一种基于行为树的辅助监控系统控制策略实现方法及系统
CN116503627A (zh) * 2023-06-25 2023-07-28 贵州省交通科学研究院股份有限公司 基于多源数据的遥感管理系统及方法
CN116503627B (zh) * 2023-06-25 2023-09-26 贵州省交通科学研究院股份有限公司 基于多源数据的模型构建系统及方法

Also Published As

Publication number Publication date
US20180190377A1 (en) 2018-07-05
US20190206546A1 (en) 2019-07-04

Similar Documents

Publication Publication Date Title
US20190206546A1 (en) Modeling and learning character traits and medical condition based on 3d facial features
US9087232B2 (en) 3D object recognition
US10321747B2 (en) Makeup assistance device, makeup assistance system, makeup assistance method, and makeup assistance program
US9311564B2 (en) Face age-estimation and methods, systems, and software therefor
CN107145857B (zh) 人脸属性识别方法、装置和模型建立方法
KR20210048523A (ko) 이미지 처리 방법, 장치, 전자 기기 및 컴퓨터 판독 가능 기억 매체
CN113570684B (zh) 图像处理方法、装置、计算机设备和存储介质
CN107871098B (zh) 人脸特征点的获取方法和装置
CN110598638A (zh) 模型训练方法、人脸性别预测方法、设备及存储介质
CN111401219B (zh) 一种手掌关键点检测方法和装置
US20230116779A1 (en) Computer implemented methods and devices for determining dimensions and distances of head features
US12147519B2 (en) User authentication based on three-dimensional face modeling using partial face images
KR102495889B1 (ko) 반자동 라벨링에 따라 학습된 딥러닝 기반 주름 검출 모델을 이용하여 얼굴 주름을 검출하기 위한 방법 및 장치
JP7225731B2 (ja) 多変数データシーケンスの画像化
CN115862120B (zh) 可分离变分自编码器解耦的面部动作单元识别方法及设备
KR20210067815A (ko) 사용자의 건강 상태를 측정하기 위한 방법 및 이를 위한 장치
CN113762117B (zh) 一种图像处理模型的训练方法、图像处理模型及计算机设备
Deng et al. View-invariant gait recognition based on deterministic learning and knowledge fusion
Wang Face feature dynamic recognition method based on intelligent image
EP1810216B1 (fr) Reconnaissance d'objets en 3d
In Kim et al. Predictor combination at test time
WO2024192561A1 (fr) Systèmes et procédés de détermination du nombre de cheveux sur une personne
Zohner Feature Learning and Bayesian Functional Regression for High-Dimensional Complex Data
KR102806105B1 (ko) 얇은 객체를 검출하기 위한 방법
CN116631019B (zh) 基于面部图像的口罩适合性检测方法及装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18734031

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18734031

Country of ref document: EP

Kind code of ref document: A1