CN119791629A - A method, system, device and medium for contactless heart rate detection of human side face based on rPPG - Google Patents
A method, system, device and medium for contactless heart rate detection of human side face based on rPPG Download PDFInfo
- Publication number
- CN119791629A CN119791629A CN202411881073.6A CN202411881073A CN119791629A CN 119791629 A CN119791629 A CN 119791629A CN 202411881073 A CN202411881073 A CN 202411881073A CN 119791629 A CN119791629 A CN 119791629A
- Authority
- CN
- China
- Prior art keywords
- heart rate
- side face
- person
- rppg
- roi
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Landscapes
- Measuring And Recording Apparatus For Diagnosis (AREA)
- Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
Abstract
一种基于rPPG的人侧脸无接触心率检测方法、系统、设备及介质，其方法通过摄像头捕捉人的侧脸，通过Mediapipe进行人侧脸关键点检测，获取人侧脸的所有关键点；然后确定用于心率检测的ROI(region ofinteresting,感兴趣区域)，通过rPPG算法对ROI内的图像数据提取RGB图像中的GREEN通道和RED通道的信号并将其加权结合作为bvp(血容量脉搏)信号，对bvp信号进行处理得到人的心率信息，并显示到界面上，实现通过检测人的侧脸实现对人心率的无接触测量；其系统、设备及介质基于rPPG的人侧脸无接触心率检测方法实现人侧脸无接触心率检测；本发明在人脸正面可能被遮挡或者角度不利于检测的情况下，仍旧能实时检测人的心率，具有方便快捷、适应性强、准确率高的优点，同时避免了传统接触式检测可能带来的不便和感染风险。
A method, system, device and medium for contactless heart rate detection of a person's side face based on rPPG. The method uses a camera to capture a person's side face, and uses Mediapipe to detect key points of the person's side face to obtain all key points of the person's side face; then determines an ROI (region of interest) for heart rate detection, extracts signals of a GREEN channel and a RED channel in an RGB image from image data within the ROI through an rPPG algorithm, and weightedly combines them as a BVP (blood volume pulse) signal; processes the BVP signal to obtain the person's heart rate information, and displays it on an interface, thereby realizing contactless measurement of the person's heart rate by detecting the person's side face; the system, device and medium realize contactless heart rate detection of the person's side face based on the method for contactless heart rate detection of the person's side face based on rPPG; the present invention can still detect the person's heart rate in real time when the front of the person's face may be blocked or the angle is not conducive to detection, and has the advantages of convenience, fastness, strong adaptability and high accuracy, while avoiding the inconvenience and infection risk that may be caused by traditional contact detection.
Description
Technical Field
The invention relates to the technical field of biomedical detection, in particular to a method, a system, equipment and a medium for detecting a human side face contactless heart rate based on rPPG.
Background
Heart rate is one of key indexes reflecting physiological states of human bodies, and plays a vital role in a plurality of fields such as medical diagnosis, health monitoring, exercise training and the like. Traditional heart rate detection methods mainly include Electrocardiogram (ECG), stethoscope measurements, etc. Although the electrocardiographic detection is accurate, the electrodes are required to be directly attached to the skin of a human body and connected to a detection instrument through a complex lead system, and the contact type measurement mode brings inconvenience to a detected person in some cases. For example, for burn patients, skin allergy sufferers or individuals who need continuous monitoring of heart rate over a long period of time (e.g. patients in intensive care units), sticking of electrodes may cause skin discomfort or even damage. The stethoscope requires the operation of professional medical staff, is greatly influenced by environmental noise, and cannot realize continuous and automatic heart rate monitoring for a long time.
With the development of technology, non-contact heart rate detection technology is becoming a research hotspot. Among them, technologies based on the Principle of Photoplethysmography (PPG) are receiving a great deal of attention. Traditional PPG obtains heart rate information by contacting a Light Emitting Diode (LED) and a photodetector to the skin surface to measure light absorption changes. However, this approach still requires contact with the skin, and has certain limitations. Based on this, remote photoplethysmography (rpg) has been developed which uses a camera to capture image information of a skin region of a human body, and extracts a heart rate signal by analyzing skin color changes without direct contact with the human body. The method has greatly improved convenience and comfort, and can be widely applied to the scenes of home health monitoring, telemedicine and the like.
Currently, rPPG-based non-contact heart rate detection is mostly focused on the frontal area of the face. This is because the front of the face is relatively flat, facilitating image acquisition and analysis, and the vascularity under the face skin has a significant correlation with heart rate variability. However, in the case of face-side detection, the influence of factors such as facial expression change, head movement, and interference of ambient light is likely. For example, when a person speaks, smiles, or frowns, the movement of facial muscles may alter the optical properties of the skin, thereby creating noise interference with the heart rate detection signal. The human side face part provides new possibility for non-contact heart rate detection, has the advantages of relatively stable vascularity and less influence of expression, and provides conditions for more accurate heart rate detection.
Patent application CN202410600014.0 adopts CardiA Net method, combines a self-attention convolution mixed network (ACmix) network and a attention-based long-short-period memory (ALSTM) network to detect heart rate, and has the defects that the method is only aimed at a front face, the number of facial feature points is too small, the positioning of the ROI is inaccurate, and the heart rate detection of a side face cannot be realized.
Disclosure of Invention
In order to overcome the defects in the prior art, the invention aims to provide a rPPG-based human side face contactless heart rate detection method, a rPPG-based human side face contactless heart rate detection system, rPPG-based human side face contactless heart rate detection equipment and rPPG-based human side face contactless heart rate detection medium, wherein a camera captures a human side face, and human side face key point detection is carried out through MEDIAPIPE to obtain all key points of the human side face; and then determining an ROI (region of interesting, region of interest) for heart rate detection, extracting signals of a GREEN channel and a RED channel in an RGB image from image data in the ROI through an rPPG algorithm, taking the signals as bvp (blood volume pulse) signals, processing the bvp signals to obtain heart rate information of a person, and displaying the heart rate information on an interface to realize non-contact measurement of the heart rate of the person through detecting the side face of the person.
In order to achieve the above purpose, the technical scheme adopted by the invention is as follows:
A human side face contactless heart rate detection method based on rPPG specifically comprises the following steps:
Step 1, acquiring a side face image of a person through a camera, wherein the acquired side face image is clear and complete;
step 2, carrying out region positioning on the side face image acquired in the step 1, and determining a region of interest (ROI) for heart rate detection;
step 3, calculating the instantaneous heart rate, the confidence coefficient and the final heart rate after weighted summation of each ROI by using an rPPG algorithm;
And 4, displaying the image captured by the camera in the step 1 and the instantaneous heart rate, the confidence coefficient and the weighted and summed final heart rate of each ROI calculated in the step 3 on an interface.
The step1 is characterized in that natural light is adopted for illumination when the side face image of the person is acquired, the illumination condition is stable, and the skin surface layer can be penetrated, so that the capturing capability of blood flow signals is enhanced.
The specific method of the step 2 is as follows:
all key points of the human side face are acquired through the visualization tool MEDIAPIPE, namely coordinates of each part of the side face are accurately positioned, and then the ROI for heart rate detection is selected.
The specific method of the step 3 is as follows:
Step 3.1, reading the image data in the ROI determined in the step 2, obtaining RGB channel histograms of all the ROIs, subtracting signals of a GREEN (GREEN) channel and a RED (RED) channel in the RGB channel histograms to obtain absolute values of differences of the GREEN (GREEN) channel and the RED (RED) channel as bvp signals, filtering data with deviation larger than a preset deviation in bvp signals, filtering by using a band-pass filter to obtain filtered signals, performing Fast Fourier Transform (FFT) on the obtained filtered signals to obtain filtered signal spectrums, and then performing peak detection to obtain instantaneous heart rates of all the ROIs;
Step 3.2, the instantaneous heart rate obtained in the step 3.1 is updated by using a formula BPM (n) =alpha.BPM (n-1) +1-alpha BPMSig (n) in a weighting way, and alpha=0.95 is taken to ensure that the heart rate result of each ROI does not jump;
Step 3.3, comparing the peak value of bvp signals of all ROIs obtained in step 3.1 with the obtained filtered signals, calculating a Mean Square Error (MSE) of a frequency spectrum to represent the frequency spectrum leakage degree by using a function, recording as a leakage, calculating the BPM confidence of all ROIs by using confidence i＝1/ 1+leakage, and calculating a final heart rate result by using the following formula for weighted summation based on the confidence and the instantaneous heart rate which is calculated and will not jump in step 3.2:
An rPPG-based human side face contactless heart rate detection system, comprising:
the image acquisition module is used for shooting by using a camera in the step 1 to collect side face pictures;
the ROI confirming module is used for carrying out side face detection on the collected side faces in the step 2, identifying all key points of the side faces and determining an ROI region;
and the signal processing module is used for extracting and processing the signals of the ROI determined in the step (2) to realize heart rate calculation.
An rPPG-based human side face contactless heart rate detection device, comprising:
a memory for storing a computer program;
And the processor is used for realizing the rPPG-based human side face contactless heart rate detection method in the steps 1 to 4 when executing the computer program.
A computer readable storage medium storing a computer program which, when executed by a processor, is capable of performing rPPG-based human face contactless heart rate detection based on the rPPG-based human face contactless heart rate detection method described in steps 1 to 4.
Compared with the prior art, the invention has the beneficial effects that:
1, the invention realizes the non-contact heart rate detection through the human side face, avoids the inconvenience and the discomfort of the traditional contact detection method, can be detected in a natural state without wearing any equipment by a user, and is particularly suitable for special groups such as the elderly, infants, skin sensitive groups and the like.
2, Compared with the existing non-contact detection method mainly aiming at the front face of the human face, the side face detection function has the advantage in more scenes. For example, under the conditions of resting on the side, turning around and the like of a detected person, the heart rate can still be accurately detected, and the application range of detection is enlarged.
The invention can effectively extract bvp signals of the ROIs through an advanced rPPG algorithm, carries out filtering and Fourier transformation and then carries out peak detection to obtain the instantaneous heart rate of each ROI, weights and updates the instantaneous heart rate to increase the detection accuracy, calculates a final heart rate result based on confidence coefficient and the obtained instantaneous heart rate which cannot jump, has high processing efficiency and high accuracy, and simultaneously, the system has anti-interference capability, can stably work under natural illumination conditions and environmental interference to a certain extent, and provides reliable technical support for long-term and continuous heart rate monitoring.
And 4, the heart rate monitoring can be realized by only one camera, the requirement on equipment is low, the application threshold is reduced, and the application range is enlarged. The system is suitable for various scenes, such as home health management, remote medical monitoring, an online education platform and the like.
In conclusion, the invention has the advantages that the user can accept detection in a natural state without wearing any equipment and in more scenes, and meanwhile, the invention has high processing efficiency, strong anti-interference capability and high accuracy through an advanced rPPG algorithm.
Drawings
FIG. 1 is a scatter plot of the results of a simulation experiment of the present invention.
FIG. 2 is a flow chart of the present invention for calculating a person's heart rate.
Fig. 3 is a flow chart of the rpg algorithm of the present invention for calculating the heart rate of a person.
Detailed Description
Referring to fig. 2 and 3, a method for detecting a contactless heart rate of a human side face based on rPPG specifically includes the following steps:
And step 1, acquiring a side face image of a person, and acquiring the image of the side face of the person by using high-resolution and high-frame-rate optical imaging equipment such as a camera. The camera frame rate needs to be greater than 30fps to ensure that the side faces are clearly and completely presented in the image. The acquisition process is required to be carried out under stable illumination conditions, and natural light is adopted to reduce the interference of ambient light.
And 2, carrying out region positioning on the side face image acquired in the step 1, determining a region of interest (ROI) for heart rate detection, and carrying out key point detection on the side face of the human body through MEDIAPIPE to obtain 478 key points of the side face of the human body, namely accurately positioning the coordinates of each part of the side face, including the key parts such as forehead, nose, chin, cheek and the like. Then, according to preset proportion and position relation, a region of interest (ROI) for heart rate detection is determined according to the key point coordinates, wherein the region of interest comprises a temple, a side cheek and a chin side face.
And 3, analyzing the image data in the ROI determined in the step 2 by using an rPPG algorithm, extracting physiological signals related to the heart rate, filtering to remove noise and other interference factors, extracting pulse wave signals reflecting heart rate variation, and calculating the heart rate by detecting peak points or periods of the pulse wave signals.
Step 3.1, reading image data of three ROIs of a temple, a side cheek and a chin side face, calculating RGB channel histograms of the ROIs by using functions of an opencv library, subtracting signals of a GREEN (GREEN) channel and a RED (RED) channel in the extracted RGB channel histograms to obtain absolute values of differences of the signals as bvp signals, filtering data with larger deviation in bvp signals by using a detrend tool, filtering by using a 5-order Butterworth band-pass filter to obtain filtered signals, obtaining a filtered signal spectrum after FFT of the filtered signals, and then carrying out peak detection by using an argmax function to obtain instantaneous heart rate of the ROIs;
step 3.2, the instantaneous heart rate is updated by weighting the formula BPM (n) =alpha.BPM (n-1) +1-alpha BPMSig (n), and alpha=0.95 is taken to ensure that the heart rate result of each ROI does not jump;
Step 3.3, comparing the peak value of bvp signals of each ROI extracted in step 3.1 with the filtered signals, calculating a Mean Square Error (MSE) of the spectrum using a function to represent the spectrum leakage degree, recording as leakage, calculating BPM confidence of each ROI using confidence i =1/1+leakage, and calculating a final heart rate result based on the confidence and the instantaneous heart rate without jump calculated in step 3.2 using the following formula weighted summation:
Simulation experiment
Simulation conditions and apparatus
1. Environmental conditions
The simulation environment is a general indoor environment, no environmental noise exists, the environmental temperature is kept at 20-25 ℃, the humidity is kept at 40-60% RH, and the human body is selected to be healthy, and the human body has no obvious cardiovascular disease, respiratory system disease or skin disease. The tested person should keep emotion stable before experiment to avoid tension, anxiety or excitation, and has clean skin, no cosmetic residue, no obvious scar or skin rash.
In the experimental process, the human subject keeps sitting posture, the head is naturally right, the side face is vertical to the camera lens, the eyes look in front of the head, and in the experimental process, the human subject is required to reduce the facial muscle movements (such as blinking, frowning, smiling and the like) and the large-amplitude actions of other parts of the body as much as possible.
2. Simulation device
1) The high-definition camera has 1080p resolution and 50fps frame rate, and can be provided with a bracket.
2) High performance computers, with specialized data processing software, python, etc.
Emulation content
And (3) aiming the camera at the side face of the person, and collecting data, wherein the duration of each collection is 3-5 minutes. The ROI is then confirmed and heart rate calculation is performed by rpg-based human side face contactless heart rate detection method. And comparing and analyzing the calculated heart rate result with a real heart rate, wherein the real heart rate result is a medical oximeter result.
The simulation experiment results are as follows:
a scatter diagram is drawn, see fig. 1, with the horizontal axis representing the heart rate detected by the present invention and the vertical axis representing the heart rate measured by a medical pulse oximeter. The calculated R square (used for measuring the fitting degree of the predicted value to the true value, the value is called perfect fitting when the value is 1, and the fitting effect is very good when the value is more than or equal to 0.9 generally) value is 0.898, which shows that the invention not only can realize the face detection of the side face, but also has very high accuracy of heart rate prediction and good fitting effect.
Claims (7)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202411881073.6A CN119791629B (en) | 2024-12-19 | 2024-12-19 | RPPG-based human side face contactless heart rate detection method, system, equipment and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202411881073.6A CN119791629B (en) | 2024-12-19 | 2024-12-19 | RPPG-based human side face contactless heart rate detection method, system, equipment and medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN119791629A true CN119791629A (en) | 2025-04-11 |
| CN119791629B CN119791629B (en) | 2025-09-26 |
Family
ID=95263551
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202411881073.6A Active CN119791629B (en) | 2024-12-19 | 2024-12-19 | RPPG-based human side face contactless heart rate detection method, system, equipment and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN119791629B (en) |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6393311B1 (en) * | 1998-10-15 | 2002-05-21 | Ntc Technology Inc. | Method, apparatus and system for removing motion artifacts from measurements of bodily parameters |
| US20190059822A1 (en) * | 2017-08-22 | 2019-02-28 | Shenzhen GOODIX Technology Co., Ltd. | Heart rate detection method and apparatus, and electronic terminal thereof |
| CN109977858A (en) * | 2019-03-25 | 2019-07-05 | 北京科技大学 | A kind of heart rate detection method and device based on image analysis |
| CN111243739A (en) * | 2020-01-07 | 2020-06-05 | 四川大学 | Anti-interference physiological parameter telemetering method and system |
| CN111325051A (en) * | 2018-12-13 | 2020-06-23 | 普天信息技术有限公司 | A face recognition method and device based on face image ROI selection |
| KR20220015779A (en) * | 2020-07-31 | 2022-02-08 | 성균관대학교산학협력단 | Methods and apparatuses for estimating robust body temperature using bio-signal of skin image |
| US20220240789A1 (en) * | 2019-02-01 | 2022-08-04 | Nec Corporation | Estimation apparatus, method and program |
| CN115719502A (en) * | 2021-08-23 | 2023-02-28 | 北京理工大学 | Head rotation motion self-adaptive non-contact robustness heart rate detection method |
| CN117281490A (en) * | 2023-09-21 | 2023-12-26 | 中国地质大学（武汉） | Method and system for telemetering vital signs of non-anesthetized animals |
| CN117426763A (en) * | 2023-11-30 | 2024-01-23 | 蔚来汽车科技(安徽)有限公司 | Heart rate monitoring method, device and storage medium based on near infrared image |
-
2024
- 2024-12-19 CN CN202411881073.6A patent/CN119791629B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6393311B1 (en) * | 1998-10-15 | 2002-05-21 | Ntc Technology Inc. | Method, apparatus and system for removing motion artifacts from measurements of bodily parameters |
| US20190059822A1 (en) * | 2017-08-22 | 2019-02-28 | Shenzhen GOODIX Technology Co., Ltd. | Heart rate detection method and apparatus, and electronic terminal thereof |
| CN111325051A (en) * | 2018-12-13 | 2020-06-23 | 普天信息技术有限公司 | A face recognition method and device based on face image ROI selection |
| US20220240789A1 (en) * | 2019-02-01 | 2022-08-04 | Nec Corporation | Estimation apparatus, method and program |
| CN109977858A (en) * | 2019-03-25 | 2019-07-05 | 北京科技大学 | A kind of heart rate detection method and device based on image analysis |
| CN111243739A (en) * | 2020-01-07 | 2020-06-05 | 四川大学 | Anti-interference physiological parameter telemetering method and system |
| KR20220015779A (en) * | 2020-07-31 | 2022-02-08 | 성균관대학교산학협력단 | Methods and apparatuses for estimating robust body temperature using bio-signal of skin image |
| CN115719502A (en) * | 2021-08-23 | 2023-02-28 | 北京理工大学 | Head rotation motion self-adaptive non-contact robustness heart rate detection method |
| CN117281490A (en) * | 2023-09-21 | 2023-12-26 | 中国地质大学（武汉） | Method and system for telemetering vital signs of non-anesthetized animals |
| CN117426763A (en) * | 2023-11-30 | 2024-01-23 | 蔚来汽车科技(安徽)有限公司 | Heart rate monitoring method, device and storage medium based on near infrared image |
Also Published As
| Publication number | Publication date |
|---|---|
| CN119791629B (en) | 2025-09-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Li et al. | The obf database: A large face video database for remote physiological signal measurement and atrial fibrillation detection | |
| Jeong et al. | Introducing contactless blood pressure assessment using a high speed video camera | |
| CN103908236B (en) | An automatic blood pressure measurement system | |
| Yu et al. | Noncontact monitoring of heart rate and heart rate variability in geriatric patients using photoplethysmography imaging | |
| US10004410B2 (en) | System and methods for measuring physiological parameters | |
| CN107358220A (en) | A kind of human heart rate and the contactless measurement of breathing | |
| McDuff et al. | Fusing partial camera signals for noncontact pulse rate variability measurement | |
| Zhou et al. | The noninvasive blood pressure measurement based on facial images processing | |
| CN110269600A (en) | Based on polynary empirical mode decomposition and the contactless video heart rate detection method for combining blind source separating | |
| CN111938622B (en) | Heart rate detection method, device and system and readable storage medium | |
| US12198468B2 (en) | Non-contact facial blood pressure measurement method based on 3D CNN | |
| CN112294282A (en) | Self-calibration method of emotion detection device based on RPPG | |
| CN114596279B (en) | A non-contact breath detection method based on computer vision | |
| CN114271800A (en) | A non-intrusive continuous blood pressure monitoring method and application in an office environment | |
| CN108771539A (en) | A kind of detection method and its device of the contactless heart rate based on camera shooting | |
| Lian et al. | Robust and remote photoplethysmography based on smartphone imaging of the human palm | |
| Alam et al. | UbiHeart: A novel approach for non-invasive blood pressure monitoring through real-time facial video | |
| Nagasawa et al. | Stress estimation using multimodal biosignal information from RGB facial video | |
| CN119791629B (en) | RPPG-based human side face contactless heart rate detection method, system, equipment and medium | |
| CN108720825B (en) | Multi-camera-based seamless detection method for non-contact vital sign parameters | |
| CN112022131B (en) | Long-term continuous non-contact heart rate measurement method and system | |
| AV et al. | Non-contact heart rate monitoring using machine learning | |
| CN116807413A (en) | Non-contact cognitive load assessment method based on infrared thermal imaging | |
| CN114469036A (en) | Remote heart rate monitoring method and system based on video images | |
| CN115245318A (en) | Automatic identification method of effective IPPG signal based on deep learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |