CN112842285B - Method and system for assisting in identifying submucosal blood vessels under endoscope - Google Patents
Method and system for assisting in identifying submucosal blood vessels under endoscope Download PDFInfo
- Publication number
- CN112842285B CN112842285B CN202011639891.7A CN202011639891A CN112842285B CN 112842285 B CN112842285 B CN 112842285B CN 202011639891 A CN202011639891 A CN 202011639891A CN 112842285 B CN112842285 B CN 112842285B
- Authority
- CN
- China
- Prior art keywords
- pixel
- value
- image
- blood volume
- fluctuation frequency
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 210000004204 blood vessel Anatomy 0.000 title claims abstract description 62
- 238000000034 method Methods 0.000 title claims abstract description 38
- 239000008280 blood Substances 0.000 claims abstract description 82
- 210000004369 blood Anatomy 0.000 claims abstract description 82
- 230000008859 change Effects 0.000 claims abstract description 48
- 238000005516 engineering process Methods 0.000 claims abstract description 28
- 238000003384 imaging method Methods 0.000 claims abstract description 23
- 238000013186 photoplethysmography Methods 0.000 claims abstract description 23
- 210000004877 mucosa Anatomy 0.000 claims abstract description 14
- 230000017531 blood circulation Effects 0.000 claims abstract description 10
- 210000001035 gastrointestinal tract Anatomy 0.000 claims abstract description 7
- 238000007781 pre-processing Methods 0.000 claims description 31
- 238000004364 calculation method Methods 0.000 claims description 17
- 238000001914 filtration Methods 0.000 claims description 10
- 238000009499 grossing Methods 0.000 claims description 10
- 238000010183 spectrum analysis Methods 0.000 claims description 9
- 238000012545 processing Methods 0.000 claims description 8
- 238000004422 calculation algorithm Methods 0.000 claims description 6
- 230000002792 vascular Effects 0.000 claims description 5
- 230000002087 whitening effect Effects 0.000 claims description 5
- 230000008569 process Effects 0.000 abstract description 6
- 238000010586 diagram Methods 0.000 description 10
- 238000004590 computer program Methods 0.000 description 9
- 230000006870 function Effects 0.000 description 9
- 238000001839 endoscopy Methods 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 102000001554 Hemoglobins Human genes 0.000 description 2
- 108010054147 Hemoglobins Proteins 0.000 description 2
- 230000000740 bleeding effect Effects 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 230000008447 perception Effects 0.000 description 2
- 206010001497 Agitation Diseases 0.000 description 1
- 238000010521 absorption reaction Methods 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 210000000467 autonomic pathway Anatomy 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000036772 blood pressure Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 201000010099 disease Diseases 0.000 description 1
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 1
- 230000023597 hemostasis Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000029058 respiratory gaseous exchange Effects 0.000 description 1
- 238000007619 statistical method Methods 0.000 description 1
- 238000002560 therapeutic procedure Methods 0.000 description 1
- 238000012800 visualization Methods 0.000 description 1
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/02—Detecting, measuring or recording for evaluating the cardiovascular system, e.g. pulse, heart rate, blood pressure or blood flow
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/0059—Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence
- A61B5/0082—Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence adapted for particular medical purposes
- A61B5/0084—Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence adapted for particular medical purposes for introduction into the body, e.g. by catheters
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/02—Detecting, measuring or recording for evaluating the cardiovascular system, e.g. pulse, heart rate, blood pressure or blood flow
- A61B5/026—Measuring blood flow
- A61B5/0295—Measuring blood flow using plethysmography, i.e. measuring the variations in the volume of a body part as modified by the circulation of blood therethrough, e.g. impedance plethysmography
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
Landscapes
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Engineering & Computer Science (AREA)
- Surgery (AREA)
- General Health & Medical Sciences (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Heart & Thoracic Surgery (AREA)
- Medical Informatics (AREA)
- Molecular Biology (AREA)
- Physics & Mathematics (AREA)
- Animal Behavior & Ethology (AREA)
- Pathology (AREA)
- Public Health (AREA)
- Veterinary Medicine (AREA)
- Physiology (AREA)
- Cardiology (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Psychiatry (AREA)
- Signal Processing (AREA)
- Hematology (AREA)
- Endoscopes (AREA)
Abstract
The invention provides a method and a system for assisting in identifying submucosal blood vessels under an endoscope, belonging to the technical field of blood vessel identification, wherein a time sequence image of a part to be detected, which is acquired in real time, is preprocessed, and the pixel value of the time sequence image is converted into a zero mean value and a unit variance; extracting blood volume waves from the preprocessed time sequence images based on an imaging type photoplethysmography technology, and determining corresponding blood volume fluctuation frequency; based on an imaging type photoplethysmography technology, extracting a pixel change value of each pixel point from the preprocessed time sequence image, and determining the pixel fluctuation frequency of the corresponding pixel point; and determining the blood vessel covering area under the mucosa according to the blood volume wave, the blood volume fluctuation frequency, the pixel change value and the pixel fluctuation frequency. The invention can accurately extract the blood flow information of the digestive tract in real time and accurately identify the blood vessels under the mucosa without additional equipment and prolonging the operation time in the endoscope operation process, thereby ensuring the operation safety.
Description
Technical Field
The invention relates to the technical field of blood vessel identification, in particular to a method and a system for assisting in identifying submucosal blood vessels under an endoscope.
Background
With the development of endoscope technology, more and more diseases which originally need surgical operation can be treated by endoscope minimally invasive therapy. However, during the operation, the bleeding during the operation is inevitably caused by various reasons such as unclear visualization of the blood vessels under the mucosa. The mild case can be treated under the endoscope, the severe case needs surgical hemostasis, and few patients even die. At present, no method for assisting in identifying submucosal blood vessels under an endoscope exists. The imaging type photoplethysmography (IPPG) technology can allow a smart phone to realize non-contact measurement of heart rate, respiration rate, autonomic nerve excitability and blood pressure of a human body by recording videos of faces and skins and by the principle that hemoglobin absorbs and reflects light, and reflects blood flow information of the body surface of the human body in an image.
Disclosure of Invention
The invention aims to provide an endoscopic auxiliary submucosal blood vessel identification method and system which can accurately extract blood flow information of a digestive tract and accurately identify submucosal blood vessels, so as to solve at least one technical problem in the background technology.
In order to achieve the purpose, the invention adopts the following technical scheme:
in one aspect, the invention provides a method for assisting in identifying submucosal blood vessels under an endoscope, which comprises the following steps:
acquiring a plurality of frames of RGB time sequence images of a part to be detected; calculating the hash values of the front and rear frames of images in real time by using a perceptual hash algorithm, and when the difference between the hash values of the front and rear frames of images is greater than a threshold value, recording RGB time sequence image information again;
selecting G channel component data in an RGB time sequence image of a part to be detected, which is acquired in real time, preprocessing the G channel component data, and converting a pixel value of the time sequence image into a zero mean value and a unit variance;
extracting blood volume waves from the preprocessed time sequence images based on an imaging type photoplethysmography technology, and determining corresponding blood volume fluctuation frequency;
based on an imaging type photoplethysmography technology, extracting a pixel change value of each pixel point from the preprocessed time sequence image, and determining the pixel fluctuation frequency of the corresponding pixel point;
and determining the actual blood vessel coverage area under the mucosa according to the blood volume wave, the blood volume fluctuation frequency, the pixel change value and the pixel fluctuation frequency.
Preferably, the method further comprises obtaining time series images of the portion to be detected in real time from the endoscopy.
Preferably, the method further comprises displaying the actual vessel coverage area by using a ghost outline method.
Preferably, the preprocessing the time-series image of the to-be-detected part acquired in real time includes:
and the G channel component data is used as original data for extracting the blood flow information image of the digestive tract, and the pixel value is converted into zero mean value and unit variance by combining Gaussian blurring and whitening processing.
Preferably, extracting the blood volume wave comprises:
the pixel value of each frame of image is regarded as a continuous function of time, and the calculation formula of the blood volume wave is as follows:
Preferably, determining the blood volume fluctuation frequency comprises:
smoothing the blood volume wave by using time domain filtering, and performing energy spectrum analysis by using fast Fourier transform, wherein the frequency corresponding to the highest energy point is the blood volume fluctuation frequency.
Preferably, the extracting the pixel variation value of each pixel point includes:
the pixel change value of the pixel point, namely the sum of absolute values of the pixel value differences of the coordinate points corresponding to every two adjacent frames of images in the continuous N frames of images, is calculated according to the following formula:
Preferably, determining the pixel fluctuation frequency of the corresponding pixel point includes:
calculating the change rule of green component G data of each pixel in the continuous N frames of image data, and setting the value of G (x, y) component value G (x, y) of a pixel point with certain coordinate (x, y) in the continuous N frames of images to form a G component sequence { G (x, y) [ k ] }, wherein k represents the k frame of image, and k is more than or equal to 0 and less than N;
and smoothing the component sequence by using time-domain filtering, and performing energy spectrum analysis by using fast Fourier transform, wherein the frequency corresponding to the highest energy point is the pixel fluctuation frequency.
Preferably, determining the actual sub-mucosal vascular coverage area comprises:
and judging whether the pixel change value of the pixel is greater than a first threshold, if so, determining that the pixel is a suspected blood vessel coverage area, and when the difference between the pixel fluctuation frequency and the blood volume fluctuation frequency of the pixel is less than a second threshold, determining that the pixel is an actual blood vessel coverage area.
In a second aspect, the present invention provides a system for assisting in identifying submucosal blood vessels under an endoscope, comprising:
the preprocessing module is used for preprocessing a time sequence image of the part to be detected, which is acquired in real time, and converting the pixel value of the time sequence image into a zero mean value and a unit variance;
the first calculation module is used for extracting blood volume waves from the preprocessed time sequence images based on an imaging type photoplethysmography technology and determining corresponding blood volume fluctuation frequency;
the second calculation module is used for extracting the pixel change value of each pixel point from the preprocessed time sequence image based on the imaging type photoplethysmography technology and determining the pixel fluctuation frequency of the corresponding pixel point;
and the judging module is used for determining the blood vessel coverage area under the mucosa according to the blood volume wave, the blood volume fluctuation frequency, the pixel change value and the pixel fluctuation frequency.
The invention has the beneficial effects that: in the endoscope operation process, on the premise of not using additional equipment and not prolonging the operation time, the digestive tract blood flow information is accurately extracted in real time, blood vessels under mucosa are accurately identified, and the operation safety is ensured.
Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1 is a functional block diagram of a system for assisting in identifying submucosal blood vessels under an endoscope according to embodiment 1 of the present invention.
Fig. 2 is a functional block diagram of a system for assisting in identifying submucosal blood vessels under an endoscope according to embodiment 2 of the present invention.
Detailed Description
Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below by way of the drawings are illustrative only and are not to be construed as limiting the invention.
It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.
It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the prior art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.
As used herein, the singular forms "a", "an", "the" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, various embodiments or examples and features of different embodiments or examples described in this specification can be combined and combined by one skilled in the art without contradiction.
For the purpose of facilitating an understanding of the present invention, the present invention will be further explained by way of specific embodiments with reference to the accompanying drawings, which are not intended to limit the present invention.
It should be understood by those skilled in the art that the drawings are merely schematic representations of embodiments and that the elements shown in the drawings are not necessarily required to practice the invention.
Example 1
As shown in fig. 1, embodiment 1 of the present invention provides a system for assisting in identifying a submucosal blood vessel under an endoscope, including: the preprocessing module is used for preprocessing a time sequence image of the part to be detected, which is acquired in real time, and converting the pixel value of the time sequence image into a zero mean value and a unit variance;
the first calculation module is used for extracting blood volume waves from the preprocessed time sequence images based on an imaging type photoplethysmography technology and determining corresponding blood volume fluctuation frequency;
the second calculation module is used for extracting the pixel change value of each pixel point from the preprocessed time sequence image based on the imaging type photoplethysmography technology and determining the pixel fluctuation frequency of the corresponding pixel point;
and the judging module is used for determining the actual blood vessel coverage area under the mucosa according to the blood volume wave, the blood volume fluctuation frequency, the pixel change value and the pixel fluctuation frequency.
In this embodiment 1, based on the above system for assisting in identifying submucosal blood vessels under an endoscope, a method for assisting in identifying submucosal blood vessels under an endoscope is implemented, and the method is based on the existing equipment facilities and endoscope environment, so that the position prediction of blood vessels under a white-light endoscope is implemented, and the bleeding problem in an operation is reduced to a certain extent. The method comprises the following steps:
preprocessing a time sequence image of a part to be detected, which is acquired in real time, and converting a pixel value of the time sequence image into a zero mean value and a unit variance;
extracting blood volume waves from the preprocessed time sequence images based on an imaging type photoplethysmography technology, and determining corresponding blood volume fluctuation frequency;
based on an imaging type photoplethysmography technology, extracting a pixel change value of each pixel point from the preprocessed time sequence image, and determining the pixel fluctuation frequency of the corresponding pixel point;
and determining the actual blood vessel coverage area under the mucosa according to the blood volume wave, the blood volume fluctuation frequency, the pixel change value and the pixel fluctuation frequency.
In this embodiment 1, the preprocessing the time-series image of the to-be-detected portion acquired in real time includes:
and selecting green component G data from the read RGB time sequence image data of the part to be detected as original data for extracting the blood flow information image of the alimentary tract, and converting the pixel value into a zero mean value and a unit variance by combining Gaussian blurring and whitening processing.
Extracting a blood volume wave comprises:
the pixel value of each frame of image is regarded as a continuous function of time, and the calculation formula of the blood volume wave is as follows:
wherein v iskRepresenting the blood volume value of the k frame image, k is more than or equal to 0<N, N represents the total number of frames of the image; p is a radical ofkxyRepresenting the preprocessing value of green component G data of a pixel point with the k frame image coordinate of (x, y); w represents the width of the image; h represents the height of the image.
Determining the blood volume fluctuation frequency comprises:
smoothing the blood volume wave by using time domain filtering, and performing energy spectrum analysis by using fast Fourier transform, wherein the frequency corresponding to the highest energy point is the blood volume fluctuation frequency.
The extracting of the pixel change value of each pixel point includes:
the pixel change value of the pixel point, namely the sum of absolute values of the pixel value differences of the coordinate points corresponding to every two adjacent frames of images in the continuous N frames of images, is calculated according to the following formula:
wherein S isxyRepresenting pixel change values of pixel points of coordinate violation (x, y) in continuous N frames of images; p is a radical ofkxyAnd (3) representing the preprocessing value of the green component G data of the pixel point with the k frame image coordinate of (x, y). S abovexyThe calculation formula (2) can not only obtain the fluctuation frequency, but also predict the main area influencing the fluctuation frequency.
Determining the pixel fluctuation frequency of the corresponding pixel point comprises:
calculating the change rule of green component G data of each pixel in the continuous N frames of image data, and setting the value of G (x, y) component value G (x, y) of a pixel point with certain coordinate (x, y) in the continuous N frames of images to form a G component sequence { G (x, y) [ k ] }, wherein k represents the k frame of image, and k is more than or equal to 0 and less than N;
and smoothing the component sequence by using time-domain filtering, and performing energy spectrum analysis by using fast Fourier transform, wherein the frequency corresponding to the highest energy point is the pixel fluctuation frequency.
Determining the actual sub-mucosal vascular coverage area includes:
and judging whether the pixel change value of the pixel is greater than a first threshold, if so, determining that the pixel is a suspected blood vessel coverage area, and when the difference between the pixel fluctuation frequency and the blood volume fluctuation frequency of the pixel is less than a second threshold, determining that the pixel is an actual blood vessel coverage area.
Example 2
As shown in fig. 2, embodiment 2 of the present invention provides a system for assisting in identifying a submucosal blood vessel endoscopically, including:
the image acquisition module is used for acquiring a time sequence image of the part to be detected in real time from endoscopy;
the preprocessing module is used for preprocessing a time sequence image of the part to be detected, which is acquired in real time, and converting the pixel value of the time sequence image into a zero mean value and a unit variance;
the first calculation module is used for extracting blood volume waves from the preprocessed time sequence images based on an imaging type photoplethysmography technology and determining corresponding blood volume fluctuation frequency;
the second calculation module is used for extracting the pixel change value of each pixel point from the preprocessed time sequence image based on the imaging type photoplethysmography technology and determining the pixel fluctuation frequency of the corresponding pixel point;
the judgment module is used for determining the actual blood vessel coverage area under the mucosa according to the blood volume wave, the blood volume fluctuation frequency, the pixel change value and the pixel fluctuation frequency;
and the display module is used for displaying the determined actual blood vessel coverage area by adopting a virtual contour method.
In this embodiment 2, a method for assisting in identifying a submucosal blood vessel under an endoscope is implemented based on the above system for assisting in identifying a submucosal blood vessel under an endoscope, and the method includes the following steps:
preprocessing a time sequence image of a part to be detected, which is acquired in real time, and converting a pixel value of the time sequence image into a zero mean value and a unit variance;
extracting blood volume waves from the preprocessed time sequence images based on an imaging type photoplethysmography technology, and determining corresponding blood volume fluctuation frequency;
based on an imaging type photoplethysmography technology, extracting a pixel change value of each pixel point from the preprocessed time sequence image, and determining the pixel fluctuation frequency of the corresponding pixel point;
and determining the actual blood vessel coverage area under the mucosa according to the blood volume wave, the blood volume fluctuation frequency, the pixel change value and the pixel fluctuation frequency.
Before preprocessing the time sequence image of the part to be detected acquired in real time, the time sequence image of the part to be detected is acquired in real time from endoscopy, and the method comprises the following steps: and keeping the lens stable for 2-3s by using an endoscope camera to obtain N frames of RGB images to be detected. In the process, a perception hash algorithm (perceptual hash algorithm) is used for calculating the lens shaking degree in real time, and when the difference between the hash values of the two frames of images is larger than a certain threshold value, the image information needs to be recorded again.
The method further comprises displaying the vessel coverage area by adopting a ghost outline method for the determined actual vessel coverage area.
In this embodiment 2, the preprocessing the time-series image of the to-be-detected portion acquired in real time includes:
and selecting green component G data from the read RGB time sequence image data of the part to be detected as original data for extracting the blood flow information image of the alimentary tract, and converting the pixel value into a zero mean value and a unit variance by combining Gaussian blurring and whitening processing.
Extracting a blood volume wave comprises:
the pixel value of each frame of image is regarded as a continuous function of time, and the calculation formula of the blood volume wave is as follows:
wherein v iskRepresenting the blood volume value of the k frame image, k is more than or equal to 0<N, N represents the total number of frames of the image; p is a radical ofkxyRepresenting the preprocessing value of green component G data of a pixel point with the k frame image coordinate of (x, y); w represents the width of the image; h represents the height of the image.
Determining the blood volume fluctuation frequency comprises:
smoothing the blood volume wave by using time domain filtering, and performing energy spectrum analysis by using fast Fourier transform, wherein the frequency corresponding to the highest energy point is the blood volume fluctuation frequency.
The extracting of the pixel change value of each pixel point includes:
the pixel change value of the pixel point, namely the sum of absolute values of the pixel value differences of the coordinate points corresponding to every two adjacent frames of images in the continuous N frames of images, is calculated according to the following formula:
wherein S isxyRepresenting pixel change values of pixel points of coordinate violation (x, y) in continuous N frames of images; p is a radical ofkxyAnd (3) representing the preprocessing value of the green component G data of the pixel point with the k frame image coordinate of (x, y).
Determining the pixel fluctuation frequency of the corresponding pixel point comprises:
calculating the change rule of green component G data of each pixel in the continuous N frames of image data, and setting the value of G (x, y) component value G (x, y) of a pixel point with certain coordinate (x, y) in the continuous N frames of images to form a G component sequence { G (x, y) [ k ] }, wherein k represents the k frame of image, and k is more than or equal to 0 and less than N;
and smoothing the component sequence by using time-domain filtering, and performing energy spectrum analysis by using fast Fourier transform, wherein the frequency corresponding to the highest energy point is the pixel fluctuation frequency.
Determining the actual sub-mucosal vascular coverage area includes:
and judging whether the pixel change value of the pixel is greater than a first threshold, if so, determining that the pixel is a suspected blood vessel coverage area, and when the difference between the pixel fluctuation frequency and the blood volume fluctuation frequency of the pixel is less than a second threshold, determining that the pixel is an actual blood vessel coverage area.
Example 3
The embodiment 3 of the invention provides a method for assisting in identifying submucosal blood vessels under an endoscope, which comprises the following steps:
step 1: obtaining time sequence image of detection part in real time from endoscope examination
And keeping the lens stable for 2-3s by using an endoscope camera to obtain N frames of RGB images to be detected. In the process, a perception hash algorithm (perceptual hash algorithm) is used for calculating the lens shaking degree in real time, and when the difference between the hash values of the two frames of images is larger than a certain threshold value, the image information needs to be recorded again.
Step 2: preprocessing acquired images
Green component G image data is selected from the read RGB image data as original data of the digestive tract blood flow information image. The reason is that the absorption rate of the oxygenated hemoglobin to green light is the largest, and experimental comparison shows that the change of the pixel value of the G channel caused by the blood filling amount is the largest.
In this embodiment 3, in order to filter out thin capillary vessel signal interference, keep thick and big blood vessel and reduce camera sensor self-borne noise, adopt gaussian blur and whitening to handle, convert pixel value into zero mean and unit variance.
And step 3: extraction of blood volume wave and fluctuation frequency from time sequence image based on IPPG technology
The pixel value of each frame of image is regarded as a continuous function of time, and the calculation formula of the blood volume wave is as follows:
wherein v iskRepresenting the blood volume value of the k frame image, k is more than or equal to 0<N, N represents the total number of frames of the image; p is a radical ofkxyRepresenting the preprocessing value of green component G data of a pixel point with the k frame image coordinate of (x, y); w represents the width of the image; h represents the height of the image.
Smoothing the blood volume wave obtained by the formula by using time domain filtering, and performing energy spectrum analysis by using fast Fourier transform, wherein the frequency corresponding to the highest energy point is the fluctuation frequency fv。
And 4, step 4: extraction of pixel change value and fluctuation frequency of each pixel point from image based on IPPG technology
Step 4.1: pixel fluctuation frequency of pixel
Calculating the change rule of the green component of each pixel in the continuous N frames of image data, and setting the value of the G component value G (x, y) of a certain coordinate (x, y) pixel in the continuous N frames of images to form a G component sequence { G (x, y) [ k ] }, wherein k represents the k-th frame of image, and k is more than or equal to 0 and less than N.
Smoothing the component sequence by using time domain filtering, and performing energy map analysis by using fast Fourier transform, wherein the frequency corresponding to the highest energy point is the pixel fluctuation frequency f of the pixel pointxy。
Step 4.2: pixel variation value of pixel
The pixel change value of the pixel point is the sum of absolute values of the pixel value differences of the coordinate points corresponding to every two adjacent frames of images in the continuous N frames of images, and the calculation formula is as follows: the pixel change value of the pixel point, namely the sum of absolute values of the pixel value differences of the coordinate points corresponding to every two adjacent frames of images in the continuous N frames of images, is calculated according to the following formula:
The existing IPPG technology can only divide time sequence imagesAnd analyzing to obtain the fluctuation frequency of the color. In the present embodiment 3, S is as described abovexyThe calculation formula (2) can not only obtain the fluctuation frequency, but also predict the main area influencing the fluctuation frequency.
And 5: further statistical analysis is carried out to determine the blood vessel region
Pixel point pixel variation value SxyThe larger the point is, the more abundant the pixel change is, when SxyIf the value is larger than the first threshold value, the point is suspected to be a blood vessel coverage area. When the pixel fluctuation frequency f is at the pointxyWith the frequency f of the fluctuation of the blood volumevWhen the difference is within a certain threshold range, the pixel point can be determined as an actual blood vessel coverage area.
Step 6: and converting the coordinate point set in the actual blood vessel coverage area into the actual coverage area by using a display module, and displaying the actual coverage area on a computer screen by using a virtual outline.
Example 4
The embodiment 4 of the present invention provides a computer device, including a memory and a processor, where the processor and the memory are in communication with each other, the memory stores a program instruction executable by the processor, and the processor calls the program instruction to execute a method for assisting in identifying submucosal blood vessels under an endoscope, including the following steps:
preprocessing a time sequence image of a part to be detected, which is acquired in real time, and converting a pixel value of the time sequence image into a zero mean value and a unit variance;
extracting blood volume waves from the preprocessed time sequence images based on an imaging type photoplethysmography technology, and determining corresponding blood volume fluctuation frequency;
based on an imaging type photoplethysmography technology, extracting a pixel change value of each pixel point from the preprocessed time sequence image, and determining the pixel fluctuation frequency of the corresponding pixel point;
and determining the blood vessel covering area under the mucosa according to the blood volume wave, the blood volume fluctuation frequency, the pixel change value and the pixel fluctuation frequency.
Example 5
An embodiment 5 of the present invention provides a computer-readable storage medium, in which a computer program is stored, and the computer program, when executed by a processor, implements a method for assisting in identifying a submucosal blood vessel under an endoscope, including the steps of:
preprocessing a time sequence image of a part to be detected, which is acquired in real time, and converting a pixel value of the time sequence image into a zero mean value and a unit variance;
extracting blood volume waves from the preprocessed time sequence images based on an imaging type photoplethysmography technology, and determining corresponding blood volume fluctuation frequency;
based on an imaging type photoplethysmography technology, extracting a pixel change value of each pixel point from the preprocessed time sequence image, and determining the pixel fluctuation frequency of the corresponding pixel point;
and determining the blood vessel covering area under the mucosa according to the blood volume wave, the blood volume fluctuation frequency, the pixel change value and the pixel fluctuation frequency.
In summary, the method and the system for assisting in identifying the submucosal blood vessel under the endoscope according to the embodiments of the present invention can accurately extract the blood flow information of the digestive tract in real time and accurately identify the submucosal blood vessel without using additional equipment and prolonging the operation time in the operation process of the endoscope, thereby ensuring the operation safety.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The above description is only a preferred embodiment of the present disclosure and is not intended to limit the present disclosure, and various modifications and changes may be made to the present disclosure by those skilled in the art. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present disclosure should be included in the protection scope of the present disclosure.
Although the present disclosure has been described with reference to the specific embodiments shown in the drawings, it is not intended to limit the scope of the present disclosure, and it should be understood by those skilled in the art that various modifications and variations can be made without inventive faculty based on the technical solutions disclosed in the present disclosure.
Claims (8)
1. A method for assisting in identifying submucosal blood vessels under an endoscope is characterized by comprising the following steps:
acquiring a plurality of frames of RGB time sequence images of a part to be detected; calculating the hash values of the front and rear frames of images in real time by using a perceptual hash algorithm, and when the difference between the hash values of the front and rear frames of images is greater than a threshold value, recording RGB time sequence image information again;
selecting G channel component data in an RGB time sequence image of a part to be detected, which is acquired in real time, preprocessing the G channel component data, and converting a pixel value of the time sequence image into a zero mean value and a unit variance;
extracting blood volume waves from the preprocessed time sequence images based on an imaging type photoplethysmography technology, and determining corresponding blood volume fluctuation frequency; the extracting a blood volume wave comprises:
the pixel value of each frame of image is regarded as a continuous function of time, and the calculation formula of the blood volume wave is as follows:
based on an imaging type photoplethysmography technology, extracting a pixel change value of each pixel point from the preprocessed time sequence image, and determining the pixel fluctuation frequency of the corresponding pixel point;
determining the actual blood vessel coverage area under the mucosa according to the blood volume wave, the blood volume fluctuation frequency, the pixel change value and the pixel fluctuation frequency; the determining of the actual sub-mucosal vascular coverage area comprises: and judging whether the pixel change value of the pixel is greater than a first threshold, if so, determining that the pixel is a suspected blood vessel coverage area, and when the difference between the pixel fluctuation frequency and the blood volume fluctuation frequency of the pixel is less than a second threshold, determining that the pixel is an actual blood vessel coverage area.
2. The method for endovascularly assisted identification of submucosal blood vessels according to claim 1, further comprising obtaining time series images of the site to be detected in real time from an endoscopic examination.
3. The method for endovascularly assisted identification of submucosal blood vessels according to claim 1, further comprising displaying the actual vessel coverage area using a ghost outline approach.
4. The method for endovascularly assisted identification of submucosal blood vessels according to claim 1, wherein the preprocessing of the real-time acquired time series images of the site to be detected comprises:
and the G channel component data is used as original data for extracting the blood flow information image of the digestive tract, and the pixel value is converted into zero mean value and unit variance by combining Gaussian blurring and whitening processing.
5. The method of endoscopically assisted identification of submucosal blood vessels, according to claim 1, wherein determining the blood volume fluctuation frequency comprises:
smoothing the blood volume wave by using time domain filtering, and performing energy spectrum analysis by using fast Fourier transform, wherein the frequency corresponding to the highest energy point is the blood volume fluctuation frequency.
6. The method of claim 1, wherein extracting the pixel variation value of each pixel point comprises:
the pixel change value of the pixel point, namely the sum of absolute values of the pixel value differences of the coordinate points corresponding to every two adjacent frames of images in the continuous N frames of images, is calculated according to the following formula:
7. The method of endoscopically assisted submucosal blood vessel identification according to claim 6, wherein determining the pixel fluctuation frequency for the corresponding pixel site comprises:
calculating the change rule of green component G data of each pixel in the continuous N frames of image data, and setting the value of G (x, y) component value G (x, y) of a pixel point with certain coordinate (x, y) in the continuous N frames of images to form a G component sequence { G (x, y) [ k ] }, wherein k represents the k frame of image, and k is more than or equal to 0 and less than N;
and smoothing the component sequence by using time-domain filtering, and performing energy spectrum analysis by using fast Fourier transform, wherein the frequency corresponding to the highest energy point is the pixel fluctuation frequency.
8. A system for assisting in identifying submucosal blood vessels under an endoscope, comprising:
the preprocessing module is used for preprocessing a time sequence image of the part to be detected, which is acquired in real time, and converting the pixel value of the time sequence image into a zero mean value and a unit variance;
the first calculation module is used for extracting blood volume waves from the preprocessed time sequence images based on an imaging type photoplethysmography technology and determining corresponding blood volume fluctuation frequency; the extracting a blood volume wave comprises:
the pixel value of each frame of image is regarded as a continuous function of time, and the calculation formula of the blood volume wave is as follows:
the second calculation module is used for extracting the pixel change value of each pixel point from the preprocessed time sequence image based on the imaging type photoplethysmography technology and determining the pixel fluctuation frequency of the corresponding pixel point;
the judgment module is used for determining the actual blood vessel coverage area under the mucosa according to the blood volume wave, the blood volume fluctuation frequency, the pixel change value and the pixel fluctuation frequency; the determining of the actual sub-mucosal vascular coverage area comprises: and judging whether the pixel change value of the pixel is greater than a first threshold, if so, determining that the pixel is a suspected blood vessel coverage area, and when the difference between the pixel fluctuation frequency and the blood volume fluctuation frequency of the pixel is less than a second threshold, determining that the pixel is an actual blood vessel coverage area.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011639891.7A CN112842285B (en) | 2020-12-31 | 2020-12-31 | Method and system for assisting in identifying submucosal blood vessels under endoscope |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011639891.7A CN112842285B (en) | 2020-12-31 | 2020-12-31 | Method and system for assisting in identifying submucosal blood vessels under endoscope |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112842285A CN112842285A (en) | 2021-05-28 |
| CN112842285B true CN112842285B (en) | 2021-09-24 |
Family
ID=76000590
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011639891.7A Active CN112842285B (en) | 2020-12-31 | 2020-12-31 | Method and system for assisting in identifying submucosal blood vessels under endoscope |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112842285B (en) |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109363630A (en) * | 2018-09-03 | 2019-02-22 | 浙江大华技术股份有限公司 | A kind of vital sign information measurement method and device |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9750575B2 (en) * | 2012-06-28 | 2017-09-05 | Koninklijke Philips N.V. | Evaluation of patency using photo-plethysmography on endoscope images |
| EP3456243A1 (en) * | 2017-09-14 | 2019-03-20 | Koninklijke Philips N.V. | Improved vessel geometry and additional boundary conditions for hemodynamic ffr/ifr simulations from intravascular imaging |
| CN111801704B (en) * | 2018-01-03 | 2025-02-11 | 皇家飞利浦有限公司 | Systems and methods for detecting abnormal tissue using vascular features |
| CN110279402A (en) * | 2019-07-31 | 2019-09-27 | 杭州泽铭睿股权投资有限公司 | A kind of imaging method of veins beneath the skin optical video image |
| CN111387966B (en) * | 2020-03-20 | 2022-08-26 | 中国科学院深圳先进技术研究院 | Signal wave reconstruction method and heart rate variability information detection device |
| CN111898416A (en) * | 2020-06-17 | 2020-11-06 | 绍兴埃瓦科技有限公司 | Video stream processing method and device, computer equipment and storage medium |
-
2020
- 2020-12-31 CN CN202011639891.7A patent/CN112842285B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109363630A (en) * | 2018-09-03 | 2019-02-22 | 浙江大华技术股份有限公司 | A kind of vital sign information measurement method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112842285A (en) | 2021-05-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107529646B (en) | Non-contact heart rate measurement method and device based on Euler image amplification | |
| US8811698B2 (en) | Image processing apparatus, image processing method, and computer-readable recording medium | |
| CN105825516B (en) | A kind of DICOM images blood flow analysis system | |
| US7248724B2 (en) | Method for monitoring wounds | |
| US9053533B2 (en) | Image processing apparatus, image processing method, and computer-readable recording medium for extracting a dark portion area from an intraluminal image | |
| CN103702014B (en) | Non-contact physiological parameter detection method, system and device | |
| US20110317043A1 (en) | Image processing device and information storage medium | |
| US20100277650A1 (en) | Scene-change detection device | |
| CN111938622B (en) | Heart rate detection method, device and system and readable storage medium | |
| CN113693573B (en) | Video-based non-contact multi-physiological-parameter monitoring system and method | |
| Rokade et al. | Automatic detection of hard exudates in retinal images using haar wavelet transform | |
| JP2010512173A (en) | Computer-aided analysis using video from an endoscope | |
| EP2405397A1 (en) | Processing of images captured inside a body lumen | |
| CN115187596B (en) | Neural intelligent auxiliary recognition system for laparoscopic colorectal cancer surgery | |
| CN105118070A (en) | Time series based method for positioning bleeding segments in WCE (wireless capsule endoscope) video | |
| CN117158926A (en) | A long-distance non-contact physiological parameter detection method, system and device | |
| KR20200046775A (en) | Surgical assist device and method for 3D analysis based on liver cancer area in CT image | |
| CN118592919B (en) | Medical auxiliary monitoring method and device based on monitor screen waveform camera | |
| CN111223090A (en) | Identification system of tumor image in human lung CT image | |
| CN112842285B (en) | Method and system for assisting in identifying submucosal blood vessels under endoscope | |
| CN114677713A (en) | Near-infrared light-based arm vein blood sampling point identification method and system | |
| JP2005312937A (en) | Medical image processing apparatus and medical image processing method | |
| CN119184591A (en) | Bleeding point positioning method, positioning device, electronic equipment and medium | |
| CN116012763A (en) | A method, device and storage medium for processing an ablation image | |
| US12272052B2 (en) | Method and system for image normalisation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |