JP2008093213A - Medical image processing apparatus and medical image processing method - Google Patents

Abstract

An object of the present invention is to accurately construct a three-dimensional image from a two-dimensional image and improve detection accuracy when detecting a lesion having a local raised shape.
An endoscope system according to the present embodiment includes a medical observation apparatus, a medical image processing apparatus, and a monitor, and a main part is configured. The CPU 22 of the medical image processing apparatus includes a three-dimensional model estimation unit 22a, a detection target region setting unit 22b, a shape feature amount calculation unit 22c as a shape feature amount calculation unit, a model correction unit 22d, a three-dimensional shape detection unit 22e, and a polyp. It consists of each function part of the determination part 22f.
[Selection] Figure 2

Description

  The present invention relates to a medical image processing apparatus and a medical image processing method, and in particular, a medical image processing apparatus and a medical use for estimating three-dimensional model data of a living tissue based on two-dimensional image data of a living tissue image. The present invention relates to an image processing method.

  Conventionally, in the medical field, observation using an image pickup device such as an X-ray diagnostic apparatus, CT, MRI, ultrasonic observation apparatus, and endoscope apparatus has been widely performed. Among such imaging devices, an endoscope apparatus has an insertion part that can be inserted into a body cavity, for example, and an image inside the body cavity formed by an objective optical system arranged at the distal end of the insertion part Is picked up by an image pickup means such as a solid-state image pickup device and output as an image pickup signal, and based on the image pickup signal, an image of the image in the body cavity is displayed on a display means such as a monitor. Then, based on the image of the image in the body cavity displayed on the display unit such as a monitor, the user observes an organ or the like in the body cavity, for example.

  In addition, the endoscope apparatus can directly capture an image of the digestive tract mucosa. Therefore, the user can comprehensively observe, for example, the color tone of the mucous membrane, the shape of the lesion, and the fine structure of the mucosal surface.

  Furthermore, the endoscope apparatus is described in, for example, Japanese Patent Application Laid-Open No. 2005-192880 (Patent Document 1) as an image processing method capable of detecting a predetermined image in which a lesion having a local raised shape exists. It is also possible to detect an image including a lesion site such as a polyp by using the existing image processing method.

  The image processing method described in Patent Document 1 can extract a contour of an input image and detect a lesion having a local ridge shape in the image based on the shape of the contour. .

Conventionally, in the colon polyp detection process, three-dimensional data is estimated from a two-dimensional image, and a colon polyp is detected using a three-dimensional feature (Shape Index / Curvedness) (Non-Patent Document 1). This three-dimensional feature amount is realized by calculating a partial differential coefficient at a reference point from three-dimensional data and using the partial differential coefficient. In the colon polyp detection process, a polyp candidate is detected by performing threshold processing on the three-dimensional feature value.
JP 2005-192880 A The Institute of Electronics, Information and Communication Engineers, IEICE Technical Report (MI2003-102), Examination of automatic detection method for colorectal polyps from 3D abdominal CT images based on shape information Kimura, Hayashi, Kitasaka, Mori, Suecho pp. 29-34, 2004

  However, the “Shape From Shading” method, which has been conventionally used as a method for estimating three-dimensional data, has a problem that accuracy is deteriorated in a portion imaged as an edge in a two-dimensional image.

  Specifically, when a three-dimensional image such as a polyp is picked up by an endoscope, an edge region is defined as a boundary at the boundary between a curved surface on the three-dimensional image or a boundary between a visible range and an occlusion (visible hiding or invisible) range. Will occur. The “Shape From Shading” method for estimating three-dimensional data from a two-dimensional image is a method for estimating a three-dimensional position based on pixel values of the two-dimensional image. Since the pixel value of the edge region is lower than the pixel value of the adjacent region, the “Shape From Shading” method uses a three-dimensional position as if a “groove (concave)” exists in the edge region. The estimated value is calculated.

  For example, with respect to a large intestine endoscopic image having a polyp 500 as shown in FIG. 26 as an imaging target, the coordinates of the three-dimensional data points are estimated by the “Shape From Shading” method, and a three-dimensional surface is obtained from the three-dimensional data points. When generating, ideally, a three-dimensional image as shown in FIG. 27 should be generated.

  However, the cross section when the three-dimensional surface is actually cut out by the “Shape From Shading” method at the plane x = x0 (dotted line position in FIG. 26) is shown in the edge region portion of the two-dimensional image data as shown in FIG. Thus, it is estimated as if “grooves (concaves)” exist. In the intestinal tract, such “grooves (concaves)” do not exist. Therefore, the conventional “Shape From Shading” method estimates nonexistent “grooves (concaves)” as three-dimensional data. Therefore, there is a problem that the accuracy of estimation of the three-dimensional image is deteriorated, and the conventional colon polyp detection processing using the three-dimensional image estimation has problems such as a decrease in detection accuracy and occurrence of erroneous detection.

  The present invention has been made in view of the above circumstances, and can accurately construct a three-dimensional image from a two-dimensional image and improve the detection accuracy when detecting a lesion having a local raised shape. An object is to provide a medical image processing apparatus and a medical image processing method.

The medical image processing apparatus of the present invention is
Three-dimensional model estimation means for estimating a three-dimensional model of the living tissue from a two-dimensional image of the image of the living tissue in the body cavity input from the medical imaging device;
Image boundary detection means for detecting an image boundary of an image area constituting the two-dimensional image,
A line width calculating means for calculating a line width of the image boundary line;
Correction means for correcting the estimation result of the three-dimensional model of the image boundary line in the three-dimensional model estimation means based on the line width;
And is configured.

Moreover, the medical image processing method of the present invention includes:
A three-dimensional model estimation step for estimating a three-dimensional model of the living tissue from a two-dimensional image of the image of the living tissue in the body cavity input from the medical imaging device;
An image boundary detection step for detecting an image boundary of an image region constituting the two-dimensional image image;
A line width calculating step of calculating a line width of the image boundary line;
A correction step of correcting an estimation result of the three-dimensional model of the image boundary line in the three-dimensional model estimation step based on the line width;
And is configured.

  According to the present invention, there is an effect that a three-dimensional image is accurately constructed from a two-dimensional image, and the detection accuracy in detecting a lesion having a local raised shape can be improved.

  Embodiments of the present invention will be described below with reference to the drawings.

  1 to 16 relate to a first embodiment of the present invention, FIG. 1 is a diagram showing an example of the overall configuration of an endoscope system in which a medical image processing apparatus is used, and FIG. 2 is a functional configuration of the CPU of FIG. FIG. 3 is a diagram showing a storage information configuration of the hard disk in FIG. 1, FIG. 4 is a flowchart showing a processing flow of the CPU in FIG. 2, and FIG. 5 is a flow of correction processing of the three-dimensional model data in FIG. 6 is a first diagram for explaining the processing of FIG. 1, FIG. 7 is a second diagram for explaining the processing of FIG. 1, and FIG. 8 is a third diagram for explaining the processing of FIG. 9 is a fourth diagram for explaining the processing of FIG. 1, FIG. 10 is a flowchart showing the flow of correction processing of three-dimensional point sequence data between edge end points as the edge region of FIG. 5, and FIG. 11 is for explaining the processing of FIG. FIG. 12 is a sixth diagram illustrating the process of FIG. 1, and FIG. FIG. 14 is a flowchart showing the processing flow of the second modification of FIG. 10, FIG. 15 is a first diagram for explaining the processing of FIG. 16 is a second diagram for explaining the processing of FIG.

  As shown in FIG. 1, the endoscope system 1 according to the present embodiment includes a medical observation device 2, a medical image processing device 3, and a monitor 4, and a main part is configured.

  The medical observation apparatus 2 is an observation apparatus that captures an image of a subject and outputs a two-dimensional image of the image of the subject. The medical image processing apparatus 3 is constituted by a personal computer or the like, and performs image processing on the video signal of the two-dimensional image output from the medical observation apparatus 2 and the video after the image processing is performed. An image processing apparatus that outputs a signal as an image signal. Furthermore, the monitor 4 is a display device that displays an image based on an image signal output from the medical image processing device 3.

  The medical observation apparatus 2 includes an endoscope 6, a light source device 7, a camera control unit (hereinafter abbreviated as CCU) 8, and a monitor 9, and a main part is configured.

  The endoscope 6 is inserted into a body cavity of a subject and images a subject such as a living tissue existing in the body cavity and outputs it as an imaging signal. The light source device 7 supplies illumination light for illuminating a subject imaged by the endoscope 6. The CCU 8 performs various controls on the endoscope 6, performs signal processing on the imaging signal output from the endoscope 6, and outputs the processed signal as a video signal of a two-dimensional image. The monitor 9 displays an image of a subject imaged by the endoscope 6 based on a video signal of a two-dimensional image output from the CCU 8.

  The endoscope 6 includes an insertion portion 11 that is inserted into a body cavity and an operation portion 12 that is provided on the proximal end side of the insertion portion 11. A light guide 13 for transmitting illumination light supplied from the light source device 7 is inserted into a portion from the proximal end side in the insertion portion 11 to the distal end portion 14 on the distal end side in the insertion portion 11. Yes.

  The light guide 13 has a distal end side disposed at the distal end portion 14 of the endoscope 6 and a rear end side connected to the light source device 7.

  Since the light guide 13 has such a configuration, the illumination light supplied from the light source device 7 is transmitted by the light guide 13 and then provided on the distal end surface of the distal end portion 14 of the insertion portion 11 (not shown). It is emitted from the illumination window. Then, illumination light is emitted from an illumination window (not shown) to illuminate a living tissue or the like as a subject.

  At the distal end portion 14 of the endoscope 6, an objective optical system 15 attached to an observation window (not shown) adjacent to an illumination window (not shown) and an imaging position of the objective optical system 15 are arranged. An image pickup unit 17 having an image pickup element 16 constituted by (element) or the like is provided. With such a configuration, the subject image formed by the objective optical system 15 is captured by the image sensor 16 and then output as an image signal. Note that the image sensor 16 is not limited to a CCD, and may be constituted by a C-MOS sensor, for example.

  The image sensor 16 is connected to the CCU 8 through a signal line. The image sensor 16 is driven based on the drive signal output from the CCU 8 and outputs an image signal corresponding to the captured subject image to the CCU 8.

  In addition, the image pickup signal input to the CCU 8 is subjected to signal processing in a signal processing circuit (not shown) provided inside the CCU 8 to be converted and output as a video signal of a two-dimensional image. The video signal of the two-dimensional image output from the CCU 8 is output to the monitor 9 and the medical image processing apparatus 3. Thereby, the monitor 9 displays the image of the subject based on the video signal output from the CCU 8 as a two-dimensional image.

  The medical image processing device 3 performs an A / D conversion on the video signal of the two-dimensional image output from the medical observation device 2, and outputs the video output from the image input unit 21. CPU 22 as a central processing unit that performs image processing on a signal, a processing program storage unit 23 in which a processing program related to the image processing is written, and an image that stores a video signal output from the image input unit 21 The storage unit 24 and an analysis information storage unit 25 that stores calculation results and the like in image processing performed by the CPU 22 are configured.

  The medical image processing apparatus 3 includes a storage device interface (I / F) 26, image data as a result of image processing of the CPU 22 via the storage device I / F 26, various data used by the CPU 22 for image processing, and the like. Display processing for displaying the image data on the monitor 4 on the basis of the image data as the image processing result of the CPU 22 and the hard disk 27 as a storage device. The display processing unit 28 outputs image data as an image signal, and includes a pointing device such as a keyboard or a mouse that allows a user to input parameters for image processing performed by the CPU 22 and operation instructions for the medical image processing apparatus 3. And an input operation unit 29. The monitor 4 displays an image based on the image signal output from the display processing unit 28.

  Each of the image input unit 21, the CPU 22, the processing program storage unit 23, the image storage unit 24, the analysis information storage unit 25, the storage device interface 26, the display processing unit 28, and the input operation unit 29 of the medical image processing apparatus 3 is provided. Are connected to each other via a data bus 30.

  As shown in FIG. 2, the CPU 22 includes a three-dimensional model estimation unit 22a as a three-dimensional model estimation unit, a detection target region setting unit 22b, a shape feature amount calculation unit 22c, a line width calculation unit, and a model correction unit as a correction unit. 22d, a three-dimensional shape detection unit 22e, and a polyp determination unit 22f.

  In the present embodiment, these functional units are realized by software executed by the CPU 22. The detailed operation of these functional units will be described later.

  Further, as shown in FIG. 3, the hard disk 27 has a plurality of storage areas for storing various data generated by each process performed by the CPU 22. Specifically, the hard disk 27 has an edge image storage area 27a, an edge thinned image storage area 27b, a three-dimensional point sequence data storage area 27c, a correspondence table storage area 27d, and a detected lesion part storage area 27e. Details of the data stored in each of these storage areas will be described later.

  Next, regarding the operation of the endoscope system 1 of the present embodiment configured as described above, referring to FIGS. 6 to 9, 11, and 12, using the flowcharts of FIGS. 4, 5, and 10. I will explain.

  First, after the user turns on the power of each part of the endoscope system 1, the user inserts the insertion part 11 of the endoscope 6 into the body cavity of the subject, for example, into the large intestine.

  When the insertion unit 11 is inserted into the large intestine of the subject by the user, for example, a subject such as a living tissue existing in the large intestine, for example, an image of the polyp 500 shown in FIG. Imaging is performed by the provided imaging unit 17. The subject image captured by the imaging unit 17 is output to the CCU 8 as an imaging signal.

  The CCU 8 converts the imaging signal as a video signal of a two-dimensional image by performing signal processing on the imaging signal output from the imaging device 16 of the imaging unit 17 in a signal processing circuit (not shown). The monitor 9 displays the subject image captured by the imaging unit 17 as a two-dimensional image based on the video signal output from the CCU 8. Further, the CCU 8 outputs a video signal of a two-dimensional image obtained by performing signal processing on the imaging signal output from the imaging element 16 of the imaging unit 17 to the medical image processing apparatus 3.

  The two-dimensional image video signal output to the medical image processing apparatus 3 is A / D converted by the image input unit 21 and then input to the CPU 22.

  Then, as shown in FIG. 4, the three-dimensional model estimation unit 22a of the CPU 22 uses, for example, a “Shape From Shading” method or the like on the two-dimensional image output from the image input unit 21 in step S1. A three-dimensional model corresponding to the two-dimensional image is estimated by performing a process such as geometric conversion based on the luminance information of the two-dimensional image. At this time, the three-dimensional model estimation unit 22a generates a correspondence table in which the three-dimensional data point sequence indicating the intestinal tract surface of the large intestine is associated with the two-dimensional image data, and stores the correspondence table in the correspondence table storage area 27d of the hard disk 27. .

  Then, the model correction unit 22d of the CPU 22 performs model correction processing to be described later on the three-dimensional model estimated in step S2, and the coordinates of each data point of the corrected three-dimensional model are stored via the storage device I / F 26. And stored in the three-dimensional point sequence data storage area 27 c of the hard disk 27.

  Next, the detection target region setting unit 22b of the CPU 22 detects the color tone change of the two-dimensional image output from the image input unit 21 in step S3 and the bulge change of the three-dimensional model corrected by the process of step S2. By doing this, a target region that is a detection target region is set as a target region to which a process for detecting a lesion having a raised shape in the three-dimensional model is applied.

  Specifically, the detection target area setting unit 22b of the CPU 22 converts, for example, the two-dimensional image output from the image input unit 21 into each plane of an R (red) image, a G (green) image, and a B (blue) image. After separation into images, a bulge change is detected based on the data of the three-dimensional model estimated according to the R image, and a color tone change is detected based on the chromaticity of the R image and the G image. Then, the detection target area setting unit 22b of the CPU 22 uses, as the target area, an area where both the bulge change and the color change are detected based on the detection result of the bulge change and the detection result of the color change. Set.

  Thereafter, the shape feature amount calculation unit 22c of the CPU 22 calculates a local partial differential coefficient of the target region in step S4. Specifically, the shape feature amount calculation unit 22c of the CPU 22 applies the R pixel value f in the local region (curved surface) including the noted three-dimensional position (x, y, z) to the calculated three-dimensional shape. First-order partial differential coefficients fx, fy, fz and second-order partial differential coefficients fx, fyy, fzz, fxy, fyz, fxz are calculated.

  Further, the shape feature value calculation unit 22c of the CPU 22 calculates, based on the local partial differential coefficient, as a shape feature value of (3D shape) for each data point existing in the processing target area of the 3D model in step S5. Processing to calculate the Shape Index value and the Curvedness value is performed.

  That is, using these local partial differential coefficients, the shape feature amount calculation unit 22c of the CPU 22 calculates the Gaussian curvature K and the average curvature H.

On the other hand, the principal curvatures k1 and k2 (k1 ≧ k2) of the curved surface are expressed as follows using the Gaussian curvature K and the average curvature H: k1 = H + (H ² −K) ^1/2 k2 = H− (H ² −K) ^{1 / 2} (1)
It is expressed.

In addition, Shape Index SI and Curvedness CV, which are feature quantities representing the curved surface shape in this case, are SI = 1 / 2− (1 / π) arc tan [(k1 + k2) / (k1−k2)] (2)
CV = ((k1 ² + k2 ² ) / 2) ^1/2 (3)
It becomes.

  In this way, the shape feature amount calculation unit 22c of the CPU 22 calculates the Shape Index SI and Curvedness CV of each three-dimensional curved surface as the three-dimensional shape information, and stores it in the analysis information storage unit 25.

  The above-described Shape Index value is a value for indicating the state of unevenness at each data point of the three-dimensional model, and is indicated as a numerical value in the range of 0 to 1. Specifically, at each data point existing in the three-dimensional model, when the Shape Index value is close to 0, the presence of a concave shape is suggested, and when the Shape Index value is close to 1, the convex shape The existence of a shape is suggested.

  The above-mentioned Curvedness value is a value for indicating the curvature at each data point of the three-dimensional model. Specifically, at each data point in the 3D model, the smaller the Curvedness value, the more sharply curved surface is suggested, and the larger the Curvedness value, the more gently curved surface. Is suggested.

  Next, in step S6, the three-dimensional shape detection unit 22d of the CPU 22 determines each value of the shape index value and the curvedness value at each data point existing in the target region of the three-dimensional model, and a predetermined threshold value set in advance. By performing comparison processing with T1 and T2, among the data points, a data group having a raised shape is detected. Specifically, the CPU 22 selects, for example, a plurality of data points whose shape index value is larger than the threshold value T1 and whose curvedness value is larger than the threshold value T2 among the data points existing in the processing target area of the three-dimensional model. It detects as a data group which has a protruding shape.

  Then, the polyp determination unit 22f of the CPU 22 is a data point in which each of a plurality of data points detected as a data group having a raised shape in the three-dimensional model in step S7 corresponds to a raised shape derived from a lesion such as a polyp. A raised shape discriminating process for discriminating whether or not is performed.

  Thereafter, the polyp determining unit 22f of the CPU 22 determines a region having a data group composed of data points corresponding to the raised shape derived from the lesion as a polyp region in step S8, and detects a polyp that is a lesion region.

  Then, the CPU 22 stores the detection result in association with the endoscopic image to be detected, for example, in the detected lesion storage area 27e of the hard disk 27 in FIG. 1, and detects it on the monitor 4 via the display processing unit 28, for example. Display alongside the target endoscopic image.

  As a result, a three-dimensional model of the subject is displayed on the monitor 4 such that the user can easily recognize the position where the raised shape derived from a lesion such as a polyp exists.

  Next, the model correction process by the model correction unit 22d in step S2 will be described. As shown in FIG. 5, the model correction unit 22d of the CPU 22 performs edge extraction processing on the input two-dimensional image in step S21 to generate an edge image, and stores it in the edge image storage area 27a of the hard disk 27. Store. In step S21, the model correction unit 22d performs thinning processing on the generated edge image to generate an edge thinned image, and stores it in the edge thinned image storage area 27b of the hard disk 27.

  If the two-dimensional edge image is shown in FIG. 6, the edge thinned image is an image obtained by thinning each edge region of the edge image to a line width of 1 pixel by thinning processing.

  Next, the model correction unit 22d of the CPU 22 sets the parameters i and j to 1 in step S23. Subsequently, the model correction unit 22d of the CPU 22 acquires the i-th edge line Li of the edge thinned image in step S24, and in step S25, the model correction unit 22d receives the j-th (point of interest) on the i-th edge line Li. Edge point Pi, j.

  Then, the model correction unit 22d of the CPU 22 generates an edge orthogonal line Hi, j that passes through the jth edge point Pi, j and is orthogonal to the i-th edge line Li in step S26. Subsequently, the model correction unit 22d of the CPU 22 determines the three-dimensional data points corresponding to the respective points (xi, j, yi, j, zi, j) on the two-dimensional image on the edge orthogonal line Hi, j in step S27. Get column from correspondence table. As described above, this correspondence table is stored in the correspondence table storage area 27d of the hard disk 27 by the three-dimensional model estimation unit 22a of the CPU 22 in step S1.

  Then, the model correcting unit 22d of the CPU 22 determines the end points Ai, j, Bi, j (see the enlarged view of FIG. 6) of the edges on the edge orthogonal line Hi, j in step S28. Specifically, in this embodiment, as shown in FIG. 7, pixel values corresponding to the respective points on the two-dimensional image are acquired, and values whose pixel values are smaller than a predetermined threshold t are set to the points on the edge. The edge end points Ai, j, Bi, j are obtained by making a determination one by one in each direction from each point on the thinned edge.

  Next, the model correction unit 22d of the CPU 22 obtains the distance Di, j between the edge endpoints Ai, j, Bi, j in step S29. In step S30, the model correction unit 22d of the CPU 22 determines whether the distance Di, j between the edge endpoints Ai, j, Bi, j is smaller than a predetermined value D0 (Di, j <D0).

  If it is determined that the distance Di, j between the edge endpoints Ai, j, Bi, j is smaller than the predetermined value D0 (Di, j <D0), the model correction unit 22d of the CPU 22 determines the edge endpoint Ai in step S31. , j, Bi, j are determined to be portions represented as edge regions, the three-dimensional point sequence data is corrected, and the process proceeds to step S33. Details of the correction process of the three-dimensional point sequence data in step S31 will be described later.

  If the distance Di, j between the edge points Ai, j, Bi, j is determined to be equal to or greater than the predetermined value D0, the model correction unit 22d of the CPU 22 determines between the edge points Ai, j, Bi, j in step S32. Is determined as a portion expressed as an occlusion area, the three-dimensional point sequence data between the edge endpoints Ai, j, Bi, j is deleted, and the process proceeds to step S33. Specifically, when the model correction unit 22d determines in step S32 that the distance Di, j between the edge endpoints Ai, j, Bi, j is equal to or greater than a predetermined value D0, as shown in FIG. The three-dimensional point sequence data line between the end points Ai, j, Bi, j of the edge is deleted as shown in FIG. 9 and corrected as an occlusion area. Is generated.

  Then, the model correction unit 22d of the CPU 22 determines whether or not the parameter j is less than all the points Nj on the edge line Li in step S33. If the parameter j is less than all the points Nj on the edge line Li, step S33 is performed. In step S34, the parameter j is incremented and the process returns to step S25, and the above steps S25 to S34 are repeated for all points on the edge line Li.

  When the above steps S25 to S33 are performed for all points on the edge line Li, the model correction unit 22d of the CPU 22 determines whether or not the parameter i is less than the number of all edge lines Ni in step S35. If the number of edge lines is less than Ni, the parameter i is incremented in step S36 and the process returns to step S24, and the above steps S24 to S36 are repeated for all edge lines.

  Next, the three-dimensional point sequence data correction process in step S31 will be described. In this three-dimensional point sequence data correction process, as shown in FIG. 10, the model correction unit 22d of the CPU 22 performs an edge (which is a point of interest) on the three-dimensional space having the three-dimensional point sequence data in step S41. An N × N × N cubic region centered on the point Pi, j is formed, and an average value of coordinates of the three-dimensional point sequence data included in the N × N × N cubic region is calculated. In step S42, the model correction unit 22d of the CPU 22 smoothes the coordinate data of the three-dimensional point sequence data on the point sequence data line of the edge portion using the calculated average value.

  The processing in FIG. 10 will be described in detail. For example, as shown in FIG. 11, N = 5 and a 3 × 5 × 5 cubic region centered on the edge point Pi, j (which is the point of interest) is 3 It is formed on a three-dimensional space having dimension point sequence data. Then, the average value of the coordinates of the three-dimensional point sequence data included in the 5 × 5 × 5 cubic region is calculated, and the coordinate data of the three-dimensional point sequence data on the point sequence data line of the edge portion is calculated by the calculated average value. By smoothing, it is possible to obtain a corrected point sequence data line as shown in FIG.

  The 3D point sequence data correction process is not limited to the process shown in FIG. 10, and the 3D point sequence data correction process of the following modifications 1 and 2 may be performed.

(Modification 1)
In the three-dimensional point sequence data correction process according to the first modification, the model correction unit 22d of the CPU 22 sets N = 5 (is an attention point) edge point Pi, j in step S41a as shown in FIG. Is formed on the two-dimensional image output from the image input unit 21. In step S42a, the model correction unit 22d of the CPU 22 calculates an average value of the pixel values (light / dark values) of the coordinate points of the two-dimensional image included in the 5 × 5 square area.

  In the case of this modification, after all the averaging (smoothing) processing is performed, the 3D point sequence data is recalculated for the edge region, and the 3D point sequence data is updated.

(Modification 2)
In the correction process of the three-dimensional point sequence data of the second modification, the model correction unit 22d of the CPU 22 determines from the edge endpoints Ai, j, Bi, j to the edge endpoints in step S41b as shown in FIG. Acquire the coordinates of consecutive N points each including Ai, j, Bi, j (however, the N points in the direction away from the edge point Pi, j (which is the point of interest)) and approximate them to a straight line. Project onto the x = 0 plane and find the intersection of the two projected approximate lines. Specifically, for example, when N = 3, as shown in FIG. 15, two approximate lines projected onto the x = 0 plane are approximated by straight lines including edge endpoints Ai, j, Bi, j, respectively. Find the intersection point Qi, j.

  Subsequently, in step S42b, the model correction unit 22d of the CPU 22 compares, for example, the z coordinate of each point in the edge region with the z coordinate of the intersection point Qi, j and uses it for correction according to the magnitude relationship. Determine an approximate line. Specifically, as shown in FIG. 16, when the y coordinate of each point in the edge region is smaller than the y coordinate of the intersection point Qi, j, approximation is performed using coordinate values smaller than the y coordinate of the intersection point Qi, j. If the y-coordinate of each point in the edge region is larger than the y-coordinate of the intersection point Qi, j, the intersection point Qi is used. , j is used as an approximate straight line approximated using a coordinate value larger than the y coordinate (in FIG. 16, an approximate straight line on the edge end point Bi, j side of the edge). Then, after determining the approximate straight line to be used, the correction point is determined by substituting (replacement) the y coordinate of the point of the edge region to be corrected into the approximate straight line.

  As described above, in the present embodiment and the first and second modifications, the threshold value is corrected using the position (Z coordinate) at the point of interest in the three-dimensional data. The threshold value excluding the influence can be used for the polyp detection process, and the detection accuracy of the polyp candidate can be improved. Therefore, it is possible to prompt the user to improve the polyp candidate discovery rate in the colonoscopy.

  17 to 25 relate to the second embodiment of the present invention, FIG. 17 is a diagram showing a storage information configuration of the hard disk, FIG. 18 is a first diagram for explaining the second embodiment, and FIG. 19 is for explaining the second embodiment. FIG. 2 is a third diagram for explaining the second embodiment, FIG. 21 is a flowchart showing the processing flow of the CPU according to the second embodiment, and FIG. 22 is a flowchart of the three-dimensional model data correction process of FIG. FIG. 23 is a first diagram for explaining the processing of FIG. 22, FIG. 24 is a second diagram for explaining the processing of FIG. 22, and FIG. 25 is a third diagram for explaining the processing of FIG. is there.

  Since the configuration of the present embodiment is almost the same as that of the first embodiment, only different points will be described.

  In this embodiment, as shown in FIG. 17, the hard disk 27 includes an edge image storage area 27a, an edge thinned image storage area 27b, a three-dimensional point sequence data storage area 27c, a correspondence table storage area 27d, and a detected lesion part storage area. In addition to 27e, it further has a morphology parameter map storage area 27f. Other configurations are the same as those of the first embodiment.

  As described above, in the “Shape From Shading” method, when there is an edge on a two-dimensional image, since the number of pixels on the edge takes a low value, an estimated value is obtained as if a groove exists in the edge portion. For example, even if the correction as in the first embodiment is performed, spike-shaped or edge-shaped noise may occur due to the leakage of the correction.

  In general, a combination of dilation processing and erosion processing, which is one morphological transformation, is widely used as a noise removal method.

  This morphological conversion is a conversion method for outputting the center of a sphere when the sphere is rolled on a three-dimensional surface. The dilation process rolls the sphere to the surface of the intestine and the erosion process rolls the sphere to the back of the intestine.

  The center of the sphere in the morphological transformation is determined by the size of the target noise. For example, as shown in FIG. 18, for example, when a sphere 300a having a diameter of 5 is rolled on a noisy surface having a width of one pixel, the locus of the center is a straight line, and thus the sphere of the same size from the back side of the surface formed by the locus. The noise can be filled by rolling

  However, as shown in FIG. 19, for example, when the sphere 300a having a diameter of 5 is rolled on the surface of the noise surface having a width of 5 pixels, there is a noise dent on the surface of the surface formed by the locus of the center. Even if you roll the ball from the back side, you can not remove the noise.

  When noise on Gomasio is placed on the entire three-dimensional data, the magnitude of the noise is small, but the magnitude of spike-like noise or edge-like noise existing on the edge depends on the thickness of the edge. Therefore, at the position where the edge exists in the two-dimensional image, it is necessary to change the diameter of the sphere for smoothing, that is, the smoothing parameter.

  For example, as shown in FIG. 20, when a sphere 300b having a sufficiently large sphere size at the time of morphological transformation is present, smoothing is performed from a noise size optimum for the sphere size to a smaller size noise. However, since the processing speed decreases as the sphere size increases, the processing speed decreases unless the sphere size optimum for the noise level is determined and executed.

  In the present embodiment, as illustrated in FIG. 21, the model correction unit 22d of the CPU 22 executes the morphology processing using the morphology parameter Wi in step S10 after the model correction processing in step S2 described in the first embodiment. The process proceeds to step S3 described in 1.

  Specifically, in this embodiment, in the model correction process in step S2, the model correction unit 22d of the CPU 22 creates a three-dimensional space as shown in FIG. 23 in steps S100 and S101 as shown in FIG. A map capable of storing morphological parameters in each small area is created by dividing into cubic small areas having a constant interval, and the morphological transformation parameters Wi, j as shown in FIG. 25 for the edge line width Di, j shown in FIG. The correspondence table is stored in the morphology parameter map storage area 27 f of the hard disk 27.

  That is, in step S100, the model correction unit 22d selects and acquires one edge of the edge thinned image as a processing target, and calculates the edge width Wi, j at one selected point of the edge line. Subsequently, the model correction unit 22d acquires the morphology conversion parameter W using the correspondence table of the edge line width Di, j and the morphology conversion parameter Wi, j in FIG.

  In step S101, the model correction unit 22d acquires the three-dimensional coordinates corresponding to the selected point on the edge line on which the process of step S100 has been performed from the corresponding point table recorded on the hard disk 27, and acquires the acquired tertiary. The coordinate position of the three-dimensional small area map corresponding to the coordinates is obtained. The morphology transformation parameter W is added and stored at the coordinate position of the obtained three-dimensional small area map, and one count value at the coordinate position is added.

  Here, the edge line width Di, j is the width of the corresponding edge of the edge image when a line orthogonal to the edge line is drawn at a point on the edge thin line of the edge thinned image as shown in FIG. Indicates. In addition, the morphological transformation parameter Wi, j in the edge image indicates the diameter of the sphere (see FIGS. 18 to 20).

  In step S10, the morphology transformation parameter W at the time of morphology transformation is determined based on the three-dimensional coordinates, and the dilation processing and erosion processing which are the morphology transformation processing are continuously executed to perform the noise removal processing.

  After the above processing is performed on all points of all edges on the image, the morphological transformation parameter W at each coordinate position of the three-dimensional small region map is averaged by the count value at each coordinate position. By performing such processing, it is possible to obtain the optimum parameter for smoothing by referring to the three-dimensional small area map based on the three-dimensional coordinate position.

  The present invention is not limited to the above-described embodiments, and various changes and modifications can be made without departing from the scope of the present invention.

The figure which shows an example of the whole structure of the endoscope system in which the medical image processing apparatus which concerns on Example 1 of this invention is used. Functional block diagram showing the functional configuration of the CPU of FIG. The figure which shows the storage information structure of the hard disk of FIG. The flowchart which shows the flow of a process of CPU of FIG. The flowchart which shows the flow of the correction process of the three-dimensional model data of FIG. 1st figure explaining the process of FIG. 2nd figure explaining the process of FIG. 3rd figure explaining the process of FIG. 4th figure explaining the process of FIG. The flowchart which shows the flow of the correction process of the three-dimensional point sequence data between edge endpoints as an edge area | region of FIG. 5th figure explaining the process of FIG. 6th figure explaining the process of FIG. The flowchart which shows the flow of a process of the 1st modification of FIG. The flowchart which shows the flow of a process of the 2nd modification of FIG. First diagram for explaining the processing of FIG. FIG. 15 is a second diagram for explaining the processing of FIG. The figure which shows the storage information structure of the hard disk based on Example 2 of this invention. 1st figure explaining Example 2 The 2nd figure explaining Example 2 The 3rd figure explaining Example 2 The flowchart which shows the flow of processing of CPU which concerns on Example 2. FIG. The flowchart which shows the flow of the correction process of the three-dimensional model data of FIG. First diagram for explaining the processing of FIG. The second figure explaining processing of Drawing 22 3rd figure explaining the process of FIG. 1st figure explaining the subject of a prior art example 2nd figure explaining the subject of a prior art example 3rd figure explaining the subject of a prior art example

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Endoscope system 2 ... Medical observation apparatus 3 ... Medical image processing apparatus 4, 9 ... Monitor 6 ... Endoscope 7 ... Light source apparatus 8 ... CCU
DESCRIPTION OF SYMBOLS 11 ... Insertion part 12 ... Operation part 13 ... Light guide 14 ... Tip part 15 ... Objective optical system 16 ... Imaging element 17 ... Imaging part 21 ... Image input part 22 ... CPU
22a ... 3D model estimation unit 22b ... detection target region setting unit 22c ... shape feature amount calculation unit 22d ... model correction unit 22e ... 3D shape detection unit 22f ... polyp determination unit 23 ... processing program storage unit 24 ... image storage unit 25 ... Analysis information storage unit 26 ... Storage device I / F
27 ... Hard disk 28 ... Display processing unit 29 ... Input operation unit 30 ... Data bus

Claims (6)

Three-dimensional model estimation means for estimating a three-dimensional model of the living tissue from a two-dimensional image of the image of the living tissue in the body cavity input from the medical imaging device;
Image boundary detection means for detecting an image boundary of an image area constituting the two-dimensional image,
A line width calculating means for calculating a line width of the image boundary line;
Correction means for correcting the estimation result of the three-dimensional model of the image boundary line in the three-dimensional model estimation means based on the line width;
A medical image processing apparatus comprising: The correction means includes
Compare the line width value with a predetermined value,
If the value of the line width is within the predetermined value, the image boundary line is set as a first correction target image,
When the line width value exceeds the predetermined value, the image boundary line is set as a second correction target image,
The estimation result of the three-dimensional model of the image boundary line in the three-dimensional model estimation means is corrected according to the first correction target image or the second correction target image. The medical image processing apparatus described. The first correction target image is a groove image in which the image boundary line is regarded as a groove,
The medical image processing apparatus according to claim 2, wherein the second correction target image is an occlusion image in which the image boundary line is regarded as occlusion. The correction unit corrects an estimation result of the three-dimensional model of the image boundary line based on an average value of pixel values of the image boundary line in the three-dimensional model estimation unit. The medical image processing apparatus according to any one of 2 and 3. The correction means approximates the pixel value of the image boundary line in the three-dimensional model estimation means, and corrects the estimation result of the three-dimensional model of the image boundary line. 4. The medical image processing apparatus according to any one of 3. A three-dimensional model estimation step for estimating a three-dimensional model of the living tissue from a two-dimensional image of the image of the living tissue in the body cavity input from the medical imaging device;
An image boundary detection step for detecting an image boundary of an image region constituting the two-dimensional image image;
A line width calculating step of calculating a line width of the image boundary line;
A correction step of correcting an estimation result of the three-dimensional model of the image boundary line in the three-dimensional model estimation step based on the line width;
A medical image processing method comprising:

Images