JP2015047309A - Medical observation support device and medical observation support program - Google Patents

Abstract

A medical observation support apparatus that realizes intuitive observation support by utterance is provided.
An observation control unit for changing a viewpoint position and a line-of-sight direction for observation of an organ in three dimensions, and a voice input from a voice input device are recognized, and an anatomical name of the organ corresponding to the voice. And the observation control unit 50 determines the viewpoint position and the line-of-sight direction related to the observation of the organ based on the anatomical name of the organ determined by the voice recognition unit 60. Therefore, by speaking the anatomical name of the site to be observed, it is possible to control the organ to be displayed easily. That is, it is possible to provide a medical observation support apparatus that realizes intuitive observation support by utterance.
[Selection] Figure 2

Description

  The present invention relates to a medical observation support apparatus and a medical observation support program that support observation of an organ in a subject, and more particularly, to an improvement for realizing intuitive observation support by utterance.

  2. Description of the Related Art Medical observation support apparatuses that support observation of organs in a subject are known. For example, surgical navigation described in Non-Patent Document 1 is an example. In such a technique, in order to enhance the safety of endoscopic surgery, three-dimensional image data such as a CT (Computed Tomography) apparatus and a magnetic resonance (MRI) apparatus that have been imaged before surgery are used. A virtual three-dimensional image is generated and displayed following the movement of the endoscope. For example, the follow-up display is performed by using a position sensor attached to an endoscope. When it is desired to observe a predetermined part in detail, the virtual three-dimensional image is observed while being translated or rotated.

  When the surgical navigation is performed during surgery, it is difficult for the surgeon himself to touch an operation device such as a mouse, a keyboard, or a touch panel display due to hygiene problems. For this reason, a non-contact type user interface has been proposed. For example, the technique described in Non-Patent Document 2 is an example. According to this technique, speech recognition is introduced into surgical navigation, and by recognizing voice commands such as left, right, up and down, the surgical navigation can be controlled and a view required by the operator can be obtained.

Eiji Ito, Masazumi Fujii, Yuichiro Hayashi, Zhengang Jiang, Tetsuya Nagatani, Kiyoshi Saito, Yugo Kishida, Kensaku Mori, and Toshihiko Wakabayashi, "Magnetically Guided 3-Dimensional Virtual Neuronavigation for Neuroendoscopic Surgery: Technique and Clinical Experience, ″ NEUROSURGERY, Vol. 66, pp.ons342-353 (2010/06) Masayasu Hirata, Keisuke Michimitsu, Yoshito Mekada, Kensaku Mori, "Intuitive operation using speech recognition in navigation systems," IEICE Technical Report, MI2012-113, Vol.112, No.411, pp.267 -270, Naha City Bunka Tembus Hall (Okinawa), 2013/01/25 (2013/01)

  By the way, in the observation of organs during surgery, there has been a demand for a doctor to directly specify a site to be observed, for example, by inputting an anatomical name using an operating device. However, as described above, it is difficult for the surgeon himself to touch the operating device during surgery because of sterilization problems. Although the conventional technique can determine viewpoint positions such as left, right, up, down, etc. related to observation of an organ by utterance, it is possible to realize surgical navigation such that a doctor directly designates a site to be observed could not. That is, the medical observation support apparatus and the medical observation support program that realize intuitive observation support by utterance have not been developed yet.

  The present invention has been made in the background of the above circumstances, and an object of the present invention is to provide a medical observation support apparatus and a medical observation support program that realize intuitive observation support by utterance.

  In order to achieve such an object, the gist of the first invention is a medical observation support apparatus for supporting observation of an organ in a subject, wherein a viewpoint position and a line-of-sight direction related to the observation of the organ are set to 3. An observation control unit that changes in dimension, and a voice recognition unit that recognizes a voice input from a voice input device and determines an anatomical name of an organ corresponding to the voice, and the observation control unit includes: Based on the anatomical name of the organ determined by the speech recognition unit, the viewpoint position and the line-of-sight direction related to the observation of the organ are determined.

  According to the first aspect of the present invention, the observation control unit that changes the viewpoint position and the line-of-sight direction related to the observation of the organ in three dimensions and the voice input from the voice input device are recognized, and the anatomy of the organ corresponding to the voice is recognized. A speech recognition unit that determines a scientific name, and the observation control unit determines a viewpoint position and a line-of-sight direction for observation of the organ based on an anatomical name of the organ determined by the speech recognition unit. Since it is determined, by speaking the anatomical name of the site to be observed, it is possible to control such that the organ is simply displayed. That is, it is possible to provide a medical observation support apparatus that realizes intuitive observation support by utterance.

  The gist of the second invention, which is dependent on the first invention, is that the observation control unit is obstructed by another organ corresponding to the viewpoint position and the line-of-sight direction related to the observation of the organ. In the case of determination, at least one of the viewpoint position and the line-of-sight direction is adjusted. In this way, observation of the target organ can be made more suitable.

  The gist of the third invention subordinate to the first invention or the second invention is that the observation control unit generates a three-dimensional virtual image of the organ based on the three-dimensional image data of the organ acquired in advance. Based on the anatomical name of the organ determined by the voice recognition unit, the viewpoint position and the line-of-sight direction related to the three-dimensional virtual image corresponding to the organ are determined. In this way, for example, in the virtual endoscopic image display control, the anatomical name of the part desired to be observed is spoken, thereby making it possible to easily display the three-dimensional virtual image of the organ.

  The gist of the fourth invention subordinate to the first invention or the second invention is that the observation control section captures an actual image of the organ, and the organ determined by the voice recognition section. The viewpoint position and the line-of-sight direction related to the imaging of the organ are determined based on the anatomical name. In this way, for example, in surgical navigation in a surgical operation using a so-called surgical robot, it is possible to control the imaging of the organ simply by speaking the anatomical name of the site to be observed.

  The gist of the fifth invention according to any one of the first to fourth inventions is that the observation control unit is updated when the anatomical name of the organ determined by the voice recognition unit is updated. First, the viewpoint position is once separated from the current observation position and then moved to the viewpoint position related to the observation of the organ. In this way, the movement of the viewpoint position can be recognized more easily by the user.

  The gist of the sixth invention according to any one of the first to fifth inventions is that the observation control unit is configured to determine a viewpoint position related to observation of an organ in the subject based on the direction of the subject. And the direction of the line of sight. In this way, observation of the target organ can be made more suitable.

  The gist of the seventh invention dependent on any one of the first to sixth inventions is that the observation control unit is configured to determine a viewpoint position and a line-of-sight direction related to observation of the organ based on the longitudinal direction of the organ. Is to determine. In this way, observation of the target organ can be made more suitable.

  In order to achieve the above object, the gist of the eighth aspect of the present invention is to provide a control device provided in a medical observation support apparatus that supports the observation of an organ, and to change the viewpoint position and the line-of-sight direction related to the observation of the organ. An observation control unit that changes in dimension, and a voice that is input from a voice input device, and that functions as a voice recognition unit that determines an anatomical name of an organ corresponding to the voice; The medical observation support program is characterized in that, based on an anatomical name of an organ determined by a voice recognition unit, a viewpoint position and a line-of-sight direction related to the observation of the organ are determined. In this way, by speaking the anatomical name of the site to be observed, it is possible to control such that the organ is simply displayed. That is, it is possible to provide a medical observation support program that realizes intuitive observation support by utterance.

It is a figure which illustrates the structure of the medical observation assistance apparatus which is one Example of this invention. It is a functional block diagram explaining the principal part of the control function with which the medical observation assistance apparatus of FIG. 1 was equipped. It is a figure which illustrates the structure of the blood vessel database with which the medical observation assistance apparatus of FIG. 1 was equipped. It is a figure which illustrates the characteristic information which concerns on the blood vessel branch b as an example of the information regarding the blood vessel of the subject which is the observation object by the medical observation support apparatus of FIG. It is a figure explaining the viewpoint position and visual line direction of a virtual camera regarding observation by the medical observation assistance apparatus of FIG. It is a figure explaining the viewpoint position and visual line direction which concern on the three-dimensional virtual image of an organ using the virtual camera of FIG. The screen in the case where voice recognition is performed for the utterance “CA” in the observation by the medical observation support apparatus of FIG. 1 is illustrated. The screen in the case where speech recognition is performed for the utterance “CHA” in the observation by the medical observation support apparatus of FIG. 1 is illustrated. FIG. 7 is a diagram schematically illustrating FIG. 6 as a cross-sectional view in order to explain the occupation ratio of the virtual image in the observation by the medical observation support apparatus of FIG. 1. FIG. 7 is a diagram schematically illustrating FIG. 6 as a cross-sectional view in order to explain the occupation ratio of the virtual image in the observation by the medical observation support apparatus of FIG. 1. It is a figure explaining the occupation rate of a virtual image in case another organ exists in the position which obstruct | occludes the organ which is the observation object of the medical observation assistance apparatus of FIG. It is a figure explaining the adjustment of a gaze direction in observation of the medical observation assistance apparatus of FIG. FIG. 2 is a diagram for conceptually explaining a difference in appearance of a virtual image according to a viewpoint position and a line-of-sight direction in the observation by the medical observation support apparatus in FIG. 1. 3 is a flowchart for explaining a main part of an example of observation support control by a CPU of the medical observation support apparatus of FIG. 1. It is a flowchart explaining the principal part of an example of a viewpoint position and a gaze direction calculation process in the control shown in FIG. It is a flowchart explaining the principal part of an example of the gaze direction correction process in the control shown in FIG. It is a flowchart explaining the principal part of an example of the correction process of the viewpoint position and the gaze direction which considered the shielding by the obstruction object in the control shown in FIG. It is a flowchart explaining the principal part of an example of the blood vessel branch b drawing rate calculation process in the control shown in FIG. It is a functional block diagram explaining the principal part of the other example of the apparatus with which the medical observation assistance apparatus of FIG. 1 was equipped, and a control function. 15 is a flowchart for explaining a main part of an example of a viewpoint position / gaze direction calculation process in the control shown in FIG. 14 and corresponds to display control of a virtual endoscopic image.

  DESCRIPTION OF EMBODIMENTS Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a diagram illustrating a configuration of a medical observation support apparatus 10 (hereinafter simply referred to as support apparatus 10) according to an embodiment of the present invention. As shown in FIG. 1, the support device 10 of this embodiment includes a CPU 12 that is a central processing unit, a ROM 14 that is a read-only memory, a RAM 16 that is a write / read memory as needed, a storage device 18, and an image. The display device 20, the input device 22, and the voice input device 24 are provided. The CPU 12 is a so-called microcomputer that processes and controls electronic information based on a predetermined program stored in advance in the ROM 14 while using the temporary storage function of the RAM 16.

  The storage device 18 is preferably a known storage device (storage medium) such as a hard disk drive capable of storing information. It may be a removable storage medium such as a small memory card or an optical disk. The video display device 20 is preferably a known display device (display) such as a TFT (Thin Film Transistor Liquid Crystal), and is connected to the CPU 12 or the like via an output interface 26. The input device 22 is a known input device such as a keyboard or a mouse, and is connected to the CPU 12 or the like via an input interface 28. The voice input device 24 is preferably a known microphone that converts vibrations corresponding to sound waves including voice into electrical signals, and is connected to the CPU 12 and the like via an A / D converter 30. That is, sound waves such as voice input from the voice input device 24 are converted into digital signals by the A / D converter 30 and input to the CPU 12 or the like.

  FIG. 2 is a functional block diagram for explaining the main part of the control function provided in the support apparatus 10. The observation control unit 50, voice recognition unit 60, anatomical name information generation unit 62, and image composition display unit 64 shown in FIG. 2 are preferably all functionally provided in the CPU 12. However, each control unit may be individually provided in a plurality of CPUs and computers and configured to be able to communicate with each other. A part of each control unit may be provided in a control device different from the support device 10. For example, the image composition display unit 64 may be provided in a device different from the support device 10 (for example, a display-dedicated device). Alternatively, the voice recognition unit 60 may be provided in a device different from the support device 10 (for example, a device dedicated to voice recognition).

  The support apparatus 10 according to the present embodiment generates a three-dimensional virtual image of the organ based on three-dimensional image data obtained in advance by an X-ray CT, a magnetic resonance apparatus, or the like regarding the organ of the subject. The support device 10 supports observation (display) of the three-dimensional virtual image corresponding to the organ. Preferably, the three-dimensional virtual image corresponding to an organ having a tree structure such as a blood vessel, a bronchus, and an esophagus is generated, and observation of the three-dimensional virtual image is supported. Or you may observe and support the three-dimensional virtual image corresponding to luminal organs (hollow organ), such as large intestine, small intestine, duodenum, stomach, bile duct, pancreatic duct, lymphatic duct.

  As shown in FIG. 2, the storage device 18 provided in the support device 10 is provided with various databases such as an image database 32 and a blood vessel database 34. These databases are preferably provided in the storage device 18, but may be provided in a storage device separate from the storage device 18. Furthermore, a part or all of the image database 32 and the blood vessel database 34 may be provided in a storage device provided in a control device separate from the support device 10.

  The image database 32 stores three-dimensional image data of a subject obtained in advance by a known X-ray CT (Computed Tomography) apparatus, magnetic resonance (MRI) apparatus, or the like. In the case where the three-dimensional image data is obtained by X-ray CT, X-rays are irradiated to the subject from multiple directions, the X-rays transmitted through the subject are detected by a detector, and transmitted X The dose information is processed by a computer and reconstructed as a three-dimensional image. The image database 32 stores, for example, a three-dimensional grayscale image of a subject obtained by X-ray CT, a magnetic resonance apparatus, or the like, in other words, a pixel value (density value) for each voxel that is a unit volume of the subject. Is done. The image database 32 preferably stores three-dimensional image data of a luminal organ obtained by an X-ray CT, a magnetic resonance apparatus or the like.

  FIG. 3 is a diagram illustrating the configuration of the blood vessel database 34. As shown in FIG. 3, the blood vessel database 34 stores various types of information related to the organ to be observed. That is, the blood vessel database 34 shown in FIG. 3 is an example of an organ database that stores various types of information related to the organ of the subject. In FIG. 3, as an example of the organ, an example in which various types of information related to blood vessels that are luminal organs having a tree structure are stored in the blood vessel database 34 is shown. In the following embodiments, control for supporting observation of blood vessels, which are examples of organs in a subject, by the support apparatus 10 will be described.

As shown in FIG. 3, in the blood vessel database 34, as information on the blood vessels of the subject, a blood vessel name N (for example, “Superior mesenteric artery”), a blood vessel Japanese name J ^N (for example, superior mesenteric artery), Blood vessel name abbreviation C ^N (for example, SMA), average observation direction d ^N (for example, (0, 1, 0)), average observation upward direction u ^N (for example, (0, -1, 0)), observation direction standard Information such as deviation Δs ^N (for example, 2 degrees) and observation upward standard deviation Δu ^N (for example, 1 degree) is stored. Preferably, in addition to the information, information such as an average observation position p ^N (for example, predetermined coordinates) and an average observation upward position q ^N (for example, predetermined coordinates) are stored. In other words, corresponding to each of a plurality of blood vessels identified by a predetermined anatomical name, anatomical name information such as a blood vessel name, a blood vessel sum name, and a blood vessel name abbreviation, an average observation direction, an average observation upward Observation viewpoint position / gaze direction information and the like such as direction, observation direction standard deviation, observation upward standard deviation, average observation position, and average upward observation position are stored. That is, the blood vessel database 34 includes an anatomical name database that stores an anatomical name of each organ for each organ to be observed by the support apparatus 10. For each organ (organ part) that is an observation target of the support apparatus 10, a viewpoint position / line-of-sight direction database that stores an observation viewpoint position and a line-of-sight direction corresponding to each organ is included. The anatomical name database and the viewpoint position / gaze direction database may be provided as separate databases.

The blood vessel database 34 preferably stores characteristic information such as the running direction, length, and cross-sectional area of each blood vessel corresponding to each of a plurality of blood vessels identified by a predetermined anatomical name. FIG. 4 is a diagram illustrating feature information related to the blood vessel branch b as an example of information related to the organ of the subject. The blood vessel database 34 is, for example, a database of branch serial numbers of blood vessel branch structures forming a tree structure, and the start point v ^b ₁ , end point v ^b _n , and center position o (= ( v ^b ₁ + v ^b _n ) / 2), route {v ^b ₁ ,..., v ^b _n }, route length n, length l ^b (= | v ^b ₁ ,..., v ^b _n |), travel direction s _{^{b (= (v b n -v}} b 1) / | v b 1, ..., v b n |), and the radius (cross sectional radius) r ^b such information is stored.

  As shown in FIG. 2, the observation control unit 50 includes a three-dimensional virtual image generation unit 52, a viewpoint position / gaze direction setting unit 54, an organ identification unit 56, and an organ overlap determination unit 58. The three-dimensional virtual image generation unit 52 generates a three-dimensional virtual image of the organ based on the three-dimensional image data acquired in advance and stored in the image database 32. For example, when observing a predetermined organ, 3D image data obtained in advance corresponding to the organ and stored in the image database 32 is read, and a 3D virtual image is generated based on the 3D image data. The three-dimensional image data stored in the image database 32 is obtained by continuously feeding the subject in the body axis direction while continuously rotating the X-ray irradiator / detector using, for example, a CT apparatus. This is three-dimensional image data obtained by performing a spiral continuous scan (helical scan) on a three-dimensional region of a subject. The three-dimensional virtual image generation unit 52 reads out such three-dimensional image data and creates a three-dimensional virtual image of an organ by stacking tomographic images of successive slices in a three-dimensional region. In the present embodiment, display control of a three-dimensional virtual image showing a virtual outer shape of an organ to be observed (for example, an image representing an appearance viewed from the front side) will be described. The present invention is also suitable for display control in which a three-dimensional virtual endoscopic image corresponding to an image obtained by inserting an endoscope into a lumen is displayed based on the three-dimensional image data. Applied.

  The viewpoint position / gaze direction setting unit 54 sets (determines) a viewpoint position and a gaze direction related to the three-dimensional virtual image of the organ generated by the three-dimensional virtual image generation unit 52. FIG. 5 is a diagram illustrating the virtual camera 70 corresponding to the viewpoint position and the line-of-sight direction according to the three-dimensional virtual image of the organ. In the virtual camera 70 shown in FIG. 5, the viewpoint position and the line-of-sight direction are defined. For example, the virtual camera 70 corresponds to the viewpoint position related to the three-dimensional virtual image of the organ. The line-of-sight direction of the virtual camera 70 corresponds to the line-of-sight direction related to the three-dimensional virtual image of the organ. The viewpoint position / gaze direction setting unit 54 conceptually moves the virtual camera 70 in a three-dimensional direction relative to the three-dimensional virtual image of the organ generated by the three-dimensional virtual image generation unit 52. By doing so, the viewpoint position and the line-of-sight direction related to the three-dimensional virtual image of the organ are set.

  The voice recognition unit 60 recognizes voice input from the voice input device 24. Specifically, the voice information input by the voice input device 24 and converted into a digital signal by the A / D converter 30 is analyzed by analyzing the frequency or the like by a known technique, thereby corresponding to the voice information. Recognize phoneme text (spoken character strings). The voice recognition unit 60 determines an anatomical name of the organ corresponding to the recognized phoneme text. The blood vessel database 34 stores an anatomical name of an organ and a phoneme text in association with each organ (blood vessel branch) to be observed by the support apparatus 10. That is, the phoneme text corresponding to the voice when the anatomical name of each blood vessel branch is uttered corresponding to each of a plurality of predetermined blood vessel branches is stored in a database. The speech recognition unit 60 preferably uses the correspondence stored in the blood vessel database 34 based on the phoneme text recognized corresponding to the speech information, and the anatomy of the organ corresponding to the speech information. Determine the name. That is, the blood vessel database 34 includes a phoneme text database that stores phoneme text corresponding to the anatomical name of each organ for each organ that is the observation target of the support apparatus 10. Such a phoneme text database may be provided as a database or table different from the blood vessel database 34.

  The voice recognition unit 60 may further recognize phoneme text corresponding to various commands related to the observation of the organ. For example, a phoneme text corresponding to a command indicating a line-of-sight direction related to observation of the organ, such as “upward” or “downward”, may be recognized. Furthermore, the phonetic text corresponding to the command for moving the viewpoint position related to the observation in the support device 10 such as “right”, “left”, “up”, “down”, etc. may be recognized.

  The organ identifying unit 56 identifies an organ to be observed. Preferably, with respect to a plurality of organs identified by predetermined anatomical names, each part in the three-dimensional virtual image of the organ generated by the three-dimensional virtual image generation unit 52 is which organ (part of the organ). Identify whether it corresponds to. Preferably, features such as the running direction, length, and cross-sectional area of each part in the three-dimensional virtual image of the organ generated by the three-dimensional virtual image generation unit 52 are extracted and stored in the blood vessel database 34. By referring to the information, it is identified which organ the target region corresponds to. For example, regarding the feature information stored in the blood vessel database 34, the organ having the closest feature such as the traveling direction, length, cross-sectional area, etc. is identified as the organ corresponding to the target region.

  The organ identifying unit 56 preferably uses the three-dimensional virtual image of the organ generated by the three-dimensional virtual image generating unit 52 as a volume region of VOI (Volume Of Interest). And organ information to be observed may be identified by extracting region information and structure information of the organ in the VOI. Alternatively, each part in the three-dimensional virtual image of the organ generated by the three-dimensional virtual image generation unit 52 is associated with an organ by human processing, for example, an anatomical name is input and associated with the input device 22. May be identified. In such an aspect, each part in the three-dimensional virtual image of the organ and the identified organ (for example, its anatomical name) are associated with each other and stored in the storage device 18 or the like for the following control. Used.

  Based on the anatomical name of the organ determined by the voice recognition unit 60, the observation control unit 50 determines a viewpoint position and a line-of-sight direction related to the observation of the organ. As described above, the anatomical name of each part in the three-dimensional virtual image of the organ generated by the three-dimensional virtual image generation unit 52 is identified by the organ identification unit 56. In the blood vessel database 34, the average observation direction, the average observation upward direction, the observation direction standard deviation, the observation upward direction standard deviation, the average observation position, and the average observation upward corresponding to each organ (blood vessel branch) to be observed. Information on the viewpoint position such as the position and the line-of-sight direction is determined and stored in advance. When the voice recognition unit 60 determines the anatomical name of a predetermined organ, the viewpoint position / gaze direction setting unit 54 stores the anatomical name of the organ stored in the blood vessel database 34. Information on the viewpoint position and the line-of-sight direction is read out, and the viewpoint position and the line-of-sight direction related to the observation of the organ are determined based on the information.

  FIG. 6 is a diagram for explaining the viewpoint position and the line-of-sight direction related to the three-dimensional virtual image of the organ using the virtual camera 70. When a three-dimensional virtual image 72 of a blood vessel branch b (hereinafter simply referred to as a virtual image 72) is considered as a three-dimensional virtual image of an organ in a subject, the relative viewpoint position and line of sight of the virtual camera 70 with respect to the virtual image 72 When the direction is set, the position, direction, and occupancy rate of the virtual image 72 in the virtual screen 74 displayed on the video display device 20 by the image composition display unit 64 described later (occupied area of the virtual image 72 in the entire screen) Etc. are determined. That is, the viewpoint position / gaze direction setting unit 54 determines a viewpoint position and a gaze direction related to the virtual image 72 corresponding to the organ based on the anatomical name of the organ determined by the voice recognition unit 60. To do.

  The viewpoint position / line-of-sight direction setting unit 54 preferably determines a viewpoint position and a line-of-sight direction related to observation of an organ in the subject based on the direction of the subject. The direction of the subject is, for example, the body axis direction of the subject (the direction from the head to the leg, the vertical direction when the subject is upright). The viewpoint position / line-of-sight direction setting unit 54 preferably observes the organ in the subject so that at least the vertical direction of the subject matches the vertical direction of the virtual image 72 in the virtual screen 74. The viewpoint position and the line-of-sight direction are determined.

The viewpoint position / line-of-sight direction setting unit 54 preferably determines a viewpoint position and a line-of-sight direction related to observation of the organ based on the longitudinal direction of the organ. The longitudinal direction of the organ is an axial direction in a longitudinal organ such as a luminal organ, and corresponds to, for example, the traveling direction s ^b of the blood vessel branch b shown in FIG. In the viewpoint position / line-of-sight direction setting unit 54, preferably, the longitudinal direction of the virtual image 72 in the virtual screen 74 is a direction corresponding to the spread of the screen, such as the horizontal direction or the vertical direction of the virtual screen 74. In this manner, the viewpoint position and the line-of-sight direction related to the observation of the organ in the subject are determined. In other words, the viewpoint position and the line-of-sight direction for observing the organ in the subject are determined so that the longitudinal direction of the virtual image 72 in the virtual screen 74 does not become the depth direction of the virtual screen 74.

  Specifically, the viewpoint position / gaze direction setting unit 54 stores the average observation direction, average observation upward direction, observation direction standard deviation, observation upward direction standard deviation, average observation position, and average stored in the blood vessel database 34. Based on the information such as the upward observation position, the viewpoint position and the line-of-sight direction for observing the organ are determined. For example, the average observation position stored in the blood vessel database 34 corresponding to the organ to be observed is read, and the viewpoint position related to the observation of the organ is set based on the average observation position. The average observation direction and the observation direction standard deviation stored in the blood vessel database 34 corresponding to the organ to be observed are read, and the line-of-sight direction relating to the observation of the organ is determined based on the average observation direction and the observation direction standard deviation. To do. For example, when observing an organ to be observed upward, such as when a phoneme text corresponding to “upward” is recognized by the speech recognition unit 60, the blood vessel database 34 corresponds to the organ to be observed. The average upward observation position stored in is read out, and the viewpoint position related to the observation of the organ is set based on the average upward observation position. The average upward observation direction and the observation upward direction standard deviation stored in the blood vessel database 34 corresponding to the organ to be observed are read, and the observation of the organ is performed based on the average upward observation direction and the observation upward direction standard deviation. Determine the line-of-sight direction. In observing an organ that is difficult to visually recognize, a viewpoint position and a line-of-sight direction are set based on the longitudinal direction of the virtual image corresponding to the organ so that the entire organ can be visually recognized.

  The viewpoint position / line-of-sight direction setting unit 54 preferably calculates the viewpoint position and line-of-sight direction so that the virtual image 72 corresponding to the organ to be observed is positioned at the approximate center of the virtual screen 74. Preferably, the viewpoint position is set so that the occupation ratio (ratio of the area occupied by the virtual image 72) of the virtual image 72 corresponding to the organ to be observed in the entire virtual screen 74 is equal to or greater than a predetermined threshold. And the line-of-sight direction is calculated. The viewpoint position / line-of-sight direction setting unit 54 does not necessarily set the viewpoint position and line-of-sight direction related to observation of the organ based on information on the viewpoint position and line-of-sight direction stored in advance in the blood vessel database 34 or the like. Good. When the organ to be observed is identified by the organ identifying unit 56, the viewpoint position and the line-of-sight direction may be calculated each time based on the characteristics of the organ.

  The viewpoint position / line-of-sight direction setting unit 54 preferably analyzes the anatomy of an organ different from the organ currently being observed, when the anatomical name of the organ determined by the speech recognition unit 60 is updated. When the scientific name is determined by the speech recognition unit 60 and the moving distance of the viewpoint position is equal to or greater than a predetermined threshold (when moving relatively far), the viewpoint position is observed at the present time. After being separated from the position once, it is moved to the viewpoint position related to the observation of the organ. For example, when the anatomical name of the organ determined by the voice recognition unit 60 is updated, the viewpoint position is moved from the current observation position to a predetermined reference viewpoint position, and then the organ is observed. Move to that viewpoint position. The reference viewpoint position corresponds to, for example, a viewpoint position that gives an overview of an entire organ to be observed (for example, the entire blood vessel). Alternatively, the viewpoint position is zoomed out by an amount corresponding to a predetermined distance from the current observation position, and then moved to the viewpoint position related to the observation of the organ. By such control, it is possible to make the observer recognize the movement of the observation position without feeling uncomfortable while suppressing an increase in the time required for the movement of the observation position.

  The anatomical name information generation unit 62 generates anatomical name information of an organ to be observed. For example, text data (character image data) corresponding to the anatomical name of the organ determined by the voice recognition unit 60 is generated. The image synthesis display unit 64 synthesizes the text data generated by the anatomical name information generation unit 62 with the 3D virtual image of the organ generated by the 3D virtual image generation unit 52, The image is displayed on the video display device 20. Preferably, in the three-dimensional virtual image generated by the three-dimensional virtual image generation unit 52, an anatomical name of the organ is assigned to a site corresponding to the organ to be observed identified by the organ identification unit 56. Corresponding text data is synthesized and displayed on the video display device 20.

  7 and 8 are diagrams illustrating examples of screens displayed on the video display device 20 when the support device 10 observes an organ. 7 and 8, a virtual screen including a three-dimensional virtual image is displayed on the left side of the screen, and a cross-sectional image corresponding to the right side of the screen, that is, image data (CT image data) that is the basis of the three-dimensional virtual image is displayed. ) Is shown as an example. FIG. 7 exemplifies a screen when the speech recognition unit 60 recognizes the utterance “CA” and corresponds to the anatomical vessel name abbreviation “CA”. The virtual image of the celiac artery is set at the approximate center of the virtual screen. In other words, the viewpoint position and the line-of-sight direction are set so that the virtual image corresponding to the anatomical name of the organ determined by the voice recognition unit 60 is approximately in the center of the virtual screen. FIG. 8 exemplifies a screen when the speech recognition unit 60 recognizes the utterance “CHA”, and the anatomical blood vessel name is abbreviated as “CHA”. The corresponding virtual image of the common hepatic artery is the approximate center of the virtual screen. However, since the virtual image of the gastroduodenal artery corresponding to the anatomical blood vessel name abbreviation “GDA” is reflected in front of the target virtual image, the virtual image is changed to the anatomical blood vessel name abbreviation “CHA”. The corresponding virtual image of the common hepatic artery is obstructed, making it difficult to observe.

  The image overlap determination unit 58 determines whether or not the organ is obstructed by another organ corresponding to the viewpoint position and the line-of-sight direction related to the observation of the organ set by the viewpoint position / line-of-sight direction setting unit 54. judge. Specifically, using a prescribed evaluation function, the occupation ratio (ratio of the area occupied by the virtual image 72) of the virtual image 72 corresponding to the organ to be observed in the entire virtual screen 74 is calculated. When the occupation ratio is less than a predetermined threshold, it is determined that the organ to be observed is blocked by another organ as described above with reference to FIG.

FIG. 9 and FIG. 10 are schematic views of FIG. 6 as a cross-sectional view for explaining the occupation ratio of the virtual image 72. FIG. 9 is a cross-sectional view (radial direction) perpendicular to the axis of the blood vessel branch b. 10 corresponds to a cross section (longitudinal cross section) including the axis of the blood vessel branch b. 9 and 10, the viewpoint position of the virtual camera 70 is indicated by p ^b , and the axis (center position) of the blood vessel branch b is indicated by c ^b . As shown in FIG. 9, the screen occupancy ratio α in the radial direction of the blood vessel branch b in the entire virtual screen 74 of the virtual image 72 represents the virtual distance from the virtual camera 70 to the axis of the blood vessel branch b. , the radius of the vessel branch b (virtual radius) and r ^b, the view angle of the virtual camera 70 as theta, expressed by the following equation (1). As shown in FIG. 10, the screen occupancy ratio β in the longitudinal direction of the blood vessel branch b in the entire virtual screen 74 of the virtual image 72 is the virtual distance from the virtual camera 70 to the axis of the blood vessel branch b. The length dimension (virtual length) of the blood vessel branch b in the longitudinal direction is l ^b and the viewing angle of the virtual camera 70 is θ, which is expressed by the following equation (2).

α = r ^b / (k × tan (θ / 2)) × 100 (1)
β = l ^b / (k × 2 × tan (θ / 2)) × 100 (2)

FIG. 11 is a diagram for explaining the occupation ratio of the virtual image 72 when another organ is present at a position that blocks the organ to be observed. In the example shown in FIG. 11, when the virtual camera 70 that determines the viewpoint position and the line-of-sight direction for observing the virtual image 72 is used as a reference, the front side (front side) of the virtual image 72 to be observed is the other. There is a three-dimensional virtual image 76 (hereinafter simply referred to as a virtual image 76) corresponding to a disturbing blood vessel branch that is an organ of the present. In such a state, when a plurality of path points (indicated by white circles in FIG. 11) v ^b _{1 to} v ^b _n are set on the path (axial center) of the virtual image 72 to be observed. , Some of them are blocked by the virtual image 76 and become invisible from the virtual camera 70. That is, the image is not drawn in the virtual screen 74 displayed on the video display device 20. Here, the drawing rate of the virtual image 72 is obtained by dividing, for example, the number of route points drawn on the virtual screen 74 among the route points of the virtual image 72 by the total number of route points of the virtual image 72. Value. That is, the drawing ratio of the virtual image 72 = (number of route points to be drawn) / (number of route points of the blood vessel branch).

The viewpoint position / line-of-sight direction setting unit 54 corresponds to the viewpoint position and line-of-sight direction related to the observation of the organ, when it is determined that the organ is blocked by another organ, that is, by the image overlap determination unit 58 If the determination is affirmative, at least one of the viewpoint position and the line-of-sight direction is adjusted (changed). FIG. 12 is a diagram for explaining the adjustment of the gaze direction by the viewpoint position / gaze direction setting unit 54. In the example shown in FIG. 12, the virtual line-of-sight direction d ^N of the observation image 72 in a predetermined angular change from a state which is d ^b, the longitudinal direction (running direction) perpendicular to the s ^b of the virtual image 74 To the state of the line-of-sight direction d' ^b . That is, the viewpoint position / line-of-sight direction setting unit 54, for example, based on the longitudinal direction s ^b of the virtual image 74 to adjust the viewing direction with respect to the longitudinal direction s ^b. Preferably, as shown in FIG. 12, while maintaining the gaze position in the virtual image 72 (a point on the extension line of the visual line direction on the virtual image 72), the viewpoint position and the visual line direction related to the observation of the virtual image 72 Adjust. For example, the gaze direction that maximizes the drawing rate of the virtual image 72 is calculated, and the viewpoint position at which the gaze position in the virtual image 72 is maintained in the gaze direction is calculated.

  FIG. 13 is a diagram conceptually illustrating a difference in appearance of the virtual image 72 according to the viewpoint position and the line-of-sight direction related to the observation of the organ. Virtual screens 74a to 74e shown in FIG. 13 are virtual cameras that change (search) the viewpoint position and the line-of-sight direction related to the observation of the virtual image 72 in the vertical direction and the horizontal direction while maintaining the gaze position in the virtual image 72. These correspond to the viewpoint positions and the line-of-sight directions of 70a to 70e, respectively. As shown in FIG. 13, when the virtual image 76 corresponding to another organ exists on the front side of the virtual image 72 to be observed, the viewpoint position and the line-of-sight direction related to the observation of the virtual image 72 are When the virtual cameras 70a to 70e are changed, the relative positional relationship between the virtual image 72 and the virtual image 76 is changed like the virtual screens 74a to 74e. In the example shown in FIG. 12, in the virtual screens 74a, 74c to 74e corresponding to the virtual cameras 70a and 70c to 70e, a part of the virtual image 72 is obstructed by the virtual image 76 and cannot be seen. On the virtual screen 74b corresponding to the virtual camera 70b, the entire virtual image 72 can be seen without being blocked by the virtual image 76. Therefore, in the example shown in FIG. 12, the virtual image 72 can be suitably observed by adopting the viewpoint position and the line-of-sight direction corresponding to the virtual camera 70b.

  FIG. 14 is a flowchart for explaining a main part of an example of the observation support control by the CPU 12 of the support apparatus 10 and is repeatedly executed at a predetermined cycle.

  First, in step S1 (hereinafter, step is omitted), three-dimensional image data (CT image data or MR image data) stored in the image database 32 and corresponding to an organ to be observed is read. Next, in S2, the three-dimensional virtual image corresponding to the three-dimensional image data read in S1 is divided by the volume region VOI, and blood vessel region information and structure information in the VOI are extracted. . Next, in S3, the anatomical name (blood vessel name) of each blood vessel is recognized based on the blood vessel region information and structure information extracted in S2. Next, in S4, tree structure data of the blood vessel is generated based on the anatomical name of each blood vessel recognized in S3.

  In parallel with the processes of S1 to S4, the processes of S5 to S8 are executed. In S5, the voice information input by the voice input device 24 and converted into a digital signal by the A / D converter 30 is recognized (analyzed) by a known technique. Next, in S6, the phoneme text (spoken character string) corresponding to the speech information recognized in S5 is recognized. Phoneme text corresponding to a command such as “upward” is recognized. Next, in S7, the anatomical name of the blood vessel corresponding to the phoneme text recognized in S6 is determined, and the accompanying command is cut out. Next, in S8, a blood vessel corresponding to the anatomical name determined in S7 is set as a target blood vessel to be observed, and a target blood vessel name command for designating the target blood vessel is output.

  Next, in S9, a blood vessel branch b corresponding to the target blood vessel name command output in S8 is searched for as an observation target organ. Next, in SA, the viewpoint position / gaze direction calculation process shown in FIG. 15 is executed. Next, in SB, the correction process of the viewpoint position and the line-of-sight direction in consideration of the shielding by the obstructing object shown in FIG. 17 is executed. Next, in S10, the viewpoint position and the line-of-sight direction calculated in SA and corrected in SB are set as the viewpoint position and the line-of-sight direction related to the observation of the blood vessel branch b. Next, in S11, the virtual image 72 of the blood vessel branch b is drawn in correspondence with the viewpoint position and the line-of-sight direction calculated in SA and corrected in SB, and indicates the anatomical name of the blood vessel branch b. After the text data is generated and the virtual image 72 and the text data are combined and displayed (drawn) on the video display device 20, this routine is terminated.

  FIG. 15 is a flowchart for explaining a main part of an example of the viewpoint position / gaze direction calculation process in the SA of the control shown in FIG. First, in SA1, a target blood vessel branch name N corresponding to the target blood vessel name command output in S8 is acquired. In parallel with the processing of SA1, information related to the blood vessel branch b such as the tree structure data of the blood vessel generated in S4 is acquired in SA2. Next, in SA3, the virtual image corresponding to the target vessel branch name N acquired in SA1 is used as an image of the organ that is the observation target, based on the information related to the blood vessel branch b acquired in SA2, and the blood vessel A radial direction screen occupancy rate α and a longitudinal direction screen occupancy rate β on the virtual screen 74 of the virtual image 72 corresponding to the branch b are calculated. A maximum gaze point distance k at which the radial direction screen occupancy α is equal to or greater than a predetermined threshold value tα% and the longitudinal direction screen occupancy rate β is equal to or greater than a predetermined threshold value tβ% is calculated.

Next, in SA4, the center position (gaze point position) c ^b stored in the blood vessel database 34 corresponding to the target blood vessel branch N is referred to. Next, in SA5, the average observation direction (viewpoint direction) d ^b = d ^N stored in the blood vessel database 34 corresponding to the target blood vessel branch ^N is referred to. When an upward observation is performed, such as when a phoneme text corresponding to “upward” is recognized in S6, an average observation upward direction (virtual virtual) stored in the blood vessel database 34 corresponding to the target blood vessel branch N is obtained. Screen upward direction) Reference is made to u ^b = u ^N. Next, in SAA, the line-of-sight direction correction process shown in FIG. 16 is executed. Next, in SA6, the viewpoint position p ^b (= c ^b −kd ^b ) is calculated corresponding to the line-of-sight direction corrected in SAA. Next, in SA7, the virtual screen drawing parameters p ^b d ^b u ^{b and the} like are output corresponding to the line-of-sight direction corrected in SAA and the viewpoint position p ^b calculated in SA6, and then in FIG. It is returned to the control shown.

FIG. 20 is a flowchart for explaining a main part of an example of the viewpoint position / gaze direction calculation processing in SA of the control shown in FIG. 14, and corresponds to the display control of the virtual endoscopic image. The control shown in FIG. 20 is executed in place of the control shown in FIG. 15 described above in the control in which the support device 10 supports the observation of the virtual endoscopic image. In the control shown in FIG. 20, steps common to the control shown in FIG. 15 described above are denoted by the same reference numerals and description thereof is omitted. In the control shown in FIG. 20, in SA5 ′ following the process of SA4, the traveling direction s ^b stored in the blood vessel database 34 corresponding to the target blood vessel branch N is referred to, and d ^b = s ^b is set. Etc. If the phoneme text corresponding to "up" in the S6 is recognized, when the upward observation is performed, the virtual screen upward direction u ^b are calculated. For example, u ^b is calculated in any direction perpendicular to d ^b . Next, in SA6 ′, the viewpoint position p ^b (= c ^b ) is calculated corresponding to the line-of-sight direction referred to in SA5 ′. That is, in the display control of the virtual endoscopic image, preferably, the viewpoint position is set as the gazing point position, and the line-of-sight direction is matched with the traveling direction of the luminal organ. Subsequently, the processing from SA7 onward is executed.

FIG. 16 is a flowchart for explaining a main part of an example of the gaze direction correction process in the SAA of the control shown in FIG. First, in SAA1, referenced line-of-sight direction at the SA5 (average viewing direction) d ^b is obtained. In parallel with the processing of SAA1, in SAA2, the traveling direction s ^b of the blood vessel branch b stored in the blood vessel database 34 corresponding to the target blood vessel branch N is read. Then, in SAA3, in a plane spanned by the running direction s ^b read by line-of-sight direction d ^b and SAA2 obtained at SAA1, as d ^b is perpendicular to the s ^b, s ^{A process} of rotating d ^b by an angle (π / 2−cos ⁻¹ (s ^b d ^b )) to obtain a new line-of-sight direction d ^b is performed with ^b × d ^b (outer product of s ^b and d ^b ) as a rotation axis. Is called. Then, in SAA4, new viewing direction d ^b calculated in SAA3, after being outputted as the result gaze direction correcting and allowed to return to the control shown in FIG. 15.

FIG. 17 is a flowchart illustrating a main part of an example of the correction process of the viewpoint position and the line-of-sight direction in consideration of the shielding by the obstructing object in the control SB illustrated in FIG. First, in SB1, information related to the blood vessel branch b such as the tree structure data of the blood vessel generated in S4 is acquired. In parallel with the processing of SB1, the virtual screen drawing parameters p ^b d ^b u ^{b and the} like output at SA7 are acquired at SB2. In parallel with the processing of SB1 and SB2, in SB3, three-dimensional image data (CT image data or MR image data or the like) stored in the image database 32 and corresponding to the organ to be observed is read. Next, at SB4, the parameter p ^'b, d' ^b, the u ^'b, the virtual screen drawing parameter p ^b acquired in SB2, d ^b, u ^b are respectively substituted.

Next, in SBA, the blood vessel branch b drawing rate calculation process (horizontal drawing rate calculation process) shown in FIG. 18 is executed. Next, in SB5, it is determined whether or not the drawing rate ρ of the blood vessel branch b calculated in SBA is the maximum. If the determination in SB5 is positive, in SB6, the parameter p _m ^b, d _m ^b, the u _m ^b, the parameter p ^'b, d' defined in SB4 ^b, u ^'b is respectively substituted After SB7, the processing below SB7 is executed. If the determination at SB5 is negative, it is determined at SB7 whether or not the entire range has been checked with respect to the lateral search range of the blood vessel branch b. The If the determination in SB7 is positive, SB11 although the following process is performed, if the determination of SB7 is negative, then, at SB8, [Delta] [theta] _h is updated. Next, in SB9, the process of those rotation [Delta] [theta] _h a d ^b u ^b as the rotation axis u ^b and d ^'b u' ^b is performed. Next, at SB 10, after the p ^'b is c ^b -kd' is a ^b, SBA following processing is executed again.

In SB11, the parameter p ^b, d ^b, the u ^b, the parameter p _m ^b defined in _{^{SB6, d m b, u m}} b are respectively substituted. Next, in SBA, the blood vessel branch b drawing rate calculation process (vertical drawing rate calculation process) shown in FIG. 18 is executed. Next, in SB12, it is determined whether or not the drawing rate ρ of the blood vessel branch b calculated in SBA is the maximum (maximum). If the determination at SB12 is positive, in SB13, the parameter p _m ^b, d _m ^b, the u _m ^b, the parameter p ^'b, d' defined in SB4 ^b, u ^'b is respectively substituted After that, the processing from SB14 is executed. However, if the determination in SB12 is negative, it is determined in SB14 whether or not the entire range has been checked with respect to the vertical direction search range of the blood vessel branch b. The If the determination at SB14 is positive, SB18 although the following process is performed, if the determination of SB14 is no, SB15, [Delta] [theta] _v is updated. Next, in SB16, the process of the d ^'b u' ^b what was [Delta] [theta] _v rotates d ^b u ^b as a rotation axis d ^b × u ^b is performed. Then, in SB17, after the p ^'b is c ^b -kd' is a ^b, SBA following processing is performed again. In SB18, the parameter p ^b, d ^b, the u ^b, the parameter p _m ^b defined by _{^{SB13, d m b, u m}} b are respectively substituted. Next, in SB19, after the virtual screen drawing parameter correction results p ^b d ^b u ^b corrected in SB11 and SB18 are output, the control returns to the control shown in FIG.

FIG. 18 is a flowchart for explaining a main part of an example of the blood vessel branch b drawing rate calculation process in the SBA of the control shown in FIG. First, in SBA1, information related to the blood vessel branch b such as the tree structure data of the blood vessel generated in S4 is acquired. In parallel with the processing of SBA1, in SBA2, the virtual screen drawing parameters p ^b d ^b u ^{b and the} like output in SA7 are acquired. In parallel with the processing of SBA1 and SBA2, in SBA3, the three-dimensional image data (CT image data or MR image data, etc.) stored in the image database 32 corresponding to the organ to be observed is read. Next, in SBA4, 1 is assigned to the parameter i and 0 is assigned to the parameter q. Next, in SBA5, a ray connecting the viewpoint positions p ^b and v ^b _i is generated. Next, in SBA6, it is determined whether or not an obstructing object (for example, a virtual image of another organ) exists on the ray generated in SBA5. When the determination of SBA6 is affirmed, the processing below SBA8 is executed, but when the determination of SBA6 is negative, 1 is added to the parameter q in SBA7 (set to q + 1). Thereafter, in SBA8, it is determined whether i = n. When this SBA8 is affirmed, the processing below SBA10 is executed. However, when the determination of SBA8 is negative, 1 is added to the parameter i (i + 1 is set) at SBA9, The processing below SBA5 is executed again. In SBA10, the drawing rate ρ (= q / b) is calculated. This b is the total number of route points of the virtual image 72 corresponding to the blood vessel branch b, for example. Next, in SBA11, after the drawing rate ρ calculated in SBA10 is output, the control is returned to the control shown in FIG.

  In the control of FIGS. 14 to 18 described above, S1 to S11, SA, and SB are the processes of the observation control unit 50, and S1 and S11 are the processes of the three-dimensional virtual image generation unit 52. SB is the process of the viewpoint position / gaze direction setting unit 54, S2 to S4 are the process of the organ identification unit 56, SBA is the process of the organ overlap determination unit 58, and S5 to S7 are the process of the voice recognition unit 60. S7 and S11 correspond to the processing of the anatomical name information generation unit 62, and S11 corresponds to the processing of the image composition display unit 64, respectively.

  As described above, according to this embodiment, the observation control unit 50 (S1 to S11, SA, and SB) that changes the viewpoint position and the line-of-sight direction related to the observation of the organ in three dimensions and the voice input device 24 are input. A speech recognition unit 60 (S5 to S7) that recognizes the speech to be performed and determines the anatomical name of the organ corresponding to the speech, and the observation control unit 50 is determined by the speech recognition unit 60. Based on the anatomical name of the target organ, the viewpoint position and line-of-sight direction for observing the organ are determined. Control such as display is possible. That is, it is possible to provide the support device 10 that realizes intuitive observation support by utterance.

  In response to the viewpoint position and the line-of-sight direction related to the observation of the organ, the observation control unit 50 determines at least one of the viewpoint position and the line-of-sight direction when it is determined that the organ is blocked by another organ. Since the adjustment is made, observation of the target organ can be made more suitable.

  The observation control unit 50 generates a virtual image 72 of the organ based on the three-dimensional image data of the organ acquired in advance, and uses the anatomical name of the organ determined by the speech recognition unit 60. Since the viewpoint position and the line-of-sight direction related to the virtual image 72 corresponding to the organ are determined based on this, the anatomical name of the part to be observed is spoken in, for example, virtual endoscopic image display control Thus, it is possible to perform control such that the virtual image 72 of the organ is simply displayed.

  When the anatomical name of the organ determined by the voice recognition unit 60 is updated, the observation control unit 50 temporarily separates the viewpoint position from the current observation position, and then observes the organ. Since it is moved to such a viewpoint position, the movement of the viewpoint position can be recognized more easily by the user.

  The observation control unit 50 determines the viewpoint position and the line-of-sight direction related to the observation of the organ in the subject based on the direction of the subject, and thus makes the observation of the target organ more suitable. be able to.

  Since the observation control unit 50 determines the viewpoint position and the line-of-sight direction related to the observation of the organ based on the longitudinal direction of the organ, the observation of the target organ can be made more suitable. .

  According to the present embodiment, the CPU 12 provided in the support device 10 that supports the observation of the organ, the observation control unit 50 that changes the viewpoint position and the line-of-sight direction related to the observation of the organ in three dimensions, and the voice input device The speech control unit 50 recognizes the voice input from the voice 24 and functions as a voice recognition unit 60 that determines the anatomical name of the organ corresponding to the voice. Based on the anatomical name, it is a medical observation support program characterized by determining the viewpoint position and line-of-sight direction related to the observation of the organ, so by speaking the anatomical name of the part to be observed, It is possible to perform control such that the organ is simply displayed. That is, it is possible to provide a medical observation support program that realizes intuitive observation support by utterance.

  Next, another preferred embodiment of the present invention will be described in detail with reference to the drawings. In the following description, portions common to the embodiments are denoted by the same reference numerals and description thereof is omitted.

  FIG. 19 is a functional block diagram illustrating a main part of an example of the devices and control functions provided in the support device 10. As shown in FIG. 19, the support apparatus 10 of this embodiment includes a medical camera 80 and a medical camera drive unit 82. The medical camera 80 is a digital camera provided with an imaging element such as a CCD (Charge Coupled Device), for example, and is an imaging device that captures an actual image of an organ to be observed by the support device 10. The medical camera drive unit 82 is a device that moves the medical camera 80 in a three-dimensional direction with respect to the organ to be observed. The medical camera drive unit 82 moves the medical camera 80 in a three-dimensional direction with respect to the organ according to a command supplied from the CPU 12. The medical camera 80 and the medical camera drive unit 82 are preferably provided in a so-called master slave type medical robot or the like that performs an operation on a subject according to an operation in an operation unit (not shown). The medical camera 80 and the medical camera drive unit 82 are preferably endoscopes such as known electronic scopes or fiberscopes used for observing luminal organs.

  The observation control unit 50 in the support apparatus 10 of the present embodiment captures an actual image of an organ to be observed via the medical camera 80. For this reason, the observation control unit 50 includes an imaging control unit 66 that controls imaging by the medical camera 80. The imaging control unit 66 moves the medical camera 80 in a three-dimensional direction with respect to the organ to be observed via the medical camera driving unit 82. That is, the viewpoint position and the line-of-sight direction related to the imaging of the organ by the medical camera 80 are controlled. The viewpoint position / gaze direction setting unit 54 provided in the observation control unit 50 according to the present embodiment sets the viewpoint position and the gaze direction related to imaging of the organ by the medical camera 80 according to the control of the imaging control unit 66. To do.

  The organ identifying unit 56 provided in the observation control unit 50 according to the present embodiment identifies an actual organ to be imaged by the medical camera 80. Preferably, for each of a plurality of organs identified by a predetermined anatomical name, each organ in the organ imaged by the medical camera 80 corresponds to which organ. Preferably, three-dimensional image data of an organ acquired in advance corresponding to a subject to be observed is referred to, and the position corresponding to which organ is imaged by the medical camera 80 is identified. . Preferably, features such as the running direction, length, cross-sectional area, etc., of each part of the organ that is a subject to be imaged by the medical camera 80 are extracted from the corresponding three-dimensional image data and stored in the blood vessel database 34. By referring to, it is identified which organ corresponds to the target region. For example, regarding the feature information stored in the blood vessel database 34, the organ having the closest feature such as the traveling direction, length, cross-sectional area, etc. is identified as the organ corresponding to the target region.

  As described above, the blood vessel database 34 includes an average observation direction, an average observation upward direction, an observation direction standard deviation, an observation upward direction standard deviation, and an average observation position corresponding to each organ (blood vessel branch) to be observed. And information regarding the viewpoint position and the line-of-sight direction, such as the average observation upward position, are stored in advance. When the voice recognition unit 60 determines the anatomical name of a predetermined organ, the viewpoint position / gaze direction setting unit 54 stores the anatomical name of the organ stored in the blood vessel database 34. Information on the viewpoint position and the line-of-sight direction is read out, and the viewpoint position and the line-of-sight direction related to the observation of the organ are determined based on the information. That is, the viewpoint position / gaze direction setting unit 54 determines a viewpoint position and a gaze direction related to imaging of the organ by the medical camera 80 based on the anatomical name of the organ determined by the voice recognition unit 60. To do.

  The image composition display unit 64 provided in the observation control unit 50 of the present embodiment is generated by the anatomical name information generation unit 62 with respect to an actual image of the organ imaged by the medical camera 80. The text data is synthesized and displayed on the video display device 20. Preferably, in the actual image of the organ imaged by the medical camera 80, the part corresponding to the organ to be observed identified by the organ identifying unit 56 corresponds to the anatomical name of the organ. The text data is synthesized and displayed on the video display device 20.

  The organ overlap determination unit 58 provided in the observation control unit 50 of the present embodiment sets the viewpoint position and the line-of-sight direction related to imaging of the organ by the medical camera 80 set by the viewpoint position / line-of-sight direction setting unit 54. Correspondingly, it is determined whether or not the actual image of the organ is obstructed by another organ. Specifically, using the method described in detail in the above embodiment, the occupancy ratio (the ratio of the area occupied by the actual image) of the actual image corresponding to the organ to be observed in the entire imaging screen is calculated. To do. When the occupation ratio is less than a predetermined threshold, it is determined that the organ to be observed is obstructed by another organ.

  In addition, when the organ overlap determination unit 58 determines that the organ is obstructed by another organ, control for adjusting at least one of the viewpoint position and the line-of-sight direction, and the organ determined by the speech recognition unit 60 When the anatomical name is updated, the viewpoint position is once separated from the current observation position, and then moved to the viewpoint position related to the observation of the organ, based on the direction of the subject. The control for determining the viewpoint position and the line-of-sight direction related to the observation of the organ in the subject, the control for determining the viewpoint position and the line-of-sight direction related to the observation of the organ based on the longitudinal direction of the organ, and the like in the embodiment. The control described above is similarly applied to this embodiment.

  Thus, according to the present embodiment, the observation control unit 50 captures an actual image of the organ, and based on the anatomical name of the organ determined by the speech recognition unit 60, Since the viewpoint position and the line-of-sight direction for imaging the organ are determined, for example, in surgical navigation using a so-called surgical robot, by speaking the anatomical name of the site to be observed, It is possible to perform control such as simply capturing an image.

  The preferred embodiments of the present invention have been described in detail with reference to the drawings. However, the present invention is not limited to these embodiments, and may be implemented in other modes.

  For example, in Example 1 described above, an example in which the present invention is applied to control for generating a 3D virtual image of an organ based on 3D image data of the organ acquired in advance is described in Example 2 described above. Have been described with respect to examples in which the present invention is applied to control for capturing an actual image of the organ, but these controls may be applied in combination. For example, in a medical observation support apparatus that executes synchronously control for generating a three-dimensional virtual image of an organ based on pre-acquired three-dimensional image data of the organ and control for capturing an actual video of the organ. The present invention may be applied to control of both. Furthermore, display control of a three-dimensional virtual image showing a virtual outer shape of an organ to be observed (for example, an image representing an appearance viewed from the front side), and internal (inner lumen) of the organ to be observed. In a medical observation support apparatus that synchronously executes display control for displaying a three-dimensional virtual endoscopic image corresponding to an image taken by inserting an endoscope based on the three-dimensional image data The present invention may be applied to control of both.

  In the above-described embodiment, the control for determining the viewpoint position and the line-of-sight direction related to the observation of the organ based on the anatomical name of the one organ determined by the voice recognition unit 60 has been described. Based on the anatomical names of two or more organs determined by the recognition unit 60, the viewpoint position and the line-of-sight direction for observing these organs may be determined. For example, in response to the utterance of “branch portion of CA and LGA”, a three-dimensional virtual image at the branch portion of the anatomical names “CA” and “LGA” in the organ to be observed is displayed (or The viewpoint position and the line-of-sight direction may be set so that an actual image of such a part is captured.

  In the embodiment described above, the control for observing the organ to be observed upward when the phoneme text corresponding to “upward” is recognized by the speech recognition unit 60 has been described. You may perform control which instruct | indicates the site | part used as object more in detail. For example, in response to the utterance of “CA from the right”, the control for setting the viewpoint position and the line-of-sight direction for observing the organ corresponding to the anatomical name “CA” from the right side to the left side, “increase CA” , The control may be performed to set the viewpoint position and the line-of-sight direction for magnifying and observing the organ corresponding to the anatomical name “CA”. Alternatively, the history related to the observation by the support device 10 is stored in the RAM 16 or the like, and the viewpoint position and the line-of-sight direction in the history stored in the RAM 16 or the like are set again in response to the utterance of “previous”. Control may be performed.

  In the above-described embodiment, the control for setting the viewpoint position and the line-of-sight direction related to the observation of the organ corresponding to the anatomical name according to the utterance of the abbreviation of the anatomical name such as “CMA” has been described. The same control may be performed according to utterances of Japanese names such as “superior mesenteric artery” or utterances of official names such as “Superior mesenteric artery”.

  In addition, although not illustrated one by one, the present invention is implemented with various modifications within a range not departing from the gist thereof.

  10: medical observation support device, 24: voice input device, 50: observation control unit, 60: voice recognition unit, 72, 76: three-dimensional virtual image

Claims (8)

A medical observation support apparatus for supporting observation of an organ in a subject,
An observation control unit that changes the viewpoint position and the line-of-sight direction for observation of the organ in three dimensions;
A voice recognition unit that recognizes a voice input from a voice input device and determines an anatomical name of an organ corresponding to the voice;
The observation control unit is configured to determine a viewpoint position and a line-of-sight direction related to observation of an organ based on an anatomical name of the organ determined by the voice recognition unit. . The observation control unit adjusts at least one of the viewpoint position and the line-of-sight direction when it is determined that the organ is obstructed by another organ corresponding to the viewpoint position and the line-of-sight direction related to the observation of the organ. The medical observation support apparatus according to claim 1. The observation control unit generates a three-dimensional virtual image of the organ based on the three-dimensional image data of the organ acquired in advance, and is based on the anatomical name of the organ determined by the voice recognition unit. The medical observation support apparatus according to claim 1 or 2, wherein a viewpoint position and a line-of-sight direction related to the three-dimensional virtual image corresponding to the organ are determined. The observation control unit captures an actual image of the organ, and determines a viewpoint position and a line-of-sight direction related to the imaging of the organ based on an anatomical name of the organ determined by the voice recognition unit The medical observation support apparatus according to claim 1 or 2. When the anatomical name of the organ determined by the voice recognition unit is updated, the observation control unit temporarily separates the viewpoint position from the current observation position, and then performs a viewpoint related to the observation of the organ. The medical observation support apparatus according to any one of claims 1 to 4, wherein the medical observation support apparatus is moved to a position. The medical observation support according to any one of claims 1 to 5, wherein the observation control unit determines a viewpoint position and a line-of-sight direction related to observation of an organ in the subject based on the direction of the subject. apparatus. The medical observation support apparatus according to any one of claims 1 to 6, wherein the observation control unit determines a viewpoint position and a line-of-sight direction for observation of the organ based on a longitudinal direction of the organ. A control device provided in a medical observation support device that supports observation of an organ,
An observation control unit for changing the viewpoint position and the line-of-sight direction in the three-dimensional view related to the observation of the organ;
And recognizes the voice input from the voice input device, and functions as a voice recognition unit that determines the anatomical name of the organ corresponding to the voice,
The said observation control part determines the viewpoint position and gaze direction which concern on observation of the said organ based on the anatomical name of the organ determined by the said voice recognition part, The medical observation assistance program characterized by the above-mentioned .