CN102302357A - Saccade endoscope and attitude sensing system used for same - Google Patents

Abstract

The invention relates to the technical field of attitude sensing of medical implanted micromodule equipment, and discloses a saccule endoscope and an attitude sensing system used for the same. The attitude sensing system comprises an in-vivo device used for collecting and processing attitude information of the saccule endoscope inside a human body and an in-vitro device used for generating a magnetic field outside the human body, wherein the in-vivo device performs communication with the in-vitro device and comprises an attitude sensor unit, an attitude sensor control unit and an attitude computation unit; the attitude sensor unit comprises one or a plurality varieties of attitude sensors and is used for collecting attitude information of the saccule endoscope inside the human body; the attitude sensor control unit is used for configuring the working modes of the attitude sensors; and the attitude computation unit is used for computing an attitude angle of the saccule endoscope by adopting a corresponding method after filtering and other processing of the attitude information from the attitude sensor unit. According to the saccule endoscope and the attitude sensing system provided by the invention, the attitude of the saccule endoscope in an alimentary tract can be accurately detected under the premises of small size and low power consumption.

Description

Attitude sensing system for balloon endoscope and balloon endoscope

technical field

The invention relates to the technical field of posture perception of medical implantable micro-equipment, in particular to a posture perception system for a balloon endoscope and the balloon endoscope.

Background technique

Capsule-type endoscope is a breakthrough in endoscopic technology. From the overall structure, the traditional cable insertion type is replaced by a pill type, which can enter the digestive tract in a swallowed way, realizing a truly non-invasive diagnosis and treatment. At the same time, it can Observing the images of the patient's digestive tract greatly expands the scope and field of view of the entire digestive tract examination. During the digestive tract examination, the wireless capsule endoscope is swallowed from the mouth of the subject, traverses the entire digestive tract through the peristalsis of the gastrointestinal tract, and collects images of the gastrointestinal tract from the inside.

At present, most capsule endoscopes on the market use 1-2 cameras, so the observation field of view is small during the digestive tract examination, which may easily cause missed detection of the stomach and intestinal tract. According to statistics, the shooting blind area of the digestive tract of the capsule endoscope with only one camera is about 20%. For example, in patents US2002/0109774, JP2001112710, US2003/0023150, etc., no more than two cameras are used. Although different methods (comprising methods such as mechanical rotation) for enlarging its field of view have been adopted, these inventions all cannot realize the image collection of full viewing angle.

In the application numbers 200910008885 and 200910080350 Chinese invention patent applications, a balloon-shaped device (referred to as a balloon endoscope) is proposed, which uses a multi-camera image acquisition solution inside. Taking a balloon endoscope with 6 cameras as an example, in order to achieve image acquisition without blind spots, the balloon endoscope system must ensure a shooting speed of at least 6 frames per second (six cameras simultaneously perform image acquisition). To achieve such a shooting speed, the working current of the system is at least about 100mA. Under such a working current requirement, the battery commonly used in existing wireless endoscopes can only be used for power supply for a short time, so it is impossible to realize the Examination of the entire gastrointestinal tract, especially the small intestine. If wireless energy supply is used for power supply, one is that there is no mature wireless energy supply system that can be practically applied to wireless endoscopy, and the other is to calculate the wireless energy supply efficiency reported in the current literature. The energy supply system needs to emit at least tens of watts to hundreds of watts of wireless energy. Whether this electromagnetic radiation is harmful to the human body and its degree of harm are still inconclusive. At the same time, the external energy supply system needs to be equipped with refrigeration equipment, which will limit the patient's life. event space.

If the device in the body is powered by a battery, in order to enable the system in the body to work with low power consumption and prolong the working time of the battery, the system in the body needs to shoot at a speed lower than 6 frames per second. At the same time, it is better for the 6 cameras to take turns to collect images to avoid the situation of multiple cameras working at the same time - multiple cameras working at the same time will greatly increase the peak operating current of the system, thereby reducing the service life of the battery. On the premise that the balloon is relatively stationary relative to the digestive tract, the strategy of image acquisition with six cameras in turn can achieve image acquisition without blind spots. However, the movement of the balloon in the digestive tract is caused by the combined action of gravity and peristalsis of the digestive tract. During the process of image acquisition by the six cameras in turn, it is very likely that the balloon has moved or rotated. If you continue to use the previous strategy at this time, it will inevitably cause a blind spot in image acquisition. Therefore, in order to reduce the power consumption of the balloon, the policy of taking turns to collect images will still have the problem of shooting blind spots in the case of reducing the shooting frame rate to save power consumption.

In the Chinese invention patent application with the application number 201010189438, a multi-view image acquisition and storage system and method used in a biological cavity is proposed. If the posture of the balloon endoscope relative to the digestive tract is known, image acquisition without blind spots can be achieved under the premise of ensuring that the balloon works with low power consumption. The so-called posture perception system refers to a system that detects the posture and movement of the carrier by using a motion sensor or a posture sensor or a combination of the two. However, the above-mentioned patent does not relate to the specific implementation method of the gesture perception system.

To sum up, there is currently no effective and practical technical means to detect the attitude of the wireless balloon endoscope.

Contents of the invention

(1) Technical problems to be solved

The technical problem to be solved by the present invention is: how to accurately detect the attitude of the balloon endoscope in the digestive tract under the premise of small size and low power consumption.

(2) Technical solution

In order to solve the above technical problems, the present invention provides a posture sensing system for balloon endoscope, the system includes an in-vivo device for collecting and processing the posture information of the balloon endoscope in the body and a user Extracorporeal devices for generating magnetic fields outside the body;

Wherein, the in-vivo device communicates with the in-vitro device, and includes:

The attitude sensor unit includes one or more attitude sensors, which are used to collect the attitude information of the balloon endoscope in the body;

an attitude sensor control unit, configured to configure the working mode of the attitude sensor;

The attitude calculation unit is used to calculate the attitude angle of the balloon endoscope by using a corresponding method after performing filtering and other processing on the attitude information from the attitude sensor unit.

Preferably, the attitude sensor unit includes one or a combination of acceleration sensors, magnetic field sensors and angular velocity sensors.

Preferably, said extracorporeal device comprises magnetic field generating means, said magnetic field generating means being an electromagnetic coil or a permanent magnet.

Preferably, the electromagnetic coils are three pairs of mutually orthogonal electromagnetic coils, which sequentially generate uniform magnetic fields in the directions of the X-axis, Y-axis and Z-axis of the absolute coordinate system.

Preferably, the attitude angle includes a pitch angle, a roll angle and a heading angle, and when the attitude sensor unit includes an acceleration sensor and a magnetic field sensor, the attitude of the balloon endoscope is calculated using a method based on the principle of gravitational acceleration and the geomagnetic field angle, specifically:

S1, using the acceleration sensor to measure the acceleration of gravity of the balloon endoscope, and using the magnetic field sensor to measure the magnetic field strength of the balloon endoscope;

S2, judge whether the balloon endoscope is moving according to the collected posture information, if so, return to step S1, otherwise execute step S3;

S3, calculating the pitch angle, roll angle and heading angle of the balloon endoscope according to the gravitational acceleration and magnetic field strength;

S4. Calculate the direction vector of the camera in the balloon endoscope according to the pitch angle, roll angle and heading angle.

Preferably, when the attitude sensor unit includes an acceleration sensor, a magnetic field sensor, and a gyroscope, the attitude angle of the balloon endoscope is calculated using a method based on the principle of gravitational acceleration, angular velocity, and geomagnetic field, specifically:

S1', using steps S1-S3 to calculate the pitch angle, roll angle and heading angle of the balloon endoscope;

S2', using the calculation result of step S1' as the initial attitude angle of the balloon endoscope, using a gyroscope to measure the angular velocity of the balloon endoscope, integrating the angular velocity, and calculating the subsequent attitude of the balloon endoscope horn;

S3', periodically adopt steps S1-S3 to calculate the attitude angle, and replace the calculation result with the attitude angle calculated in step S2'.

Preferably, when the attitude sensor unit includes a magnetic field sensor and a gyroscope, a method based on angular velocity and magnetic field principles is used to calculate the attitude angle of the balloon endoscope, specifically:

S1", using a method based on the magnetic field principle to measure the attitude angle: after the in vitro device generates a uniform magnetic field in the direction of the X-axis, Y-axis and Z-axis of the absolute coordinate system at time A, time B and time C in sequence, the magnetic field sensor Detect the magnetic field strength of the balloon endoscope in the X' axis, Y' axis and Z' axis direction of the carrier coordinate system at time A, B and C respectively, and use the magnetic field strength at time A, B and C to calculate The initial attitude angle of the balloon endoscope;

S2", using the gyroscope to measure the angular velocity of the balloon endoscope, integrating the angular velocity, and estimating the attitude angle of the balloon endoscope at each moment thereafter;

S3", using the method based on the magnetic field principle to calculate the attitude angle, and periodically replacing the attitude angle calculated by step S2" with the calculation result.

The present invention also provides a balloon endoscope, which includes a geometric body, the system, and a camera, the system is located inside the geometric body, and the cameras are distributed on the outer surface of the geometric body.

(3) Beneficial effects

The attitude sensing system proposed by the present invention is designed based on the principles of gravitational acceleration, angular velocity and magnetic field, and a series of low power consumption technologies and MEMS technologies are easy to be adopted in the design based on the principles. Among them, the posture sensor unit in the body adopts a number of small size and low power consumption posture sensors to collect the posture information of the balloon; the posture calculation unit also adopts a low power consumption method to realize. Therefore, the posture sensing system has the characteristics of small size and low power consumption, which is beneficial to be integrated into the balloon endoscopy system.

When the attitude sensing system proposed by the present invention is applied to the balloon endoscope system, a specific camera can be selected to work according to the detected attitude angle, so as to realize full-view image acquisition of the digestive tract; at the same time, repeated image acquisition of the same part can be avoided , which greatly reduces the redundancy of images, thereby reducing the power consumption of the balloon endoscope.

Three kinds of attitude methods in the system of the present invention are respectively: (1) the attitude perception method based on the gravitational acceleration and the geomagnetic field principle, the method has a small amount of calculation, is easy to realize by hardware and detects the attitude angle in real time; (2) is based on the gravitational acceleration , an attitude sensing method based on the principle of angular velocity and magnetic field, which has high precision and is easy to implement in hardware, and can be applied in environments with severe movements; (3) and an attitude sensing method based on the principle of angular velocity and magnetic field, which has high precision and is easy to implement in hardware , requiring extracorporeal devices.

Description of drawings

Fig. 1 is a three-dimensional schematic diagram of the internal geometry of a wireless balloon endoscope according to an embodiment of the present invention;

Fig. 2 is a schematic structural diagram of a posture sensing system for a wireless balloon endoscope according to an embodiment of the present invention;

Fig. 3 is a flow chart of detecting the attitude angle of the balloon according to the embodiment of the present invention;

Fig. 4 is a schematic diagram of the posture perception system used in human digestive tract inspection in Embodiment 1 of the present invention;

Fig. 5 is a schematic structural diagram of a posture sensing system for a wireless balloon endoscope in Embodiment 1 of the present invention;

FIG. 6 is a flow chart of attitude method A adopted in Embodiment 1 of the present invention;

Fig. 7 is a schematic structural diagram of a posture sensing system for a wireless balloon endoscope in Embodiment 2 of the present invention;

Fig. 8 is a schematic structural diagram of attitude method B adopted in Embodiment 2 of the present invention;

Fig. 9 is a schematic diagram of the posture perception system in the third embodiment of the present invention being used for human digestive tract inspection;

Fig. 10 is a schematic structural diagram of a posture sensing system for a wireless balloon endoscope in Embodiment 3 of the present invention;

FIG. 11 is a schematic structural diagram of attitude method C adopted in Embodiment 3 of the present invention.

Detailed ways

The specific implementation manners of the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

The present invention is suitable for multi-camera balloon endoscopes (also referred to as balloons for short). The inner core part of the balloon endoscope (that is, the posture perception system of the present invention) is wrapped by a flexible PCB board to form a separate geometric body, and a plurality of cameras are evenly placed on the outer surface of the geometric body, and full-view image acquisition is performed through these cameras . Wherein, the geometry may be a regular tetrahedron, a cube, a sphere, etc. as required. In the present invention, the case where the geometric body is a regular cube is described, but the present invention is also applicable to other geometric bodies. FIG. 1 is a schematic perspective view of the internal geometric body 107 of the wireless balloon endoscope, and the cameras 101-106 are respectively located at the exact center of each surface of the cube.

The attitude sensing system for balloon endoscope proposed by the present invention is mainly composed of two parts: an in-vivo device for collecting and processing in-vivo attitude information and an in-vitro device for configuring an in-vitro magnetic field environment. As shown in Figure 2, the in-body device includes: an attitude sensor unit, including one or more attitude sensors, to realize the collection of attitude information of the balloon endoscope; an attitude sensor control unit, used to configure the working mode of the attitude sensor; The posture calculation unit is used to process the balloon posture information from the posture sensor unit; the wireless transceiver unit realizes the communication with the external device; and the posture system control unit controls the working mode of the entire internal device. The external device includes: a magnetic field control unit, which controls the external magnetic field generating device to generate a uniform magnetic field; a wireless transceiver unit, which realizes communication with the internal device; and an external attitude system control unit, which is used to control and schedule the configuration of the entire external system.

The present invention is applied to a balloon endoscope system, and the posture perception system can be used to judge the direction that the six cameras of the balloon are facing. The balloon collects images at a certain position in the digestive tract, and the method of sequentially collecting images from all directions is still adopted by the camera. If the posture of the balloon changes due to the peristalsis of the digestive tract or other reasons during the acquisition process, the strategy of acquiring images in turn should be corrected. According to the posture perception system, the direction each camera is facing can be detected, and one or some cameras are selected to collect images of the areas that are not collected, so that the non-blind area image collection of the position can be completed. In addition, after the image acquisition of the position by the balloon is completed, the image acquisition of the balloon can be stopped until the position of the balloon in the digestive tract changes. In this way, due to the introduction of the posture perception system, repeated image acquisition of some parts of the digestive tract is avoided, thereby greatly saving energy consumption. This can make the battery life nearly doubled, and under the premise of no missed detection, a full digestive tract inspection will become possible.

和航向角α；而摄像头的方向矢量定义为6个摄像头在绝对坐标系中指向的方向(采用3维矢量描述)。一般球囊所在参考系称为载体坐标系(由坐标轴X’，Y’和Z’确定，参见图4)，地球所在参考系称为绝对坐标系(地理北极为X轴，东为Y轴，重力方向为Z轴，参见图4)。姿态角和摄像头的方向矢量是两种等效的描述方式，它们之间存在确定的转换关系式。姿态感知系统进行一次姿态角检测主要包含以下几个步骤：Through the attitude sensing system, combined with the corresponding attitude methods (to be introduced below, including attitude methods A, B, and C), the attitude angle of the balloon endoscope can finally be obtained. Generally, the attitude angle or the direction vector of the camera is used to describe the attitude of the balloon. Attitude angle is defined as roll angle θ, pitch angle

and the heading angle α; and the direction vector of the camera is defined as the direction that the six cameras point to in the absolute coordinate system (described by a 3-dimensional vector). Generally, the reference system where the balloon is located is called the carrier coordinate system (determined by the coordinate axes X', Y' and Z', see Figure 4), and the reference system where the earth is located is called the absolute coordinate system (the geographic north pole is the X axis, and the east is the Y axis , the direction of gravity is the Z axis, see Figure 4). The attitude angle and the direction vector of the camera are two equivalent description methods, and there is a definite conversion relationship between them. An attitude angle detection by the attitude perception system mainly includes the following steps:

A1, the attitude system control unit of the internal device initiates attitude angle detection, that is, sends an attitude angle detection command to the external device through the wireless transceiver unit;

A2, the external device is configured with an external magnetic field environment, and after the environment configuration is completed, an instruction is sent to notify the attitude system control unit;

A3, the attitude system control unit initializes and configures the attitude sensor through the attitude sensor control unit;

A4, one or more sensors in the attitude sensor unit complete the collection of balloon attitude information in a certain way;

A5, the attitude calculation unit receives the balloon attitude information from the attitude sensor unit, and performs Kalman filtering on the attitude signal;

A6. The attitude calculation unit uses the filtered attitude signal to calculate the attitude angle of the balloon endoscope using an attitude method.

See flow chart 3 for the detailed process.

Among them, Kalman filtering is performed on the attitude signal in A5. There is a random error in the attitude signal measured by the attitude sensor, and the Kalman filter algorithm can reduce the error.

Wherein, the attitude sensor included in the attitude sensor unit can be one of a MEMS (micro-electromechanical system) accelerometer (accelerometer is also called an acceleration sensor), a MEMS gyroscope, and a MEMS magnetic field strength meter (a magnetic field strength meter is also called a magnetic field sensor). one or a combination of several. However, different sensor combinations correspond to different in vitro devices. At the same time, the pose methods adopted for different combinations are also different. The composition and working principle of the posture perception system will be described in detail below with three embodiments.

Embodiment one

As shown in Fig. 5, preferably, in this embodiment, the in-body device of the attitude sensing system includes: an acceleration sensor, which can measure the gravity components in the three axes of the gravity acceleration carrier coordinate system by using a 3-axis acceleration sensor; a magnetic field sensor, Using a 3-axis magnetic field sensor can measure the magnetic field strength components on the 3 axes of the geomagnetic field carrier coordinate system; the attitude sensor control unit is used to configure the attitude sensor working mode; the attitude calculation unit is used to process the information from the attitude sensor unit Balloon attitude information; in this embodiment, the geomagnetic field is used as a reference physical quantity for attitude perception, so no additional external devices are required to configure the magnetic field environment.

When detecting the attitude angle of the balloon endoscope, the following method should be followed. First, the attitude system control unit initiates attitude angle measurement; the attitude sensor control unit configures the measurement frequency, measurement range and working mode of the acceleration sensor and the magnetic field sensor; after the attitude sensor control unit completes the configuration of the attitude sensor, the acceleration sensor and the magnetic field sensor The gravitational acceleration and magnetic field strength of each axis can be collected periodically, and A/D sampling, storage and other operations are performed on the collected analog values; the acceleration sensor and the magnetic field sensor transmit the acceleration value and magnetic field value to the attitude calculation unit, and the attitude The calculation unit performs Kalman filtering on the attitude signal to reduce the measurement error caused by the sensor; then, the attitude calculation unit can obtain the attitude vector of the balloon endoscope or the direction vector of the camera by using the attitude method. To further illustrate this embodiment, each component in the gesture sensing system will be described in detail below.

Accelerometers measure the acceleration of moving objects. There are three types of MEMS accelerometers with mature technology: piezoelectric, capacitive, and thermal. The most widely used is the capacitive MEMS accelerometer. Look, it's a standard plate capacitor. The change of acceleration drives the movement of the movable mass, thereby changing the distance between the two poles of the plate capacitor and the facing area, and the acceleration is calculated by measuring the change in capacitance. When the chip accelerates to the right, the movable mass block in the middle moves to the left relative to the other two capacitor plates, and the capacitance of the two parallel plate capacitors changes, thereby measuring the acceleration of the chip movement. When the 3-axis accelerometer is stationary, it measures the acceleration components of the acceleration of gravity in the 3 axes. When the 3-axis accelerometer is in motion, it measures the vector sum of the acceleration of gravity and the acceleration of motion.

Magnetic field strength sensor, which can measure the magnetic field strength in all directions. According to the principle of electric field and magnetic field, when a current is applied in the length direction of the ferromagnetic alloy thin strip, if a magnetic field is applied in the direction perpendicular to the current, there will be a heterogeneous phenomenon of reluctance in the ferromagnetic material, which will cause The resistance value of the alloy strip itself changes. The magnetic field strength sensor based on this principle can measure the magnetic field component strength of the earth's magnetic field in all directions. This type of magnetic field strength sensor is called a magnetoresistive sensor.

In this embodiment, the attitude sensor unit is composed of a MEMS accelerometer and a MEMS magnetic field sensor. Both accelerometers and magnetic field sensors feature small size, low power consumption, and high accuracy, making them ideal for low-power implantable medical microdevices like wireless balloon endoscopes. Referring to Fig. 4, a 3-axis MEMS accelerometer 403 and a 3-axis MEMS magnetic field sensor 404 are installed on a certain surface of the wireless balloon endoscope internal geometry 107 (also can be inside the geometry), to ensure that the accelerometer and the The three measurement axes of the magnetic field sensor coincide with the three axes of the carrier coordinate system respectively. The human body 401 swallows the balloon endoscope from the oral cavity, and the balloon endoscope generally does not do strenuous exercise during image acquisition in the digestive tract, so the acceleration value measured by the 3-axis accelerometer is the acceleration of gravity in the carrier coordinates is the gravity component of the three axes (g _x′ , g _y′ , g _z′ ); the magnetic field strength measured by the 3-axis magnetic field sensor is the magnetic field strength component of the earth’s magnetic field in the three axes of the carrier coordinate system (B _x′ , B _y′ , B _z′ ).

The attitude sensor control unit needs initial configuration of the accelerometer and magnetic field sensor after the wireless balloon endoscope is started. The initial configuration mainly includes: setting the measurement range and measurement accuracy of the sensor; setting the acquisition frequency of the sensor for attitude information; and other interrupt processing. After completing the initialization of the sensor, the accelerometer and the magnetic field sensor will periodically measure the acceleration and magnetic field strength of the three axes. During the image acquisition process of the balloon endoscope in the digestive tract, the working mode of the attitude sensor can be reconfigured through the attitude sensor control unit as required.

The attitude system control unit is responsible for the overall scheduling and control of the internal devices of the balloon endoscope system. When the balloon endoscope is not activated, the entire internal device is in a dormant state, and the attitude sensor is also in a dormant state. After the balloon endoscope is started, the attitude system control unit is responsible for completing the initialization of the attitude sensor and other related internal devices. In the process of image information collection, it is responsible for scheduling the attitude sensor's attitude information acquisition, attitude angle calculation and other related work.

The attitude calculation unit calculates the attitude angle of the balloon endoscope through the gravitational acceleration component measured by the accelerometer and the magnetic field sensor and the strength component of the earth's magnetic field. This is the core component of the entire posture perception system. Currently, there are some methods in academia to calculate the attitude angle. However, most of the methods have defects such as high computational complexity and high requirements for hardware data processing capabilities, so they cannot be applied to small embedded devices.

The gesture method in this embodiment (referred to as gesture method A).

Attitude method A can calculate the attitude angle of the balloon endoscope and the direction vector of the camera quickly, accurately and with low hardware cost. Below, step by step how to calculate the attitude angle of the balloon.

(g为当地的重力加速度值，一般取9.8m/s²)，说明球囊运动平缓，可以采用S2计算姿态角；否则，说明球囊运动剧烈，回到S1。S1: Measure the acceleration and magnetic field strength to judge whether the balloon endoscope is doing strenuous exercise. g _x′ , g _y′ , and g _z′ are measured by the acceleration sensor. if

(g is the local gravitational acceleration value, generally 9.8m/s ² ), indicating that the balloon motion is gentle, and S2 can be used to calculate the attitude angle; otherwise, it indicates that the balloon motion is violent, and return to S1.

如果

那么

其中 $\sin {\mathrm{\θ}}^{,}=-\frac{g_{y^{,}}}{g},$ $\sin \mathrm{\φ}=-\frac{g_{z^{,}}}{g};$ 否则，

S2: Calculate pitch angle

if

So

in $\sin {\mathrm{\θ}}^{,}=-\frac{g_{{\mathrm{the\; y}}^{,}}}{g},$ $\sin \mathrm{\φ}=-\frac{g_{z^{,}}}{g};$ otherwise,

其中：如果

$\sin \mathrm{\φ}=-\frac{g_{z^{,}}}{g};$ 否则， $\sin {\mathrm{\θ}}^{,}=-\frac{g_{y^{,}}}{g}.$ S3: Calculate the roll angle θ.

where: if

$\sin \mathrm{\φ}=-\frac{g_{z^{,}}}{g};$ otherwise, $\sin {\mathrm{\θ}}^{,}=-\frac{g_{{\mathrm{the\; y}}^{,}}}{g}.$

S4: Calculate the heading angle α.

Through the above steps, the attitude angle of the balloon in the digestive tract relative to the absolute coordinate system can be calculated more accurately. There is a certain mathematical relationship between the direction vectors of the six cameras in the absolute coordinate system and the attitude angle of the balloon endoscope, and the mathematical relationship can be determined by the following method.

θ＝0时，参见图1：Set at attitude angle α=0,

When θ=0, see Figure 1:

The direction vector of camera 101 is

The direction vector of camera 102 is

The direction vector of camera 103 is

The direction vector of camera 104 is

The direction vector of camera 105 is

The direction vector of camera 106 is

θ时，相应的摄像头的方向矢量可以通过以下式子确定：When the attitude angle of the balloon endoscope becomes α,

When θ, the direction vector of the corresponding camera can be determined by the following formula:

The direction vector of camera 101 is

The direction vector of camera 102 is

The direction vector of camera 103 is

The direction vector of camera 104 is

The direction vector of camera 105 is

The direction vector of camera 106 is

in,

The specific process of the above method is shown in flowchart 6.

So far, the direction vector of each camera in the balloon endoscope can be determined through a 3-axis accelerometer and a 3-axis magnetic field sensor. The direction vector can provide the most important reference basis for the balloon full-view image acquisition strategy. In this embodiment, the internal geometry of the balloon endoscope is a cube, so the attitude angle and the direction vector of the camera can be equivalently converted by the above mathematical relationship. When the internal geometry of the balloon endoscope is in other shapes, the corresponding conversion relation can still be found through the above-mentioned ideas.

Embodiment two

As shown in Figure 7, preferably, in this embodiment, the in-body device of the attitude sensing system includes: an acceleration sensor, which can measure the gravity component of the acceleration of gravity in the 3-axis of the carrier coordinate system by using a 3-axis acceleration sensor; a magnetic field sensor, Using a 3-axis magnetic field sensor can measure the magnetic field intensity components of the geomagnetic field in the 3 axes of the carrier coordinate system; the MEMS gyroscope can measure the 3 axes of the balloon endoscope in the carrier coordinate system angular acceleration; the attitude sensor control unit is used to configure the attitude sensor working mode; the attitude calculation unit is used to process the sensor information from the attitude sensor unit. In this embodiment, the earth's magnetic field is used as a reference physical quantity for attitude perception, so no additional external devices are needed to configure the magnetic field environment.

Compared with the first embodiment, this embodiment adds an attitude sensor—MEMS gyroscope. Two conditions must be met when performing attitude angle detection in Embodiment 1: (1) the human body cannot perform violent movements, and the movement of the balloon endoscope in the digestive tract is relatively stable to ensure that the acceleration value measured by the accelerometer is within the range of gravity Gravity components in three directions in the carrier coordinate system; (2) The electromagnetic environment outside the human body is relatively stable, and there cannot be other magnetic interference sources affecting the geomagnetic field. However, these two conditions may not be met in many cases. For example, when the balloon passes through the esophagus of the human body, the acceleration is relatively high; it is difficult for the human body not to move around during the examination process; there may also be electromagnetic sources (speakers) and magnetically conductive media that change the geomagnetic field outside the human body. In this embodiment, after the MEMS gyroscope is added, the posture of the balloon endoscope can still be detected even if the above two conditions are not met during the inspection process.

When detecting the attitude angle (or camera direction vector) of the balloon endoscope, the following method should be followed. First, the attitude system control unit initiates attitude angle measurement; the attitude sensor control unit configures the measurement frequency, measurement range, and working mode of the acceleration sensor, magnetic field sensor, and gyroscope; after the attitude sensor control unit completes the configuration of the attitude sensor, the acceleration sensor The magnetic field sensor and gyroscope can periodically collect the gravitational acceleration, magnetic field strength and angular velocity of each axis, and perform A/D sampling and storage of the collected analog values; the acceleration sensor, magnetic field sensor and gyroscope will accelerate value, magnetic field value and angular velocity value are transmitted to the attitude calculation unit, and the attitude calculation unit can obtain the attitude vector of the balloon endoscope or the direction vector of the camera through the corresponding attitude angle method. To further illustrate this embodiment, each component in the gesture sensing system will be described in detail below.

The acceleration sensor is the same as the first embodiment.

The magnetic field sensor is the same as the first embodiment.

The MEMS gyroscope can measure the angular velocity of the balloon endoscope in three axes in the carrier coordinate system. The MEMS gyroscope is based on the Coriolis force. The mass block is packaged into the chip, and the driving speed is applied to the mass block in the radial direction. According to the Coriolis force, the mass block will generate acceleration in the tangential direction, which is proportional to the rotation speed of the carrier. . The carrier rotation speed can thus be calculated by measuring this acceleration. With its small size and low power consumption, MEMS gyroscopes can be applied to attitude perception systems.

The attitude sensor control unit needs initial configuration of the accelerometer, magnetic field sensor and MEMS gyroscope after the wireless balloon endoscope is started. The initial configuration mainly includes: setting the measurement range and measurement accuracy of the sensor; setting the acquisition frequency of the sensor for attitude information; and interrupt processing. After completing the initialization of the sensor, the accelerometer, magnetic field sensor and MEMS gyroscope will periodically measure the acceleration, magnetic field strength and angular velocity of the three axes. During the image acquisition process of the balloon endoscope in the digestive tract, the working mode of the attitude sensor can be reconfigured through the attitude sensor control unit as required.

The attitude system control unit is responsible for the overall scheduling and control of the internal devices of the balloon endoscope system. When the balloon endoscope is not activated, the entire internal device is in a dormant state, and the attitude sensor is also in a dormant state. After the balloon endoscope is started, the attitude system control unit is responsible for completing the scheduling of the initialization of the attitude sensor and other related internal devices. In the process of image information acquisition, it is responsible for scheduling the attitude information acquisition of the attitude sensor and the calculation of the attitude angle (direction vector of the camera).

The attitude calculation unit calculates the attitude of the balloon endoscope (direction vector of the camera) through the gravitational acceleration component, the geomagnetic field strength component and the angular velocity.

The posture method of Embodiment 2 (referred to as posture method B).

Compared with Embodiment 1, the calculation of the attitude angle at this time is relatively complicated, but the attitude method proposed here is more accurate. There are two ways to directly calculate the attitude angle of the balloon endoscope:

(1) Utilize the acceleration value and the magnetic field strength value, adopt attitude method A to calculate balloon attitude, see embodiment one;

(2) Real-time integration of the angular velocity values of the three axes of the balloon to detect the change of the balloon attitude.

There is an error in the angular velocity measured by the MEMS gyroscope, and the long-term integration of the angular velocity will cause error accumulation, so that the obtained attitude angle will drift relative to the correct value. However, the accuracy of the attitude angle obtained by integrating the angular velocity in a short time is relatively high. The posture method combining the above two methods is the so-called posture method B.

In this embodiment, the actual practice of measuring the attitude angle of the balloon endoscope by the attitude sensing system is:

S1': First calculate the attitude angle by using the attitude method A according to the acceleration value and the magnetic field strength value. However, it is necessary to ensure that $g_{x^{\′}}^2+g_{{\mathrm{the\; y}}^{\′}}^2+g_{z^{\′}}^2\≈g^2$ as well as $B_{x^{\′}}^2+B_{{\mathrm{the\; y}}^{\′}}^2+B_{z^{\′}}^2\≈B^2$ (where g is the local gravitational acceleration value, generally 9.8m/s ² ; B is the local geomagnetic field strength value). The attitude angle cannot be calculated until the acceleration and magnetic field strength meet the above two conditions;

α_t，当前陀螺仪的角速度采样值为ω_x，ω_y，ω_z，那么可以计算出在T这段时间中球囊内窥镜绕X’，Y’和Z’轴分别旋转了ω_x·T，ω_y·T，ω_z·T(即积分过程)，通过该旋转角度ω_x·T，ω_y·T，ω_z·T就可以得到该时刻的姿态角θ_t+1，

α_t+1；S2': The calculation result is used as the initial posture of the balloon endoscope, the angular velocity value of the balloon is integrated, and the posture of the balloon endoscope at a subsequent time is calculated. Integrate the angular velocity value of the balloon, and calculate the attitude of the balloon endoscope at the next moment: the sampling period of the gyroscope is T, the attitude angle at the previous moment is _θt ,

α _t , the angular velocity sampling value of the current gyroscope is ω _x , ω _y , ω _z , then it can be calculated that the balloon endoscope rotates ω _x around the X', Y' and Z' axes during the period T T, ω _y T, ω _z T (that is, the integration process), through the rotation angle ω _x T, ω _y T, ω _z T, the attitude angle θ _t+1 at this moment can be obtained,

αt ₊₁ ;

S3': At the same time, in order to avoid the drift of the attitude angle, the attitude method A should be used periodically to correct the attitude angle.

A schematic structural diagram of the attitude method B can be referred to in FIG. 8 .

In Embodiment 1, two conditions must be met when performing attitude angle detection: (1) the human body cannot perform strenuous exercise, while the movement of the balloon endoscope in the digestive tract is relatively stable; (2) the electromagnetic environment outside the human body is relatively stable , there can be no other magnetic interference sources affecting the earth's magnetic field. In this embodiment, at certain moments, even if the balloon endoscope does not meet the above two conditions, it can still estimate the attitude of the balloon through the MEMS gyroscope.

Embodiment three

As shown in FIG. 9 , preferably, in this embodiment, the posture sensing system is composed of two parts: an internal device and an external device. Among them, the device in the body includes: a magnetic field sensor 701, which can measure the magnetic field strength components of the geomagnetic field on three axes in the carrier coordinate system by using a 3-axis magnetic field sensor; MEMS gyroscope 304, which can measure the The angular acceleration of the three axes in the carrier coordinate system; the attitude sensor control unit is used to configure the attitude sensor working mode; the attitude calculation unit is used to process the sensor information from the attitude sensor unit; the wireless transceiver unit is used to realize and Wireless communication between in vitro devices. The external device includes: an electromagnetic control unit configured with a magnetic field generating device outside the human body 301; a wireless transceiver unit for realizing wireless communication with the internal device; an external posture system control unit for scheduling the external magnetic field generating device and the internal device. information exchange. Magnetic field sensor 701 and MEMS gyroscope 304 are located within geometry 207 .

When detecting the attitude angle of the balloon endoscope, the following steps should be followed. First, the attitude system control unit initiates attitude angle measurement; the internal wireless transceiver unit sends instructions to the external device, ordering it to configure the magnetic field generating device; after receiving the configuration command through the wireless transceiver unit, the external attitude system control unit commands the magnetic field control The unit takes corresponding operations; the magnetic field control unit controls the magnetic field generating device to periodically generate a uniform magnetic field; the external attitude system control unit notifies the internal device through the wireless transceiver unit that the configuration of the magnetic field generating device has been completed; the attitude sensor control unit configures the magnetic field sensor and MEMS gyroscope The measurement frequency, measurement range, and related working modes of the instrument; after the attitude sensor control unit completes the configuration of the attitude sensor, the magnetic field sensor and MEMS gyroscope can periodically collect the magnetic field strength and angular velocity of each axis, and The collected analog values are subjected to A/D sampling, storage and other operations; the attitude calculation unit receives the measured values from the sensor and performs related processing to calculate the attitude angle of the balloon (the direction vector of the camera).

In order to further illustrate this embodiment, each component of the gesture sensing system will be described in detail below in conjunction with FIG. 10 .

The magnetic field generating device has two implementation schemes: the method of electromagnetic coil and the method of permanent magnet.

The method of electromagnetic coils: 3 pairs of orthogonal electromagnetic coils are installed outside the body, and the directions of these 3 pairs of electromagnetic coils are respectively the same as the X-axis, Y-axis and Z-axis, see FIG. 9 . During the digestive tract examination, the patient is required to stay in a non-magnetic environment where the earth's magnetic field has been shielded by the material mounted on the wall. Therefore, when a current is passed through the coil, an approximate uniform magnetic field can be generated in the area where the human body is located. When performing attitude measurement, the three pairs of orthogonal electromagnetic coils are operated in turn, and at most one electromagnetic coil can be in working state at any time. Therefore, uniform electromagnetic fields of the X-axis, Y-axis and Z-axis will be generated sequentially in the area where the human body is located. The timing control of the operation of the three electromagnetic coils must be very strict, and the wireless transceiver unit periodically communicates with the internal device to let the internal device know the working status of the electromagnetic coil.

The permanent magnet method: A uniform magnetic field is generated in the area where the human body is located by a pair of permanent magnets. The pair of permanent magnets can rotate, so that the magnetic fields in the X-axis, Y-axis and Z-axis directions can be generated sequentially through the rotation.

The magnetic field control unit can control the conduction and cut-off of the current in the electromagnetic coil, and can also adjust the strength of the electromagnetic field by controlling the magnitude of the conduction current; rotate the direction of the permanent magnet pair to generate a magnetic field in the desired direction. When measuring an attitude angle, uniform magnetic fields in the directions of the X-axis, Y-axis and Z-axis are sequentially generated. It may be assumed that the generation of a uniform magnetic field in the X-axis direction corresponds to time A; the generation of a uniform magnetic field in the Y-axis direction corresponds to time B; the generation of a uniform magnetic field in the Z-axis direction corresponds to time C. It is necessary to ensure that the time intervals between time A, time B and time C are relatively short.

The external wireless transceiver unit realizes the wireless communication with the internal device.

The external posture system control unit controls the work of the external electromagnetic coil and realizes the communication with the internal device.

The magnetic field sensor is the same as the first embodiment.

The MEMS gyroscope is the same as the second embodiment.

The sensor control unit needs initial configuration of the magnetic field sensor and MEMS gyroscope after the wireless balloon endoscope is started. The initial configuration mainly includes: setting the measurement range and measurement accuracy of the sensor; setting the acquisition frequency of the sensor for attitude information; and interrupt processing. After completing the initialization of the sensor, the magnetic field sensor and MEMS gyroscope will periodically measure the magnetic field strength and angular velocity of the three axes. The difference from the second embodiment is that when the magnetic field sensor is used to measure the attitude angle, the magnetic field sensor needs to measure the magnetic field strength of the balloon in three axial directions at time A, time B and time C sequentially. During the image acquisition process of the balloon endoscope in the digestive tract, the working mode of the attitude sensor can be reconfigured through the attitude sensor control unit as required.

The attitude system control unit is responsible for the overall scheduling of the internal devices of the balloon endoscope system and the communication with external devices. When the balloon endoscope is not activated, the entire internal device is in a dormant state, and the attitude sensor is also in a dormant state. After the balloon endoscope is started, the attitude system control unit is responsible for completing the initialization and scheduling of attitude sensors and other related internal devices. In the process of image information collection, it is responsible for scheduling the attitude information acquisition of the attitude sensor, the calculation of the attitude angle (direction vector of the camera), and other related work.

The wireless transceiver unit in the body realizes the communication with the wireless transceiver unit outside the body.

The attitude calculation unit, combined with the external device, calculates the attitude of the balloon by using the magnetic field strength and angular velocity.

The posture method corresponding to the third embodiment (referred to as posture method C).

There are two ways to independently calculate the pose of the balloon in the digestive tract:

1) Real-time integration of the angular velocity values of the three axes of the balloon to detect the change of the balloon attitude. There is an error in the angular velocity measured by the MEMS gyroscope, and the long-term integration of the angular velocity will cause error accumulation, so that the obtained attitude angle will drift relative to the correct value. However, the accuracy of the attitude angle obtained by integrating the angular velocity in a short time is relatively high.

2) Combining the external device and the magnetic field sensor to calculate the attitude of the balloon endoscope. When using this method to measure the attitude angle, the external device needs to generate uniform magnetic fields of the X-axis, Y-axis and Z-axis at time A, time B and time C in sequence, and then the magnetic field sensor detects time A, time B and time C respectively. The magnetic field strength of the balloon endoscope in the X' axis, Y' axis and Z' axis. The attitude of the balloon endoscope can be estimated by using the magnetic field intensity component values at time A, time B and time C. Specific method: use the magnetic field strength value measured by the magnetic field sensor at time A to obtain the angle between the X axis in the absolute coordinate system and the X' axis, Y' axis, and Z' axis of the balloon endoscope; use the magnetic field sensor at time B to measure The angle between the Y axis in the absolute coordinate system and the X' axis, Y' axis and Z' axis of the balloon endoscope can be obtained by using the magnetic field strength value; the absolute coordinate system can be obtained by using the magnetic field strength value measured by the magnetic field sensor at time C The included angle between the central Z axis and the X' axis, Y' axis, and Z' axis of the balloon endoscope; combined with the above information, the attitude angle of the balloon in the digestive tract can be easily obtained.

However, if the balloon endoscope only uses the method 2) to measure the attitude angle, frequent communication between the external device and the internal device is required, and a magnetic field needs to be generated outside the body, so the cost is relatively high. A feasible solution is to combine the above two methods to detect the attitude of the balloon endoscope in the digestive tract, which is called attitude method C. in particular:

S1": Initially adopt method 2) to obtain the initial attitude angle of the balloon endoscope;

S2 ": Then, integrate the sampling value of the 3-axis MEMS gyroscope, estimate the attitude angle of the balloon endoscope at each moment thereafter (similar to the integration step of step S2');

S3": At the same time, method 2) is also required to periodically correct the attitude angle of the balloon endoscope.

When this solution is adopted, the operating frequency of the external magnetic field generating device is greatly reduced, and the communication frequency between the internal device and the external device is also greatly reduced.

The method structural diagram of attitude method C can be referred to FIG. 11 .

The above is only a preferred embodiment of the present invention, it should be pointed out that for those of ordinary skill in the art, without departing from the technical principle of the present invention, some improvements and modifications can also be made, these improvements and modifications It should also be regarded as the protection scope of the present invention.

Claims (8)

1. A posture sensing system for a balloon endoscope is characterized by comprising an in-vivo device for acquiring and processing the posture information of the balloon endoscope in vivo and an in-vitro device for generating a magnetic field in vitro;

wherein the in-vivo device is in communication with the in-vitro device and comprises:

the attitude sensor unit comprises one or more attitude sensors and is used for acquiring the attitude information of the balloon endoscope in vivo;

the attitude sensor control unit is used for configuring the working mode of the attitude sensor;

and the attitude calculation unit is used for calculating the attitude angle of the balloon endoscope by adopting a corresponding method after filtering and other processing is carried out on the attitude information from the attitude sensor unit.

2. The system of claim 1, wherein the attitude sensor unit comprises a combination of one or more of an acceleration sensor, a magnetic field sensor, and an angular velocity sensor.

3. The system of claim 2, wherein the extracorporeal device comprises a magnetic field generating device, the magnetic field generating device being an electromagnetic coil or a permanent magnet.

4. The system of claim 3, wherein the electromagnetic coils are 3 pairs of mutually orthogonal electromagnetic coils, which generate uniform magnetic fields in the X-axis, Y-axis and Z-axis directions of the absolute coordinate system in order.

5. The system of claim 2, wherein the attitude angles include a pitch angle, a roll angle, and a heading angle, and when the attitude sensor unit includes an acceleration sensor and a magnetic field sensor, the attitude angle of the balloon endoscope is calculated by a method based on the principles of gravitational acceleration and geomagnetic field, specifically:

s1, measuring the gravity acceleration of the balloon endoscope by using the acceleration sensor, and measuring the magnetic field intensity of the balloon endoscope by using the magnetic field sensor;

s2, judging whether the balloon endoscope moves according to the acquired posture information, if so, returning to the step S1, otherwise, executing the step S3;

s3, calculating a pitch angle, a roll angle and a course angle of the balloon endoscope according to the gravity acceleration and the magnetic field intensity;

and S4, calculating the direction vector of the camera in the balloon endoscope according to the pitch angle, the roll angle and the course angle.

6. The system according to claim 5, wherein when the attitude sensor unit includes an acceleration sensor, a magnetic field sensor, and a gyroscope, the attitude angle of the balloon endoscope is calculated using a method based on the principles of gravitational acceleration, angular velocity, and geomagnetic field, specifically:

s1', calculating the pitch angle, the roll angle and the course angle of the balloon endoscope by utilizing the steps S1-S3;

s2 ', taking the calculation result of step S1' as the initial attitude angle of the balloon endoscope, measuring the angular velocity of the balloon endoscope by a gyroscope, integrating the angular velocity, and calculating the attitude angle of the balloon endoscope thereafter;

s3 ', periodically calculates the attitude angle using steps S1 to S3, and replaces the calculation result with the attitude angle calculated at step S2'.

7. The system of claim 6, wherein when the attitude sensor unit comprises a magnetic field sensor and a gyroscope, the attitude angle of the balloon endoscope is calculated using a method based on the principles of angular velocity and magnetic field, in particular:

s1 ", performing attitude angle measurement using a method based on the magnetic field principle: after the in-vitro device sequentially generates uniform magnetic fields in the X-axis direction, the Y-axis direction and the Z-axis direction of an absolute coordinate system at the time A, the time B and the time C, the magnetic field sensors respectively detect the magnetic field strengths of the balloon endoscope at the time A, the time B and the time C in the X ' -axis direction, the Y ' -axis direction and the Z ' -axis direction of a carrier coordinate system, and the initial attitude angle of the balloon endoscope is calculated by using the magnetic field strengths at the time A, the time B and the time C;

s2 ″ measuring the angular velocity of the balloon endoscope by using the gyroscope, integrating the angular velocity, and estimating the attitude angle of the balloon endoscope at each time thereafter;

s3 ″ using the method based on the magnetic field principle to calculate the attitude angle, and periodically replacing the calculation result with the attitude angle calculated in step S2 ″.

8. A balloon endoscope comprising a geometry, a system according to any of claims 1-7, and cameras, the system according to any of claims 1-7 being located inside the geometry, the cameras being distributed on the outer surface of the geometry.

Images