CN114494374A - Method for determining fusion error of three-dimensional model and two-dimensional image and electronic equipment - Google Patents

Abstract

The present application discloses a method and electronic device for determining the fusion error of a three-dimensional model and a two-dimensional image, which are used to solve the actual problem that the existing technology cannot accurately calculate the three-dimensional model of biological tissue and the two-dimensional image of biological tissue in laparoscopic video. The problem of fusion error. In this application, the two-dimensional image and the three-dimensional model of the biological tissue are registered and displayed by using the registration algorithm in the perspective of the virtual camera; the first outer contour of the biological tissue in the two-dimensional image and the three-dimensional model of the biological tissue are projected onto The second outer contour of the biological tissue in the reference two-dimensional image obtained by the two-dimensional plane; determining the pixel distance between the first outer contour and the second outer contour; using the predetermined correspondence between the unit pixel distance and the physical distance, The physical distance corresponding to the pixel distance between the first outer contour and the second outer contour is determined as the fusion error, so that the actual fusion error of the three-dimensional model of the biological tissue and the two-dimensional image can be accurately calculated.

Description

Method and electronic device for determining fusion error of 3D model and 2D image

technical field

The present application relates to the technical field of image processing, and in particular, to a method and electronic device for determining a fusion error of a three-dimensional model and a two-dimensional image.

Background technique

In laparoscopic surgical navigation, it is necessary to register the preoperatively reconstructed 3D model with the biological tissue in the laparoscopic video, and perform fusion display, so that the actual blood vessels and lesions in the video can be marked with the preoperatively reconstructed blood vessels and lesions. Location.

In the related art, one method to measure the matching degree between the three-dimensional model of biological tissue and the two-dimensional image of biological tissue in laparoscopic video is based on image similarity, and the measurement method based on image similarity cannot directly give the actual matching between the two. The fusion error, the actual fusion error of the two matching, is based on the premise that the rotation angle, zoom, and translation have accurate solutions, and the error between the three-dimensional model of the biological tissue and the accurate solution under the registered camera view is calculated. However, in actual use, it is generally impossible to give accurate solutions for the rotation angle, scaling and translation, that is, the optimal rotation angle, scaling and translation parameters cannot be obtained, resulting in the inability to accurately calculate the 3D model of biological tissue and the biological data in laparoscopic videos. The actual fusion error of the two-dimensional image of the tissue, so how to accurately calculate the actual fusion error of the three-dimensional model of the biological tissue and the two-dimensional image of the biological tissue in the laparoscopic video is an urgent problem to be solved.

SUMMARY OF THE INVENTION

The purpose of this application is to provide a method and electronic device for determining the fusion error of a three-dimensional model and a two-dimensional image, so as to solve the problem that the prior art cannot accurately calculate the three-dimensional model of biological tissue and the two-dimensional image of biological tissue in laparoscopic video. the actual fusion error.

In a first aspect, the present application provides a method for determining a fusion error of a three-dimensional model and a two-dimensional image, the method comprising:

The two-dimensional image and the three-dimensional model of the biological tissue are fused and displayed after registration using a registration algorithm under the perspective of the virtual camera;

acquiring the first outer contour of the biological tissue in the two-dimensional image, and acquiring the second outer contour of the biological tissue in the reference two-dimensional image obtained by projecting the three-dimensional model of the biological tissue onto the two-dimensional plane;

determining the pixel distance between the first outer contour and the second outer contour;

Using the predetermined correspondence between the unit pixel distance and the physical distance, the physical distance corresponding to the pixel distance between the first outer contour and the second outer contour is determined as the fusion error.

In a possible implementation manner, the determining the pixel distance between the first outer contour and the second outer contour includes:

Based on the first outer contour and the second outer contour, determine the pixel point pair used to describe the same position in the first outer contour and the second outer contour;

Based on the pixel point pairs, an average pixel distance error of the first outer contour and the second outer contour is determined as a pixel distance between the first outer contour and the second outer contour.

In a possible implementation manner, based on the first outer contour and the second outer contour, the pixel point pair used to describe the same position in the first outer contour and the second outer contour is determined ,include:

Calculate the context information corresponding to each pixel in the first outer contour and the second outer contour; the context information is the neighborhood structure of the pixel;

Calculate the cost value of any two pixels in the first outer contour and the second outer contour;

A group of pixel point pairs with the lowest overall cost value of the first outer contour and the second outer contour is counted by using the Hungarian algorithm; wherein the group of pixel point pairs includes a plurality of pixel point pairs at the same position.

In a possible implementation manner, the determining an average pixel distance error of the first outer contour and the second outer contour based on the pixel point pair includes:

Under the same coordinate system, determine the coordinates of each pixel in the first outer contour and the second outer contour; the same coordinate system includes X-axis coordinates and Y-axis coordinates;

Subtract the coordinates on the X axis of each pixel pair in the first outer contour and the second outer contour to obtain the pixel distance error of all pixel pairs on the X axis;

Divide the pixel distance error of all pixel point pairs on the X axis by the number of pixel point pairs on the X axis to obtain the average pixel distance error on the X axis;

Subtract the coordinates on the Y-axis of each pixel pair in the first outer contour and the second outer contour to obtain the pixel distance error of all pixel pairs on the Y-axis;

The average pixel distance error on the Y axis is obtained by dividing the pixel distance error of all pixel point pairs on the Y axis by the number of pixel point pairs on the Y axis.

In a possible implementation manner, determining the correspondence between the unit pixel distance and the physical distance specifically includes:

determining the minimum three-dimensional coordinate value and the maximum three-dimensional coordinate value of the biological tissue in the three-dimensional model in the world coordinate system;

Under the same viewing angle of the virtual camera, project the three-dimensional coordinate minimum value and the three-dimensional coordinate maximum value to a two-dimensional plane to obtain two-dimensional coordinates corresponding to the three-dimensional coordinate minimum value and the three-dimensional coordinate maximum value;

Based on the world coordinates of the minimum three-dimensional coordinate and the maximum three-dimensional coordinate in the world coordinate system, calculate the physical distance between the minimum three-dimensional coordinate and the maximum three-dimensional coordinate;

Based on the two-dimensional coordinates corresponding to the three-dimensional coordinate minimum value and the three-dimensional coordinate maximum value, calculate the pixel distance between the two-dimensional coordinates corresponding to the three-dimensional coordinate minimum value and the three-dimensional coordinate maximum value;

Based on the physical distance between the three-dimensional coordinate minimum value and the three-dimensional coordinate maximum value and the pixel distance between the two-dimensional coordinates corresponding to the three-dimensional coordinate minimum value and the three-dimensional coordinate maximum value, determine the X-axis in the two-dimensional coordinate system. The physical distance corresponding to the unit pixel distance, and the physical distance corresponding to the unit pixel distance on the Y axis.

In a possible implementation manner, the determining the physical distance corresponding to the pixel distance between the first outer contour and the second outer contour specifically includes:

Multiply the average pixel distance error on the X axis by the physical distance corresponding to the unit pixel distance on the X axis to obtain the distance between the first outer contour and the second outer contour on the X axis. The physical distance corresponding to the pixel distance;

Multiply the average pixel distance error on the Y-axis by the physical distance corresponding to the unit pixel distance on the Y-axis to obtain the distance between the first outer contour and the second outer contour on the Y-axis. The physical distance corresponding to the pixel distance;

The first outer contour is obtained based on the physical distance corresponding to the pixel distance on the X axis and the physical distance corresponding to the pixel distance on the Y axis between the first outer contour and the second outer contour the physical distance corresponding to the pixel distance between the contour and the second outer contour;

The physical distance corresponding to the pixel distance between the first outer contour and the second outer contour is used as the fusion error.

In a possible implementation manner, acquiring the first outer contour of the biological tissue in the two-dimensional image, and acquiring the biological tissue in the reference two-dimensional image obtained by projecting the three-dimensional model of the biological tissue onto a two-dimensional plane. The second outer contour of the tissue, specifically including:

Using an artificial intelligence algorithm to obtain the first outer contour of the biological tissue in the two-dimensional image;

Create a virtual window with the same video playback size as the two-dimensional image playback size, and use the virtual camera on the virtual window to project the three-dimensional model of the biological tissue on the virtual camera from the same viewing angle. two-dimensional plane;

acquiring a two-dimensional image of the two-dimensional plane to obtain the reference two-dimensional image;

Edge detection is performed on the biological tissue in the reference two-dimensional image to obtain a second outer contour of the biological tissue.

In a possible implementation manner, before the determining the pixel distance between the first outer contour and the second outer contour, the method further includes:

Using a curve data compression algorithm, the first outer contour and the second outer contour are down-sampled to obtain the first outer contour and the second outer contour with the same number of pixel points.

In a second aspect, the present application provides an electronic device, including a processor and a memory:

the memory for storing a computer program executable by the processor;

The processor is connected to the memory and is configured to execute the instructions to implement the method for determining a fusion error of a three-dimensional model and a two-dimensional image according to any one of the first aspects above.

In a third aspect, the present application provides a computer-readable storage medium, which, when an instruction in the computer-readable storage medium is executed by an electronic device, enables the electronic device to execute any one of the above-mentioned first aspects. A method for determining the fusion error of a 3D model and a 2D image.

In a fourth aspect, the present application provides a computer program product, including a computer program:

When the computer program is executed by the processor, the method for determining the fusion error of the three-dimensional model and the two-dimensional image according to any one of the above-mentioned first aspects is realized.

The technical solutions provided by the embodiments of the present application bring at least the following beneficial effects:

In the embodiment of the present application, the actual physical distance corresponding to the unit pixel distance is firstly determined, and then the two-dimensional image and the three-dimensional model of the biological tissue are registered and displayed by using the registration algorithm in the perspective of the virtual camera; The second outer contour of the biological tissue in the reference two-dimensional image obtained by projecting the first outer contour of the tissue and the three-dimensional model of the biological tissue to the two-dimensional plane; determining the pixel distance between the first outer contour and the second outer contour; The determined correspondence between the unit pixel distance and the physical distance is determined, and the physical distance corresponding to the pixel distance between the first outer contour and the second outer contour is determined as a fusion error. Therefore, the present application uses the predetermined correspondence between the unit pixel distance and the physical distance, the first outer contour of the biological tissue in the two-dimensional image and the second outer contour of the biological tissue in the two-dimensional projection image of the three-dimensional model of the biological tissue. , which can more accurately quantify the error between the two-dimensional map and the three-dimensional model of the same biological tissue, improve the accuracy of the actual fusion error between the three-dimensional model of the biological tissue and the two-dimensional image of the biological tissue in the laparoscopic video, so as to be more accurate The fusion error of the 3D model and the 2D image can be given accurately, which can improve the user experience.

Other features and advantages of the present application will be set forth in the description which follows, and in part will be apparent from the description, or may be learned by practice of the present application. The objectives and other advantages of the application may be realized and attained by the structure particularly pointed out in the written description, claims, and drawings.

Description of drawings

In order to illustrate the technical solutions of the embodiments of the present application more clearly, the following briefly introduces the drawings that need to be used in the embodiments of the present application. Obviously, the drawings introduced below are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can also be obtained from these drawings without any creative effort.

1a is a schematic diagram of a two-dimensional image of biological tissue in a laparoscopic video provided by an embodiment of the application;

Fig. 1b is a schematic diagram of a three-dimensional model of a preoperatively reconstructed biological tissue provided by an embodiment of the present application;

Fig. 1c is a fusion display effect diagram of the fusion display after the registration of the three-dimensional model of the biological tissue and the two-dimensional image provided by the embodiment of the application;

2 is a schematic flowchart of a method for determining a fusion error of a three-dimensional model and a two-dimensional image provided by an embodiment of the present application;

3a is a schematic diagram of a first outer contour of a biological tissue in a two-dimensional image provided by an embodiment of the present application;

Fig. 3b is a schematic diagram of the second outer contour of the biological tissue in the reference two-dimensional image obtained by projecting the three-dimensional model of the biological tissue provided by the embodiment of the application to the two-dimensional plane;

Fig. 3c is a schematic diagram after downsampling of the first outer contour of biological tissue in the two-dimensional image provided by the embodiment of the present application;

FIG. 3d is a schematic diagram after downsampling of the second outer contour of the biological tissue in the reference two-dimensional image provided by the embodiment of the present application;

4 is a schematic flowchart of a method for determining a pixel distance between a first outer contour and a second outer contour provided by an embodiment of the present application;

5 is a schematic diagram of a group of pixel point pairs with the lowest overall cost value of the first outer contour and the second outer contour provided by an embodiment of the present application;

6 is a schematic flowchart of a method for determining an average pixel distance error of a first outer contour and a second outer contour provided by an embodiment of the present application;

7 is a schematic flowchart of a method for determining a correspondence between a unit pixel distance and a physical distance according to an embodiment of the present application;

8 is a schematic flowchart of a method for determining a physical distance corresponding to a pixel distance between a first outer contour and a second outer contour according to an embodiment of the present application;

FIG. 9 is a schematic structural diagram of an electronic device provided by an embodiment of the present application.

Detailed ways

In order to make the purposes, technical solutions and advantages of the embodiments of the present application more clear, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present application. The described embodiments are part of the embodiments of the present application, but not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

In addition, in the description of the embodiments of the present application, unless otherwise specified, “/” means or means, for example, A/B can mean A or B; “and/or” in the text is only a description of the associated object The association relationship indicates that there can be three kinds of relationships, for example, A and/or B can indicate that A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of this application , "plurality" means two or more than two.

Hereinafter, the terms "first" and "second" are only used for descriptive purposes, and should not be construed as implying or implying relative importance or implying the number of indicated technical features. Therefore, the features defined with "first", "second", and "second" may expressly or implicitly include one or more of the features, and in the description of the embodiments of the present application, unless otherwise stated, "a plurality of" The meaning is two or more.

In laparoscopic surgical navigation, it is necessary to register the biological tissue in the laparoscopic video as shown in Figure 1a with the preoperatively reconstructed 3D model as shown in Figure 1b, and perform fusion display (as shown in Figure 1c). , so that the actual blood vessels and lesions in the video can be marked with the preoperatively reconstructed blood vessels and lesions. In the process of matching the preoperatively reconstructed 3D model with the biological tissue in the laparoscopic video, 3D/2D registration needs to be involved.

Among them, the 3D/2D matching 2D image method is different dimensional matching. It is necessary to raise the liver area in the laparoscopic video to 3D to achieve 3D/3D registration or to reduce the dimension of the preoperative 3D model and project it to 2D to achieve 2D/ 2D registration. In 2D/2D registration, it is necessary to project a 3D model to a 2D image, generate a 2D contour, and segment the biological tissue area from the 2D image of the biological tissue in the laparoscopic video to obtain the contour of the biological tissue so that the It is matched with the projected contour library of preoperative biological tissue, and the best matching contour is selected from the contour library, and then the rotation, translation, zoom and other parameters of the three-dimensional model are obtained, so as to convert the three-dimensional model to a suitable position to achieve Calibration of blood vessels and lesions in the video. In 3D/3D registration, because the laparoscope is often zoomed, the current camera internal parameters cannot be obtained in real time, so the depth information of the pixel points cannot be obtained, and the surface of the biological tissue is very smooth and there are no obvious marking points. Therefore, in this application The registration method used is 2D/2D registration.

In the related art, there are mainly two methods to measure the matching degree between the three-dimensional model of biological tissue and the two-dimensional image of biological tissue in the laparoscopic video: one is based on image similarity, and the other is based on edge information. Among them, the measurement method based on image similarity cannot directly give the actual fusion error of the two matching. The error of the model from the exact solution at the registered camera view. However, in actual use, it is generally impossible to provide accurate solutions for rotation angle, scaling, and translation, that is, the optimal rotation angle, scaling, and translation parameters, resulting in the inability to accurately calculate the three-dimensional model of biological tissue and the biological tissue in laparoscopic videos. The actual fusion error of the two-dimensional image, so how to accurately calculate the actual fusion error of the three-dimensional model of the biological tissue and the two-dimensional image of the biological tissue in the laparoscopic video is an urgent problem to be solved.

In view of this, the present application provides a method and electronic device for determining the fusion error of a three-dimensional model and a two-dimensional image, so as to solve the problem that the existing technology cannot accurately calculate the three-dimensional model of the biological tissue and the biological tissue in the laparoscopic video. The problem of the actual fusion error of dimensional images.

The inventive concept of the present application can be summarized as follows: in the embodiment of the present application, the actual physical distance corresponding to the unit pixel distance is firstly determined, and then the two-dimensional image and the three-dimensional model of the biological tissue are registered using the registration algorithm from the perspective of the virtual camera. Fusion display; obtaining the first outer contour of the biological tissue in the two-dimensional image and the second outer contour of the biological tissue in the reference two-dimensional image obtained by projecting the three-dimensional model of the biological tissue to the two-dimensional plane; determining the first outer contour and the second outer contour The pixel distance between the contours; using the predetermined correspondence between the unit pixel distance and the physical distance, the physical distance corresponding to the pixel distance between the first outer contour and the second outer contour is determined as the fusion error. Therefore, the present application uses the predetermined correspondence between the unit pixel distance and the physical distance, the first outer contour of the biological tissue in the two-dimensional image and the second outer contour of the biological tissue in the two-dimensional projection image of the three-dimensional model of the biological tissue. , which can more accurately quantify the error between the two-dimensional map and the three-dimensional model of the same biological tissue, improve the accuracy of the actual fusion error between the three-dimensional model of the biological tissue and the two-dimensional image of the biological tissue in the laparoscopic video, so as to be more accurate The fusion error of the 3D model and the 2D image can be given accurately, which can improve the user experience.

It should be noted that the present application is not only applicable to determining the fusion of a 3D model and a laparoscopic video, but also applicable to the calculation of fusion errors between a 3D model of biological tissue and any 2D medical image. Biological tissues in the examples of the present application are internal organs such as liver, stomach, kidney, etc.

After introducing the main inventive idea of the embodiments of the present application, the following describes the method for determining the fusion error of the three-dimensional model and the two-dimensional image provided by the embodiments of the present application with reference to the accompanying drawings.

FIG. 2 is a schematic flowchart of a method for determining a fusion error of a three-dimensional model and a two-dimensional image according to an embodiment of the present application. As shown in Figure 2, the method includes the following steps:

In step 201, the two-dimensional image of the biological tissue and the three-dimensional model are fused and displayed after being registered using a registration algorithm from the perspective of the virtual camera.

Specifically, the method for determining the fusion error of the three-dimensional model and the two-dimensional image provided by the embodiments of the present application can be implemented based on the open source software platform of Mitk (Medical Image Interaction Toolkit). In this method, it is first necessary to load dicom (Digital Imaging and Communications in Medicine, digital imaging and communication in medicine) data and the data of the three-dimensional model of the biological tissue into the open source software platform of Mitk, and at the same time load the biological tissue in the laparoscopic surgical video The 2D image is rendered into the 3D window, and then the 3D model of the biological tissue shown in Figure 1b is registered with the region of the biological tissue in the laparoscopic video shown in Figure 1a using the 2D/3D registration algorithm. , the fusion display is performed, and the fusion display effect diagram is shown in Figure 1c.

In step 202, the first outer contour of the biological tissue in the two-dimensional image is acquired, and the second outer contour of the biological tissue in the reference two-dimensional image obtained by projecting the three-dimensional model of the biological tissue onto the two-dimensional plane is acquired.

In a possible implementation, in the example of this application, the first outer contour of the biological tissue in the two-dimensional image is acquired, and the first outer contour of the biological tissue in the reference two-dimensional image obtained by projecting the three-dimensional model of the biological tissue onto the two-dimensional plane is acquired. The second outer contour can be specifically implemented as follows: first, use artificial intelligence algorithm to obtain the first outer contour of the biological tissue in the two-dimensional image, then create a virtual window with the video playback size consistent with the two-dimensional image playback size, and use the virtual window in step 201. The camera acts on the virtual window and projects the 3D model of the biological tissue to a 2D plane under the same viewing angle; then acquires a 2D image of the 2D plane to obtain a reference 2D image; finally, the biological tissue is analyzed in the reference 2D image. Edge detection is performed to obtain the second outer contour of the biological tissue.

The first outer contour is the set of all pixel points that constitute the edges of the leading and trailing edges of the biological tissue in the two-dimensional image, and the second outer contour is the edges that constitute the leading and trailing edges of the biological tissue in the reference two-dimensional image. The collection of all pixels.

Exemplarily, taking the liver as a biological tissue, the first outer contour of the liver in the two-dimensional image in the laparoscopic video is segmented by using an artificial intelligence algorithm, as shown in Figure 3a, the first outer contour refers to the liver in the two-dimensional The leading and trailing edges in the image, the thick line is the trailing edge of the liver, and the thin line is the trailing edge of the liver. Next, it is necessary to obtain the projection outline of the three-dimensional liver model under the same viewing angle of the virtual camera in step 201. By creating a virtual window whose video playback size is the same as the size of the playback window of the laparoscopic video, the virtual camera can be used to act on the virtual window, The reference two-dimensional image obtained after the three-dimensional model of the liver is projected onto the two-dimensional plane is obtained, and the second outer contour of the projected image of the liver is obtained, as shown in Figure 3b, the thick line is the rear edge of the liver, and the thin line is the front edge of the liver. Therefore, using a virtual window with the same video playback size as the two-dimensional image playback size can ensure that the projected image size of the three-dimensional image is consistent with the size of the video image.

In a possible implementation, in order to reduce the algorithm complexity in the process of matching the pixels of the first outer contour and the second outer contour and speed up the calculation time of the algorithm, curve data compression (Douglas–Peucker ) algorithm, down-sampling the first outer contour and the second outer contour to obtain the first outer contour and the second outer contour with the same number of pixels, as shown in Figure 3c and Figure 3d. Thereby, the number of pixel points in the first outer contour and the second outer contour can be reduced, and at the same time, it can be ensured that the pixel points in the first outer contour and the second outer contour can be in one-to-one correspondence.

In step 203, the pixel distance between the first outer contour and the second outer contour is determined.

In a possible implementation manner, the pixel distance between the first outer contour and the second outer contour is determined in the present application, which can be specifically implemented as the steps shown in FIG. 4 :

In step 401, based on the first outer contour and the second outer contour, a pixel point pair used to describe the same position in the first outer contour and the second outer contour is determined.

In a possible implementation, the embodiment of the present application may use a shape context algorithm (shapecontext) to obtain the pixel point pair used to describe the same position in the first outer contour and the second outer contour, which may be specifically implemented as: first calculate the first outer contour The context information corresponding to each pixel in the first outer contour and the second outer contour; then calculate the cost value of any two pixels in the first outer contour and the second outer contour; finally use the Hungarian algorithm to count the first outer contour and A group of pixel pairs with the lowest overall cost value of the second outer contour, as shown in FIG. 5 , the white pixels are the pixels of the first outer contour, and the gray pixels are the pixels of the second outer contour. The context information is the neighborhood structure of pixel points, and a group of pixel point pairs includes multiple pixel point pairs at the same position.

In step 402, based on the pixel point pair, an average pixel distance error of the first outer contour and the second outer contour is determined as the pixel distance between the first outer contour and the second outer contour.

In a possible implementation manner, in the embodiment of the present application, the average pixel distance error of the first outer contour and the second outer contour is determined based on the pixel point pair, which may be specifically implemented as the steps shown in FIG. 6 :

In step 601, under the same coordinate system, determine the coordinates of each pixel point in the first outer contour and the second outer contour; the same coordinate system includes X-axis coordinates and Y-axis coordinates;

In step 602, the coordinates on the X axis of each pixel pair in the first outer contour and the second outer contour are subtracted to obtain the pixel distance error of all pixel pairs on the X axis;

In step 603, the pixel distance error of all pixel point pairs on the X axis is divided by the number of pixel point pairs on the X axis to obtain the average pixel distance error on the X axis;

In step 604, the coordinates on the Y-axis of each pixel pair in the first outer contour and the second outer contour are subtracted to obtain the pixel distance error of all pixel pairs on the Y-axis;

In step 605, the pixel distance error of all pixel point pairs on the Y axis is divided by the number of pixel point pairs on the Y axis to obtain the average pixel distance error on the Y axis.

其中第i个像素点的二维坐标为

表示像素点

在二维图像坐标系下的X轴上的坐标为

在Y轴上的坐标为

其中坐标原点为虚拟窗口的左上角；m表示第一外轮廓的像素点的数量；与第一外轮廓的像素点集合对应的第二外轮廓的像素点集合为

其中第i个像素点的二维坐标为

表示像素点

在三维模型投影得到的参考二维图像坐标系下的X轴上的坐标为

在Y轴上的坐标为

因为前文得到的生物组织的二维图像与生物组织的三维模型投影得到的参考二维图像尺寸一致，并且重合，所以第一外轮廓的像素点集合与第二外轮廓的像素点集合在同一坐标系下，均以三维模型的虚拟窗口的左上角为坐标原点。Exemplarily, it is assumed that the set of pixel points of the first outer contour is

The two-dimensional coordinates of the i-th pixel are

Represents a pixel

The coordinates on the X-axis in the two-dimensional image coordinate system are

The coordinates on the Y axis are

The origin of the coordinates is the upper left corner of the virtual window; m represents the number of pixels of the first outer contour; the set of pixels of the second outer contour corresponding to the set of pixels of the first outer contour is

The two-dimensional coordinates of the i-th pixel are

Represents a pixel

The coordinates on the X-axis under the reference two-dimensional image coordinate system obtained by the projection of the three-dimensional model are

The coordinates on the Y axis are

Because the two-dimensional image of the biological tissue obtained above is the same size as the reference two-dimensional image projected from the three-dimensional model of the biological tissue and overlaps, the set of pixels of the first outer contour and the set of pixels of the second outer contour are at the same coordinates Under the system, the upper left corner of the virtual window of the 3D model is used as the coordinate origin.

具体可以实施为，先将X轴上所有像素点对的X轴上坐标的差值相加，得到X轴上的像素距离误差，如

然后再除以像素点对的数量m，得到X轴上的平均像素距离误差；同样的，先将Y轴上所有像素点对的Y轴上坐标的差值相加，得到Y轴上的像素距离误差，如

然后再除以像素点对的数量m，得到Y轴上的平均像素距离误差,最后得到第一外轮廓和第二外轮廓的平均像素距离误差为

You can use the coordinates of the pixels of the first outer contour and the coordinates of the pixels of the second outer contour to calculate the distance between each pixel point pair at the same position, and obtain the distance between the first outer contour and the second outer contour pixel point pair The distance is

Specifically, it can be implemented as follows: first, the difference values of the coordinates on the X axis of all pixel point pairs on the X axis are added to obtain the pixel distance error on the X axis, such as

Then divide by the number m of pixel pairs to get the average pixel distance error on the X axis; similarly, first add the difference of the coordinates on the Y axis of all pixel pairs on the Y axis to get the pixels on the Y axis distance error, such as

Then divide by the number m of pixel pairs to get the average pixel distance error on the Y axis, and finally get the average pixel distance error of the first outer contour and the second outer contour as

Therefore, the average pixel distance error of the first outer contour and the second outer contour can be calculated from the coordinates of each pixel pair of the first outer contour and the second outer contour, and the difference between the first outer contour and the second outer contour can be determined. pixel distance between.

In step 204, using the predetermined correspondence between the unit pixel distance and the physical distance, the physical distance corresponding to the pixel distance between the first outer contour and the second outer contour is determined as the fusion error.

In a possible implementation manner, in order to obtain the physical distance corresponding to the pixel distance between each pixel pair of the first outer contour and the second outer contour, it is necessary to obtain the distance between the pixel pairs in the two-dimensional image in the virtual window. The physical distance represented by the unit pixel distance, so the corresponding relationship between the unit pixel distance and the physical distance is determined in the embodiment of the present application, which can be specifically implemented as the steps shown in Figure 7:

In step 701, the minimum and maximum three-dimensional coordinates of the biological tissue in the three-dimensional model are determined in the world coordinate system;

In step 702, under the same viewing angle of the virtual camera, the three-dimensional coordinate minimum value and the three-dimensional coordinate maximum value are projected to a two-dimensional plane to obtain the two-dimensional coordinates corresponding to the three-dimensional coordinate minimum value and the three-dimensional coordinate maximum value;

In step 703, based on the world coordinates of the minimum three-dimensional coordinate and the maximum three-dimensional coordinate in the world coordinate system, calculate the physical distance between the minimum three-dimensional coordinate and the maximum three-dimensional coordinate;

In step 704, based on the two-dimensional coordinates corresponding to the three-dimensional coordinate minimum value and the three-dimensional coordinate maximum value, calculate the pixel distance between the two-dimensional coordinates corresponding to the two three-dimensional coordinate minimum value and the three-dimensional coordinate maximum value;

In step 705, based on the physical distance between the three-dimensional coordinate minimum value and the three-dimensional coordinate maximum value and the pixel distance between the two-dimensional coordinates corresponding to the three-dimensional coordinate minimum value and the three-dimensional coordinate maximum value, determine the X-axis in the two-dimensional coordinate system. The physical distance corresponding to the unit pixel distance of , and the physical distance corresponding to the unit pixel distance on the Y axis.

Exemplarily, since the three-dimensional model of the biological tissue and the two-dimensional image of the biological tissue in the present application have been registered and displayed in fusion, in the embodiment of the present application, the VTK (Visualization Toolkit, Visualization Toolkit, Visualization Toolkit, which renders the three-dimensional model of the biological tissue) is used. Library) virtual camera and VTK coordinate transformation, map the pixels in the three-dimensional model of the biological tissue in the world coordinate system to the two-dimensional coordinate system, and then find the pixel point of the minimum three-dimensional coordinate in the world coordinate and the maximum value of the three-dimensional coordinate. The coordinates of the pixel after mapping to the two-dimensional coordinate system. For example, the minimum bounding box of the rendered 3D model of biological tissue in the world coordinate system can be determined, and then the coordinates of each pixel in the minimum bounding box can be determined, and the pixel with the minimum 3D coordinate and the pixel with the maximum 3D coordinate can be found. Assuming that the range of the minimum bounding box is [Min _x ,Max _x ,Min _y ,Max _y ,Min _z ,Max _z ], then determine the minimum and maximum three-dimensional coordinates of the biological tissue in the three-dimensional model in the world coordinate system They are Min=[Min _x , Min _y , Min_z] and Max=[Max _x , Max _y , Max _z ], respectively. Under the same viewing angle of the virtual camera, using VTK coordinate transformation, project these two pixels to a two-dimensional plane, and obtain the two-dimensional coordinates corresponding to the minimum three-dimensional coordinate and the maximum three-dimensional coordinate as:

和

and

Then, according to the two pixel points of the minimum three-dimensional coordinate and the maximum three-dimensional coordinate of the biological tissue in the three-dimensional model in the world coordinate system, use formula (1) to calculate the physical distance between the minimum three-dimensional coordinate and the maximum three-dimensional coordinate. The distance is:

Then, according to the two-dimensional coordinates corresponding to the minimum three-dimensional coordinates and the maximum three-dimensional coordinates, use formula (2) to calculate the pixel distance between the two-dimensional coordinates corresponding to the two minimum three-dimensional coordinates and the maximum three-dimensional coordinates:

Finally, the physical distance between the minimum three-dimensional coordinate and the maximum three-dimensional coordinate calculated according to formula (1) and the pixel distance between the two-dimensional coordinate corresponding to the minimum three-dimensional coordinate calculated by formula (2) and the maximum three-dimensional coordinate , use formula (3) to calculate the physical distance corresponding to the unit pixel distance on the X axis in the two-dimensional coordinate system, and use formula (4) to calculate the physical distance corresponding to the unit pixel distance on the Y axis:

是获取三维坐标最小值的像素点和三维坐标最大值的像素点对应的两个坐标之间的直线距离上单位像素距离对应的实际物理距离，而

表示二维坐标下X轴的占比，因此公式(3)就表示二维坐标系中X轴上的单位像素距离对应的物理距离，而

表示二维坐标下Y轴的占比，因此公式(4)就表示二维坐标系中Y轴上的单位像素距离对应的物理距离。in,

is the actual physical distance corresponding to the unit pixel distance on the straight-line distance between the two coordinates corresponding to the pixel point of the minimum three-dimensional coordinate and the pixel of the maximum three-dimensional coordinate, and

Represents the proportion of the X-axis in the two-dimensional coordinate system, so formula (3) represents the physical distance corresponding to the unit pixel distance on the X-axis in the two-dimensional coordinate system, and

Represents the proportion of the Y-axis in the two-dimensional coordinate system, so formula (4) represents the physical distance corresponding to the unit pixel distance on the Y-axis in the two-dimensional coordinate system.

Thus, the correspondence between the unit pixel distance and the physical distance can be determined, that is, the physical distance represented by the unit pixel distance between the pixel pairs is:

Therefore, by rendering the two endpoints of the minimum bounding box of the three-dimensional model of biological tissue in the world coordinate system and the coordinate transformation, the physical distance represented by the unit pixel distance between the pixel pairs in the two-dimensional image in the virtual window can be calculated. .

Finally, in the embodiment of the present application, the average pixel distance error Dist _pixel of the first outer contour and the second outer contour obtained according to the steps shown in FIG. 6 and the unit between the pixel point pairs obtained according to the steps shown in FIG. 7 The physical distance pixel _spacing represented by the pixel distance determines the physical distance corresponding to the pixel distance between the first outer contour and the second outer contour, which can be specifically implemented as the steps shown in Figure 8:

In step 801, multiply the average pixel distance error on the X axis by the physical distance corresponding to the unit pixel distance on the X axis to obtain the physical distance corresponding to the pixel distance on the X axis between the first outer contour and the second outer contour distance;

In step 802, multiply the average pixel distance error on the Y axis by the physical distance corresponding to the unit pixel distance on the Y axis to obtain the physical distance corresponding to the pixel distance on the Y axis between the first outer contour and the second outer contour distance;

In step 803, the first outer contour and the second outer contour are obtained based on the physical distance corresponding to the pixel distance on the X axis and the physical distance corresponding to the pixel distance on the Y axis between the first outer contour and the second outer contour The physical distance corresponding to the pixel distance between contours;

In step 804, the physical distance corresponding to the pixel distance between the first outer contour and the second outer contour is taken as the fusion error.

Exemplarily, the average pixel distance error Dist _pixel of the first outer contour and the second outer contour obtained according to the steps shown in FIG. 6 and the unit pixel distance between the pixel pairs obtained according to the steps shown in FIG. 7 are represented by Physical distance pixel _spacing , calculate the physical distance corresponding to the pixel distance between the first outer contour and the second outer contour according to formula (5), and finally calculate the calculated pixel distance between the first outer contour and the second outer contour The corresponding physical distance is used as the fusion error.

即

where Dist _pixel [0] represents the average pixel distance error on the X axis, and Dist _pixel [1] represents the average pixel distance error on the Y axis, i.e.

which is

Thus, the average pixel distance of all pixel point pairs of the second outer contour of the biological tissue in the reference two-dimensional image obtained by calculating the first outer contour of the biological tissue in the two-dimensional image and the projection of the three-dimensional model of the biological tissue to the two-dimensional plane error, and the physical distance represented by the unit pixel distance between pixel pairs in the world coordinate system, so as to calculate the physical distance corresponding to the pixel distance between the first outer contour and the second outer contour, which is given on the two-dimensional image Based on the visual registration accuracy, the fusion error of the 3D model and the 2D image can be given very directly, which can improve the user experience.

Based on the foregoing description, the embodiment of the present application will first determine the actual physical distance corresponding to the unit pixel distance, and then fuse and display the two-dimensional image and three-dimensional model of the biological tissue using a registration algorithm from the perspective of the virtual camera after registration; The second outer contour of the biological tissue in the reference two-dimensional image obtained by projecting the first outer contour of the biological tissue and the three-dimensional model of the biological tissue to the two-dimensional plane in the two-dimensional image; determining the distance between the first outer contour and the second outer contour; Pixel distance: Using the predetermined correspondence between the unit pixel distance and the physical distance, the physical distance corresponding to the pixel distance between the first outer contour and the second outer contour is determined as the fusion error. Therefore, the present application uses the predetermined correspondence between the unit pixel distance and the physical distance, the first outer contour of the biological tissue in the two-dimensional image and the second outer contour of the biological tissue in the two-dimensional projection image of the three-dimensional model of the biological tissue. , which can more accurately quantify the error between the two-dimensional map and the three-dimensional model of the same biological tissue, improve the accuracy of the actual fusion error between the three-dimensional model of the biological tissue and the two-dimensional image of the biological tissue in the laparoscopic video, so as to be more accurate The fusion error of the 3D model and the 2D image can be given accurately, which can improve the user experience.

The electronic device 900 according to this embodiment of the present application is described below with reference to FIG. 9 . The electronic device 900 shown in FIG. 9 is only an example, and should not impose any limitations on the functions and scope of use of the embodiments of the present application.

As shown in FIG. 9, the electronic device 900 takes the form of a general electronic device. The components of the electronic device 900 may include, but are not limited to: the above-mentioned at least one processor 901 , the above-mentioned at least one memory 902 , and a bus 903 connecting different system components (including the memory 902 and the processor 901 ).

Bus 903 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a processor, or a local bus using any of a variety of bus structures.

Memory 902 may include readable media in the form of volatile memory, such as random access memory (RAM) 9021 and/or cache memory 9022 , and may further include read only memory (ROM) 9023 .

The memory 902 may also include a program/utility 9025 having a set (at least one) of program modules 9024 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, which An implementation of a network environment may be included in each or some combination of the examples.

Electronic device 900 may also communicate with one or more external devices 904 (eg, keyboards, pointing devices, etc.), may also communicate with one or more devices that enable a user to interact with electronic device 900, and/or communicate with the electronic device 900 can communicate with any device (eg, router, modem, etc.) that communicates with one or more other electronic devices. Such communication may take place through input/output (I/O) interface 905 . Also, the electronic device 900 may communicate with one or more networks (eg, a local area network (LAN), a wide area network (WAN), and/or a public network such as the Internet) through a network adapter 906 . As shown, network adapter 906 communicates with other modules for electronic device 900 via bus 903 . It should be understood that, although not shown, other hardware and/or software modules may be used in conjunction with electronic device 900, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives and data backup storage systems.

In an exemplary embodiment, a computer-readable storage medium including instructions is also provided, such as a memory 902 including instructions, and the instructions can be executed by the processor 901 to complete the method for determining the fusion error of the three-dimensional model and the two-dimensional image. . Alternatively, the storage medium may be a non-transitory computer-readable storage medium, for example, the non-transitory computer-readable storage medium may be ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage equipment, etc.

In an exemplary embodiment, a computer program product is also provided, including a computer program that, when executed by the processor 901, implements any one of the methods for determining fusion errors of a three-dimensional model and a two-dimensional image provided by the present application method.

In an exemplary embodiment, various aspects of the method for determining a fusion error of a three-dimensional model and a two-dimensional image provided by the present application can also be implemented in the form of a program product, which includes program code, and when the program product is stored in a computer device When running on the computer, the program code is used to cause the computer device to execute the steps in the method for determining the fusion error of the three-dimensional model and the two-dimensional image described above in the present specification according to various exemplary embodiments of the present application.

The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or a combination of any of the above. More specific examples (non-exhaustive list) of readable storage media include: electrical connections with one or more wires, portable disks, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), optical fiber, portable compact disk read only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

The program product for the determination method of the fusion error of the three-dimensional model and the two-dimensional image of the embodiment of the present application may adopt a portable compact disk read only memory (CD-ROM) and include program codes, and may be executed on an electronic device. However, the program product of the present application is not limited thereto, and in this document, a readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

A readable signal medium may include a propagated data signal in baseband or as part of a carrier wave, carrying readable program code therein. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. A readable signal medium can also be any readable medium, other than a readable storage medium, that can transmit, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.

Program code embodied on a readable medium may be transmitted using any suitable medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

Program code for performing the operations of the present application may be written in any combination of one or more programming languages, including object-oriented programming languages—such as Java, C++, etc., as well as conventional procedural programming Language - such as the "C" language or similar programming language. The program code may execute entirely on the user's electronic device, partly on the user's device, as a stand-alone software package, partly on the user's electronic device and partly on a remote electronic device, or entirely on the remote electronic device or service Execute on the end. In the case of remote electronic equipment, the remote electronic equipment may be connected to the user electronic equipment through any kind of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to external electronic equipment (eg, using an Internet service provider business via an Internet connection).

It should be noted that although several units or sub-units of the apparatus are mentioned in the above detailed description, this division is merely exemplary and not mandatory. Indeed, according to embodiments of the present application, the features and functions of two or more units described above may be embodied in one unit. Conversely, the features and functions of one unit described above may be further subdivided to be embodied by multiple units.

Furthermore, although the operations of the methods of the present application are depicted in the figures in a particular order, this does not require or imply that the operations must be performed in the particular order, or that all illustrated operations must be performed to achieve desirable results. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined to be performed as one step, and/or one step may be decomposed into multiple steps to be performed.

Similar parts between the embodiments provided in the present application may be referred to each other. The specific embodiments provided above are just a few examples under the general concept of the present application, and do not constitute a limitation on the protection scope of the present application. For those skilled in the art, any other implementations expanded according to the solution of the present application without creative work fall within the protection scope of the present application.

As will be appreciated by those skilled in the art, the embodiments of the present application may be provided as a method, a system, or a computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to the present application. It will be understood that each flow and/or block in the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to the processor of a general purpose computer, special purpose computer, embedded processor or other programmable data processing device to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing device produce Means for implementing the functions specified in a flow or flow of a flowchart and/or a block or blocks of a block diagram.

These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory result in an article of manufacture comprising instruction means, the instructions The apparatus implements the functions specified in the flow or flow of the flowcharts and/or the block or blocks of the block diagrams.

These computer program instructions can also be loaded on a computer or other programmable data processing device to cause a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process such that The instructions provide steps for implementing the functions specified in the flow or blocks of the flowcharts and/or the block or blocks of the block diagrams.

Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims (10)

1. a method for determining the fusion error of a three-dimensional model and a two-dimensional image, wherein the method comprises: The two-dimensional image and three-dimensional model of biological tissue are fused and displayed after registration using the registration algorithm from the perspective of the virtual camera; acquiring the first outer contour of the biological tissue in the two-dimensional image, and acquiring the second outer contour of the biological tissue in the reference two-dimensional image obtained by projecting the three-dimensional model of the biological tissue onto the two-dimensional plane; determining the pixel distance between the first outer contour and the second outer contour; Using the predetermined correspondence between the unit pixel distance and the physical distance, the physical distance corresponding to the pixel distance between the first outer contour and the second outer contour is determined as the fusion error. 2. The method according to claim 1, wherein the determining the pixel distance between the first outer contour and the second outer contour comprises: Based on the first outer contour and the second outer contour, determine the pixel point pair used to describe the same position in the first outer contour and the second outer contour; Based on the pixel point pairs, an average pixel distance error of the first outer contour and the second outer contour is determined as a pixel distance between the first outer contour and the second outer contour. 3 . The method according to claim 2 , wherein determining the first outer contour and the second outer contour based on the first outer contour and the second outer contour is used for describing the Pairs of pixels at the same location, including: Calculate the context information corresponding to each pixel in the first outer contour and the second outer contour; the context information is the neighborhood structure of the pixel; Calculate the cost value of any two pixels in the first outer contour and the second outer contour; A group of pixel point pairs with the lowest overall cost value of the first outer contour and the second outer contour is counted by using the Hungarian algorithm; wherein the group of pixel point pairs includes a plurality of pixel point pairs at the same position. 4. The method according to claim 2, wherein the determining the average pixel distance error of the first outer contour and the second outer contour based on the pixel point pair comprises: Under the same coordinate system, determine the coordinates of each pixel in the first outer contour and the second outer contour; the same coordinate system includes X-axis coordinates and Y-axis coordinates; Subtract the coordinates on the X axis of each pixel pair in the first outer contour and the second outer contour to obtain the pixel distance error of all pixel pairs on the X axis; Divide the pixel distance error of all pixel point pairs on the X axis by the number of pixel point pairs on the X axis to obtain the average pixel distance error on the X axis; Subtract the coordinates on the Y-axis of each pixel pair in the first outer contour and the second outer contour to obtain the pixel distance error of all pixel pairs on the Y-axis; The average pixel distance error on the Y axis is obtained by dividing the pixel distance error of all pixel point pairs on the Y axis by the number of pixel point pairs on the Y axis. 5. The method according to claim 4, wherein determining the correspondence between the unit pixel distance and the physical distance, specifically comprising: determining the minimum three-dimensional coordinate value and the maximum three-dimensional coordinate value of the biological tissue in the three-dimensional model in the world coordinate system; Under the same viewing angle of the virtual camera, project the three-dimensional coordinate minimum value and the three-dimensional coordinate maximum value to a two-dimensional plane to obtain two-dimensional coordinates corresponding to the three-dimensional coordinate minimum value and the three-dimensional coordinate maximum value; Based on the world coordinates of the minimum three-dimensional coordinate and the maximum three-dimensional coordinate in the world coordinate system, calculate the physical distance between the minimum three-dimensional coordinate and the maximum three-dimensional coordinate; Based on the two-dimensional coordinates corresponding to the three-dimensional coordinate minimum value and the three-dimensional coordinate maximum value, calculate the pixel distance between the two-dimensional coordinates corresponding to the three-dimensional coordinate minimum value and the three-dimensional coordinate maximum value; Based on the physical distance between the three-dimensional coordinate minimum value and the three-dimensional coordinate maximum value and the pixel distance between the two-dimensional coordinates corresponding to the three-dimensional coordinate minimum value and the three-dimensional coordinate maximum value, determine the X-axis in the two-dimensional coordinate system. The physical distance corresponding to the unit pixel distance, and the physical distance corresponding to the unit pixel distance on the Y axis. 6. The method according to claim 5, wherein the determining the physical distance corresponding to the pixel distance between the first outer contour and the second outer contour specifically comprises: Multiply the average pixel distance error on the X axis by the physical distance corresponding to the unit pixel distance on the X axis to obtain the distance between the first outer contour and the second outer contour on the X axis. The physical distance corresponding to the pixel distance; Multiply the average pixel distance error on the Y-axis by the physical distance corresponding to the unit pixel distance on the Y-axis to obtain the distance between the first outer contour and the second outer contour on the Y-axis. The physical distance corresponding to the pixel distance; The first outer contour is obtained based on the physical distance corresponding to the pixel distance on the X axis and the physical distance corresponding to the pixel distance on the Y axis between the first outer contour and the second outer contour the physical distance corresponding to the pixel distance between the contour and the second outer contour; The physical distance corresponding to the pixel distance between the first outer contour and the second outer contour is used as the fusion error. 7 . The method according to claim 1 , wherein, acquiring the first outer contour of the biological tissue in the two-dimensional image, and acquiring a reference two obtained by projecting the three-dimensional model of the biological tissue onto a two-dimensional plane. 8 . The second outer contour of the biological tissue in the dimensional image, specifically including: Using an artificial intelligence algorithm to obtain the first outer contour of the biological tissue in the two-dimensional image; Create a virtual window with the same video playback size as the two-dimensional image playback size, and use the virtual camera on the virtual window to project the three-dimensional model of the biological tissue on the virtual camera from the same viewing angle. two-dimensional plane; acquiring a two-dimensional image of the two-dimensional plane to obtain the reference two-dimensional image; Edge detection is performed on the biological tissue in the reference two-dimensional image to obtain a second outer contour of the biological tissue. 8. The method according to claim 7, wherein before the determining the pixel distance between the first outer contour and the second outer contour, the method further comprises: Using a curve data compression algorithm, the first outer contour and the second outer contour are down-sampled to obtain the first outer contour and the second outer contour with the same number of pixel points. 9. An electronic device, comprising a processor and a memory: the memory for storing a computer program executable by the processor; The processor is connected to the memory and is configured to execute the instructions to implement the method for determining a fusion error of a three-dimensional model and a two-dimensional image according to any one of claims 1-8. 10. A computer-readable storage medium, characterized in that, when the instructions in the computer-readable storage medium are executed by an electronic device, the electronic device is enabled to implement any one of claims 1-8 A method for determining the fusion error of a 3D model and a 2D image.

Images