CN114693640A - A method, system, electronic device and storage medium for classifying and processing ultrasound image lesion attributes based on video sequence - Google Patents

Abstract

The invention discloses an ultrasonic image focus attribute classification processing method, system, electronic equipment and storage medium based on a video sequence, and belongs to the technical field of intelligent medical diagnosis. The method avoids the mode of directly judging the attribute category by a target detector, firstly determines the position of a focus and the position of the focus in order to adapt to the attribute classification scenes of different conditions, acquires the tracking sequence of the focus target under different frames, then improves the diagnosis accuracy on difficult attributes according to the focus target attribute category and the integral confidence corresponding to the focus target attribute category, and then improves the accuracy of the focus attribute diagnosis result by integrating multi-frame result information.

Description

A method, system, electronic device and storage medium for classifying and processing ultrasound image lesion attributes based on video sequence

technical field

The invention belongs to the technical field of intelligent medical diagnosis, and in particular relates to a method, system, electronic device and storage medium for classifying and processing the attributes of ultrasonic image lesions based on video sequences.

Background technique

The classification of lesion attributes in ultrasound images refers to further judging the attributes of the lesions on the specific localized lesions. For example, the orientation attributes of thyroid nodules include vertical and horizontal attributes; the posterior echo attributes include echo enhancement and echo uniformity. Judging the attributes of the lesions is of great significance for doctors to give the specific diagnosis results of the lesions in the later stage.

At present, most of the ultrasonic image automatic diagnosis technologies are generally based on a single frame image for identification. This way of working requires doctors to find the best viewing angle, interact with the ultrasound equipment, fix the ultrasound image, and then start the automatic diagnosis software to identify the lesions. Obviously, this method has a strong dependence on the doctor's scanning technique. The better the scanning angle, the higher the diagnostic accuracy. In addition, when diagnosing a lesion, a doctor often needs to synthesize information from different angles of observation of the lesion to give a diagnosis result, so as to improve the accuracy of the diagnosis. However, this method cannot synthesize scanning information from different angles, and the judgment given is one-sided to a certain extent. At present, some technologies have been partially improved. For example, by increasing the speed of the model, the algorithm can make a diagnosis on the scanned image in real time and give the diagnosis result. Although this method can give the diagnosis results in real time, due to the amount of training data and the fitting difference of the model itself, the model has a certain error rate, and the diagnosed attribute classification will be unstable, which will affect the diagnosis of doctors. The result is some disturbance.

SUMMARY OF THE INVENTION

Aiming at the problem that there is an error in the attribute category output by the current diagnosis software, which affects the diagnosis result, the present invention avoids the method of directly judging the attribute category by the target detector. position, obtain the tracking sequence of the lesion target in different frames, and then classify the lesion attribute according to the lesion target attribute category and the overall confidence level corresponding to the lesion target attribute category, so as to improve the diagnosis accuracy rate on difficult attributes, It imitates the doctor's diagnosis process, integrates multi-frame result information, gives the final diagnosis result, and improves the accuracy of the diagnosis result of the lesion attribute.

In order to achieve the above objects, the present invention adopts the following technical solutions.

A first aspect of the present invention provides a method for classifying and processing ultrasound image lesion attributes based on a video sequence, the method comprising:

S102: Obtain an ultrasound image to be tested, and extract a lesion image at one or more locations in the ultrasound image to be tested as a lesion target;

S104: Track the lesion target in the ultrasound image to be tested, and obtain multiple sets of tracking sequences corresponding to the lesion target;

S106: Acquire attribute information of the lesion target in the tracking sequence, and add the attribute information of the lesion target to the tracking sequence;

S108 : Acquire attribute information of the current lesion target in the tracking sequence, calculate the overall confidence level corresponding to the lesion target in the tracking sequence; determine the attribute category of the lesion target according to the overall confidence level, and output.

As a preferred solution, the step S102 includes:

Use the target detection model to extract the image of one or more lesions in the ultrasound image to be tested as the target of the lesion, and obtain the lesion position and type of the lesion of the target; the lesion detection model uses the annotated bounding box samples to perform train.

As a preferred solution, the step S106 includes:

Identify the attribute category of the lesion target in the tracking sequence by using the classifier model, and calculate the confidence level of the target attribute category corresponding to the attribute category of the lesion target;

The attribute confidence of the lesion target in the ultrasound image, the bounding box of the lesion target, and the attribute information category of the lesion target are all added to the tracking sequence.

As a preferred solution, the step S108 further includes:

According to the high attribute classification confidence of the lesion target, the different attribute categories of the two frames in the ultrasound image, and the overlap of the bounding boxes of the two frames in the ultrasound image, the overall confidence mathematical model was constructed;

Calculate the target attribute category of the lesion and an overall confidence level corresponding to the target attribute category of the lesion.

As a preferred solution, the overall confidence is compared with a first threshold and a second threshold, wherein the first threshold is greater than the second threshold;

If the overall confidence level is greater than the first threshold, placing the target attribute category of the lesion corresponding to the overall confidence level in an output queue;

If the overall confidence is less than the first threshold but greater than the second threshold, retaining the target attribute category of the lesion corresponding to the overall confidence in the tracking sequence;

If the overall confidence level is less than a second threshold, the lesion target attribute category corresponding to the overall confidence level is deleted from the tracking sequence.

As a preferred solution, the construction of the overall confidence mathematical model is as follows:

表示跟踪序列中病灶目标是否连续出现的赋值；

表示病灶目标属性置信度权重；

表示病灶目标重叠度权重；γ表示权重偏置。Among them, w ^j represents the overall confidence corresponding to the lesion target in the tracking sequence;

An assignment indicating whether the target of the lesion appears consecutively in the tracking sequence;

Represents the confidence weight of the target attribute of the lesion;

Represents the weight of the overlap of the lesion target; γ represents the weight bias.

A second aspect of the present invention provides a system for classifying and processing ultrasound image lesion attributes based on a video sequence, comprising:

a lesion identification module, which is used for obtaining an ultrasound image to be tested, and extracting a lesion image at one or more locations in the ultrasound image to be tested as a lesion target;

a lesion tracking module, which is used to track the lesion target in the ultrasound image to be tested, and obtain multiple sets of tracking sequences corresponding to the lesion target;

an attribute identification module, configured to acquire attribute information of the lesion target in the tracking sequence, and add the attribute information of the lesion target to the tracking sequence;

a judgment module, which is used for acquiring attribute information of the current lesion target in the tracking sequence, calculating the overall confidence level corresponding to the lesion target in the tracking sequence; determining the attribute category of the lesion target according to the overall confidence level, and outputting it.

As a preferred solution, the judging module includes a first judging unit, a second judging unit and a third judging unit;

A first judging unit, configured to compare the overall confidence with a first threshold, where the first threshold is greater than the second threshold; if it is greater than the first threshold, place the target attribute category of the lesion corresponding to the overall confidence in the output queue;

a second judgment unit, configured to compare the overall confidence with a first threshold and a second threshold, if the overall confidence is less than the first threshold but greater than the second threshold, it will correspond to the overall confidence The lesion target attribute category is retained in the tracking sequence;

a third judging unit, configured to compare the overall confidence with a second threshold, and if the overall confidence is less than the second threshold, remove the target attribute category of the lesion corresponding to the overall confidence from the tracking sequence delete.

A third aspect of the present invention provides an electronic device, including a processor, an input device, an output device, and a memory, wherein the processor, the input device, the output device, and the memory are connected in sequence, and the memory is used to store a computer program, and the computer The program includes program instructions, the processor being configured to invoke the program instructions to perform the method as described above.

A fourth aspect of the present invention provides a readable storage medium storing a computer program, the computer program including program instructions, the program instructions, when executed by a processor, cause the processor to perform the method as described above .

Compared with the prior art, the beneficial effects of the present invention are:

The embodiment of the present invention avoids the method of directly judging the attribute category by the target detector. In order to be able to adapt to the attribute classification scenarios of different situations, the lesion and the location of the lesion are firstly determined, and the tracking sequence of the lesion target in different frames is obtained. The target attribute category of the lesion and the confidence level corresponding to the target attribute category of the lesion, the method of classifying the lesion attribute, improving the diagnosis accuracy rate on the difficult attribute, imitating the diagnosis process of the doctor, synthesizing the multi-frame result information, and giving the final diagnosis As a result, the accuracy of the diagnosis results of the lesion attribute is improved; in addition, in the process of constructing the confidence mathematical model, the first one takes the confidence factor of the classifier into account, so that the target with high attribute classification confidence contributes more, and the second is the modeling of the bounding box. The process considers that the target shape change in two consecutive frames should not be large. If it is too large, the probability that the two belong to the same target will be lower, and the contribution to the overall confidence will be lower. Third, if the two frames before and after the target do not belong to the same target An attribute category, the lower the probability of correct classification of this attribute, the contribution of the overall confidence should be reduced. Through these three points to ensure the stability and continuity of the confidence of the final tracking sequence, the accuracy of the classification results can be greatly improved.

Description of drawings

The above and other objects, features and advantages of the present application will become more apparent by describing the embodiments of the present application in more detail in conjunction with the accompanying drawings. The accompanying drawings are used to provide a further understanding of the embodiments of the present application, constitute a part of the specification, and are used to explain the present application together with the embodiments of the present application, and do not constitute a limitation to the present application. In the drawings, the same reference numbers generally refer to the same components or steps. In the attached picture:

FIG. 1 is a flowchart of a method for classifying and processing ultrasound image lesion attributes based on a video sequence provided by an embodiment of the present invention;

2 is an ultrasound image of a thyroid nodule provided by an embodiment of the present invention;

FIG. 3 is a block diagram of a system for classifying and processing ultrasound image lesion attributes based on a video sequence according to an embodiment of the present invention;

4 illustrates a block diagram of an electronic device according to an embodiment of the present application;

FIG. 5 is a structural block diagram of an improved classifier structure provided by an embodiment of the present invention.

Detailed ways

Hereinafter, exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited by the example embodiments described herein.

Exemplary method

As shown in FIG. 1 , this example discloses a method for classifying and processing ultrasound image lesion attributes based on a video sequence, and the method includes the following steps:

S102: Obtain an ultrasound image to be tested, and extract a lesion image at one or more locations in the ultrasound image to be tested as a lesion target.

Specifically, the ultrasonic image to be tested in this example is to scan the human body with ultrasonic sound beams, and obtain images of internal organs or a certain characteristic area by receiving and processing reflected signals. The ultrasound image to be tested can be the ultrasound image of the thyroid gland and breast, and the method of acquiring the ultrasound image can be taken by a medical imaging device based on ultrasound (ultrasound), such as reading the video stream of the display interface of the ultrasound device, and dividing the video stream by frame. Decode into a continuous thyroid ultrasound image for the unit. It can also be acquired in non-real time, for example, receiving thyroid ultrasound images pre-stored in a device such as a server, or receiving thyroid ultrasound images transmitted from other devices. Figure 2 shows a thyroid nodule with a hypoechoic image.

To extract the lesion image of the ultrasound image to be tested, a target detection model based on deep learning is used to first identify the type of the lesion in the ultrasound image, and then accurately locate the position of the lesion. The target detection model here is, for example, yolo-v5, faster-rcnn and other detection network models. It should be noted that this example requires a bounding box as the subsequent calculation of the overall confidence, so it needs to be trained by using ultrasound image samples with annotated bounding boxes. For example, a thyroid nodule detector needs to use a large number of bounding boxes containing thyroid nodules. The samples of the neural network are trained so that the detector has the ability to detect thyroid nodules.

S104: Track the lesion target in the ultrasound image to be tested, and obtain multiple sets of tracking sequences corresponding to the lesion target.

Specifically, the target tracker is used to track the lesion target, and the target tracker is used to track a plurality of the acquired lesion targets. At this time, the lesion target becomes the tracking target of the target tracker, and Obtain the trajectory of the tracking target within the time t1-t2 and the tracking target result to obtain a tracking queue. It should be understood that an object detector is used to detect whether the predicted position of the next frame is an object, and then the training set is updated with the new detection results to update the object detector. When training the target detector, the target area is generally selected as a positive sample, and the surrounding area of the target is a negative sample. Of course, the area closer to the target is more likely to be a positive sample.

Here, target trackers such as IOU and KCF are used to track the detected lesion targets. When the IOU single target tracker is used, it is mainly based on the overlap of the bounding boxes of the two targets before and after to determine whether the two belong to the same target. of. When the KCF single target tracker is used, it mainly judges whether the two belong to the same target according to the similarity of the target features in the bounding boxes of the front and rear targets. Due to the same moment, in one ultrasound image, multiple different lesions may be detected. The single target tracker can determine that the multiple target targets detected in the previous and subsequent frames are the same target target during the continuous ultrasound scanning process.

S106: Acquire attribute information of the lesion target in the tracking sequence, and add the attribute information of the lesion target to the tracking sequence.

Specifically, the classifier model is used to identify the attribute category of the latest lesion target in the tracking sequence, and the confidence level of the target attribute category corresponding to the attribute category of the lesion target is calculated; the classifier here can use resnet, vgg, inception, etc. Mainstream target classification network; the attribute confidence of the lesion target in the ultrasound image, the bounding box of the lesion target and the attribute information category of the lesion target are added to the tracking sequence.

S108 : Acquire attribute information of the current lesion target in the tracking sequence, calculate the overall confidence level corresponding to the lesion target in the tracking sequence; determine the attribute category of the lesion target according to the overall confidence level, and output.

Specifically, the overall confidence mathematical model is pre-built here, and then the overall confidence corresponding to the lesion target in the tracking sequence is calculated, and the attribute category of the lesion target is judged according to the overall confidence, so as to output the attribute category of the current lesion target, which should be Understand that the judgment based on the overall confidence level here can be realized through a single threshold or through multiple threshold control, so as to improve the output accuracy of the attribute category of the current lesion target.

This example can adapt to attribute classification scenarios in different situations. First, the lesion and the location of the lesion are determined, and the tracking sequence of the lesion target in different frames is obtained. Then, according to the attribute category of the lesion target and the overall confidence corresponding to the attribute category of the lesion target It can improve the accuracy of diagnosis on difficult attributes, imitate the diagnosis process of doctors, integrate multi-frame result information, give the final diagnosis results, and improve the stability of the diagnosis results of lesion attributes. The difficult attribute here refers to the fact that some attributes are difficult to judge the attribute type from a single ultrasound image, and need to integrate multiple frames of information to judge, that is, to combine the tracking sequence of the lesion image and the corresponding attribute characteristics, such as internal calcification attributes, In some angles, calcification points cannot be observed, and different angles of observation and judgment are required.

As a preferred embodiment, this example controls the accuracy of the attribute category of the lesion target by setting multiple sets of thresholds.

Firstly, the influence of the attribute information of the lesion target on the overall confidence is considered, and the overall confidence is mathematically constructed according to the high attribute classification confidence of the lesion target, the different attribute categories of the two frames before and after the ultrasound image, and the overlap of the bounding boxes of the two frames before and after the ultrasound image. A model; calculating the target attribute category of the lesion and the overall confidence level corresponding to the target attribute category of the lesion.

Specifically, it is assumed that the target tracker in step S106 outputs the tracking sequences corresponding to N lesion targets, T={t ₁ ,t ₂ ,...,t _N }, T represents the target tracking sequence, and each target tracking sequence in The number of targets is D={d ₁ , d ₂ ,...,d _N }; according to the results of the target detector and the target classifier, each target in the target tracking sequence has three kinds of information: the lesion target attribute category C _j , the lesion target The bounding box b _j , the confidence level P _j of the target attribute category of the lesion, where j is the j-th target in a certain tracking sequence.

Then the process of constructing the confidence mathematical model is considered as follows:

(1) Considering that the higher the confidence of the target attribute category, the higher the judgment accuracy of the target attribute category will be, so the logarithm is used to transform the confidence. Since the confidence cannot be a negative value, the max value is used for the result.

表示病灶目标属性置信度权重，ɑ为一个固定的权重值，P_j表示病灶目标属性类别置信度。in,

Represents the confidence weight of the target attribute of the lesion, ɑ is a fixed weight value, and P _j represents the confidence of the attribute category of the lesion target.

(2) Considering that the change rate of the bounding box of the lesion target in several consecutive video frames of the image should not be very large, the IOU is the intersection over union of the two bounding boxes, which calculates two bounding boxes. The ratio of the intersection and union of the bounding boxes is mainly used to quantify the overlap of the target bounding boxes before and after the frame. β is a fixed weight value, and the sum of ɑ and β is 1.

表示病灶目标重叠度权重。b _j-1 represents the coordinate information (x ₁ , y ₁ , x ₂ , y ₂ ) of the j-1th target bounding box in the tracking sequence of the lesion target, and x ₁ , y ₁ represent the horizontal and vertical coordinates of the upper left corner of the bounding box , x ₂ , y ₂ represent the horizontal and vertical coordinates of the lower right corner of the bounding box; b _j represents the coordinate information of the j-th target bounding box in the lesion target sequence;

Indicates the weight of the lesion target overlap degree.

(3) Considering that the attribute categories of the lesion target appearing continuously in the continuous video frames of the image are the same, it is more likely to be the expected output attribute category, while the continuous occurrence of the target attribute has a lower probability of the expected output attribute. Therefore, the following ways are adopted to distinguish the contributions of the two.

表示跟踪序列中病灶目标是否连续出现的赋值。c _j-1 represents the attribute category of the j-1th target in the target sequence,

An assignment representing whether the foci targets appear consecutively in the tracking sequence.

If an attribute category does not appear consecutively in the next frame, the overall confidence is reduced by a fixed value γ.

Combining the above three points, the first confidence model takes into account the confidence factor of the classifier, so that the target contribution of high attribute classification confidence is greater. The second bounding box modeling process considers that the target shape change in two consecutive frames should not be large. If it is too large, the probability that the two belong to the same target will be lower, and the contribution to the overall confidence will be lower. The third rule is that if the objects in the two frames before and after do not belong to the same attribute category, the probability of correct classification of this attribute is lower, and the contribution of the overall confidence should be reduced. These three points ensure the stability and continuity of the confidence of the final tracking sequence, which can greatly improve the accuracy of the classification results.

Then the overall confidence of the target sequence is modeled as follows:

表示跟踪序列中病灶目标是否连续出现的赋值；

表示病灶目标属性置信度权重；

表示病灶目标重叠度权重；γ表示权重偏置。Among them, w ^j represents the overall confidence corresponding to the lesion target in the tracking sequence;

An assignment indicating whether the target of the lesion appears consecutively in the tracking sequence;

Represents the confidence weight of the target attribute of the lesion;

Represents the weight of the overlap of the lesion target; γ represents the weight bias.

It should be noted that after the confidence of different attributes is calculated, the attribute category with the largest overall confidence is selected for output. After the overall confidence is output, a threshold needs to be set for comparison. In this example, the first threshold thr can be set according to the attribute category of the lesion target, for example, 5, 6, 7, or 8; the second threshold is 0.

comparing the overall confidence with a first threshold and a second threshold, wherein the first threshold is greater than the second threshold;

If it is greater than the first threshold, the target attribute category of the lesion corresponding to the overall confidence level is placed in the output queue; the result is outputted.

If the overall confidence level is less than the first threshold but greater than the second threshold, the target attribute category of the lesion corresponding to the overall confidence level is retained in the tracking sequence, which is convenient for later calculation of the overall confidence level.

If the overall confidence level is less than a second threshold, the lesion target attribute category corresponding to the overall confidence level is deleted from the tracking sequence.

Specifically, according to the above calculation of the overall confidence, the attribute target of the 0th category has not appeared in multiple consecutive frames, and the attribute confidence of this category is less than 0; the attribute target of the first category has appeared in three consecutive frames, and the attribute confidence of this category is 5.1, The attribute target of class 2 does not appear for three consecutive frames, and the attribute confidence of this class is 1.2. After the overall judgment, the confidence of the first category is the highest, and it is greater than the threshold of 3.0, then its output is displayed. The 0th class attribute is less than 0, so all the targets of this attribute in this queue are deleted. The second type of target confidence is less than the threshold 3.0, but greater than 0, it is still kept in the queue; after processing, the first target sequence queue is updated to {1,1,2,1,1,1}.

As another preferred implementation, the classifier model in this example is different from the traditional classifier. The traditional classifier can only classify a single attribute. Here is an improved multi-class classifier based on the traditional classifier. Task classifier, which can classify different attributes at the same time. Such as nodule echo, nodule edge two attributes. The nodule echo was subdivided into hyperechoic, hypoechoic and anechoic, and the nodule boundary was divided into clear and unclear.

As shown in Figure 5, the specific classifier structure is improved as follows, in which CNN is the feature extraction network of the recognition network, which is the same as the feature extraction network of the traditional deep classifier, and the FC is the fully connected layer, which is the same as the fully connected layer of the traditional deep classifier. . The final output category. If you want to identify two attributes, such as nodule boundary, nodule echo two attributes. The pre-improvement classifier requires two separate CNN networks, trained separately using their respective attribute data. The improved classifier requires only one CNN network and can be trained using both attribute data simultaneously. In this way, the CNN network of the improved classifier can learn the information of the two kinds of data. It is also proved by experiments that under the same amount of data, the improved classifier is more accurate than the pre-improved classifier on the respective attributes. Sex increased by 6%.

In this example, the traditional classifier model has been transformed into the network structure, so that it has the function of multi-task recognition at the same time. The advantage of this is that different tasks can promote each other, enhance the ability of the backbone network in the classifier to extract the features of the picture, and achieve the recognition ability of the goal of mutual promotion and improvement.

Exemplary System

As shown in FIG. 3 , a system for classifying and processing ultrasound image lesion attributes based on a video sequence is characterized in that it includes:

A lesion identification module 20, which is used for obtaining an ultrasound image to be tested, and extracting the lesion image at one or more locations in the ultrasound image to be tested as a lesion target;

a lesion tracking module 30, which is used for tracking the lesion target in the ultrasound image to be measured, and obtaining multiple sets of tracking sequences corresponding to the lesion target;

an attribute identification module 40, configured to acquire attribute information of the lesion target in the tracking sequence, and add the attribute information of the lesion target to the tracking sequence;

The judgment module 50 is configured to obtain the attribute information of the current lesion target in the tracking sequence, calculate the overall confidence level corresponding to the lesion target in the tracking sequence; determine the attribute category of the lesion target according to the overall confidence level, and output.

The judging module 50 includes a first judging unit, a second judging unit and a third judging unit;

A first judging unit, configured to compare the overall confidence with a first threshold, where the first threshold is greater than the second threshold; if it is greater than the first threshold, place the target attribute category of the lesion corresponding to the overall confidence in the output queue;

a second judgment unit, configured to compare the overall confidence with a first threshold and a second threshold, if the overall confidence is less than the first threshold but greater than the second threshold, it will correspond to the overall confidence The lesion target attribute category is retained in the tracking sequence;

a third judging unit, configured to compare the overall confidence with a second threshold, and if the overall confidence is less than the second threshold, remove the target attribute category of the lesion corresponding to the overall confidence from the tracking sequence delete.

In order to adapt to attribute classification scenarios in different situations, this example firstly determines the lesion and the location of the lesion, obtains the tracking sequence of the lesion target in different frames, and then uses the lesion target attribute category and the confidence corresponding to the lesion target attribute category. It can improve the accuracy of diagnosis on difficult attributes, imitate the diagnosis process of doctors, integrate multi-frame result information, give the final diagnosis results, and improve the stability of the diagnosis results of lesion attributes.

Exemplary Electronics

Hereinafter, an electronic device according to an embodiment of the present application will be described with reference to FIG. 4 . The electronic device may be the movable device itself, or a stand-alone device independent of it, which may communicate with the movable devices to receive collected input signals from them and transmit to them selected target decision-making behaviors.

FIG. 4 illustrates a block diagram of an electronic device according to an embodiment of the present application.

As shown in FIG. 4 , the electronic device 10 includes one or more processors 11 and a memory 12 .

Processor 11 may be a central processing unit (CPU) or other form of processing unit having data processing capabilities and/or instruction execution capabilities, and may control other components in electronic device 10 to perform desired functions.

Memory 12 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and/or cache memory, or the like. The non-volatile memory may include, for example, read only memory (ROM), hard disk, flash memory, and the like. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 11 may execute the program instructions to implement the decision-making behavior decision method and/or the above-described various embodiments of the present application Other desired features.

In one example, the electronic device 10 may also include an input device 13 and an output device 14 interconnected by a bus system and/or other form of connection mechanism (not shown). For example, the input device 13 may include various devices such as a camera, CT, MRI, and ultrasound imaging devices. The input device 13 may also include, for example, a keyboard, a mouse, and the like. The output device 14 may include, for example, displays, speakers, printers, and communication networks and their connected remote output devices, among others.

Of course, for simplicity, only some of the components in the electronic device 10 related to the present application are shown in FIG. 4 , and components such as buses, input/output interfaces, and the like are omitted. Besides, the electronic device 10 may also include any other suitable components according to the specific application.

Exemplary computer program product and computer readable storage medium

In addition to the methods and apparatuses described above, embodiments of the present application may also be computer program products comprising computer program instructions that, when executed by a processor, cause the processor to perform the "exemplary methods" described above in this specification The steps in the decision-making method according to various embodiments of the present application described in the section.

The computer program product can write program codes for performing the operations of the embodiments of the present application in any combination of one or more programming languages, including object-oriented programming languages, such as Java, C++, etc. , also includes conventional procedural programming languages, such as "C" language or similar programming languages. The program code may execute entirely on the user computing device, partly on the user device, as a stand-alone software package, partly on the user computing device and partly on a remote computing device, or entirely on the remote computing device or server execute on.

In addition, embodiments of the present application may also be computer-readable storage media having computer program instructions stored thereon, the computer program instructions, when executed by a processor, cause the processor to perform the above-mentioned "Example Method" section of this specification Steps in the decision-making method according to various embodiments of the present application described in .

The computer-readable storage medium may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may include, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses or devices, or a combination of any of the above. More specific examples (non-exhaustive list) of readable storage media include: electrical connections with one or more wires, portable disks, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), optical fiber, portable compact disk read only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

The basic principles of the present application have been described above in conjunction with specific embodiments. However, it should be pointed out that the advantages, advantages, effects, etc. mentioned in the present application are only examples rather than limitations, and these advantages, advantages, effects, etc., are not considered to be Required for each embodiment of this application. In addition, the specific details disclosed above are only for the purpose of example and easy understanding, rather than limiting, and the above-mentioned details do not limit the application to be implemented by using the above-mentioned specific details.

The block diagrams of devices, apparatus, apparatuses, and systems referred to in this application are merely illustrative examples and are not intended to require or imply that the connections, arrangements, or configurations must be in the manner shown in the block diagrams. As those skilled in the art will appreciate, these means, apparatuses, apparatuses, systems may be connected, arranged, configured in any manner. Words such as "including", "including", "having" and the like are open-ended words meaning "including but not limited to" and are used interchangeably therewith. As used herein, the words "or" and "and" refer to and are used interchangeably with the word "and/or" unless the context clearly dictates otherwise. As used herein, the word "such as" refers to and is used interchangeably with the phrase "such as but not limited to".

It should also be pointed out that in the apparatus, equipment and method of the present application, each component or each step can be decomposed and/or recombined. These disaggregations and/or recombinations should be considered as equivalents of the present application.

The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other aspects without departing from the scope of the application. Therefore, this application is not intended to be limited to the aspects shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

The foregoing description has been presented for the purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of the application to the forms disclosed herein. Although a number of example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims (10)

1. An ultrasound image focus attribute classification processing method based on a video sequence is characterized by comprising the following steps:

s102: obtaining an ultrasonic image to be detected, and extracting a focus image at one or more positions in the ultrasonic image to be detected as a focus target;

s104: tracking the focus target in an ultrasonic image to be detected to obtain a plurality of groups of tracking sequences corresponding to the focus target;

s106: acquiring the attribute information of a focus target in the tracking sequence, and adding the attribute information of the focus target into the tracking sequence;

s108: acquiring attribute information of a current focus target in the tracking sequence, and calculating the integral confidence degree corresponding to the focus target in the tracking sequence; and determining the attribute category of the lesion target according to the overall confidence and outputting the attribute category.

2. The method for classifying and processing the lesion property of the ultrasound image based on the video sequence as claimed in claim 1, wherein the step S102 comprises:

extracting a focus image at one or more positions in the ultrasonic image to be detected by using a target detection model as a focus target, and acquiring a focus position and a focus type of the focus target; the lesion detection model is trained using labeled bounding box samples.

3. The method for classifying and processing the lesion property of the ultrasound image based on the video sequence as claimed in claim 1, wherein the step S106 comprises:

identifying the attribute category of the focus target in the tracking sequence by utilizing a classifier model, and calculating the confidence coefficient of the target attribute category corresponding to the attribute category of the focus target;

and adding the attribute confidence of the focus target in the ultrasonic image, the bounding box of the focus target and the attribute information category of the focus target into the tracking sequence.

4. The method as claimed in claim 3, wherein the step S108 further includes:

constructing an integral confidence mathematical model according to the high attribute classification confidence of the focus target, the different attribute categories of the front frame and the rear frame in the ultrasonic image and the overlapping degree of surrounding frames of the front frame and the rear frame in the ultrasonic image;

calculating the lesion target attribute category and an overall confidence corresponding to the lesion target attribute category.

5. The method according to claim 3, wherein the overall confidence is compared with a first threshold and a second threshold, wherein the first threshold is greater than the second threshold;

if the overall confidence is larger than a first threshold, placing the focus target attribute category corresponding to the overall confidence in an output queue;

if the overall confidence is less than a first threshold but greater than a second threshold, then retaining a lesion target attribute category corresponding to the overall confidence in the tracking sequence;

deleting a lesion target attribute category corresponding to the overall confidence from the tracking sequence if the overall confidence is less than a second threshold.

6. The method according to claim 4, wherein the overall confidence mathematical model is constructed as follows:

wherein, w^jRepresenting the integral confidence corresponding to the focus target in the tracking sequence;

an assignment indicating whether a focal object appears continuously in the tracking sequence;

representing a lesion target attribute confidence weight;

representing a focus target overlap degree weight; γ represents a weight bias.

7. An ultrasonic image focus attribute classification processing system based on a video sequence is characterized by comprising

The system comprises a focus identification module, a focus detection module and a focus detection module, wherein the focus identification module is used for obtaining an ultrasonic image to be detected and extracting focus images at one or more positions in the ultrasonic image to be detected as a focus target;

the focus tracking module is used for tracking the focus target in an ultrasonic image to be detected to obtain a plurality of groups of tracking sequences corresponding to the focus target;

the attribute identification module is used for acquiring the attribute information of the focus target in the tracking sequence and adding the attribute information of the focus target into the tracking sequence;

the judging module is used for acquiring the attribute information of the current focus target in the tracking sequence and calculating the integral confidence degree corresponding to the focus target in the tracking sequence; and determining the attribute category of the lesion target according to the overall confidence and outputting the attribute category.

8. The system of claim 7, wherein the determining module comprises a first determining unit, a second determining unit and a third determining unit;

the first judging unit is used for comparing the overall confidence with a first threshold, wherein the first threshold is larger than a second threshold; if the overall confidence is greater than the first threshold, placing the focus target attribute category corresponding to the overall confidence in an output queue;

a second judging unit, configured to compare the overall confidence with a first threshold and a second threshold, and if the overall confidence is smaller than the first threshold but larger than the second threshold, keep the lesion target attribute category corresponding to the overall confidence in the tracking sequence;

and the third judging unit is used for comparing the overall confidence with a second threshold, and deleting the focus target attribute category corresponding to the overall confidence from the tracking sequence if the overall confidence is smaller than the second threshold.

9. An electronic device comprising a processor, an input device, an output device, and a memory, the processor, the input device, the output device, and the memory being connected in series, the memory being configured to store a computer program comprising program instructions, the processor being configured to invoke the program instructions to perform the method of any of claims 1-6.

10. A readable storage medium, characterized in that the storage medium stores a computer program comprising program instructions which, when executed by a processor, cause the processor to carry out the method according to any one of claims 1-6.

Images