CN117642774A - Method, device and computer storage medium for automatically generating annotated images - Google Patents

Abstract

Methods, devices and computer storage media for automatically generating annotated images, the method includes: generating a three-dimensional scene model (201) based on one or more CAD models corresponding to one or more objects; based on the three-dimensional scene model, generating A set of two-dimensional projected images about one or more objects (202); automatically annotating each image in the two-dimensional projected image set to generate annotation information associated with unoccluded objects (203); summing the annotation information with Background images related to the production line of one or more objects are fused to generate an annotation image (204). The process of image generation and annotation can be automated by utilizing CAD models. Furthermore, the method can be automatically performed by a computer, thus avoiding errors, inaccuracies and lost markups.

Description

Method, device and computer storage medium for automatically generating annotated images Technical field

The present invention relates to the field of image processing, and more specifically, to methods, devices, computing devices, computer-readable storage media and computer program products for automatically generating annotated images.

Background technique

In recent years, artificial intelligence (AI) has received widespread attention and tremendous development. Various fields, including industry, are trying to leverage the power and domain knowledge of AI to solve different real-world problems, such as object recognition or detection in different scenarios. However, the development of AI applications always lags far behind the construction of production lines. Therefore, the value of AI is greatly limited by this delay, resulting in high time costs and low capacity utilization. One of the root causes is laborious data preparation, including data acquisition and data annotation. Data preparation plays a vital role as an important link in the AI application process. However, many factors in data preparation can hinder real-life AI applications.

One factor is the lack of scene-specific image data. This is especially true in industrial applications, where privacy is of paramount concern.

Another factor is that obtaining annotated image data is time-consuming and laborious. Since scene-specific image data is difficult to obtain easily, different cameras are used to capture the image data. This often consumes many labor resources and a lot of time. In addition, annotation is also a problem after capturing image data, because for different vision tasks, people need to manually annotate images to tell where different objects are in the image or what the objects are in the image. In fact, annotation usually takes no less time than the image capture process.

Yet another factor is incorrect, imprecise or missing markup. Due to the carelessness of annotators when encountering a large number of images, annotators may make errors, miss annotations, or have inaccurate annotations during the annotation process, which will affect the final performance of the trained AI model and hinder the practical application of AI.

Therefore, an improved solution for obtaining annotated images is urgently needed.

Contents of the invention

For specific scenarios in industrial applications, capturing and annotating images is a time-consuming and labor-intensive process. Although a variety of manual annotation tools have emerged, they have not solved the problem of how to obtain images, and users still need to spend a lot of time on annotation. Moreover, due to manual annotation, the accuracy of the annotation cannot be ensured. In addition, some companies are trying to automatically obtain image data sets for training and use robots to automatically capture images, but this does not solve the image annotation problem. In view of this, the present invention proposes a solution for automatically generating annotated images.

The first embodiment of the present disclosure proposes a method for automatically generating annotated images, which method includes:

Generate a three-dimensional scene model based on one or more CAD models, where each CAD model of the one or more CAD models corresponds to one of the one or more objects;

Based on the three-dimensional scene model, generate a two-dimensional projection image collection about one or more objects;

Automatically annotate each image in the set of two-dimensional projected images to generate annotation information associated with unoccluded objects, where the unoccluded objects are included in one or more objects; and

The annotation information is fused with background images related to the production line of one or more objects to generate an annotation image.

A second embodiment of the present disclosure proposes a device for automatically generating annotated images, which device includes:

a model unit configured to generate a three-dimensional scene model based on one or more CAD models, where each CAD model of the one or more CAD models corresponds to one of the one or more objects;

a projection unit configured to generate a set of two-dimensional projection images about one or more objects based on the three-dimensional scene model;

an annotation unit configured to automatically annotate each image in the two-dimensional projection image set to generate annotation information associated with an unoccluded object, wherein the unoccluded object is included in one or more objects; and

The fusion unit is configured to fuse the annotation information and the background image related to the production line of one or more objects to generate an annotation image.

A third embodiment of the present disclosure provides a computing device, which includes: a processor; and a memory for storing computer-executable instructions, which when executed, causes the processor to execute the first embodiment Methods.

A fourth embodiment of the present disclosure provides a computer-readable storage medium having computer-executable instructions stored thereon, the computer-executable instructions being used to perform the method of the first embodiment.

A fifth embodiment of the present disclosure proposes a computer program product that is tangibly stored on a computer-readable storage medium and includes computer-executable instructions that, when executed, cause at least one process The device executes the method of the first embodiment.

It can be seen from the above solution that because the present invention can automatically realize the process of image generation and image annotation by using CAD models, it significantly saves time and human resources, and can easily obtain a large amount of customized image data, thereby fully preparing for AI applications. data to train the AI model. Additionally, the image generation and annotation process can be automated by computers, thus avoiding errors, inaccuracies, and lost labeling.

Description of drawings

Features, advantages, and other aspects of embodiments of the present disclosure will become more apparent with reference to the following detailed description, taken in conjunction with the accompanying drawings, several embodiments of which are shown by way of illustration and not limitation, in which In the attached picture:

Figure 1 illustrates an exemplary three-dimensional scene model applicable in embodiments of the present disclosure;

2 illustrates a flowchart of an exemplary method for automatically generating annotated images according to an embodiment of the present disclosure;

3 illustrates an exemplary apparatus for automatically generating annotated images according to an embodiment of the present disclosure;

4 illustrates an exemplary computing device for automatically generating annotated images in accordance with embodiments of the present disclosure.

Among them, the reference signs are as follows:

100 3D scene models

101 CAD models

102 CAD models

103 CAD models

200 methods

201 steps

202 steps

203 steps

204 steps

300 devices

301 model unit

302 projection unit

303 Label unit

304 Fusion Unit

400 computing devices

401 processor

402 memory

Detailed ways

Various exemplary embodiments of the present disclosure are described in detail below with reference to the accompanying drawings. Although the exemplary methods and apparatus described below include software and/or firmware executed on hardware among other components, it should be noted that these examples are only illustrative and should not be viewed as limiting. For example, consider that any or all hardware, software and firmware components may be implemented exclusively in hardware, exclusively in software, or in any combination of hardware and software. Therefore, although exemplary methods and apparatuses have been described below, those skilled in the art will readily understand that the examples provided are not intended to limit the manner in which these methods and apparatuses may be implemented.

Furthermore, the flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operations of possible implementations of methods and systems according to various embodiments of the present disclosure. It should be noted that the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown one after the other may actually execute substantially in parallel, or they may sometimes execute in the reverse order, depending on the functionality involved. It should also be noted that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented using special purpose hardware-based systems that perform the specified functions or operations. , or can be implemented using a combination of specialized hardware and computer instructions.

The terms "including", "includes" and similar terms used herein are open-ended terms, that is, "includes/includes but is not limited to", indicating that other contents may also be included. The term "based on" means "based at least in part on." The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment" and so on.

As mentioned before, the current development of AI applications always lags far behind the construction of production lines. On the one hand, existing AI applications usually need to be implemented to replace manual operations after the production line is actually put into production, such as automatic identification and sorting of products. On the other hand, unlike open image data sets on the Internet (for example, PASCAL VOC, MS COCO, ImageNet, etc.), industrial scene images are private data, which means that companies can only obtain what they want to use based on their own production lines. Image resources to train their AI models. This often requires using different cameras to capture image data and annotating the images manually, which consumes a lot of labor resources and a lot of time, and manual annotation is prone to errors and missed tags, especially when processing large amounts of image data. In view of this, the present invention provides a solution for automatically generating annotation images to solve the above-mentioned deficiencies in the prior art.

FIG. 1 illustrates an exemplary three-dimensional scene model 100 applicable in embodiments of the present disclosure. The three-dimensional scene model 100 may be generated based on one or more CAD models, where each CAD model respectively corresponds to one of one or more objects (eg, parts, components, subassemblies, assemblies, equipment, etc.). For example, as shown in Figure 1, a three-dimensional scene model can be generated based on three CAD models 101, 102 and 103, where each CAD model 101-103 respectively corresponds to one of the three objects. The three CAD models 101 to 103 may include at least two identical CAD models or completely different CAD models. It should be understood that the number and shape of the CAD models in FIG. 1 are only exemplary and not limiting, and the three-dimensional scene model 100 can be generated by more or less of the same or different CAD models. For example, the three-dimensional scene model 100 may be generated by copying a single CAD model, or the three-dimensional scene model 100 may be generated by multiple different CAD models. In some examples, the three-dimensional scene model 100 may correspond to products, different stages of a production line (eg, different processes for parts, components, subassemblies, assemblies, equipment, etc.), and the like.

2 illustrates a flowchart of an exemplary method 200 for automatically generating annotated images in accordance with disclosed embodiments.

Referring to Figure 2, method 200 begins with step 201. In step 201, a three-dimensional scene model is generated based on one or more CAD models, where each CAD model of the one or more CAD models corresponds to one of the one or more objects. In this step, for example, the three-dimensional scene model can be generated directly after designing the CAD model of the object (for example, product, such as parts, components, subassemblies, components, equipment, etc.) without waiting for the actual production of the production line, so it can It is easy to build application scenarios of AI models along with the entire product life cycle.

Next, method 200 proceeds to step 202. In step 202, based on the three-dimensional scene model, a two-dimensional projection image set about one or more objects is generated. In this step, the three-dimensional scene model is related to one or more objects, and a two-dimensional plane image is obtained through projection, where the projections of one or more objects on the imaging plane may or may not intersect.

Next, method 200 proceeds to step 203. In step 203, each image in the two-dimensional projection image set is automatically annotated to generate annotation information associated with an unoccluded object, where the unoccluded object is included in one or more objects. In this step, since the projections of one or more objects on the imaging plane may or may not intersect, unoccluded objects can be fully identified and annotated for planar images. For example, the annotation information can be of various types, such as category annotation information, bounding box annotation information, polygon annotation information (or mask information), line annotation information, cuboid annotation information, key point annotation information, etc. .

Next, method 200 proceeds to step 204. In step 204, the annotation information and the background image related to the production line of one or more objects are fused to generate an annotation image. In this step, the image data set can be easily expanded by fusing the annotation information with background images related to the production line involving one or more objects, thereby obtaining various annotation image sets for training of the AI model. For example, the background image related to the production line involving one or more objects may be various real scene images captured in advance (for example, workshop, operation platform, etc.).

Different from the prior art solution of capturing an image from a real scene and annotating the image, the method 200 automatically realizes the image generation and annotation process by using a CAD model to automatically realize the image generation and annotation process. Furthermore, the method can be automatically performed by a computer, thus avoiding errors, inaccuracies, and missing labels, even when the size of the image data collection is very large.

In some embodiments, step 201 may include: importing one or more CAD models into the three-dimensional scene model, where each CAD model of the one or more CAD models includes model name information, structure information, and material information. In this step, a three-dimensional scene model is constructed by importing one or more CAD models. At the same time, each CAD model of the object includes model name information (for example, used to distinguish each CAD model), structural information (for example, geometric structure, Including point, line, surface information, etc.) and material information (for example, material, so that the presented model approximates the real object).

In some embodiments, step 202 may include: loading a three-dimensional scene model including model name information, structure information, and material information; rendering the loaded three-dimensional scene model based on different rendering configuration parameters to obtain a three-dimensional rendering model set; Each model in the collection of three-dimensional rendering models is projected to generate a collection of two-dimensional projected images about one or more objects. In this step, the three-dimensional scene model can be loaded into the rendering engine, and the polygon meshing of the three-dimensional scene model (for example, triangular mesh or other polygon mesh, etc.) is implemented based on the structural information and material information, and the rendering engine is based on Different rendering configuration parameters (or rendering parameters) render the three-dimensional scene model to obtain a three-dimensional rendering model set. For example, each model in a set of three-dimensional rendering models can be projected from different angles by a viewpoint to generate a set of two-dimensional projected images about one or more objects. In some examples, the transformation of the three-dimensional rendering model into the two-dimensional projection image of the imaging plane can be implemented based on the projection transformation matrix.

In some embodiments, the rendering configuration parameters may include at least one of: lighting parameters, depth parameters, pose parameters, or texture parameters. For example, lighting parameters may include light source type, light source position, light direction, light intensity, etc. For example, the depth parameter refers to the distance of each pixel in the image from the viewpoint. For example, the posture parameters may include translation parameters (eg, translation distances relative to the x, y, and z axes) and rotation parameters (eg, rotation angles relative to the x, y, and z axes). For example, texture parameters may refer to appearance information used to describe the surface of an object, such as color information, normals, roughness, etc. The three-dimensional scene model rendered based on the rendering configuration parameters can obtain the effect of approximating the objects in the real scene under various parameters, so as to easily obtain a large number of rendering models.

In some embodiments, step 203 may include determining one or more projection areas in each image, each of the one or more projection areas corresponding to one of one or more CAD models, the one or Multiple projection areas have corresponding depth values; determine a projection area associated with an unoccluded object in one or more projection areas, wherein the unoccluded object is included in one or more objects; according to the vision task type, based on The metadata automatically annotates the projected area associated with the unoccluded object in each image to generate annotation information associated with the unoccluded object. The metadata includes information about the CAD model and/or one or more of the rendering configuration parameters. indivual. For example, when a three-dimensional scene model including one or more CAD models is projected from the viewpoint at different angles, some of these projected areas will be partially or completely blocked due to different depths. Usually, the projected areas of unoccluded objects can be Fully identified and labeled. For image annotation requirements, there may be different visual annotation tasks (e.g., recognition, detection, instance segmentation, pose estimation, etc.). Different visual tasks require, for example, various metadata (from the rendering engine), including CAD model One or more of the information (eg, name information of the CAD model) and/or rendering configuration parameters (eg, depth, pose, etc.). For example, for recognition, metadata at least includes the name information of the CAD model; for detection/instance segmentation/pose estimation tasks, in addition to the name information of the CAD model and the projection of the three-dimensional scene model, the rendering configuration parameters also need to be included. depth information and attitude information.

In some embodiments, step 203 may further include: determining whether one or more intersection portions exist for one or more projection areas; and if one or more intersection portions exist, for each intersection associated with the plurality of projection areas part, retaining the projection area with the smallest depth value among the multiple projection areas and removing the remaining projection areas from the multiple projection areas to generate a projection area associated with the unoccluded object; or if one or more intersecting parts do not exist , then the one or more projection areas are determined as projection areas associated with the unoccluded object. In this step, since pixels in different projection areas have different depth values, the projection area with the smallest depth value is closest to the viewpoint and can be identified as unoccluded.

In some embodiments, step 203 may further include: selecting a visual processing algorithm according to the type of visual task; using the visual processing algorithm to determine characteristic information of the projection area associated with the unoccluded object based on metadata to generate annotation information. In this step, for image annotation needs, there may be different visual annotation tasks, so it is necessary to determine a suitable visual processing algorithm. For example, you can choose from the functions provided by open source computer vision libraries, which can be implemented efficiently. Various computer vision algorithms are used, and the selected functions and metadata are used to determine the characteristic information of the projection area associated with the unoccluded object to generate annotation information.

In some embodiments, step 203 may further include: determining contour point information of the projection area associated with the unoccluded object as feature information; generating bounding box information and/or mask associated with the unoccluded object based on the contour point information. Membrane information is used as annotation information. In this step, for vision task types of detection and instance segmentation, the contour point information of the projection area associated with the unoccluded object can be extracted and, for example, based on the upper-left corner and the lower-right corner of the contour point in the coordinate system The coordinates of the location are used to determine the bounding box information, and the projection area is masked, for example, based on the contour points to obtain the mask information.

In some embodiments, step 204 may further include: extracting the projection area associated with the unoccluded object from each image in the two-dimensional projection image set; using the projection area associated with the unoccluded object as the foreground and the projection area associated with the unoccluded object. The background images related to the production line of one or more objects are fused to obtain a fused image; the annotation information is superimposed on the fused image to generate an annotation image. In this step, by obtaining the projection area of the unoccluded object, it can be easily compared with the background image related to the production line involving one or more objects (e.g., various real scene images captured in advance (e.g., workshop, operation Taiwan, etc.) are fused to obtain an expanded image data set.

In some embodiments, the annotated images are used to train an AI application model associated with the production line of one or more objects. As mentioned before, the AI application model associated with the production line of one or more objects can include, for example, object detection, object recognition, object sorting, etc., by fusing a large number of annotated images with annotation information (for example, The masked object CAD model is fused with the background image so that the object appears to actually exist in the background image) can be used as prepared image training data to train the AI application model.

In some embodiments, one or more CAD models may include at least two identical CAD models or may include completely different CAD models. For example, different three-dimensional scene models can be constructed. In some cases, they can be scenes that include multiple objects of the same category, and in some cases, they can be scenes that include multiple objects of different categories.

FIG. 3 illustrates an exemplary apparatus 300 for automatically generating annotated images according to an embodiment of the present disclosure.

Referring to FIG. 3 , the device 300 includes a model unit 301 , a projection unit 302 , an annotation unit 303 and a fusion unit 304 .

The model unit 301 is configured to generate a three-dimensional scene model based on one or more CAD models, where each CAD model of the one or more CAD models corresponds to one of the one or more objects.

The projection unit 302 is configured to generate a set of two-dimensional projection images about one or more objects based on the three-dimensional scene model.

The annotation unit 303 is configured to automatically annotate each image in the two-dimensional projection image set to generate annotation information associated with unoccluded objects, wherein the unoccluded objects are included in the one or more objects.

The fusion unit 304 is configured to fuse the annotation information and the background image related to the production line of one or more objects to generate an annotation image.

In some embodiments, the model unit 301 may be configured to import one or more CAD models into the three-dimensional scene model, where each CAD model of the one or more CAD models includes model name information, structure information, and material information. .

In some embodiments, the projection unit 302 may be configured to: load a three-dimensional scene model including model name information, structure information, and material information; render the loaded three-dimensional scene model based on different rendering configuration parameters to obtain a three-dimensional rendering model Collection; projects each model in a collection of 3D rendering models to produce a collection of 2D projected images about one or more objects.

In some embodiments, the rendering configuration parameters may include at least one of: lighting parameters, depth parameters, pose parameters, or texture parameters.

In some embodiments, the annotation unit 303 may be configured to: determine one or more projection areas in each image, each of the one or more projection areas corresponding to one of the one or more CAD models, a or multiple projection areas have corresponding depth values; determine the projection area associated with the unoccluded object in one or more projection areas, where the unoccluded object is included in one or more objects; according to the vision task type, based on the element The data automatically annotates the projection area associated with the unoccluded object in each image to generate annotation information associated with the unoccluded object, and the metadata includes information of the CAD model and/or one or more of the rendering configuration parameters.

In some embodiments, the annotation unit 303 may be further configured to: determine whether one or more intersection portions exist in one or more projection areas; if there are one or more intersection portions, for For each intersection, retain the projection area with the smallest depth value among the multiple projection areas and remove the remaining projection areas from the multiple projection areas to generate a projection area associated with the unoccluded object; or if one or more projection areas are not present intersecting parts, one or more projection areas are determined as projection areas associated with the unoccluded object.

In some embodiments, the annotation unit 303 may be further configured to: select a visual processing algorithm according to the type of visual task; use the visual processing algorithm to determine the characteristic information of the projection area associated with the unoccluded object based on metadata to generate annotation information .

In some embodiments, the annotation unit 303 may be further configured to: determine contour point information of the projection area associated with the unoccluded object as feature information; generate bounding box information associated with the unoccluded object based on the contour point information and/ Or mask information as annotation information.

In some embodiments, the fusion unit 304 may be configured to: extract the projection area associated with the unoccluded object from each image in the two-dimensional projection image set; use the projection area associated with the unoccluded object as the foreground and Background images related to the production lines of one or more objects are fused to obtain a fused image; annotation information is superimposed on the fused image to generate an annotation image.

In some embodiments, the annotated images are used to train an AI application model associated with the production line of one or more objects.

In addition, the model unit 301 may also be configured to perform other processes as described above with respect to step 201 in the method 200, the projection unit 302 may also be configured to perform other processes as described above with respect to step 202 in the method 200, the annotation unit 303 may also be configured to perform other processes as described above with respect to step 203 in method 200, and fusion unit 304 may also be configured to perform other processes as described above with respect to step 204 in method 200.

4 illustrates a block diagram of an exemplary computing device 400 for monitoring an enclosed space environment, in accordance with embodiments of the present disclosure. Computing device 400 includes a processor 401 and a memory 402 coupled to processor 401 . The memory 402 is used to store computer-executable instructions. When executed, the computer-executable instructions cause the processor 401 to perform the methods in the above embodiments (for example, any one or more steps of the aforementioned method 200).

Compared with the existing technology, the solution for automatically generating annotated images according to the embodiments of the present disclosure as described above can have the following advantages: First, the image generation and annotation process can be performed directly after product design, so the trained AI The model can run through the entire product life cycle; second, the image generation and labeling process can be applied to each production stage, such as part-level, component-level, sub-assembly or component-level products; third, the image generation and labeling process has a high degree of automation degree, will greatly save time and human resources, and can significantly speed up the implementation of mass customization; fourth, the image generation and annotation process is automatic, so the size of the image data collection can be freely expanded, which can greatly improve the performance of the AI model; Fifth, the image generation and annotation process is performed by computers, so errors, inaccuracies, and missing labels can be avoided even for large image data sets.

Different from various manual annotation tools, if the user uses the disclosed solution to automatically generate annotated images for training, he only needs to load the CAD model, configure the rendering parameters, and set the background image he wants, and then he can automatically generate annotated images with annotated information. Images can be used to train different AI models for different industry scenarios (for example, smart production, etc.). For example, after obtaining the product design, image data sets for AI model training can be automatically generated without any actual production data. These synthetic production data can accelerate the realization of various business scenarios before the production and manufacturing stages, such as product Design, product verification, manufacturing process, production plan management, supply chain management, production line planning, production line design, product maintenance, etc.

Furthermore, alternatively, the above method can be implemented by a computer-readable storage medium. Computer-readable storage media carries thereon computer-readable program instructions for executing various embodiments of the present disclosure. Computer-readable storage media may be tangible devices that can retain and store instructions for use by an instruction execution device. The computer-readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM) or Flash memory), Static Random Access Memory (SRAM), Compact Disk Read Only Memory (CD-ROM), Digital Versatile Disk (DVD), Memory Stick, Floppy Disk, Mechanical Coding Device, such as a printer with instructions stored on it. Protruding structures in hole cards or grooves, and any suitable combination of the above. As used herein, computer-readable storage media are not to be construed as transient signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or through electrical wires. transmitted electrical signals.

Accordingly, in another embodiment, the present disclosure provides a computer-readable storage medium having computer-executable instructions stored thereon for performing various implementations of the disclosure. method in the example.

In another embodiment, the present disclosure provides a computer program product tangibly stored on a computer-readable storage medium and including computer-executable instructions that, when executed, cause At least one processor performs methods in various embodiments of the present disclosure.

Generally speaking, various example embodiments of the present disclosure may be implemented in hardware or special purpose circuits, software, firmware, logic, or any combination thereof. Some aspects may be implemented in hardware, while other aspects may be implemented in firmware or software that may be executed by a controller, microprocessor, or other computing device. While aspects of embodiments of the present disclosure are illustrated or described as block diagrams, flowcharts, or using some other graphical representation, it will be understood that the blocks, devices, systems, techniques, or methods described herein may be used as non-limiting Examples are implemented in hardware, software, firmware, special purpose circuitry or logic, general purpose hardware or controllers, or other computing devices, or some combination thereof.

Computer-readable program instructions or computer program products used to execute various embodiments of the present disclosure can also be stored in the cloud. When needed to be called, the user can access the instructions stored in the cloud for execution through the mobile Internet, fixed network or other networks. The computer-readable program instructions of an embodiment of the present disclosure are used to implement the technical solutions disclosed according to various embodiments of the present disclosure.

While embodiments of the present disclosure have been described with reference to a number of specific embodiments, it is to be understood that embodiments of the present disclosure are not limited to the specific embodiments disclosed. The disclosed embodiments are intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims. The scope of the claims is to be accorded the broadest interpretation so as to encompass all such modifications and equivalent structures and functions.

Claims (25)

1. A method for automatically generating an annotated image, the method comprising the steps of:

   generating a three-dimensional scene model based on one or more CAD models, wherein each of the one or more CAD models corresponds to one of the one or more objects;

   generating a set of two-dimensional projection images for the one or more objects based on the three-dimensional scene model;

   automatically labeling each image in the set of two-dimensional projection images to generate labeling information associated with a non-occluded object, wherein the non-occluded object is included in the one or more objects; and

   the annotation information is fused with a background image associated with the line of production of the one or more objects to generate an annotation image.
2. The method of claim 1, wherein generating the three-dimensional scene model comprises:

   and importing the one or more CAD models into the three-dimensional scene model, wherein each of the one or more CAD models includes model name information, structure information, and material information.
3. The method of claim 1 or 2, wherein generating a set of two-dimensional projection images for the one or more objects comprises:

   loading a three-dimensional scene model comprising model name information, structure information and material information;

   rendering the loaded three-dimensional scene model based on different rendering configuration parameters to obtain a three-dimensional rendering model set;

   each model in the set of three-dimensional rendering models is projected to generate a set of two-dimensional projection images for the one or more objects.
4. A method according to claim 3, wherein the rendering configuration parameters comprise at least one of: illumination parameters, depth parameters, pose parameters, or texture parameters.
5. The method of claim 1, wherein automatically labeling each image in the set of two-dimensional projection images to generate labeling information associated with a non-occluding object comprises:

   determining one or more projection areas in each image, each of the one or more projection areas corresponding to one of the one or more CAD models, the one or more projection areas having respective depth values;

   determining a projected area of the one or more projected areas associated with a non-occluding object, wherein the non-occluding object is included in the one or more objects;

   according to the visual task type, the projected areas associated with the non-occluded object in each image are automatically annotated based on metadata including one or more of information of the CAD model and/or rendering configuration parameters to generate annotation information associated with the non-occluded object.
6. The method of claim 5, wherein determining a projected area of the one or more projected areas associated with a non-occluding object comprises:

   determining whether one or more intersection portions exist for the one or more projection areas;

   if the one or more intersections exist, reserving a projection region of the plurality of projection regions having a smallest depth value for each intersection associated with the plurality of projection regions, and removing remaining projection regions of the plurality of projection regions to generate a projection region associated with the non-occluding object; or alternatively

   If the one or more intersections are not present, the one or more projection areas are determined to be projection areas associated with a non-occluding object.
7. The method of claim 5 or 6, wherein automatically labeling the projected area associated with the non-occluded object in each image based on metadata to generate labeling information associated with the non-occluded object according to the visual task type comprises:

   selecting a visual processing algorithm according to the visual task type;

   feature information of a projection region associated with the non-occluded object is determined based on the metadata to generate the annotation information using the vision processing algorithm.
8. The method of claim 7, wherein determining characteristic information of a projection region associated with a non-occluding object to generate annotation information comprises:

   determining contour point information of a projection area associated with a non-occluded object as characteristic information;

   and generating boundary box information and/or mask information associated with the non-occluded object based on the contour point information as the labeling information.
9. The method of claim 7, wherein fusing the annotation information with a background image related to a line of production of the one or more objects to generate an annotation image comprises:

   extracting a projection region associated with the non-occluding object from each image in the set of two-dimensional projection images;

   fusing a projection area associated with the non-occluded object as a foreground and a background image related to a production line of the one or more objects to obtain a fused image;

   and superposing the annotation information on the fusion image to generate the annotation image.
10. The method of claim 1 or 9, wherein the annotation image is used to train an AI application model associated with a production line of the one or more objects.
11. The method of claim 1, wherein the one or more CAD models comprise at least two identical CAD models or comprise disparate CAD models.
12. An apparatus for automatically generating an annotation image, the apparatus comprising:

    a model unit configured to generate a three-dimensional scene model based on one or more CAD models, wherein each of the one or more CAD models corresponds to one of the one or more objects;

    a projection unit configured to generate a set of two-dimensional projection images for the one or more objects based on the three-dimensional scene model;

    an annotating unit configured to automatically annotate each image in the set of two-dimensional projection images to generate annotation information associated with a non-occluded object, wherein the non-occluded object is included in the one or more objects; and

    and a fusion unit configured to fuse the labeling information with a background image related to the production line of the one or more objects to generate a labeling image.
13. The apparatus of claim 12, wherein the model unit is configured to:

    and importing the one or more CAD models into the three-dimensional scene model, wherein each of the one or more CAD models includes model name information, structure information, and material information.
14. The apparatus of claim 12 or 13, wherein the projection unit is configured to:

    loading a three-dimensional scene model comprising model name information, structure information and material information;

    rendering the loaded three-dimensional scene model based on different rendering configuration parameters to obtain a three-dimensional rendering model set;

    each model in the set of three-dimensional rendering models is projected to generate a set of two-dimensional projection images for the one or more objects.
15. The apparatus of claim 14, wherein the rendering configuration parameters comprise at least one of: illumination parameters, depth parameters, pose parameters, or texture parameters.
16. The apparatus of claim 12, wherein the labeling unit is configured to:

    determining one or more projection areas in each image, each of the one or more projection areas corresponding to one of the one or more CAD models, the one or more projection areas having respective depth values;

    determining a projected area of the one or more projected areas associated with a non-occluding object, wherein the non-occluding object is included in the one or more objects;

    according to the visual task type, the projected areas associated with the non-occluded object in each image are automatically annotated based on metadata including one or more of information of the CAD model and/or rendering configuration parameters to generate annotation information associated with the non-occluded object.
17. The apparatus of claim 16, wherein the labeling unit is further configured to:

    determining whether one or more intersection portions exist for the one or more projection areas;

    if the one or more intersections exist, reserving a projection region of the plurality of projection regions having a smallest depth value for each intersection associated with the plurality of projection regions, and removing remaining projection regions of the plurality of projection regions to generate a projection region associated with the non-occluding object; or alternatively

    If the one or more intersections are not present, the one or more projection areas are determined to be projection areas associated with a non-occluding object.
18. The apparatus according to claim 16 or 17, wherein the labeling unit is further configured to:

    selecting a visual processing algorithm according to the visual task type;

    feature information of a projection region associated with the non-occluded object is determined based on the metadata to generate the annotation information using the vision processing algorithm.
19. The apparatus of claim 18, wherein the labeling unit is further configured to:

    determining contour point information of a projection area associated with a non-occluded object as characteristic information;

    and generating boundary box information and/or mask information associated with the non-occluded object based on the contour point information as the labeling information.
20. The apparatus of claim 18, wherein the fusion unit is configured to:

    extracting a projection region associated with the non-occluding object from each image in the set of two-dimensional projection images;

    fusing a projection area associated with the non-occluded object as a foreground and a background image related to a production line of the one or more objects to obtain a fused image;

    and superposing the annotation information on the fusion image to generate the annotation image.
21. The apparatus of claim 12 or 20, wherein the annotation image is used to train an AI application model associated with a production line of the one or more objects.
22. The apparatus of claim 12, wherein the one or more CAD models comprise at least two identical CAD models or comprise disparate CAD models.
23. A computing device, the computing device comprising:

    a processor; and

    a memory for storing computer-executable instructions that, when executed, cause the processor to perform the method of any of claims 1-11.
24. A computer-readable storage medium having stored thereon computer-executable instructions for performing the method according to any of claims 1-11.
25. A computer program product tangibly stored on a computer-readable storage medium and comprising computer-executable instructions that, when executed, cause at least one processor to perform the method of any one of claims 1-11.