KR102247245B1 - Method to generate label - Google Patents

Abstract

A computer program stored in a computer-readable storage medium according to an embodiment of the present disclosure, wherein the computer program provides a user interface (UI) for displaying information for labeling when executed on one or more processors of a computing device, the The user interface includes: an image layer that displays an image including a label object object, and displays an annotation form for the label object object in response to a user annotation input for displaying the label object object; And a label layer that allows input of a user class for inputting a class for each of two or more layers included in the label of the label target object.

Description

How to create a label {METHOD TO GENERATE LABEL}

The present invention relates to a method of generating a label, and more particularly, to a method of providing a user interface for displaying information for label generation.

Various types of research on machine learning algorithms are being conducted, and as a result, various deep learning techniques such as deep neural networks, convolutional neural networks, and recurrent neural networks are used in computer vision, It is applied to fields such as speech recognition and natural language processing.

These machine learning algorithms have a complex structure and can output results through complex operations. In order to process data using machine learning algorithms, a considerable understanding of machine learning algorithms must be preceded, and accordingly, users who can use machine learning algorithms are limited.

Accordingly, there is a need in the art to allow users to easily access machine learning algorithms, and there is a need to easily generate labeled data for learning of machine learning algorithms.

Korean Patent Publication No. 2016-0012537 discloses a neural network learning method and apparatus, and a data processing apparatus.

The present disclosure has been devised in response to the above-described background technology, and an object of the present disclosure is to provide a method of displaying information for labeling.

A computer program stored in a computer-readable storage medium according to an embodiment of the present disclosure for realizing the above-described problems, wherein the computer program is executed on one or more processors of a computing device, the user displaying information for labeling Provides an interface (UI), wherein the user interface: In response to a user annotation input for displaying an image including a label object object, and for displaying the label object object, an annotation form for the label object object ( an image layer that displays the form); And a label layer that allows input of a user class for inputting a class for each of two or more layers included in the label of the label target object.

In an alternative embodiment of computer program operations to perform the following operations for providing a label generation method, the label layer comprises: a first class display object corresponding to a first layer, a second class corresponding to a second layer Display objects, layer addition objects, and layer deletion objects can be displayed.

In an alternative embodiment of computer program operations to perform the following operations for providing a label generation method, the label layer is: a third class corresponding to a third layer in response to a user input to the layer addition object. A display object may be further displayed, or the second class display object corresponding to the second layer may be deleted in response to a user input to the layer deletion object.

In an alternative embodiment of computer program operations for performing the following operations for providing a label generation method, the label layer comprises: one or more first template classes corresponding to a first layer and one or more first template classes corresponding to a second layer. A template class selection area displaying a second template class; Including, And in response to a user label input for at least one of the first template class or the second template class, a first class display object corresponding to the first layer or a second class corresponding to the second layer At least one of the first template class and the second template class corresponding to the user label input may be displayed on at least one of the class display objects, respectively.

In an alternative embodiment of computer program operations to perform the following operations for providing a label generation method, the template class selection area is: a first additional template class corresponding to the first layer or corresponding to the second layer The additional template classes may be further displayed in response to a user input to at least one of the second additional template classes.

In an alternative embodiment of computer program operations for performing the following operations for providing a label generation method, the user annotation input comprises: a region selection input for indicating a region of the label object object; And a region adjustment input for changing at least one of a position, a size, or an angle of the annotation form.

In an alternative embodiment of computer program operations for performing the following operations for providing a label generation method, the annotation form may be displayed in the form of one annotation form inputted by a user selection among one or more annotation form forms.

In an alternative embodiment of computer program operations to perform the following operations for providing a label generation method, the image layer is: in response to a user's copy input for copying a first annotation form included in the image. , A second annotation form having the same shape as the first annotation form may be additionally displayed.

In an alternative embodiment of computer program operations for performing the following operations for providing a label generation method, the image layer is: in response to displaying a first annotation form for a first label target object included in the image. , With respect to the second label target object related to the first label target object, a second annotation form based on the first annotation form may be additionally displayed on the second label target object.

In an alternative embodiment of computer program operations to perform the following operations for providing a label generation method, the processor comprises: displaying a first annotation form for a first label target object included in the image and inputting a first class In response to, determining the first class as a label for a second label target object having the same shape as the first label target object; Alternatively, for a third label target object having a different type from the first label target object, a second class that is a class different from the first class may be determined as a label.

In an alternative embodiment of computer program operations for performing the following operations for providing a label generation method, the processor: generates learning data including an object included in the annotation form and a label corresponding to the user class input. It may include an operation to generate.

A method of displaying labeling information in a user terminal according to an embodiment of the present disclosure for realizing the above-described problem, the method comprising: displaying an image including a label object object in an image layer; Displaying an annotation form for the label object object in response to a user annotation input for displaying the label object object on the image layer; And displaying, on the label layer, a display object allowing input of a user class for inputting a class for each of two or more layers included in a label of the label target object.

A user terminal according to an embodiment of the present disclosure for realizing the above-described problem, comprising: a processor including one or more cores; Memory; And an output unit providing a user interface, wherein the user interface displays an image including a label object object, and in response to a user annotation input for displaying the label object object, the label object object An image layer that displays an annotation form; And a label layer that allows input of a user class for inputting a class for each of two or more layers included in the label of the label target object.

A server according to an embodiment of the present disclosure for realizing the above-described problem, comprising: a processor including one or more cores; Network unit; And a memory, wherein the processor determines to transmit a user interface (UI) to a user terminal through the network unit, the user interface: displaying an image including a label object object, and displaying the label object object An image layer for displaying an annotation form for the label target object in response to a user annotation input for performing the labeling; And a label layer that allows input of a user class for inputting a class for each of two or more layers included in the label of the label target object.

The present disclosure may provide a method of displaying labeling related information on a user interface.

1 is a diagram illustrating a block diagram of a computing device that performs operations for displaying labeling information on a user interface according to an embodiment of the present disclosure.
2 is a diagram illustrating an image layer for labeling according to an embodiment of the present disclosure.
3 is a diagram illustrating a label layer for allowing a class input according to an embodiment of the present disclosure.
4 is a diagram illustrating an image layer for labeling according to an embodiment of the present disclosure.
5 is a diagram illustrating an image layer for labeling according to an embodiment of the present disclosure.
6 is a diagram illustrating an exemplary network function setting layer for labeling according to an embodiment of the present disclosure.
7 is a diagram illustrating an image layer for labeling according to an embodiment of the present disclosure.
8 is a flowchart illustrating a method of displaying information for labeling according to an embodiment of the present disclosure.
9 is a block diagram of a computing device according to an embodiment of the present disclosure.

Various embodiments are now described with reference to the drawings. In this specification, various descriptions are presented to provide an understanding of the present disclosure. However, it is clear that these embodiments may be implemented without this specific description.

As used herein, the terms "component", "module", "system" and the like refer to a computer-related entity, hardware, firmware, software, a combination of software and hardware, or execution of software. For example, a component may be, but is not limited to, a process executed on a processor, a processor, an object, a thread of execution, a program, and/or a computer. For example, both an application running on a computing device and a computing device may be components. One or more components may reside within a processor and/or thread of execution. A component can be localized within a single computer. A component can be distributed between two or more computers. In addition, these components can execute from a variety of computer readable media having various data structures stored therein. Components can be, for example, through a signal with one or more data packets (e.g., data from one component interacting with another component in a local system, a distributed system, and/or a signal through another system and a network such as the Internet. Depending on the data being transmitted), it may communicate via local and/or remote processes.

In addition, the term “or” is intended to mean an inclusive “or” rather than an exclusive “or”. That is, unless specified otherwise or is not clear from the context, "X employs A or B" is intended to mean one of the natural inclusive substitutions. That is, X uses A; X uses B; Or when X uses both A and B, "X uses A or B" can be applied to either of these cases. In addition, the term “and/or” as used herein should be understood to refer to and include all possible combinations of one or more of the listed related items.

In addition, the terms "comprises" and/or "comprising" should be understood to mean that the corresponding features and/or components are present. However, it is to be understood that the terms "comprising" and/or "comprising" do not exclude the presence or addition of one or more other features, elements, and/or groups thereof. In addition, unless otherwise specified or when the context is not clear to indicate a singular form, the singular in the specification and claims should be interpreted as meaning "one or more" in general.

Those of skill in the art would further describe the various illustrative logical blocks, configurations, modules, circuits, means, logics, and algorithm steps described in connection with the embodiments disclosed herein, including electronic hardware, computer software, or a combination of both. It should be recognized that it can be implemented as To clearly illustrate the interchangeability of hardware and software, various illustrative components, blocks, configurations, means, logics, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented in hardware or as software depends on the specific application and design restrictions imposed on the overall system. Skilled technicians can implement the described functionality in various ways for each particular application. However, such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure.

Description of the presented embodiments is provided so that a person of ordinary skill in the art of the present disclosure can use or implement the present invention. Various modifications to these embodiments will be apparent to those of ordinary skill in the art. The general principles defined herein may be applied to other embodiments without departing from the scope of the present disclosure. Thus, the present invention is not limited to the embodiments presented herein. The invention is to be accorded the widest scope consistent with the principles and novel features presented herein.

In an embodiment of the present disclosure, the server may include other components for executing the server environment of the server. The server may include any type of device. The server is a digital device, and may be a digital device equipped with a processor, such as a laptop computer, a notebook computer, a desktop computer, a web pad, and a mobile phone, and equipped with a memory and computing power. The server may be a web server that processes services. The types of servers described above are only examples, and the present disclosure is not limited thereto.

In this specification, neural networks, artificial neural networks, and network functions can often be used interchangeably.

1 is a diagram illustrating a block diagram of a computing device that performs operations for displaying labeling information on a user interface according to an embodiment of the present disclosure.

The computing device 100 for providing a label generation method or a user interface for displaying information for labeling according to an embodiment of the present disclosure includes a network unit 110, a processor 120, a memory 130, and an output unit ( 140) and an input unit 150 may be included.

In embodiments of the present disclosure, a layer may mean a layer that is displayed on a user interface and used to overlap and display multiple images. Two or more layers may be overlapped and displayed. When two or more layers are overlapped and displayed, one layer may hide the other layer and may not be visible. Alternatively, when two or more layers are overlapped and the upper layer is partially semi-transparent, at least a part of the lower layer may be visible. Two or more layers may have the same size. Also, two or more layers may have different sizes. The layer may include one or more regions. Also, a layer may include one or more display objects.

In embodiments of the present disclosure, a layer may be divided into a plurality of regions. The plurality of areas may be one space on the screen that does not overlap. One layer may include only one area or may include a plurality of areas. One area may include one or more display objects.

In the embodiments of the present disclosure, the object may be a set of pictures, symbols, or letters corresponding to each program, command, or data. According to an embodiment of the present disclosure, an object may be used to receive a user selection input. For example, when receiving a user input for an object, the processor 120 may execute a command or data stored in correspondence with the object and display it on the user interface. In the embodiments of the present disclosure, an object and a display object may be interpreted as having the same meaning.

In the embodiments of the present disclosure, “display” may be an operation of displaying data to a user through the output unit 140. "Display" and "display" can be used interchangeably.

Hereinafter, a method of providing a user interface displaying information for labeling according to embodiments of the present disclosure will be described.

Research is being conducted on various methods of calculating images and the like using network functions. In order to learn the network function, supervised learning, unsupervised learning, and semi supervised learning may be used. As a method for learning a network function, a teacher learning method is generally used most. Teacher learning may be a method of learning a network function while knowing the input data and the correct answer to the input data. In order to train a network function, it is necessary to generate training data including input data and a pair of correct answers to the input data. In order to generate training data, it is necessary to label the correct answers to the input data with respect to the input data. The time and cost required for a person to label the correct answer to the input data against the input data is quite large. That is, if the time and cost required to construct the training data increase, the time and cost required to learn the network function may also increase. Accordingly, there is a need for studies to easily construct learning data. In the case of constructing training data according to an embodiment of the present invention, that is, when labeling a pair of correct answers to the input data with respect to the input data, time and cost required for constructing the training data may be reduced. In addition, even for users who do not have an in-depth understanding of learning network functions, learning data can be easily constructed according to an embodiment of the present invention. The present invention can provide a user interface for increasing user convenience and reducing time and cost required for learning a network function.

The processor 120 may display an image layer and a label layer on the user interface. 2 illustrates an image layer 200 according to an embodiment of the present disclosure.

The image layer 200 may display an image including an object to be labeled. The size of the image layer 200 may be the same size as the image. The size of the image layer 200 may be different from that of the image. The image layer 200 may be displayed by adjusting the size of the image to correspond to the size of the image layer 200. The image layer may be a layer for displaying a visual representation of a label target object included in an image. The visual representation may be an annotation form for the object to be labeled. The annotation form will be described later.

The image can be the object of operation of the network function. The image may be data used to input training data. At least a portion of the image may be data to be input for training data. The image may be data input to the input layer of the network function. At least a portion of the image may be data input to the input layer of the network function. The image may be, for example, an image included in a satellite image. Images may include metadata. Meta data may include additional information related to the image. The metadata may be, for example, a latitude, longitude, or geographic coordinate system of a satellite image. The specific description of the above-described image is only an example, and the present disclosure is not limited thereto.

The image may include an object to be labeled. The label target object may be an object to be identified included in the image. The label target object may be an object to be identified using a network function. The object to be labeled may be an object to be labeled included in the image. For example, when the image is at least a part of a satellite image, the object to be labeled may be a mountain, a building, a ship, or the like. The detailed description of the label target object is only an example, and the present disclosure is not limited thereto.

The label object object may be associated with a label. Each of the two or more label target objects included in the image may be associated with a label, respectively. The label target object and the label included in the image may be paired. The label may be a characteristic of the object to be labeled. The label associated with the label target object may include a class for a hierarchy.

Labels can contain classes for one layer. Labels may contain classes for each of two or more layers. For example, the label may include a class for each of the first layer and the second layer. One layer can be associated with more than one class. The specific description of the above label is for illustrative purposes only, and the present disclosure is not limited thereto.

According to an embodiment of the present disclosure, a hierarchy may represent an upper subset relationship of classes. One layer can have a sub-set relationship with other layers. For example, the first layer may be a superset of the second layer. The first layer and the second layer may have an inclusion relationship on the image. For example, in a satellite image, classes for the second layer may be a relationship included in the class for the first layer. For example, the image may be an image of an aircraft carrier floating on the sea. Aircraft carriers may contain fighters, tanks and containers. For example, for an aircraft carrier image, labels may include a first layer and a second layer. Classes for the first tier may be sea and aircraft carriers. The classes for the second tier can be fighters, tanks and containers. The label for the object to be labeled as a fighter aircraft may be'first layer: aircraft carrier' and'second layer: fighter aircraft'. Alternatively, for example, the first layer may be an object classification, and the second layer may be a sub-classification of objects. For example, a class for the first tier may be a fighter, a tank, and a container, and a class for a second tier may be a type of a fighter, a type of tank, and a container type. For example, depending on the type of fighter, the class of the second tier may be'F-22 fighter' and'F-35 fighter'. That is, the label for the F-22 fighter label target object may be “first layer: fighter” and “second layer: F-22 fighter”. Alternatively, for example, the class for the first layer may be a semiconductor panel and a conveyor belt, and the class for the second layer may be a hole, copper, or conductor included in the semiconductor panel. Specific description of the above-described label is only an example, and the present disclosure is not limited thereto.

According to an embodiment of the present disclosure, each layer may represent a classification of a different property. For example, the first layer and the second layer may represent classifications of different properties, respectively. For example, if the object to be labeled is clothes, the class for the first layer is top, bottom, dress, outerwear, the class for the second layer is women's wear, men's wear, and the class for the third layer is A brand, B It could be a brand. For example, a label for a specific label target object may be a first layer: dress,'second layer: women's wear', and'third layer: A brand'. Specific description of the above-described label is only an example, and the present disclosure is not limited thereto.

The image layer 200 may include a display object for adjusting an image. Display objects for adjusting the image may be display objects for adjusting the size, contrast, brightness, shadow, transparency, angle, and the like of the image. The processor 120 may enlarge an image or adjust brightness in response to a user input to a display object for adjusting the image.

The image layer 200 may adjust and display an image so that objects included in the image are displayed at the center of the image layer 200. The processor 120 may analyze the image, enlarge a portion in which many objects are distributed in the image, and display it on the image layer 200. The processor 120 analyzes the image, adjusts the image so that a portion of the image where many objects are distributed is located at the center of the image layer 200 and displays the image on the image layer 200.

The image layer 200 may adjust and display an image in response to a user's image adjustment input. For example, in response to an image adjustment input of dragging an image with a mouse, the image layer 200 may adjust and display the position of the image in the dragging direction. For example, in response to an image adjustment input using an arrow on the keyboard, the image layer 200 may adjust and display the position of the image in the direction of the arrow on the keyboard. The detailed description of the above-described image layer is only an example, and the present disclosure is not limited thereto.

The image layer 200 may be displayed in the user interface together with the minimap layer. The minimap layer may be a layer different from the image layer 200. The minimap layer may be displayed by being superimposed on the image layer 200. The minimap layer can display a minimap for an image. The minimap may be a reduced whole image. For example, when the image is a satellite image, the minimap may be an image obtained by reducing the satellite image image. The mini-map may be a map for displaying coordinates of a portion of an image displayed on the image layer 200 among all images. It is possible to check which part of the image is the area currently displayed on the image layer 200 through the mini-map. The minimap layer may display a visual representation of an area displayed on the current image layer 200 among all images. For example, when a container part of the satellite image image is enlarged and displayed on the image layer 200, the minimap layer may display a visual expression with a red border on the container part of the reduced satellite image image. The detailed description of the above-described mini-map is only an example, and the present disclosure is not limited thereto.

The image layer 200 may adjust and display an image so that the corresponding coordinates are displayed on the image layer 200 in response to a user input for a specific coordinate in the minimap layer. For example, when a user drags a red border portion from the minimap layer to another area, the image layer 200 may adjust and display the image so that the other area is displayed. For example, when a user clicks a specific coordinate on the minimap layer, the red border portion can be moved so that the center of the red border portion overlaps the specific coordinate, and the image layer 200 corresponds to the image layer 200 You can display it by adjusting the image so that the part is visible. The detailed description of the above-described mini-map is only an example, and the present disclosure is not limited thereto.

The user interface may display the image layer 200 and the network function setting layer together. The network function setting layer will be described with reference to FIGS. 6 and 7. The network function setting layer 600 may change settings for one or more channels included in the network function.

The network function may include more than one channel. The network function can calculate the input image using two or more channels. Two or more channels may each use different filters to perform an operation in the convolutional layer. For example, the network function may include three channels. Each channel can be configured with a different filter, and a network function can be learned for each color channel. For example, a first channel of the network function may process a red color of an image, a second channel may process a green color, and a third channel may process a blue color (ie, R, G, B channels). The network function can calculate one output by synthesizing the result of calculating an image using a plurality of channels. The specific description of the above-described channel is only an example, and the present disclosure is not limited thereto.

The network function setting layer 600 may change characteristics of images processed by two or more channels, respectively. For example, in a network function, a red color may be processed in a first channel, a blue color may be processed in a second channel, and a green color may be processed in a third channel. The network function setting layer 600 may be changed to process specific characteristics of an image in each channel. For example, in response to a user input for a display object displayed on the network function setting layer 600, the processor 120 may change the first channel to process blue instead of red. The network function setting layer 600 may determine in which channel among a plurality of channels a specific feature of an image is to be processed. For example, in response to a user input for a display object displayed on the network function setting layer 600, the processor 120 may change the red color of the image to be processed in the third channel instead of the first channel. The left image 710 shown in FIG. 7 is an example of an image set to process red, green, and blue colors, respectively, in the first, second, and third channels of the network function. The right image 720 shown in FIG. 7 is an example of an image set to process blue, green, and red colors, respectively, in the first, second, and third channels of the network function. The detailed description of the above-described network function setting is only an example, and the present disclosure is not limited thereto.

The image layer 200 may display the annotation form 210 for the label object object in response to a user annotation input for displaying the label object object. By displaying the annotation form 210 on the image displayed on the image layer 200, the user can determine the location of the label target object included in the image, and build a training data set.

The user annotation input may be a region selection input for displaying the region of the label target object. By displaying the area of the label object object to generate training data, the processor 120 may perform the identification of the position of the label object object or the label object object through an image operation using a network function. The processor 120 may check the location and shape of the label target object by using the user annotation input.

The annotation form 210 may be displayed in the shape of one annotation form input by user selection among one or more annotation form shapes. For example, the shape of the annotation form may be a square, a circle, a polygon, or the like. Although FIG. 2 illustrates an annotation form 210 displayed based on a rectangular annotation form shape, the present disclosure is not limited thereto, and a label target object area may be displayed in various shapes. For example, when the shape of the annotation form is a rectangle, the user's region selection input may be a single click and drag input. For example, when the shape of the annotation form is a polygon, the user's region selection input may be a plurality of click-and-drag inputs. For example, when the label target object is an airplane, the user may form the annotation form 210 along the edge of the airplane object. The detailed description of the above-described annotation form is only an example, and the present disclosure is not limited thereto.

The image layer 200 may display at least one of the size or position coordinates of the label target object in response to the user's input of the annotation form 210. When the image includes metadata such as a geographic coordinate system, the image layer 200 may display the size or position coordinates of the label target object in response to the user's input of the annotation form 210. For example, the image layer 200 may display the size in meters of a portion to which the annotation form 210 is input. The image layer 200 may display coordinates, such as latitude and longitude, of a portion where the annotation form 210 is input. For example, the image layer 200 may display a container size of 113m * 219m in response to an annotation form 210 input to a container box. When the image does not contain metadata, the image layer 200 may display the size of the label target object in pixels in response to the user's input of the annotation form 210. For example, in response to the annotation form 210 input to the container box, 120pix * 180pix, which is a pixel unit of the container image, may be displayed. The detailed description of the above-described annotation form is only an example, and the present disclosure is not limited thereto.

The user annotation input may include a region adjustment input for changing at least one of a position, a size, or an angle of the annotation form 210. The user annotation input may include an input for deleting the annotation form 210. For example, the annotation form 210 may be adjusted in response to a user's click-and-drag input on the annotation form 210. For example, in response to a click-and-drag input for an area within the annotation form 210, the image layer 200 may move and display the annotation form 210 in correspondence with the drag direction. Referring to FIG. 4, in response to an input for adjusting the position of the user's annotation form, the image layer 200 moves and displays the annotation form at the first position 410 to the second position 420. can do. For example, in response to a click-and-drag input on a border or corner of the annotation form 210, the image layer 200 may display the annotation form 210 by adjusting its size to correspond to the drag direction. For example, in response to a click-and-drag input on a circle part connected to the annotation form outside the annotation form 210, the image layer 200 may display the annotation form 210 by adjusting the angle corresponding to the drag direction. have. For example, based on a click on the annotation form 210 and an input to the Delete key on a keyboard, the image layer 200 may delete the annotation form 210 display. The detailed description of the above-described annotation form is only an example, and the present disclosure is not limited thereto.

In response to the user's copy input for copying the first annotation form 510 included in the image, the image layer may additionally display a second annotation form 520 having the same form as the first annotation form 510. I can. 5 is a diagram illustrating an example of copying an annotation form. For example, based on a click-and-drag input with a right mouse button on the inner area of the first annotation form 510, the image layer 200 is the same size as the first annotation form 510 in the dragged portion, A second annotation form 520 having a shape and an angle may be displayed. An image may contain more than one object to be labeled. At least some of the label target objects included in the image may be the label target objects of the same type. When a user creates an annotation form for each of the objects to be labeled with the same type, it may take a lot of time. In particular, it may take more time if the annotation form is a polygon that is displayed along the edge of an object rather than a rectangle. Therefore, it is possible to quickly and easily perform annotation display on the label target object of the same type by using the copy function of the annotation form. In addition, when the copy function of the annotation form is used, the efficiency of generating learning data may be increased. The detailed description of the above-described annotation form is only an example, and the present disclosure is not limited thereto.

In response to the display of the first annotation form for the first label target object included in the image, the image layer 200 includes a second label target object related to the first label target object, based on the first annotation form. An annotation form may be additionally displayed on the second label target object. The processor 120 may receive a first annotation form display input for the first label target object. The processor 120 may check the second label target object having the same shape as the first label target object included in the image. The processor 120 may display the second annotation form in the same form as the first annotation form with respect to the second label target object. That is, even if the user does not separately display the annotation form for the second label target object, if the user displays the annotation form for the first label target object, the processor 120 is the same type of the second label target object. You can display the annotation form automatically in the. A plurality of label target objects of the same shape may be included in the image. For example, container boxes of the same shape may be included in a satellite image. Even if the user displays the annotation form for only one of the container boxes, the processor 120 may also display the annotation form for other container boxes having the same shape. There may be a case in which an image includes a plurality of objects to be labeled with the same type. When a user displays an annotation form for each of a plurality of label target objects, one by one, time and cost may be consumed. According to an embodiment of the present disclosure, the processor 120 generates an annotation form for the remaining label target objects of the same type in response to an input for one label target object, thereby reducing the time and cost consumption of the user. . The detailed description of the above-described annotation form is only an example, and the present disclosure is not limited thereto.

The image layer 200 may generate and display a pseudo annotation form for a plurality of label target objects included in the image layer 200.

According to an embodiment of the present disclosure, the processor 120 may receive a user input for an annotation form for some of a plurality of label target objects included in an image. The processor 120 may display a capital annotation form on label target objects related to the label target objects that have received the annotation form user input, with reference to the annotation form user input. For example, the image may include 3 fighter label target objects and 5 container label target objects. When an annotation form is input for at least one fighter label target object and at least one container label target object, the processor 120 may also annotate two fighter label target objects and four container label objects having the same shape. A form may be created and displayed on the image layer 200. The specific description of the aforementioned capital annotation form is only an example, and the present disclosure is not limited thereto.

According to an embodiment of the present disclosure, the processor 120 may identify label target objects included in an image. The processor 120 may detect edges included in the image. For example, the processor 120 may detect edges included in the image using canny edge detection, harris corner detection, or the like. The processor 120 may identify objects included in the image using edges included in the image. The processor 120 may generate a capital annotation form along the edges of the identified objects. The specific description of the aforementioned capital annotation form is only an example, and the present disclosure is not limited thereto.

The image layer 200 may display a capital annotation form for one or more label target objects, and receive a user's correct answer or an error input for the capital annotation form. For example, the image layer 200 may sequentially display two or more capital annotation forms included in an image one by one. The processor 120 may cause the user to display a normal or error input for the corresponding water supply annotation form each time the water supply annotation form is displayed one by one. Alternatively, the image layer 200 may display two or more number annotation forms included in the image. The processor 120 may select only normal capital annotation forms from among two or more capital annotation forms displayed on the image layer 200 or select only erroneous capital annotation forms. The specific description of the aforementioned capital annotation form is only an example, and the present disclosure is not limited thereto.

The user interface may display an image layer 200 and a label layer 300. Hereinafter, it will be described with reference to FIG. 3. The processor 120 may display the label layer 300 on the user interface in response to a user annotation input from the image layer 200.

The label layer 300 may be a layer separated from the image layer 200. The label layer 300 may be a layer having a size different from that of the image layer 200. The label layer 300 may be a layer having a size smaller than that of the image layer 200. The label layer 300 may be an upper layer of the image layer 200. That is, the label layer 300 is displayed above the image layer 200, and at least a part of the image layer 200 may be covered by the label layer 300.

The label layer 300 may be moved independently from the image layer 200. That is, the label layer 300 may be a floating layer. The size of the label layer 300 may be adjusted based on a user input. For example, when it is desired to check the label target object at the lower right of the image layer 200, the label layer 300 located at the lower right may be moved to a different position. The detailed description of the above-described label layer is only an example, and the present disclosure is not limited thereto.

The label layer 300 may allow input of a user class to input a class for each of two or more layers included in a label of a label target object. Learning data corresponding to the label target object may be generated based on the user class input in the label layer 300. Based on the user class input in the label layer 300, a correct answer in the network model calculation for the label target object may be determined.

The label layer 300 includes a first class display object 312 corresponding to a first layer, a second class display object 314 corresponding to a second layer, a layer addition object 322, and a layer deletion object 324. Can be displayed.

The class display objects 312, 314, and 316 may be display objects for indicating which class each of the layers of the label target object is. For example, when receiving a user input that the class for the first layer is an aircraft carrier and the class for the second layer is a container box for the label object object, the first class display object 312 may display an aircraft carrier. And the second class display object 314 may display a container box. That is, the class display objects 312, 314, and 316 may be display objects for displaying a label input by a user or generated as a capital label. The detailed description of the above-described class display object is only an example, and the present disclosure is not limited thereto.

The label layer 300 may further display a third class display object 316 corresponding to the third layer in response to a user input to the layer addition object 322. For example, when a label target object may be displayed in three layers, and only class display objects for two layers are displayed on the label layer 300, a layer may be added using the layer addition object 322 . The detailed description of the above-described class display object is only an example, and the present disclosure is not limited thereto.

The label layer 300 may delete the second class display object 314 corresponding to the second layer in response to a user input to the layer deletion object 324. For example, when class display objects corresponding to each of the first layer and the second layer are displayed on the label layer 300, and if the label is to be displayed in only one layer, the processor 120 may use the second layer. A user input for deleting the second class display object 314 corresponding to. The label layer 300 may display only the first class display object 312 in response to a user's deletion input. The detailed description of the above-described class display object is only an example, and the present disclosure is not limited thereto.

The label layer 300 may display the template class selection area 330.

According to an embodiment of the present disclosure, the template class selection area 330 may display one or more first template classes corresponding to a first layer and one or more second template classes corresponding to a second layer. The template class selection area 330 may display one or more template classes corresponding to each of one or more layers. The processor 120 may receive a user input for template classes displayed in the template class selection area 330. The processor 120 may receive a selection input for each of the first template class of the first layer and the second template class of the second layer among a plurality of template classes displayed in the template class selection area 330. For example, the template class selection area 330 may display display objects for first template classes (aircraft carrier, sea) for the first layer. The template class selection area 330 may display display objects for second template classes (tank, container, airplane) for the second layer. The user may select one template class for the first layer and one template class for the second layer. Detailed description of the above-described template class is merely an example, and the present disclosure is not limited thereto.

The label layer 300 is a first class display object 312 corresponding to the first layer or a second class display object 312 corresponding to the second layer in response to a user label input for at least one of the first template class and the second template class. At least one of a first template class or a second template class corresponding to a user label input may be displayed on at least one of the class display objects 314, respectively. That is, the class display objects 312, 314, and 316 of the label layer 300 may display a class in response to a user's template class selection input for each layer in the template class selection area 330. Detailed description of the above-described template class is merely an example, and the present disclosure is not limited thereto.

According to an embodiment of the present disclosure, the template class selection area 330 may display a template class pair corresponding to each of the first layer and the second layer. For example, the template class selection area 330 may display'aircraft carrier/tank','aircraft carrier/container', and'aircraft carrier/airplane' template class pairs for the first layer and the second layer. . The processor 120 may receive a user label input for one template class from among a plurality of template class pairs. The user may select'aircraft carrier/container' of the template class selection area 330 when determining a label for an object to be labeled with a container from an image in which the container is placed on the aircraft carrier. Detailed description of the above-described template class is merely an example, and the present disclosure is not limited thereto.

The template class selection area 330 may further display additional template classes in response to a user input to at least one of a first additional template class corresponding to the first layer or a second additional template class corresponding to the second layer. I can.

The label layer 300 may display a template class additional display object 340. Add Template Class In response to a user selection input for the display object 340, the user may add an additional template class. The processor 120 may receive at least one additional template class for at least one layer. Alternatively, the processor 120 may receive an additional pair of template classes corresponding to a plurality of layers. Detailed description of the above-described template class is merely an example, and the present disclosure is not limited thereto.

In response to an input of the first class for the first label target object included in the image, the processor 120 may determine the first class as a label for other label target objects related to the first label target object. The processor 120 may identify a label target object having the same shape as the first label target object included in the image. The processor 120 may determine a label of the label target object having the same shape as the first label target object included in the image as the first class. For example, the image may include a plurality of objects to be labeled with an airplane. In response to the user's class input for one plane label object object (first layer/second layer:'aircraft carrier/airplane'), the processor 120 also performs'aircraft labeling' for the remaining plane label object objects of the same type. Classes of the first tier and the second tier may be determined as'mothership/airplane'. The specific description of the above-described labeling is only an example, and the present disclosure is not limited thereto.

In response to the input of the first class for the first label target object included in the image, the processor 120 provides a second label target object that is not related to the first label target object, which is a different class from the first class. Class can be determined by label. The processor 120 may identify a label target object having a different shape from the first label target object included in the image. The processor 120 may determine a label of the label target object in a form different from the first label target object included in the image as a second class that is a class different from the first class. For example, the image may include an airplane label target object and a container label target object. In response to the user's class input (first layer/second layer:'aircraft carrier/airplane') for the airplane label object object, the processor 120 is different for the container label object object of a different type from the airplane label object object. The label can be determined by the class'aircraft carrier/container'. The specific description of the above-described labeling is only an example, and the present disclosure is not limited thereto.

The processor 120 may generate learning data including an object included in the annotation form and a label corresponding to a user class input.

According to an embodiment of the present disclosure, the processor 120 may construct a training data set using the entire image. That is, the processor 120 may include an image and a label generated corresponding to the image in the training data set. The processor 120 may determine the image as input data of the training data set. The processor 120 may determine the annotation form 210 indicating the location of each of the label target objects included in the image and the labels for the label target objects as correct answers of the training data set. For example, the processor 120 may determine the satellite image image extracted from the satellite image as input data of the training data. For example, the processor 120 is an annotation form 210 for a fighter label target object included in a satellite image image and an annotation for a label (first layer: aircraft carrier / second layer: fighter) and container label target object. The form 210 and the label (first layer: aircraft carrier / second layer: container) may be determined as the correct answer of the training data (ie, the label of the input data). Detailed description of the above-described training data set is only an example, and the present disclosure is not limited thereto.

According to an embodiment of the present disclosure, the processor 120 may construct a training data set by separating only a portion of the annotation form 210 generated through an annotation input. That is, the processor 120 may include a portion of the annotation form 210 and a label generated corresponding to the annotation form 210 in the training data set. For example, the processor 120 may separate a portion of the annotation form 210 of the object object to be labeled with a container from an image to input training data. The processor 120 may determine a label (first layer: aircraft carrier / second layer: container) for the object object to be labeled as a correct answer of the training data (ie, a label of input data). Detailed description of the above-described training data set is only an example, and the present disclosure is not limited thereto.

The computing device 100 according to the embodiments of the present disclosure includes a personal computer (PC), a notebook (note book), a mobile terminal, a smart phone, a tablet PC, and the like. And all types of terminals that can access wired/wireless networks.

The network unit 110 may transmit/receive an image including a label target object according to an embodiment of the present disclosure, such as another computing device, a server, or the like. In addition, the network unit 110 may enable communication between a plurality of computing devices so that learning of a network function using data labeled by each of the plurality of computing devices is distributedly performed.

The network unit 110 according to an embodiment of the present disclosure may operate based on any type of wired/wireless communication technology currently used and implemented, such as short-range (short-range), long-range, wired, and wireless communication technologies.

The processor 120 may be composed of one or more cores, a central processing unit (CPU), a general purpose graphics processing unit (GPGPU), and a tensor processing unit (TPU) of a computing device. unit) may include a processor for displaying information for labeling. The processor 120 may provide a user interface that reads a computer program stored in the memory 130 and displays information for labeling according to an embodiment of the present disclosure. At least one of the CPU, GPGPU, and TPU of the processor 120 may provide or generate a user interface for displaying information for labeling. For example, the CPU and GPGPU can together provide a user interface that displays information for labeling. In addition, according to an embodiment of the present disclosure, a user interface for displaying information for labeling may be provided by using processors of a plurality of computing devices together. In addition, the computer program executed in the computing device according to an embodiment of the present disclosure may be a CPU, a GPGPU, or a TPU executable program.

According to an exemplary embodiment of the present disclosure, the memory 130 may store information in an arbitrary form generated or determined by the processor 120 and information in an arbitrary form received by the network unit 110.

According to an embodiment of the present disclosure, the memory 130 is a flash memory type, a hard disk type, a multimedia card micro type, and a card type memory (for example, SD or XD memory), RAM (Random Access Memory, RAM), SRAM (Static Random Access Memory), ROM (Read-Only Memory, ROM), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM (Programmable Read) -Only Memory), magnetic memory, magnetic disk, and optical disk. The computing device 100 may operate in connection with a web storage that performs a storage function of the memory 130 on the Internet. The description of the above-described memory is only an example, and the present disclosure is not limited thereto.

The output unit 140 according to an embodiment of the present disclosure may display an arbitrary user interface (UI) for providing information for labeling. The output unit 140 may display a user interface as illustrated in FIGS. 2 to 7. User interfaces shown below and described below are only examples, and the present disclosure is not limited thereto.

The output unit 140 according to an embodiment of the present disclosure may output information in an arbitrary form generated or determined by the processor 120 and information in an arbitrary form received by the network unit 110.

In an embodiment of the present disclosure, the output unit 140 is a liquid crystal display (LCD), a thin film transistor liquid crystal display (TFT LCD), and an organic light-emitting diode (OLED). , It may include at least one of a flexible display (flexible display), a 3D display (3D display). Some of these display modules may be configured as a transparent type or a light-transmitting type so that the outside can be seen through them. This may be referred to as a transparent display module, and a representative example of the transparent display module is TOLED (Transparant OLED).

A user input may be received through the input unit 150 according to an embodiment of the present disclosure. The input unit 150 according to an embodiment of the present disclosure may include keys and/or buttons for receiving a user input. A computer program for providing information for labeling according to embodiments of the present disclosure may be executed according to a user input through the input unit 150.

The input unit 150 according to embodiments of the present disclosure may receive a signal by sensing a button manipulation or a touch input of a user, or may receive a voice or motion of a user through a camera or a microphone and convert it into an input signal. . To this end, speech recognition technology or motion recognition technology may be used.

The input unit 150 according to embodiments of the present disclosure may be implemented as an external input device connected to the computing device 100. For example, the input device may be at least one of a touch pad, a touch pen, a keyboard, or a mouse for receiving a user input, but this is only an example and the present disclosure is not limited thereto.

The input unit 150 according to an embodiment of the present disclosure may recognize a user touch input. The input unit 150 according to an embodiment of the present disclosure may have the same configuration as the output unit 140. The input unit 150 may be configured as a touch screen implemented to receive a user's selection input. For the touch screen, any one of a contact type capacitive method, an infrared light sensing method, a surface ultrasound (SAW) method, a piezoelectric method, and a resistive film method may be used. Detailed description of the above-described touch screen is only an example according to an embodiment of the present invention, and various touch screen panels may be employed in the computing device 100. The input unit 150 configured as a touch screen may include a touch sensor. The touch sensor may be configured to convert a change in pressure applied to a specific portion of the input unit 150 or a capacitance generated at a specific portion of the input unit 150 into an electrical input signal. The touch sensor may be configured to detect not only a touched position and area, but also a pressure at the time of touch. When there is a touch input to the touch sensor, a signal(s) corresponding thereto is transmitted to the touch controller. The touch controller processes the signal(s) and then transmits the corresponding data to the processor 120. As a result, the processor 120 can recognize whether an area of the input unit 150 has been touched or the like.

In an embodiment of the present disclosure, the server may include other components for executing the server environment of the server. The server may include any type of device. The server is a digital device, and may be a digital device equipped with a processor, such as a laptop computer, a notebook computer, a desktop computer, a web pad, and a mobile phone, and equipped with a memory and computing power.

A server (not shown) performing an operation for providing a user interface displaying information for labeling to a user terminal according to an embodiment of the present disclosure may include a network unit, a processor, and a memory.

The server may generate a user interface according to embodiments of the present disclosure. The server may be a computing system that provides information to a client (eg, a user terminal) through a network. The server may transmit the generated user interface to the user terminal. In this case, the user terminal may be any type of computing device 100 capable of accessing the server. The processor of the server may transmit the user interface to the user terminal through the network unit. The server according to the embodiments of the present disclosure may be, for example, a cloud server. The server may be a web server that processes services. The types of servers described above are only examples, and the present disclosure is not limited thereto.

Each of the network unit, the processor, and the memory included in the server according to the embodiments of the present disclosure plays the same role as the network unit 110, the processor 120, and the memory 130 included in the above-described computing device 100. Can be performed or configured identically

8 is a flowchart illustrating a method of displaying information for labeling according to an embodiment of the present disclosure.

The computing device 100 may display 810 an image including the label target object on the image layer.

The computing device 100 may display 820 an annotation form for the label target object in response to a user annotation input for displaying the label target object on the image layer.

The computing device 100 may display 830 a display object that allows a user class input for inputting a class for each of two or more layers included in a label of the label target object on the label layer. The label layer may include a template class selection area displaying one or more first template classes corresponding to the first layer and one or more second template classes corresponding to the second layer.

The computing device 100 may further display a third class display object corresponding to the third layer in response to a user input for the layer addition object on the label layer.

The computing device 100 may delete the second class display object corresponding to the second layer in response to a user input for the layer deletion object in the label layer.

In response to a user label input for at least one of the first template class or the second template class, the computing device 100 may display a first class display object corresponding to the first layer or a second class display object corresponding to the second layer in the label layer. At least one of the first template class or the second template class corresponding to the user label input may be displayed on at least one of the class display objects, respectively.

In response to a user input to at least one of a first additional template class corresponding to a first layer or a second additional template class corresponding to a second layer in the template class selection area, the computing device 100 may further add additional template classes. Can be displayed.

In response to the display of the first annotation form for the first label target object included in the image on the image layer, the computing device 100 provides the first annotation form for the second label target object related to the first label target object. The based second annotation form may be additionally displayed on the second label target object.

In response to an input of the first class for the first label target object included in the image, the computing device 100 determines the first class as a label for the second label target object related to the first label target object, or A second class, which is a class different from the first class, for a third label target object not related to the first label target object may be determined as a label.

The user interface according to an embodiment of the present disclosure may be displayed by logic, modules, circuits or means capable of driving the above method for displaying the user interface.

The data structure may store data corresponding to a network function algorithm learned by using training data according to an embodiment of the present disclosure.

A computer-readable medium storing a data structure according to an embodiment of the present disclosure is disclosed.

Data structure can refer to the organization, management, and storage of data that enables efficient access and modification of data. The data structure may refer to an organization of data for solving a specific problem (eg, data search, data storage, data modification in the shortest time). Data structures may be defined as physical or logical relationships between data elements, designed to support specific data processing functions. The logical relationship between data elements may include a connection relationship between data elements that a user thinks. The physical relationship between the data elements may include an actual relationship between the data elements physically stored in a computer-readable storage medium (eg, hard disk). The data structure may specifically include a set of data, a relationship between data, and a function or instruction applicable to the data. Through an effectively designed data structure, the computing device can perform calculations while using the resources of the computing device to a minimum. Specifically, computing devices can increase the efficiency of computation, reading, insertion, deletion, comparison, exchange, and search through an effectively designed data structure.

The data structure can be classified into a linear data structure and a non-linear data structure according to the shape of the data structure. The linear data structure may be a structure in which only one data is connected after one data. The linear data structure may include a list, a stack, a queue, and a deck. The list may mean a series of data sets in which order exists internally. The list may include a linked list. The linked list may be a data structure in which data is linked in a way that each data has a pointer and is linked in a single line. In the linked list, the pointer may include connection information with the next or previous data. The linked list can be expressed as a single linked list, a double linked list, or a circular linked list. The stack can be a data listing structure that allows limited access to data. The stack may be a linear data structure that can process (eg, insert or delete) data only at one end of the data structure. The data stored in the stack may be a data structure (LIFO-Last in First Out) that comes out sooner as it enters the stack. A queue is a data arrangement structure that allows limited access to data, and unlike a stack, a queue may be a data structure (FIFO-First in First Out) that comes out later as data stored later. A deck can be a data structure that can process data at both ends of the data structure.

The nonlinear data structure may be a structure in which a plurality of data is connected behind one data. The nonlinear data structure may include a graph data structure. The graph data structure may be defined as a vertex and an edge, and the edge may include a line connecting two different vertices. The graph data structure may include a tree data structure. The tree data structure may be a data structure in which a path connecting two different vertices among a plurality of vertices included in the tree is one. That is, it may be a data structure that does not form a loop in the graph data structure.

Throughout this specification, a computational model, a neural network, a network function, and a neural network may be used with the same meaning. (Hereinafter, it will be unified and described as a neural network.) The data structure may include a neural network. In addition, the data structure including the neural network may be stored in a computer-readable medium. The data structure including the neural network may also include data input to the neural network, weights of the neural network, hyperparameters of the neural network, data obtained from the neural network, an active function associated with each node or layer of the neural network, and a loss function for learning the neural network. have. The data structure including the neural network may include any of the components disclosed above. That is, the data structure including the neural network is all or all of the data input to the neural network, the weight of the neural network, the hyper parameter of the neural network, the data obtained from the neural network, the active function associated with each node or layer of the neural network, and the loss function for training the neural network. It may be configured to include any combination of. In addition to the above-described configurations, the data structure including the neural network may include any other information that determines the characteristics of the neural network. In addition, the data structure may include all types of data that are used or generated in a neural network operation process, and is not limited to the above-described matters. Computer-readable media may include computer-readable recording media and/or computer-readable transmission media. A neural network may consist of a set of interconnected computational units, which may generally be referred to as nodes. These nodes may also be referred to as neurons. The neural network is composed of at least one or more nodes.

The data structure may include data input to the neural network. A data structure including data input to the neural network may be stored in a computer-readable medium. The data input to the neural network may include training data input during the neural network training process and/or input data input to the neural network on which the training has been completed. Data input to the neural network may include data that has undergone pre-processing and/or data to be pre-processed. Pre-processing may include a data processing process for inputting data into a neural network. Accordingly, the data structure may include data to be pre-processed and data generated by pre-processing. The above-described data structure is only an example, and the present disclosure is not limited thereto.

The data structure may include data input to or output from the neural network. A data structure including data input to or output from the neural network may be stored in a computer-readable medium. The data structure stored in the computer-readable medium may include data input during an inference process of a neural network or output data output as a result of inference of a neural network. In addition, since the data structure may include data processed by a specific data processing method, it may include data before and after processing. Accordingly, the data structure may include data to be processed and data processed through a data processing method.

The data structure may include weights of the neural network. (In the present specification, weights and parameters may have the same meaning.) In addition, a data structure including a weight of a neural network may be stored in a computer-readable medium. The neural network may include a plurality of weights. The weight may be variable and may be changed by a user or an algorithm in order for a neural network to perform a desired function. For example, when one or more input nodes are interconnected to one output node by respective links, the output node includes values input to input nodes connected to the output node and a link corresponding to each input node. The output node value may be determined based on the parameter. The data structure described above is only an example, and the present disclosure is not limited thereto.

As an example and not a limitation, the weight may include a weight variable in a neural network training process and/or a weight on which neural network training has been completed. The variable weight in the neural network learning process may include a weight at a time point at which the learning cycle starts and/or a weight variable during the learning cycle. The weight at which the neural network training is completed may include the weight at which the learning cycle is completed. Accordingly, the data structure including the weight of the neural network may include a data structure including a weight variable during the neural network training process and/or a weight on which the neural network training has been completed. Therefore, it is assumed that the above-described weights and/or combinations of weights are included in the data structure including the weights of the neural network. The above-described data structure is only an example, and the present disclosure is not limited thereto.

The data structure including the weights of the neural network may be serialized and then stored in a computer-readable storage medium (eg, memory, hard disk). Serialization may be a process of storing data structures on the same or different computing devices and converting them into a form that can be reconstructed and used later. The computing device serializes the data structure to transmit and receive data through a network. The data structure including the weights of the serialized neural network can be reconstructed in the same or different computing devices through deserialization. The data structure including the weights of the neural network is not limited to serialization. Furthermore, the data structure including the weight of the neural network is a data structure to increase the efficiency of operation while using the resources of the computing device to a minimum (e.g., B-Tree, Trie, m-way search tree, AVL tree, Red-Black Tree). The foregoing is only an example, and the present disclosure is not limited thereto.

The data structure may include a hyper-parameter of a neural network. In addition, the data structure including the hyper parameter of the neural network may be stored in a computer-readable medium. The hyper parameter may be a variable that is variable by a user. Hyper parameters include, for example, learning rate, cost function, number of iterations of learning cycles, weight initialization (e.g., setting the range of weight values to be weighted to be initialized), Hidden Unit It may include the number (eg, the number of hidden layers, the number of nodes of the hidden layer). The above-described data structure is only an example, and the present disclosure is not limited thereto.

9 is a block diagram of a computing device according to an embodiment of the present disclosure.

9 shows a simplified and general schematic diagram of an exemplary computing environment in which embodiments of the present disclosure may be implemented.

While the present disclosure has generally been described above with respect to computer-executable instructions that can be executed on one or more computers, those skilled in the art will appreciate that the present disclosure may be implemented in combination with other program modules and/or as a combination of hardware and software. will be.

Generally, program modules include routines, programs, components, data structures, etc. that perform particular tasks or implement particular abstract data types. In addition, to those skilled in the art, the method of the present disclosure is not limited to single-processor or multiprocessor computer systems, minicomputers, mainframe computers, as well as personal computers, handheld computing devices, microprocessor-based or programmable household appliances, etc. (each of which is It will be appreciated that other computer system configurations may be implemented, including one that may operate in connection with one or more associated devices.

The described embodiments of the present disclosure may also be practiced in a distributed computing environment where certain tasks are performed by remote processing devices that are connected through a communication network. In a distributed computing environment, program modules may be located in both local and remote memory storage devices.

Computers typically include a variety of computer-readable media. Any medium accessible by the computer may be a computer-readable medium. Computer-readable media includes volatile and nonvolatile media, transitory and non-transitory media, and removable and non-removable media. By way of example and not limitation, computer-readable media may include computer-readable storage media and computer-readable transmission media. Computer-readable storage media include volatile and nonvolatile media, temporary and non-transitory media, removable and non-removable media implemented in any method or technology for storing information such as computer readable instructions, data structures, program modules or other data. Includes the medium. Computer storage media include RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital video disk (DVD) or other optical disk storage device, magnetic cassette, magnetic tape, magnetic disk storage device, or other magnetic storage device, Or any other medium that can be accessed by a computer and used to store the desired information.

Computer-readable transmission media typically embodies computer-readable instructions, data structures, program modules or other data in a modulated data signal, such as other transport mechanisms, and includes all information delivery media. do. The term modulated data signal refers to a signal in which one or more of the characteristics of the signal is set or changed so as to encode information in the signal. By way of example and not limitation, computer-readable transmission media include wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared, and other wireless media. Combinations of any of the above-described media are also intended to be included within the scope of computer-readable transmission media.

An exemplary environment 1100 is shown that implements various aspects of the present disclosure, including a computer 1102, which includes a processing unit 1104, a system memory 1106, and a system bus 1108. do. System bus 1108 couples system components, including but not limited to, system memory 1106 to processing device 1104. The processing unit 1104 may be any of a variety of commercially available processors. Dual processors and other multiprocessor architectures may also be used as processing unit 1104.

The system bus 1108 may be any of several types of bus structures that may be additionally interconnected to a memory bus, a peripheral bus, and a local bus using any of a variety of commercial bus architectures. System memory 1106 includes read-only memory (ROM) 1110 and random access memory (RAM) 1112. The basic input/output system (BIOS) is stored in non-volatile memory 1110 such as ROM, EPROM, EEPROM, etc. This BIOS is a basic input/output system that helps transfer information between components in the computer 1102, such as during startup. Includes routines. RAM 1112 may also include high speed RAM such as static RAM for caching data.

The computer 1102 also includes an internal hard disk drive (HDD) 1114 (e.g., EIDE, SATA)-this internal hard disk drive 1114 can also be configured for external use within a suitable chassis (not shown). Yes-, magnetic floppy disk drive (FDD) 1116 (for example, to read from or write to removable diskette 1118), and optical disk drive 1120 (e.g., CD-ROM For reading the disk 1122 or for reading from or writing to other high-capacity optical media such as DVD). The hard disk drive 1114, the magnetic disk drive 1116, and the optical disk drive 1120 are each connected to the system bus 1108 by a hard disk drive interface 1124, a magnetic disk drive interface 1126, and an optical drive interface 1128. ) Can be connected. The interface 1124 for implementing an external drive includes at least one or both of Universal Serial Bus (USB) and IEEE 1394 interface technologies.

These drives and their associated computer readable media provide non-volatile storage of data, data structures, computer executable instructions, and the like. In the case of computer 1102, drives and media correspond to storing any data in a suitable digital format. Although the description of the computer-readable medium above refers to a removable optical medium such as a HDD, a removable magnetic disk, and a CD or DVD, those skilled in the art may use a zip drive, a magnetic cassette, a flash memory card, a cartridge, etc. It will be appreciated that other types of media readable by a computer, such as the like, may also be used in the exemplary operating environment and that any such media may contain computer executable instructions for performing the methods of the present disclosure.

A number of program modules, including the operating system 1130, one or more application programs 1132, other program modules 1134, and program data 1136, may be stored in the drive and RAM 1112. All or part of the operating system, applications, modules and/or data may also be cached in RAM 1112. It will be appreciated that the present disclosure may be implemented on several commercially available operating systems or combinations of operating systems.

A user may input commands and information to the computer 1102 through one or more wired/wireless input devices, for example, a pointing device such as a keyboard 1138 and a mouse 1140. Other input devices (not shown) may include a microphone, IR remote control, joystick, game pad, stylus pen, touch screen, and the like. These and other input devices are often connected to the processing unit 1104 through the input device interface 1142, which is connected to the system bus 1108, but the parallel port, IEEE 1394 serial port, game port, USB port, IR interface, It can be connected by other interfaces such as etc.

A monitor 1144 or other type of display device is also connected to the system bus 1108 through an interface such as a video adapter 1146. In addition to the monitor 1144, the computer generally includes other peripheral output devices (not shown) such as speakers, printers, and the like.

Computer 1102 may operate in a networked environment using logical connections to one or more remote computers, such as remote computer(s) 1148, via wired and/or wireless communication. The remote computer(s) 1148 may be a workstation, a computing device computer, a router, a personal computer, a portable computer, a microprocessor-based entertainment device, a peer device, or other common network node, and is generally connected to the computer 1102. Although it includes many or all of the components described for simplicity, only memory storage device 1150 is shown. The logical connections shown include wired/wireless connections to a local area network (LAN) 1152 and/or to a larger network, eg, a wide area network (WAN) 1154. Such LAN and WAN networking environments are common in offices and companies, and facilitate enterprise-wide computer networks such as intranets, all of which can be connected to worldwide computer networks, for example the Internet.

When used in a LAN networking environment, the computer 1102 is connected to the local network 1152 via a wired and/or wireless communication network interface or adapter 1156. Adapter 1156 may facilitate wired or wireless communication to LAN 1152, which also includes a wireless access point installed therein to communicate with wireless adapter 1156. When used in a WAN networking environment, the computer 1102 may include a modem 1158, connected to a communication computing device on the WAN 1154, or through the Internet, to establish communication through the WAN 1154. Have other means. Modem 1158, which may be an internal or external and a wired or wireless device, is connected to the system bus 1108 through a serial port interface 1142. In a networked environment, program modules described for the computer 1102 or portions thereof may be stored in the remote memory/storage device 1150. It will be appreciated that the network connections shown are exemplary and other means of establishing communication links between computers may be used.

Computer 1102 is associated with any wireless device or entity deployed and operated in wireless communication, e.g., a printer, scanner, desktop and/or portable computer, portable data assistant (PDA), communication satellite, wireless detectable tag. It operates to communicate with any device or place, and a phone. This includes at least Wi-Fi and Bluetooth wireless technologies. Accordingly, the communication may be a predefined structure as in a conventional network or may simply be ad hoc communication between at least two devices.

Wi-Fi (Wireless Fidelity) allows you to connect to the Internet or the like without wires. Wi-Fi is a wireless technology such as a cell phone that allows such devices, for example computers, to transmit and receive data indoors and outdoors, that is, anywhere within the coverage area of a base station. Wi-Fi networks use a wireless technology called IEEE 802.11 (a, b, g, etc.) to provide a secure, reliable and high-speed wireless connection. Wi-Fi can be used to connect computers to each other, to the Internet, and to a wired network (using IEEE 802.3 or Ethernet). Wi-Fi networks can operate in unlicensed 2.4 and 5 GHz radio bands, for example, at a data rate of 11 Mbps (802.11a) or 54 Mbps (802.11b), or in products that include both bands (dual band). have.

Those of ordinary skill in the art of this disclosure will understand that information and signals may be represented using any of a variety of different technologies and techniques. For example, data, instructions, commands, information, signals, bits, symbols and chips that may be referenced in the above description are voltages, currents, electromagnetic waves, magnetic fields or particles, optical fields. Or particles, or any combination thereof.

Those of ordinary skill in the art of the present disclosure include various exemplary logical blocks, modules, processors, means, circuits and algorithm steps described in connection with the embodiments disclosed herein, electronic hardware For the sake of clarity, it will be appreciated that it may be implemented by various forms of program or design code or a combination of both (referred to herein as "software"). To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the particular application and design constraints imposed on the overall system. A person of ordinary skill in the art of the present disclosure may implement the described functions in various ways for each specific application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure.

The various embodiments presented herein may be implemented as a method, apparatus, or article of manufacture using standard programming and/or engineering techniques. The term “article of manufacture” includes a computer program or media accessible from any computer-readable device. For example, computer-readable media include magnetic storage devices (e.g., hard disks, floppy disks, magnetic strips, etc.), optical disks (e.g., CD, DVD, etc.), smart cards, and flash memory. Devices (eg, EEPROM, card, stick, key drive, etc.). In addition, the various storage media presented herein include one or more devices and/or other machine-readable media for storing information.

It is to be understood that the specific order or hierarchy of steps in the presented processes is an example of exemplary approaches. Based on the design priorities, it is to be understood that within the scope of this disclosure a specific order or hierarchy of steps in processes may be rearranged. The appended method claims provide elements of the various steps in a sample order, but are not meant to be limited to the specific order or hierarchy presented.

A description of the presented embodiments is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these embodiments will be apparent to those of ordinary skill in the art. The general principles defined herein may be applied to other embodiments without departing from the scope of the present disclosure. Thus, the present disclosure is not to be limited to the embodiments presented herein, but is to be construed in the widest scope consistent with the principles and novel features presented herein.

Claims (14)

A computer program stored in a computer-readable storage medium, the computer program providing a user interface (UI) for displaying information for labeling when executed on one or more processors of a computing device, the user interface comprising:
An image layer that displays an image including the label object object, and displays an annotation form for the label object object in response to a user annotation input for displaying the label object object; And
A label layer allowing a user class input for inputting a class for each of two or more layers included in the label of the label object object;
Including,
The label layer is:
A template class selection area displaying one or more first template classes corresponding to a first layer and one or more second template classes corresponding to a second layer;
Contains, and
In response to a user label input for at least one of the first template class or the second template class, at least one of a first class display object corresponding to the first layer or a second class display object corresponding to the second layer Displaying at least one of the first template class and the second template class corresponding to the user label input on one, respectively,
A computer program stored on a computer readable storage medium.
The method of claim 1,
The label layer is:
Displaying a first class display object corresponding to a first layer, a second class display object corresponding to a second layer, a layer addition object, and a layer deletion object,
A computer program stored on a computer readable storage medium.
The method of claim 2,
The label layer is:
In response to a user input for the layer addition object, a third class display object corresponding to the third layer is further displayed, or
Deleting the second class display object corresponding to the second layer in response to a user input to the layer deletion object,
A computer program stored on a computer readable storage medium.
delete The method of claim 1,
The template class selection area is:
In response to a user input to at least one of a first additional template class corresponding to the first layer or a second additional template class corresponding to the second layer, further displaying the additional template classes,
A computer program stored on a computer readable storage medium.
The method of claim 1,
The user annotation input is:
A region selection input for displaying the region of the label object object; And
A region adjustment input for changing at least one of a position, a size, or an angle of the annotation form;
Containing,
A computer program stored on a computer readable storage medium.
The method of claim 1,
The annotation form is:
Displayed as one user-selected annotation form shape among one or more annotation form shapes,
A computer program stored on a computer readable storage medium.
The method of claim 1,
The image layer is:
In response to a user's copy input for copying the first annotation form included in the image, further displaying a second annotation form having the same form as the first annotation form,
A computer program stored on a computer readable storage medium.
The method of claim 1,
The image layer is:
In response to displaying the first annotation form for the first label target object included in the image, a second annotation form based on the first annotation form is provided for the second label target object related to the first label target object. Additionally displayed on the second label target object,
A computer program stored on a computer readable storage medium.
The method of claim 1,
The processor is:
In response to a first class input for a first label target object included in the image,
Determining the first class as a label for a second label target object related to the first label target object; or
Determining a second class, which is a class different from the first class, as a label for a third label target object not related to the first label target object;
Containing,
A computer program stored on a computer readable storage medium.
The method of claim 1,
The processor is:
Generating learning data including an object included in the annotation form and a label corresponding to the user class input;
Containing,
A computer program stored on a computer readable storage medium.
As a method of displaying labeling information in a user terminal,
Displaying an image including the label target object on the image layer;
Displaying an annotation form for the label object object in response to a user annotation input for displaying the label object object on the image layer; And
Displaying a display object allowing input of a user class for inputting a class for each of two or more layers included in a label of the label target object on a label layer;
Including,
The label layer is:
A template class selection area displaying one or more first template classes corresponding to a first layer and one or more second template classes corresponding to a second layer;
Contains, and
In response to a user label input for at least one of the first template class or the second template class, at least one of a first class display object corresponding to the first layer or a second class display object corresponding to the second layer Displaying at least one of the first template class and the second template class corresponding to the user label input on one, respectively,
A method of displaying labeling information in a user terminal.
As a user terminal,
A processor including one or more cores;
Memory; And
An output unit providing a user interface;
Including,
The user interface,
An image layer that displays an image including the label object object, and displays an annotation form for the label object object in response to a user annotation input for displaying the label object object; And
A label layer allowing a user class input for inputting a class for each of two or more layers included in the label of the label object object;
Including,
The label layer is:
A template class selection area displaying one or more first template classes corresponding to a first layer and one or more second template classes corresponding to a second layer;
Contains, and
In response to a user label input for at least one of the first template class or the second template class, at least one of a first class display object corresponding to the first layer or a second class display object corresponding to the second layer Displaying at least one of the first template class and the second template class corresponding to the user label input on one, respectively,
User terminal.
As a server,
A processor including one or more cores;
Network unit; And
Memory;
Including,
The processor determines to transmit a user interface (UI) to the user terminal through the network unit,
The user interface is:
An image layer that displays an image including the label object object, and displays an annotation form for the label object object in response to a user annotation input for displaying the label object object; And
A label layer allowing a user class input for inputting a class for each of two or more layers included in the label of the label object object;
Including,
The label layer is:
A template class selection area displaying one or more first template classes corresponding to a first layer and one or more second template classes corresponding to a second layer;
Contains, and
In response to a user label input for at least one of the first template class or the second template class, at least one of a first class display object corresponding to the first layer or a second class display object corresponding to the second layer Displaying at least one of the first template class and the second template class corresponding to the user label input on one, respectively,
server.

Images