JP4553300B2 - Content identification device - Google Patents

Description

  The present invention relates to a content identification device, and more particularly to a content identification device that can identify content at high speed and with high accuracy using the feature amount of the content.

  2. Description of the Related Art Conventionally, in order to search for a desired content from a plurality of contents, it has been performed that metadata indicating a feature amount is assigned to each content in advance. The search for content is performed via metadata, not the content itself.

  The provision of metadata to each content is originally performed by manually describing and giving the description to each content. In addition, the content search is generally performed by presenting content to which an explanatory note that matches the text input at the time of search is presented.

  On the other hand, a method for automating extraction of metadata to be added to content has also been proposed. For example, Patent Literature 1 classifies colors into a plurality of color groups, calculates an occupancy ratio that is a ratio of pixels of each color group in the entire image, and calculates the pixel occupancy ratio of the calculated color group and the color group. It is described that the representative color of the name or color group is used as metadata. At the time of search, the similarity between images can be calculated based on the Euclidean distance between metadata, and an image with a high similarity can be presented as a search result.

  Patent Document 2 describes that a grayscale image including a face area is made into a mosaic, and the mosaic of the face area is extracted as metadata using the fact that the mosaic pattern is different inside and outside the face area. Describes that a ruled line pattern extracted by removing a part other than a ruled line from an image is quantified as metadata. In these, image search or image classification can be performed by using the inner product of normalized metadata as distances.

  Patent Document 4 describes that a color histogram, edge pixel information, and edge pixel change information of two frames are used as metadata, and a similar image is searched using a self-organizing map. Describes searching for similar images by cluster analysis using the color histogram extracted for each image of different resolution, the luminance average for each block, and the edge amount as metadata.

Further, Patent Document 6 describes that similar images are searched using a discriminant analysis method using the representative colors obtained by annealing M-estimator and their arrangement as metadata.
Japanese Patent Laid-Open No. 11-96364 JP-A-8-221547 JP-A-7-160844 JP 11-39325 A JP 2003-256427 A JP 2003-676764 A

  However, in the above-mentioned primitive method, metadata must be manually assigned to each content, and when the amount of content has reached a huge amount, the workload of adding metadata spreads and is not realistic. In addition, the explanatory text given to each content needs to be objective, but subjective judgment must be entered, and similar content is given to different content, There is a problem that a different explanation may be given.

  According to the methods described in Patent Documents 1 to 6, objective metadata can be automatically extracted, and thus the above-described problems are solved. However, the simple histogram of the pixel occupancy ratio of each color group extracted by the method described in Patent Document 1 cannot be said to accurately capture the feature of the content, resulting in a decrease in content identification accuracy. I have a problem. Further, since it is necessary to measure the Euclidean distance of individual metadata for identification, processing time becomes a problem as the number of contents increases.

  In addition, the methods described in Patent Documents 2 and 3 are such that the feature amount of an image is a mosaic pattern or ruled line pattern of a face area in particular, so that the processing target is further limited to a face image or a document image in the image. There is a problem that versatility is poor.

  Furthermore, the methods of Patent Documents 4 to 6 each have a problem that it takes a long processing time because it is necessary to extract a large amount of metadata by calculation. Further, since different types of metadata such as color histograms and edges are collectively used for identification, there is a problem that it is difficult to set parameters in consideration of mutual metadata.

  The object of the present invention is to solve the above-mentioned problems and identify whether any unknown content is content to be identified (positive example content) or non-identification content (negative example content) with high speed and high accuracy. It is an object of the present invention to provide a content identification device that can do this.

In order to solve the above problems, the present invention provides a content identification device for identifying whether the unknown content is content identification target, the content of the identification target prepared in advance (hereinafter, referred to as a positive sample teacher content .) of the feature with a prepared identification-target content (hereinafter, by performing learning based on feature amounts of referred to as negative examples teacher content.), each category of negative examples teacher content, and wherein a learning processing means for constructing a plurality of learning models for each amount, the unknown feature quantity and the learning process the unknown content identified based on the learning model selected from a plurality of learning models built by means of the content A plurality of identification means for each classification of negative example teacher content for identifying whether or not the content is a target content (hereinafter referred to as positive example content); The unknown content from the identification result by the serial plurality of identification means and an identifying identification processing means whether a positive cases content, the learning processing means, first of extracting a feature quantity of positive cases teacher content Extraction means , second extraction means for extracting feature values of negative example teacher contents, classification means for classifying the negative example teacher contents according to the feature values extracted by the second extraction means, and classification means The positive teacher content is provided based on the feature amount extracted by the first extraction means and the feature amount of each negative example teacher content classified by the classification means. and optimally classify negative example teacher content, a plurality of learning means for constructing a learning model for each feature value, and provided corresponding to each of the plurality of learning means, said classification means The optimal feature amount of the learning model corresponding to the classified individual negative examples teacher content was, provided corresponding to each of the selection to the plurality of learning means from learning model for each feature value that is constructed by the learning means It has a basic feature of having a selection means to be given to the identified identification means .

  According to the present invention, it is possible to identify with high accuracy whether unknown content is positive content or negative content. Further, by adaptively selecting the feature amount used for identification for each classified negative example teacher content, the feature amount extracted from the unknown content at the time of identification can be minimized. In addition, the identification processing is arranged in a hierarchical manner so that the distance between the feature amounts is maximized, and the identification amount is deteriorated by extracting the necessary feature amounts for each identification processing and performing the stepwise identification processing. The processing speed can be increased without any problems.

  The present invention will be described below with reference to the drawings. FIG. 1 is a functional block diagram showing an embodiment of a content identification apparatus according to the present invention. In the following description, it is assumed that the content is a still image, and the feature amount as metadata of the still image is mainly color information and shape information of the still image such as hue, color distribution, composition, and pattern. However, the present invention is not limited to these, and any content such as audio or moving images can be processed, and any feature can be implemented as metadata.

The present invention includes a learning processing unit 10 that constructs a learning model by performing learning using teacher content and an identification processing unit 20 for unknown content as basic configurations. These processing means can be configured by software, but may be configured by hardware.
(1) Learning processing means 10

  First, the learning processing means 10 will be described. In the learning process, an image that is known to be within the identification target and an image that is known to be outside the identification target are prepared as teacher content. In this specification, an image to be identified is referred to as positive example teacher content, and an image that is not to be identified is referred to as negative example teacher content. For example, when it is desired to identify whether an unknown image is a person image, the person image is a positive example teacher content, and all images other than the person image are negative example teacher contents.

  Next, positive example teacher contents and negative example teacher contents are input to the extraction means 11 and 12, respectively, and feature quantities are extracted from the respective teacher contents to form metadata. The metadata can be any feature amount, but it is compatible with the descriptors defined by the international standard MPEG-7, that is, Color Layout, Scalable Color, Dominant Color, Color Structure and Edge Histgram. This is convenient because it can realize a versatile device.

  The negative example teacher content includes, for example, various types of images other than the person image when the person image is the positive example content. Perform clustering according to distance. For this clustering, a K-mean method or a self-organizing map can be used. Also, since the types of negative example teacher content prepared in the learning process are known in advance, the negative example teacher content prepared here and the metadata extracted from it are clustered by manual operation, for example. You can also.

  From the negative example teacher contents by the clustering by the classification means 13, for example, the metadata of the sea image is classified as cluster A, and the metadata of the mountain image is classified as cluster B.

  The learning means (1) to (3) 14 to 16 use the metadata of the positive example teacher contents extracted by the extraction means 11 and the metadata of the individual negative example teacher contents clustered by the classification means 12. A separation hyperplane that is optimally separated into two groups of positive example teacher content and negative example teacher content is calculated. For example, the learning means (1) uses the metadata of the person image extracted by the extracting means 11 and the metadata of the sea image extracted by the extracting means 12 and classified by the classifying means 13, and uses the example content. A separation hyperplane that separates an image of a person and a sea image as negative example content is calculated. When there are a plurality of metadata, a separation hyperplane is calculated for each metadata. For example, a support vector machine (SVM) or discriminant analysis can be used to calculate the separation hyperplane.

  For support vector machines (SVM), for example, “Vapnik: Statistical learning theory, A Wiley-Interscience Publication, 1998”, “CC.CHANG, CW.IISU AND CJ LIN, The analysis of decomposition methods for support vector machines, IEEE Transaction” on Neural Networks, 11 (4) pp.1003-1008 ”.

   FIG. 2 is an explanatory diagram showing the concept of SVM. As shown in FIG. 2, the amounts of different elements a and b in a given metadata are plotted on the vertical and horizontal axes, respectively. Plot the position. For example, if the metadata is Scalable Color (HSV color histogram scalable representation), taking red and blue on the vertical and horizontal axes respectively, the metadata of each image of the person is plotted at the position of “ The metadata of each image is plotted at the position of “x”. Note that FIG. 2 shows a case where there are two types of elements a and b in a certain metadata.

  As shown in FIG. 2, the SVM has a separation hyperplane h that serves as a separation threshold. As described above, the separation hyperplane h gives the metadata of the positive example teacher content and the negative example teacher content to the SVM, performs learning for each metadata, and based on the learning result, the positive example teacher content and the negative example It is set so that teacher content is optimally separated. The identification accuracy of the learning model depends on the separation distance between the separation hyperplane h obtained when the metadata of various positive example teacher contents and negative example teacher contents is given to the SVM and the closest plot position. .

  As described above, a learning model for classifying positive content or negative content is constructed for each cluster and each metadata. For example, for a negative example teacher content classified as cluster A (sea image) by the classification means 13, a learning model of Color Layout for identifying whether a person image (positive example content) or a sea image (negative example content), Scalable Color learning model, Dominant Color learning model, Color Structure learning model, Edge Histgram learning model, etc. are built. Similarly, with regard to negative example teacher content classified as cluster B (mountain image), a Color Layout learning model for identifying whether a human image (positive example content) or a mountain image (negative example content), Scalable Color Learning model, Dominant Color learning model, Color Structure learning model, Edge Histgram learning model, etc. are constructed.

  Selection means (1) to (3) 17 to 19 adaptively use metadata according to the type of negative example teacher content for a plurality of metadata extracted from positive example teacher content and negative example teacher content. Therefore, the metadata and the learning model having the optimum discrimination performance with the positive teacher content are selected for each cluster. For example, for the cluster A (sea image), Color Layout metadata and its learning model are selected, and for the cluster B (mountain image), Edge Histgram metadata and its learning model are selected.

  The metadata for each cluster selected by the selection means (1) to (3) 17 to 19 and the learning model thereof are respectively stored in the corresponding identification means (1) to (3) 21 to 23 of the identification processing means 20. Given.

In addition, when the optimal metadata in a certain cluster is known in advance, for example, the optimal metadata for identifying an ocean image as negative example content is Scalable Color in advance, or in the previous ocean image If it is known from learning, only a scalable color can be extracted as metadata for a sea image for subsequent learning, and a learning model for the entire sea image can be generated or modified.
(2) Identification processing means 20

  Next, the identification processing means 20 will be described. The identification processing means 20 includes selection means (1) to (3) 17 to 19, that is, a plurality of identification means (1) to (3) 21 to 23 as many as the clusters. Unknown content is given to the identification processing means 20, and positive content or negative content is identified.

  Here, it is conceivable to perform the identification processing in the identification means (1) to (3) 21 to 23 independently, but the identification means (1) to (3) 21 to 23 are connected to each other, for example, As will be described later, stepwise identification processing is preferably performed on unknown content. An unknown content is determined to be a positive content when it is identified as a positive content by all the identification means (1) to (3) 21 to 23. Further, if any of the identification means (1) to (3) 21 to 23 is identified as negative example content, it is determined as negative example content, and at that stage, the identification process is interrupted and further processing is performed. Absent.

  FIG. 3 is a flowchart showing an example of the procedure of identification processing in the identification processing means 20. The individual identification means (1) to (3) 21 to 23 in the identification processing means 20 use only the optimum metadata for each cluster selected by the selection means (1) to (3) 17 to 19 as unknown content. Extract from Note that if this metadata has already been extracted by other identifying means, the metadata can be reused, and a process for extracting new metadata is not necessary.

  Each of the identification means (1) to (3) 21 to 23 has an unknown content generated by SVM or discriminant analysis based on the learning model derived for each cluster by the learning means (1) to (3) 14 to 16. Whether the content belongs to positive content or negative content is identified. The identification processing in each of the identification means (1) to (3) 21 to 23 is based on a learning model that separates positive example contents or negative example contents, and therefore should be performed at high speed without depending on the number of teacher contents. Can do.

  FIG. 3 shows an example in which unknown content is identified as cluster X, which is positive example content, and clusters A, B, and C, which are negative example content. The unknown content is first given to the identification means (1) 21 to be identified other than the cluster A and the cluster A. Here, the content identified as cluster A is determined as negative example content. The content identified as other than the cluster A is given to the identification means (2) 22 to be identified other than the cluster B and the cluster B. Here, the content identified as cluster B is determined to be a negative example content. Next, the contents identified as other than the cluster B are given to the identification means (3) 23 and identified as the cluster X and the cluster C. The content identified as cluster X is determined as positive example content, and the content identified as cluster C is determined as negative example content.

  Reduce the burden of identification processing by setting the application order of identification means according to the appearance frequency of each cluster in unknown content, or by setting the application order of identification means according to the identification accuracy of each learning model And high speed and high accuracy.

  For example, rather than prioritizing identification processing for identifying clusters that are known to have a high appearance frequency in unknown content as negative example content, many clusters in unknown content are identified as negative example content at an early stage. Can be excluded from the subject. Content removed by early identification is not subject to processing in later stages, and extraction of metadata from it can be omitted, so by extracting the minimum metadata from unknown content as a whole, Identification becomes possible, and high-speed identification processing can be realized. In addition, for example, by giving priority to the identification process using a learning model with high identification accuracy, the content included in the negative example content can be identified with high accuracy for each cluster.

  The identification result of unknown content can also be used for relearning. For example, when an unknown content is identified as a positive example content even though it is a negative example content, the unknown content is given to the learning means or extraction means for the cluster, and the unknown content is identified as a negative example content Let them learn again. In this case, the cluster of unknown content and the optimal metadata are known in the identification process. Therefore, if the metadata is stored, the metadata can be used for relearning. Since only the optimal metadata needs to be extracted, the metadata to be extracted can be minimized.

It is a functional block diagram which shows one Embodiment of the content identification device based on this invention. It is explanatory drawing which shows the concept of a support vector machine (SVM). It is a flowchart which shows an example of the procedure of the identification process in an identification process means.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Learning processing means 11,12 ... Extraction means, 13 ... Classification means, 14, 15, 16 ... Learning means, 17, 18, 19 ... Selection means, 20 ... Identification processing means 21, 22, 23... Identification means

Claims (12)

In a content identification device for identifying whether unknown content is content to be identified,
Advance content of provided identification target (hereinafter, referred to as positive cases teacher content.) Of the feature with a prepared identification-target content (hereinafter, referred to as negative examples teacher content.) Based on the feature amount of Learning processing means for constructing a plurality of learning models for each classification of the negative example teacher content and for each feature amount ,
The unknown feature quantity and the learning processing unit by constructed plurality of learning the unknown content based on the selected learning model from the model identification target content of the content (hereinafter, referred to as positive cases content.) Is Identification processing means for identifying whether or not the unknown content is positive example content from the identification results by the plurality of identification means And
The learning processing means, first extraction means for extracting a feature quantity of positive cases teacher content, a second extraction means for extracting a feature value of the negative examples teacher content feature amount extracted by said second extraction means And classifying means for classifying the negative example teacher content according to each of the classification means and the classification means by the classification means, each of which is classified by the feature means extracted by the first extraction means and the classification means A plurality of learning means for constructing a learning model for each feature amount, which optimally classifies the positive example teacher content and the negative example teacher content based on the feature amount of each negative example teacher content, and the plurality of learning means provided corresponding to each of the features optimal learning model corresponding to the classified individual negative examples teacher content was by the classification means, for each feature quantity constructed by the learning means learning Content identification device, characterized in that it comprises a selection means to select from Dell gives the identification means provided corresponding to each of the plurality of learning means.   The content identification apparatus according to claim 1, wherein the learning processing unit uses a descriptor defined by MPEG-7 as the feature amount.   The classifying means is configured to classify the negative example teacher content according to the feature amount using a K-mean method or a self-organizing map, or to be classified by a manual operation. The content identification device according to claim 1.   The plurality of learning units are configured to set a separation hyperplane that optimally separates from negative example teacher content for each negative example teacher content classified by the classification unit. 2. The content identification device according to 1.   5. The content identification apparatus according to claim 4, wherein the separation hyperplane is set using a support vector machine or discriminant analysis. The content identification apparatus according to claim 1, wherein the plurality of identification units are connected so as to execute identification processing in stages on the unknown content.   7. The content identification apparatus according to claim 6, wherein each of the identification means identifies unknown content into two groups, a negative example content corresponding to classification by the classification means and other content. Unknown content consists of multiple content,
The identification processing means indicates the degree of appearance of the content classified into each unknown content according to the classification by the classification means , according to the appearance frequency in each classification or the identification accuracy of the plurality of identification means The content identification apparatus according to claim 6, wherein the application order of the plurality of identification units is configured to be changeable.   7. The content identification according to claim 6, wherein the identification processing unit identifies the unknown content from the example content when all of the plurality of identification units identify the example content as the example content. apparatus.   The identification processing unit identifies the unknown content as negative example content and interrupts the processing when the unknown content is identified as negative example content by at least one of the plurality of identification units. Item 7. The content identification device according to Item 6. The identification processing means extracts the same feature quantity as that used for construction of the learning model given to each identification means from unknown content when each identification means executes the identification process. The content identification device according to claim 6.   7. The content identification apparatus according to claim 6, wherein the identification processing unit reuses the feature quantity used by each identification unit in the identification process if it has already been extracted by another identification unit.

Images