JP2004336466A - Metadata registration method - Google Patents

Abstract

To facilitate registration of metadata (personal information + subject position) for an image.
A personal information is registered in an icon, and the icon is dropped on another image. Then, the dropped position is determined as the subject position and associated with the image data. As described above, by using the icon image, it is possible to easily register the personal information and the subject position.
[Selection diagram] Fig. 1

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention belongs to the field related to a method for registering additional information (metadata) of an image.
[0002]
[Prior art]
The metadata is data attached to the data, and in the case of a still image, particularly means data such as a title, a keyword, and subject information for the still image. By using the metadata, it is possible to perform a still image search based on a keyword, automatically classify still images, and the like, thereby improving convenience. Furthermore, if information such as an e-mail address of a subject is registered as metadata, further services such as sending an e-mail by simply clicking a still image can be expected. However, in practice, it takes a great deal of effort to input subject information such as a keyword and an e-mail address for each still image using a keyboard or the like and to register the information as a still image.
[0003]
Therefore, for example, as disclosed in Japanese Patent Application Laid-Open No. H11-146317, a keyword or subject information is converted into an icon in advance, and the iconized information is registered in the still image as metadata only by associating the icon with the still image. There are also methods.
[0004]
[Problems to be solved by the invention]
As described above, if a method as disclosed in Japanese Patent Application Laid-Open No. H11-146317 is used, registration of keywords and subject information is simplified, and more metadata can be registered. However, as the registration of subject information becomes easier and the number of registered subject information increases, it becomes difficult to understand which subject information indicates which subject. To solve this problem, it is necessary to have subject area information indicating in which area in the still image data the subject linked to the subject information is located.
[0005]
The present invention has been made in view of the above problems, and has as its object to easily register personalized iconized information and area information of a subject corresponding thereto.
[0006]
[Means for Solving the Problems]
In order to achieve the above object, the present invention includes the following means.
[0007]
That is, a method for registering, in an image data, subject information associated with the icon and an area in the image data of the subject using the icon, the area designating means for designating the area in the image data of the subject. From the icon designating means, the area designating means, and the icon designating means for designating an icon corresponding to the subject, the acquired subject area information, and metadata generating means for integrating the subject information to generate metadata, Metadata registration means for associating the generated metadata with the image data;
Is provided.
[0008]
BEST MODE FOR CARRYING OUT THE INVENTION
(Example 1)
In this embodiment, a method of easily registering a subject area and personal information in an image by dragging and dropping an icon in which personal information such as a name and a mail address is registered to another image will be described.
[0009]
FIG. 1 is a block diagram showing a system configuration according to the embodiment of the present invention. The input device 101 is a device for inputting instructions and data from a user, and includes a keyboard, a mouse, operation buttons, and the like. The display device 102 is a device that displays a GUI and data, and uses a CRT, a liquid crystal display, or the like. The storage device 103 is a device for storing image data and metadata, and usually uses a removable medium such as a hard disk or a memory card.
[0010]
Reference numeral 104 denotes a CPU, which is involved in all of the above-described processing of each configuration. The ROM 105 and the RAM 106 provide programs, data, a work area, and the like necessary for the processing to the CPU 104. Further, it is assumed that a control program necessary for the processing of the subsequent flowcharts is stored in the storage device 103 or in the ROM 105. If the data is stored in the storage device 103, the data is temporarily read into the RAM 106 and then executed.
[0011]
Although there are various other components in the system configuration other than those described above, they are not the main subject of the present invention, and therefore description thereof is omitted.
[0012]
FIG. 2 is a schematic diagram showing a state in which personal information 202 such as the name and address of the person is registered in an icon 201 in which the face of the person is shown. The icon 201 is an image file 203 such as JPEG, GIF, bitmap, etc., and personal information 203 described in XML is described in a header portion thereof. The description method of the personal information and the method of associating the personal information with the icon may be a method other than the embodiment, as long as the object is achieved.
[0013]
FIG. 3 is a diagram showing a GUI of a metadata registration application 301 for registering personal information and a subject area in an image using the icon of FIG. The image display device 302 is a device that displays an image for registering personal information. The personal information display device 303 can confirm registered personal information. The personal information selection device 304 has icons 307 arranged vertically, and can select personal information to be registered in an image from the icons. If the icon to be selected is not displayed in the device, the icon can be scrolled up and down by pressing the up scroll button 306 or the down scroll button 308. The scroll operation is shown as a scroll operation 401 in FIG. By pressing the down scroll button, E below D is displayed. Further, when the upper scroll button is pressed in this state, A, which was above B, is displayed. By clicking the icon, the icon in the personal information selection device 304 can be selected. The operation at the time of clicking is shown by a click operation 402 in FIG. The selected icon is surrounded by a gray rectangle.
[0014]
FIG. 5 is a flowchart illustrating an operation of actually registering personal information and a subject area in an image using the metadata registration application 301 in FIG. In S501, the user selects an image file for registering metadata, and causes the image display device 302 to display the image file. If metadata is already registered in the image file, personal information and subject area information are read from the metadata, and the personal information registered mark 604 in FIG. 6 is displayed on the corresponding subject. This mark will be described later in detail. In S502, the user selects one subject from the displayed image, and searches the personal information selection device 304 for an icon corresponding to the subject. Here, it is assumed that the icon B 601 has been searched for the subject B 602 in FIG. When performing a search, the scroll operation shown in FIG. 4 is performed. In step S503, the user places the mouse pointer on the searched icon B 601 and presses the mouse button. Then, the icon B 601 is surrounded by a gray rectangle. In S504, the mouse pointer is moved to the position of the face of the subject B 602 while keeping the mouse button pressed, and the mouse button is released there (drag and drop). In S505, the system creates a predetermined rectangular area centered on the position where the mouse button is released, and displays it as the subject area 603 in FIG. Further, a personal information registered mark 604 indicating that personal information has been registered is displayed at the upper left of the subject area 603. In step S <b> 506, the personal information described in XML registered in the icon B 601 is read and stored in the RAM 106. The contents of the personal information stored in the RAM 106 are displayed in a subject information display area 605. Further, the area information (upper left coordinate, width, and height) of the subject area 603 is also stored in the RAM 106 at the same time. In step S507, it is determined whether the size, coordinates, and the like of the subject area 603 displayed in the predetermined size are appropriate. Here, if there is a problem such as a face protruding from the subject area 603, the process proceeds to S508, the size and coordinates of the rectangle are corrected with a mouse or the like, and the corrected area information is stored in the RAM 106 again. Proceed to S506. If there is no problem, the process proceeds to S509. In S509, it is checked whether or not to continue the work of registering personal information. To continue the operation, the process proceeds to S502. If the personal information has already been registered for the main subject, the registration work ends, and the process proceeds to S510. Whether personal information has been registered can be determined by checking the personal information registered mark. For example, in the case of FIG. 7, the subjects B and D have registered personal information, but the subjects A and C have not been registered. If the user wants to check the personal information that has already been registered, the user can press the subject selection button 701 to check one after another. In step S510, by pressing the metadata write button 305, all the personal information stored in the RAM 106 and the area information of the subject are integrated, and the metadata is written to an image file. Specifically, when the personal information of B and D is registered as shown in FIG. 7, the personal information 801 of the subject B, the area information 802, and the personal information of the subject D are stored in the RAM 106 as shown in FIG. Information 803 and area information 804 are stored. By integrating these pieces of information, metadata 805 is created, and the metadata is written in the header of the image file. In this manner, a plurality of pieces of personal information and a subject area can be easily registered in an image only by dragging and dropping the icon in which the personal information is registered to the target image.
[0015]
(Example 2)
In the present embodiment, a method for registering personal information and a subject area using a face detection technique will be described. The system configuration and the like are the same as in the first embodiment, and a description thereof will not be repeated.
[0016]
In the first embodiment, the position where the icon is dropped is recognized as the region of the subject, and necessary processing is performed. However, it may be difficult to specify the position of the face of the subject, for example, when a pointing device such as a mouse does not exist. In such a case, the registration operation can be simplified by using the face detection technique together.
[0017]
FIG. 9 is a diagram showing an outline of the face detection technique. First, two parallel black points are extracted from the original image 901, and the positions of the eyes are detected (902). Then, a skin color region is extracted around the position of the eyes, and the region is detected as a face (903).
[0018]
Note that the face detection algorithm requires more processing besides the above, but is not the focus of the present invention, and thus detailed description is omitted.
[0019]
FIG. 10 is a diagram illustrating an appearance of a digital still camera 1001 that can register personal information and a subject area in an image using a face detection function. A lens 1002 and a flash 1003 are provided on the front surface, and a shutter button 1004 and a mode dial 1006 are provided on the upper surface. The mode dial 1006 is used to switch between a shooting mode and a playback (image viewing) mode. On the back, a liquid crystal display 1007 for displaying an image and a GUI, a metadata button 1008 used for metadata registration, cursor keys 1009 used for GUI operation, an OK button 1010, a cancel button 1011 and the like are provided.
[0020]
FIG. 11 is a diagram illustrating a state in which an image captured by a digital still camera is viewed on a liquid crystal display 1007. When the mode dial 1006 is switched to the playback mode after shooting the image, the image shot earlier is displayed as a playback mode screen 1101. Here, when the metadata button 1008 is pressed, a metadata registration screen 1102 is displayed. On the metadata registration screen 1102, there are a personal information selection device 1103 for selecting an icon and an image display device 1104 for displaying a captured image. Face detection has been performed on the image displayed on the image display device 1104, and the face area detected at that time is stored in the RAM 106. The icon pointer 1105 is used when designating a subject. A personal information registered mark 1106 displayed at the upper left of the subject indicates that personal information has been registered for the subject. An icon selection cursor 1107 indicates that an icon surrounded by the selection cursor has been selected.
[0021]
FIG. 12 is a flowchart illustrating a method for registering personal information and a subject area in an image using the digital still camera 1001. In step S1201, after capturing an image, the user sets the mode dial 1006 to the playback mode. In the playback mode, a metadata button 1008 is pressed while an image whose metadata is to be registered is displayed on the liquid crystal display 1007. In step S1202, face detection is performed on the displayed image. The detected face area is stored in the RAM 106 as a rectangle (upper left coordinate, width, height). In step S1203, a metadata registration screen is displayed. In step S1204, an icon to be registered as a subject is selected. The icon is selected by moving the icon selection cursor 1108. By pressing the up and down keys of the cursor key 1009, the icon selection cursor 1108 moves up and down, and an arbitrary icon can be selected. In S1205, an OK button 1010 is pressed. Then, the icon pointer 1105 appears in the image display device 1104, and the icon selection cursor cannot be moved. In step S1206, as shown in FIG. 1301, the icon pointer 1105 is moved close to the subject using the cursor key 1009. In S1207, the OK button 1010 is pressed again. Then, as shown in FIG. 1302, the icon pointer changes to the personal information registered mark, and the icon selection cursor can be moved again. At this time, personal information registered in the icon is stored in the RAM 106. In S1208, it is determined whether to continue the work of registering personal information. If continuing, the process returns to S1204. In the case of ending, the process proceeds to S1209. In S1209, the metadata button is pressed again. Then, the area closest to the position of the personal information registered mark detected by face detection is selected as the subject area, and is associated with the personal information stored in the RAM 106 (FIG. 1303). In step S1210, the personal information and the subject area information are integrated to create metadata described in XML, and write the metadata in the header of the image file. Thereafter, the screen returns to the original reproduction mode screen 1101.
[0022]
As described above, by using the face detection technique, the personal information and the subject area can be easily registered in the image. Further, in the present embodiment, an example has been described as a face detection technique. However, any other method may be used as long as the object is achieved.
[0023]
(Example 3)
In the present embodiment, a method for easily registering personal information and a subject area in an image by photographing an image using a specific GUI in a digital still camera will be described. The system configuration and the like are the same as in the first embodiment, and a description thereof will not be repeated. Also, the configuration of the digital still camera is the same as that of the second embodiment, and the description is omitted.
[0024]
FIG. 14 is a diagram showing a GUI for special photographing. The object area registration GUI 1401 appears on a finder or a liquid crystal display by pressing a metadata button 1008 while the mode dial 1006 of the digital still camera 1001 is in a shooting mode. The photographing is performed in a state where the person is superimposed on the human-shaped area (subject area 1406). Here, an example has been given as an example of the GUI for special shooting. However, there are several other GUIs depending on the number of shots, the arrangement, and the like. In the shooting mode, each time the left or right button of the cursor key 1009 is pressed, The GUI switches.
[0025]
FIG. 15 is a flowchart illustrating a method of registering personal information and subject area information in image data using the subject area registration GUI. In step S1501, a desired GUI is selected in the shooting mode. Then, the area information of the subject area 1406 is stored in the RAM 106. In step S1502, an icon to be registered as a subject is selected using the icon selection cursor 1403 in the personal information selection device 1402. In S1503, an OK button 1010 is pressed. Then, the icon pointer 1404 is displayed on the GUI, and the icon selection cursor 1403 cannot be moved. In step S1504, the icon pointer 1404 is moved to a position near the subject area 1406 using the cursor key 1009. In S1505, the OK button 1010 is pressed again. Then, the icon pointer changes to the personal information registered mark 1405, and the icon selection cursor can be moved again. At this time, personal information registered in the icon is stored in the RAM 106. In step S1506, it is determined whether to continue registering the personal information in the subject area 1406. If continuing, the process returns to S1502. If not, the process advances to step S1507. In step S1507, a person is superimposed on the subject area 1406 to capture an image. In step S1508, captured image data is generated. In S1509, the subject area closest to the personal information registered mark is associated as subject area information of the personal information. In step S1510, the personal information stored in the RAM 106 and the subject area information are integrated to create metadata described in XML. Then, the created metadata is associated with the image data.
[0026]
As described above, by using a special GUI when capturing an image, it is possible to easily register personal information and a subject area in the image.
[0027]
【The invention's effect】
As described above, the personal information and the subject area can be easily registered in the image by using the drag and drop, the face detection, and the GUI at the time of shooting. This makes it possible to construct a new service using images and metadata, such as sending an e-mail just by clicking an image.
[Brief description of the drawings]
FIG. 1 is a configuration diagram of a system according to an embodiment.
FIG. 2 is a diagram showing an outline of an icon in which personal information is registered.
FIG. 3 is a GUI of a metadata registration application.
FIG. 4 is a diagram illustrating the operation of the personal information selection device.
FIG. 5 is a flowchart illustrating metadata registration by a drag-and-drop operation.
FIG. 6 is a schematic diagram illustrating metadata registration by a drag-and-drop operation.
FIG. 7 is a diagram illustrating a method of displaying metadata.
FIG. 8 is a diagram illustrating a state in which personal information and subject area information are joined to create metadata.
FIG. 9 is a diagram showing an outline of face detection.
FIG. 10 is a diagram illustrating an appearance of a digital still camera.
FIG. 11 is a GUI for registering metadata in a digital still camera.
FIG. 12 is a flowchart illustrating a method for performing metadata registration in a digital still camera.
FIG. 13 is a diagram illustrating an outline of an operation of performing metadata registration in a digital still camera.
FIG. 14 is a diagram illustrating a shooting screen for inputting subject area information during shooting.
FIG. 15 is a flowchart illustrating a method of registering metadata by using a special GUI at the time of shooting.

Claims (12)

Using the icon, subject information associated with the icon,
A method of registering an area in image data of a subject in image data,
An area specifying means for specifying an area in the image data of the subject;
An icon designating means for designating an icon corresponding to the subject;
A metadata generation unit that integrates the acquired subject region information and the subject information from the region designation unit and the icon designation unit to generate metadata,
Metadata registration means for associating the generated metadata with the image data;
A metadata registration method comprising: 2. The image processing apparatus according to claim 1, wherein the area specified by the area specifying means is an area in the image data centered on a position where the icon is dragged and dropped on the image data. Metadata registration method described in. 3. The metadata registration according to claim 2, wherein the area in the image data centered on the dropped position changes in size and position by using a pointing device such as a mouse. Method. 2. The metadata registration method according to claim 1, wherein the area specified by the area specifying means is an area in image data detected by face detection. The area specified by the area specifying means is an area in the image data designated by overlapping and photographing a subject at a position designated by overlaying a special GUI on the finder during image data photographing. The metadata registration method according to claim 1, wherein: 2. The metadata registration method according to claim 1, wherein subject information described in XML is registered in the icon. 2. The metadata registration method according to claim 1, wherein subject information described in V-card is registered in the icon. 2. The metadata registration method according to claim 1, wherein the icon is linked to subject information recorded in a database. The metadata registration method according to claim 1, wherein the metadata generated by the metadata generation unit is described in XML. 2. The metadata registration method according to claim 1, wherein the metadata registration unit writes the metadata in a header portion of the image data. 2. The metadata registration method according to claim 1, wherein the metadata registration unit writes the metadata at the end of the image data. 2. The metadata registration method according to claim 1, wherein the metadata registration unit attaches a link between the metadata as an XML file and the image data.

Images