WO2018199351A1

WO2018199351A1 - Method and device for generating image file including sensor data as metadata

Info

Publication number: WO2018199351A1
Application number: PCT/KR2017/004401
Authority: WO
Inventors: 최상호; 정성엽
Original assignee: 라인 가부시키가이샤
Priority date: 2017-04-26
Filing date: 2017-04-26
Publication date: 2018-11-01

Abstract

Provided is an image processing method for acquiring depth information of a target image on the basis of a predetermined pattern information including information regarding a shape of an object included in an image and information regarding relative locations between objects included in the image, and applying a dynamic effect to the target image on the basis of the acquired depth information.

Description

Method and apparatus for generating image file including sensor data as metadata

The present disclosure relates to a method and an apparatus for generating an image file according to a shooting command, and more particularly, to a method and an apparatus for generating an image file including sensor data as metadata included in an image file.

In the case of an image, only two-dimensional objects can be expressed, and therefore, liveness or realism as shown in a video cannot be expressed. Image files are metadata that contain information related to shooting conditions such as the type of camera used to take a picture or the shooting speed such as shutter speed, ISO value, and aperture value. In order to generate the, it was necessary to receive additional content such as additional sensor information or additional image files separately.

In the case of a video, the capacity is very large compared to an image, and it takes a long time to produce. In addition, a method of generating content including dynamic effects by synthesizing images and voices or synthesizing images and videos also provides a limited experience to viewers of the content.

Accordingly, there is a need for a method and apparatus for generating content including dynamic effects using an image file generated through a single shot, without input of additional data or receipt of additional content.

Korean Patent No. 10-1513999 (published April 21, 2015) separates a selected first object from a plurality of objects included in two-dimensional image data, and first object at a first position and a second position of a background. A method of displaying a two-dimensional image as a three-dimensional image by generating data relating to the arrangement of is described.

The information described above is merely for the sake of understanding, and may include content that does not form part of the prior art, and may not include what the prior art may suggest to those skilled in the art.

In one embodiment, in response to a photographing command, the sensor data associated with the object from the sensor unit is obtained, and the sensor data is added to the image file generated by photographing the object by the camera as metadata. It can provide a way to create an image file.

According to an embodiment of the present disclosure, metadata is extracted from an image file photographing an object including sensor data acquired through the sensor unit as metadata, and based on the extracted metadata, a desired content including dynamic effects is generated. It may provide a method.

In one aspect, a method for generating an image file in response to a photographing command for an object through a user terminal, the method comprising: acquiring sensor data associated with the object from at least one sensor in response to the photographing command The object is at least one of a subject and a background around the subject, and the sensor data includes information required to generate a dynamic effect associated with at least one of the subject and the background; And generating the desired image file by adding the sensor data as metadata to an image file generated by photographing the object by the camera in response to the photographing command. .

The sensor data may include at least one of information about a positional relationship between the user terminal and the object and information about a color of the object.

The sensor may include at least one of a position sensor, a gyro sensor, a distance sensor, an illuminance sensor, a depth sensor, a motion tracking sensor, and a shape identification unit.

The sensor data includes information about a viewpoint of the camera, information about a distance between the user terminal and the ground obtained from the position sensor, and an inclination of the user terminal when generating the desired image file measured by the gyro sensor. A distance from the user terminal measured by the distance sensor to at least one measurement point of the object, an illuminance value upon generation of the desired image file measured by the illuminance sensor, measured by the depth sensor It may include at least one of a depth value of the measurement point, motion tracking related data of the object obtained by the motion tracking sensor, and information about the shape of the object identified by the shape identification unit.

The dynamic effect may be a Virtual Reality (VR) effect or an Augmented Reality (AR) effect, a three-dimensional effect, or a motion effect.

The method for generating the image file may further include generating a desired content including the dynamic effect based on the metadata included in the desired image file.

The generating of the desired content may include: building a virtual space based on the metadata; And synthesizing the virtual space with a user-photographed or user-selected target object.

The virtual space may correspond to a target object background around the target object.

The target object background may have a spatial similarity to the background.

The target object background may have spatial similarity to the background by including the same at least one environment element associated with the background.

The environmental element may include a color of at least a part of the background, an object included in the background, a shape of the object, a type of the object, or a positional relationship of the subject with respect to at least a part of the background.

The building of the virtual space may include determining a location in the virtual space of at least one thing to be placed in the virtual space based on the metadata; And generating the virtual space by arranging or rendering the object based on the determined position.

The building of the virtual space may include determining a shape of an object to be disposed in the virtual space based on the information about the shape of the object included in the metadata; And generating the virtual space by arranging or rendering the object based on the determined shape of the object.

The object to be disposed or rendered may be the same object as the object or another object having the same shape as the object.

The method of generating the image file may further include building a database based on the metadata included in the desired image file.

The metadata may be stored in the desired image file in an Exchangable Image File format (Exif).

In another aspect, extracting the metadata from an image file photographing the object that includes sensor data related to an object obtained through at least one sensor as metadata, wherein the object is a subject and a surrounding of the subject. At least one of a background of the sensor data, wherein the sensor data includes information required to generate a dynamic effect associated with at least one of the subject and the background; And generating a desired content including the dynamic effect, based on the metadata.

The target object background may have a spatial similarity to the background.

In another aspect, an electronic device that generates an image file in response to a photographing command for an object through a user terminal, wherein the sensor device obtains sensor data related to an object from at least one sensor in response to the photographing command. A sensor data obtaining unit, wherein the object is at least one of a subject and a background around the subject, and the sensor data includes information required to generate a dynamic effect associated with at least one of the subject and the background; And an image file generator for generating a desired image file by adding the sensor data as metadata to an image file generated by photographing the object by the camera in response to the photographing command.

The electronic device may further include a content generation unit configured to generate desired content including the dynamic effect, based on the metadata included in the desired image file.

In photographing an object through a user terminal, a method of including sensor data as metadata in a generated image file may be provided by only performing photographing without a separate operation or setting for acquiring sensor data. .

Since a single image file can be used to generate content having a dynamic effect having a visual commonality with the corresponding image file, it is possible to easily generate content in which the atmosphere at the time of shooting is reproduced.

Using only a single image file, a VR environment or an AR environment representing an atmosphere similar to the atmosphere in which the image file is photographed can be constructed.

1 illustrates a method of generating an image file including sensor data as metadata and generating content including dynamic effects, according to an embodiment.

2 illustrates an apparatus for generating an image file including sensor data as metadata and generating content including dynamic effects, according to an embodiment.

3 is a flowchart illustrating a method of generating an image file including sensor data as metadata and generating content including dynamic effects, according to an exemplary embodiment.

4 is a flowchart illustrating a method of generating content including dynamic effects based on metadata included in an image file, according to an example.

5 is a flowchart illustrating a method of building a virtual space included in content in generating content based on metadata included in an image file according to an example.

6 is a flowchart illustrating a method of building a database based on metadata included in an image file, according to an example.

7 illustrates a method of generating content having a dynamic effect, according to an example.

8 illustrates a method of constructing a database based on metadata included in an image file and generating content having a dynamic effect based on the constructed database.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 illustrates a method of generating an image file as the object 130 is photographed through the camera 110 of the user terminal 100.

The user terminal 100 may be an electronic device including a sensor unit 120 including a camera 110 and at least one sensor. For example, the user terminal 100 may be a smartphone.

The object 130 is an object displayed in an image generated by photographing, and may represent at least one of a subject 134 to be photographed and a background 132 around the subject 134. The subject 134 may include at least one object. The background 132 may mean an area excluding the subject 134 in the image of the object 130 generated by photographing. Background 132 may also include at least one object.

In response to a user's photographing command of the user terminal 100, the sensor unit 120 may acquire sensor data related to the object 130 using at least one sensor. The sensor data may include information required for generating a dynamic effect associated with at least one of the subject 134 and the background 132. For example, sensor data can be used to generate dynamic effects such as Virtual Reality (VR) effects or Augmented Reality (AR) effects, three-dimensional effects, or motion effects on the content of an image (image) or video. Information may be included.

The user terminal 100 may generate an image file by capturing the object 130 by the camera 110 in response to the photographing command, and the sensor data obtained by the sensor unit 120 as the metadata as the image. By adding to a file, a desired image file can be generated.

That is, when photographing the object 130 through the user terminal 100, the sensor data may be included as metadata in the image file generated by the photographing without any additional manipulation or setting for acquiring the sensor data.

The user terminal 100 may transmit the generated image file or metadata included in the image file to the content generation device 140. The content generation device 140 may extract metadata from an image file from the user terminal 100, and generate content including dynamic effects using the metadata. Since the generated content includes dynamic effects generated using metadata (sensor data) included in the image file, the generated content may have a visual commonality with the image file. For example, the generated content may be content including a VR effect representing an environment (or virtual space) similar to the background 132 represented by the image file. Alternatively, the generated content may be content including AR effects in which environmental elements associated with the background 132 represented by the image file are synthesized.

That is, by using a single image file, content having an atmosphere similar to the atmosphere in which the image file is photographed may be generated, and a VR environment or AR environment having an atmosphere similar to the atmosphere in which the image file is photographed may be constructed. .

In FIG. 1, an example in which the content generating device 140 exists as a separate device outside the user terminal 100 is illustrated. However, unlike the illustrated example, the content generating device 140 is configured inside the user terminal 100. It may be. That is, the user terminal 100 may generate the content to which the dynamic effect is added by using the metadata included in the image file by using the image file generated by photographing.

Detailed configurations of the electronic device and the content generation device 140 corresponding to the user terminal 100 and their roles will be described in more detail with reference to FIGS. 2 to 8 to be described later.

Referring to FIG. 2, the electronic device 200 generating an image file including the sensor data as metadata, and the content generating device 140 generating content including dynamic effects using the metadata of the generated image file. Explain about. The electronic device 200 may correspond to the user terminal 100 described above with reference to FIG. 1.

The electronic device 200 is a device for photographing the object 130 and acquiring sensor data related to the object 130 according to a photographing command. The electronic device 200 may be a personal computer, a laptop computer, or a laptop computer. It may be a terminal used by a user such as a laptop computer, a tablet, an Internet Of Things device, or a wearable computer.

The electronic device 200 may include a communication unit 210, a controller 220, a camera 230, and a sensor unit 240. The camera 230 and the sensor unit 240 may correspond to the camera 110 and the sensor unit 120 described above with reference to FIG. 1.

The communication unit 210 may be a device for the electronic device 200 to communicate with another server or a user terminal. In other words, the communication unit 210 may be a hardware module or network device driver such as a network interface card, a network interface chip and a networking interface port of the electronic device 200 that transmits / receives data and / or information to another server or user terminal. It may be a software module such as a driver or a networking program.

The controller 220 may manage components of the electronic device 200 and may execute a program or application used by the electronic device 200. For example, the controller 220 may control the sensor unit 240 to obtain sensor data according to the received photographing command, and control the camera 230 to photograph the object 130, and to capture an image file. Can be generated. The controller 220 may execute a program or an application required for generating an image file, and may process operations required for executing the program or the application and processing data. In addition, the controller 220 may be configured to process data received from other servers and user terminals. In addition, the controller 220 may be at least one processor of the electronic device 200 or at least one core in the processor.

The controller 220 includes a sensor data acquisition unit 222 for controlling the sensor unit 240 to acquire sensor data, and an image file generation unit 224 for generating an image file according to the photographing by the camera 230. can do.

The sensor data acquisition unit 222 may acquire sensor data related to the object 130 from the sensor unit 240 including at least one sensor in response to a photographing command for the object 130.

The image file generation unit 224 adds the sensor data as metadata to an image file generated by photographing the object 130 by the camera 230 in response to a photographing command for the object 130. You can create a file.

In addition, the controller 220 may further include a content generator 226 that generates desired content including dynamic effects based on metadata included in the image file generated by the image file generator 224. Can be. The content generator 226 extracts the metadata from the image file generated by the image file generator 224, which includes sensor data related to the object 130 obtained through the sensor unit 240 as metadata. And based on the extracted metadata, it is possible to generate content including dynamic effects.

More specific functions and operations of the components 222 to 226 of the controller 220 will be described in more detail with reference to FIGS. 3 to 8 to be described later. The above-described configurations 222 through 226 of the controller 220 may be implemented in at least one processor, and the functions and operations of the configurations 222 through 226 may be executed by at least one processor.

The camera 230 may be a device for photographing the object 130. The camera 230 may include an image sensor for photographing the object 130 to generate an image (image or image).

The sensor unit 240 may include at least one sensor for measuring / acquiring data related to the object 130. For example, the sensor unit 240 may include at least one of a position sensor, a gyro sensor, a distance sensor, an illuminance sensor, a depth sensor, a motion tracking sensor, and a shape identification unit. In addition to the shape and location of the object 130, the electronic device 200 (eg, the user terminal 100) and the positional relationship between the object 130 and the color information of the object 130 may be determined. It may include any sensor.

The sensor data may include information about a viewpoint of the camera 230, information about a distance between the electronic device 200 and the ground obtained from the position sensor, and an electronic device at the time of generating a desired image file measured by the gyro sensor. Tilt of 200, distance from the electronic device 200 measured by the distance sensor to at least one measurement point of the object 130, at the time of generation of the image file measured by the illuminance sensor (ie, at the time of shooting) Information on the shape of the object 130 identified by the illuminance value of the measurement point, the depth value of the measurement point measured by the depth sensor, motion tracking related data of the object 130 obtained by the motion tracking sensor, and the shape identification unit. It may include at least one of. The image file generator 224 may parameterize the acquired sensor data, and generate the image file by adding the parameterized sensor data as metadata of the image file to be generated. The measurement point may represent any point of the object 130 or may represent any point of the object included in the object 130. The motion tracking related data and / or information about the shape may be used to identify what the object 130 (or the objects included in the object 130) is.

Although not shown, the electronic device 200 may further include a display unit for outputting data input by a user or displaying content including image files and / or dynamic effects. For example, the display unit may include a touch screen, and in this case, the display unit may be configured to include a function of an input unit for receiving a setting and a request from a user.

Although not shown, the electronic device 200 may include a storage unit as a device for storing data or information. The storage may include any memory or storage device. The storage unit may store a program or an application executed by the controller 220 and information related thereto. For example, the storage unit may store content including the generated image file and / or the generated dynamic effect.

The content generation device 140 may be a device for generating content including dynamic effects, based on metadata included in the image file generated by the image file generation unit 224. The content generation device 140 may include a communication unit 250 and a control unit 260. The content generation device 140 may be a server or other computing device that generates content that includes dynamic effects.

The communication unit 250 may be a component for communicating with other devices, including the electronic device 200. The communication unit 250 may be a network interface card, a network interface chip and a networking interface port of the server 110 that transmits / receives data and / or information to another device (database or other server) including the electronic device 200. It may be a hardware module. The communication unit 250 may be or a software module such as a network device driver or a networking program.

The controller 260 may manage the components of the content generation device 140, execute a program or an application used by the content generation device 140 to generate and provide content, and process related data and operations. Can be. The controller 260 may be at least one processor of the content generating device 140 or at least one core in the processor. The controller 260 may include a content generator 266.

Operations and functions of the content generator 266 may be the same as those of the content generator 226 described above, and thus a detailed description thereof will be omitted. In other words, the content including the dynamic effect may be generated by the electronic device 200 or may be generated by the content generating device 140 external to the electronic device 200. Alternatively, some operations (eg, virtual space creation) for generating content are performed by the external content generation device 140, and the remaining operations (combination of the target object and the virtual space) are performed by the electronic device 200. It may also be performed by.

A detailed method of generating an image file including sensor data as metadata and a detailed method of generating content including dynamic effects using the metadata will be described in more detail with reference to FIGS. 3 to 8 to be described later.

As described above, the description of the technical features described above with reference to FIG. 1 may be applied to FIG. 2 as it is, and thus redundant descriptions thereof will be omitted.

Steps 310 to 330 to be described below illustrate a method of generating an image file including sensor data as metadata by the electronic device 200 (I).

Meanwhile, steps 340 and 350 to be described below illustrate a method for generating content including dynamic effects by the electronic device 200 or the content generating device 140 (II). In order to avoid duplicate description, steps 340 and 350 are described as being performed by the content generator 226, but steps 340 and 350 may be performed by the content generator 266.

In operation 310, the sensor data acquisition unit 222 may acquire sensor data related to the object 130 from at least one sensor (that is, the sensor unit 240) in response to a photographing command. The sensor data may include information required for generating a dynamic effect associated with at least one of the subject 134 and the background 134. For example, when the user of the electronic device 200 executes an application for controlling the camera 230, the sensor data acquirer 222 may acquire data about the photographing environment of the object 130 through the sensor unit 240. It can be measured. In addition, when the user completes capturing the object 130 through the camera 230 (for example, when the user presses the shutter button), the sensor data acquirer 222 may acquire measured sensor data. .

Through the sensor unit 240, the sensor data acquired by the sensor data acquisition unit 222 may include information about the positional relationship between the electronic device 200 (eg, the user terminal 100) and the object 130. 130 may include at least one of the information about the color. Information regarding the positional relationship between the electronic device 200 and the object 130 at the time of photographing may include at least one object included in the object 130 or the distance of the electronic device 200 with respect to at least a part of the object, the electronic device 200. Height difference) and associated angles.

The sensor data may include information about a viewpoint of the camera 230, information about a distance between the electronic device 200 obtained from a position sensor and the ground, and an electronic device 200 when generating a desired image file measured by a gyro sensor. ), The distance from the electronic device 200 measured by the distance sensor to the at least one measurement point of the object 130, the illuminance at the time of generation of the image file measured by the illuminance sensor A value, a depth value of the measurement point measured by the depth sensor, motion tracking related data of the object 130 obtained by the motion tracking sensor, and information about the shape of the object 130 identified by the shape identification unit. It may include one.

In operation 320, the image file generator 224 may parameterize the sensor data acquired by the sensor data acquirer 222.

For example, the information about the viewpoint of the camera 230 is a viewType. For example, the type of the camera (mobile phone camera, wide angle camera, panoramic camera, etc.), whether zooming or not, focusing may be parameterized. The information about the distance between the electronic device 200 and the ground may be parameterized by digitizing the distance between the electronic device 200 and the ground. The inclination of the electronic device 200 is an angle, and an angle with respect to the ground of the electronic device 200 (that is, the camera 230) may be numerically parameterized.

In addition, each object included in the object 130 (that is, the object 132 or the object included in the background 134) may have an ID (eg, an integer value) as the objectId. In addition, as the shape of each object is identified by the shape identification unit, the type of each object may be parameterized as an objectType. The type of object may represent a specific object (or creature) such as, for example, a wall, sky, floor, ceiling, light source, road, river, sea, animal, person, door, chair, desk, or the like. The shape of the object 130 (or each object) is a shape, for example, an approximate shape of the object, a circle, a ball, a box, or the like may be parameterized.

In addition, the two-dimensional position in the photographing environment of the object 130 (or each object) is a map, for example, the x coordinate and the y coordinate may be parameterized. The three-dimensional position in the photographing environment of the object 130 (or each object) is a location, for example, the x coordinate, the y coordinate and the z coordinate may be parameterized. The size of the object 130 (or each thing) is size, for example, the width, the length and the depth may be parameterized.

In addition, the color of the object 130 (or each object) is color, for example, the HEX code of the average color of the object 130 (or each object) may be parameterized.

In addition, any sensor data such as motion tracking data of the object 130 (or each object), illuminance of the shooting environment, and the like may be digitized / coded as an extension and parameterized.

In step 330, in response to the photographing command, the image file generating unit 224 uses sensor data (parameterized) as metadata in the image file generated by photographing the object 130 by the camera 230. By adding, the desired image file can be generated. The sensor data may be stored in the generated image file and the metadata may be stored in an Exchangable Image File format (Exif) (eg, may be stored in an extention of Exif). Accordingly, the image file including the sensor data as metadata may be generated only by the photographing command for the object 130 through the electronic device 200.

In operation 340, the content generator 226 may extract metadata included in the image file generated in operation 330. The content generator 226 may extract information corresponding to sensor data necessary for generating a dynamic effect related to at least one of the subject 134 and the background 132 among the information included in the metadata.

In operation 350, the content generator 226 may generate desired content including dynamic effects based on the extracted metadata. The dynamic effect on the content may be a Virtual Reality (VR) effect or an Augmented Reality (AR) effect, a three-dimensional effect, or a motion effect associated with the object 130. For example, in step 350, the generated content may have a visual commonality with the image file generated in step 330. For example, the generated content may be content including a VR effect representing an environment (or virtual space) similar to the background 132 represented by the image file. Alternatively, the generated content may be content including AR effects in which environmental elements associated with the background 132 represented by the image file are synthesized. The motion effect may be to cause a shake effect, a rotation effect, or a tilt effect to occur in at least a portion of the generated content.

More specific methods for generating content including dynamic effects using metadata included in the generated image file will be described in more detail with reference to FIGS. 4 to 8 to be described later.

As described above, the description of the technical features described above with reference to FIGS. 1 and 2 may be applied to FIG. 3 as it is, and thus redundant descriptions thereof will be omitted.

Referring to FIG. 4, a method of generating content including dynamic effects based on metadata included in an image file will be described in more detail.

Steps

410 and 420 to be described below may be included in step 350 described above with reference to FIG. 3.

In operation 410, the content generator 226 (or the content generator 266) may build a virtual space based on metadata included in the image file. The virtual space may be a space in which the VR effect or the AR effect of the generated content is displayed. Alternatively, the virtual space may be a portion to which the dynamic effect is applied in the generated content. The content generator 226 may build a virtual space based on sensor data related to the subject 134 and the background 132 included in the metadata.

In operation 420, the content generator 226 may synthesize the generated virtual space and the user-photographed or user-selected target object. The virtual space may correspond to a target object background around the target object. The target object may be, for example, another object (or subject) photographed by the user. In this case, as the other subject is photographed, the content generator 226 may generate content by synthesizing the photographed subject and the generated virtual space (eg, an AR effect applied space). Alternatively, the target object may be an object corresponding to a subject who experiences a VR effect (that is, an object corresponding to the center of the viewpoint (eg, corresponding to the center of the viewpoint of the 360 degree camera). In this case, the content generator 226 may be a target. Content may be generated by synthesizing the object and the generated virtual space, and the user may experience the VR effect through the virtual space by watching the content using the VR device.

The target object background corresponding to the virtual space may have a spatial similarity to the background 132 of the image file. Spatial similarity may indicate visual commonality (similarity) between the virtual space and the background 132. That is, when comparing the generated content with the image file, the user may feel visual similarity (spatial similarity).

The target object background may have spatial similarity to the background 132 by including the same at least one environment element associated with the background 132. An environmental element may indicate a color of at least a portion of the background 132, an object included in the background 132, a shape of the object, a type of the object, or a positional relationship of the subject 134 with respect to at least a portion of the background 132. Can be. For example, the target object background includes the same or similar color as the background 132; Include objects that are the same or similar to the objects included in the background 132, or are the same or similar in shape; The position of the target object with respect to the background of the target object is determined to correspond to the position of the subject 134 with respect to the background 132, thereby having a spatial similarity with respect to the background 132. The spatial similarity to the background 132 may be secured by building the target object background using metadata included in the image file.

A method of building a virtual space of content including dynamic effects will be described in more detail with reference to FIGS. 5 and 7 to be described later.

By generating the content including the dynamic effect according to the above-described method, the content including the VR effect / AR effect of the atmosphere similar to the image file of the object 130 is generated, or the object ( It may be possible to generate an image file reproducing the atmosphere of the image file photographed 130).

As described above, the description of the technical features described above with reference to FIGS. 1 to 3 may be applied as it is to FIG. 4, and thus redundant descriptions thereof will be omitted.

Steps

510 and 520 to be described below may be included in step 410 described above with reference to FIG. 4.

In operation 510, the content generator 226 (or the content generator 266) is disposed in the virtual space based on the information about the shape of the object 130 included in the metadata included in the image file. You can determine the shape of the object to be. The content generator 226 may determine the objects of the object to be placed in the virtual space based on the information about the shape of each object included in the object 130 (the background 132 and / or the object 134) included in the metadata. The shape can be determined. According to the determined shape, the content generating unit 226 may be the same object as the object 130 (the object included in the object) or another object having the same shape as the object 130 (the object included in the object). Can be determined as an object placed or rendered within. For example, an object disposed or rendered in a virtual space may be determined based on an objectType and a shape of metadata of an image file.

In operation 520, the content generator 226 may generate a virtual space by arranging or rendering the object based on the determined shape of the object. In other words, the object determined according to the information about the shape of the object 130 included in the metadata may be disposed or rendered in the virtual space.

In addition, in step 510, the content generating unit 226 (or the content generating unit 266) is in the virtual space of at least one thing to be placed in the virtual space based on metadata included in the image file. The location can be determined. The content generator 226 corresponds to each object in the virtual space based on the location information of each object included in the object 130 (background 132 and / or the subject 134) included in the metadata. You can determine where the thing will be placed. For example, a position where the object (s) are to be disposed in the virtual space may be determined to correspond to the position of the object included in the background 132 and the position of the object included in the subject 134 in the image file. In this case, the content generator 226 may allow the above-described target object to be disposed at a position of the virtual space corresponding to the subject 134.

In operation 520, the content generator 226 may generate a virtual space by arranging or rendering a corresponding object based on the location determined in operation 510.

A detailed example of determining an object to be arranged or rendered in a virtual space and rendering and placing the object will be described in more detail with reference to FIG. 7 to be described later.

As described above, the description of the technical features described above with reference to FIGS. 1 to 4 may be applied as it is to FIG.

With respect to the image file generated in step 330 described above with reference to FIG. 3, in step 610, the content generator 226 (or the content generator 266) is added to the metadata included in the image file. You can build a database based on that. For example, the content generator 226 may build a database based on the metadata included in the image file by transmitting the image file or metadata included in the image file to a database external to the electronic device 200. have. The database may store metadata included in image files from the plurality of electronic devices 200. The metadata stored in the database may be utilized in content generation including dynamic effects by the content generation unit 226 as big data.

Specific examples of generating content including dynamic effects using information from a database storing metadata of a plurality of image files will be described in more detail with reference to FIG. 8 to be described later.

As described above, the description of the technical features described above with reference to FIGS. 1 to 5 may be applied to FIG. 6 as it is, and thus redundant descriptions thereof will be omitted.

Referring to FIG. 7, a method of generating content 720 including dynamic effects using metadata of an image file 710 including sensor data as metadata is described.

In the image file 710, the person 712 may correspond to the subject, and the door 718 and the puppy 716 may not correspond to the subject and correspond to an object included in the background 714. The metadata of the image file 710 may store information regarding the location and shape of two-dimensional and three-dimensional images in the image file 710 of the objects 712 to 716. In addition, the distance information to the electronic device 200 and at least the height and inclination information of the electronic device 200 with respect to at least one measurement point (eg, any of the walls) of the background 714 are also stored in the metadata. It may be.

The content generating unit 226 (or the content generating unit 266) uses the metadata so that the object 718 is a rectangular door, the object 716 is a puppy as an animal, and the object 712 ) Can identify you as a person. The content generator 226 may also identify that the object 712 corresponds to the subject. For example, the object in focus or the object close to the center of the image may be identified as the subject. The content generator 226 may generate the content 720 by using (relative) location information and shape information of the image files of the objects 712 to 716 included in the metadata.

For example, a rectangular door 728 corresponding to the object 718 may be disposed in the virtual space 724. Also, a cat 726 (determined as an object representing the same type (animal) as the object 716) corresponding to the object 716 may be disposed in the virtual space 724. In response to the object 712, which is the subject, the object 722 photographed or selected by the user (corresponding to the target object described above) may be disposed in the virtual space 724.

By using spatial position information in the image file of the objects 712 to 716 included in the metadata, the position of the corresponding virtual space 724 in the image file of the objects 712 to 716 is used. Things 722-726 may be arranged. Since the metadata includes information that can indicate a location in space including depth information of the objects 712 to 716, the content generator 226 may be assigned a dynamic effect related to at least one of the objects 722 to 726. Generated content 720 can be generated.

In other words, the generated content 720 may further have a dynamic effect such as a VR effect or an AR effect, a three-dimensional effect, or a motion effect, while providing an atmosphere similar to that of the image file 710, compared to the image file 710. .

As described above, the description of the technical features described above with reference to FIGS. 1 to 6 may be applied as it is to FIG. 7, and thus redundant descriptions thereof will be omitted.

The illustrated database 810 may correspond to a database constructed based on the metadata described above with reference to FIG. 6.

The database 810 may store metadata included in the image files 710-1 to 710 -N itself or the image files 710-1 to 710 -N generated by the one or more electronic devices 200. Can be.

The content generator 226 may select an appropriate one from among metadata of the image files 710-1 through 710 -N stored in the database 810, and use the generated content 720 with the dynamic effect. For example, when generating the content 720, the electronic device 200 may output a user interface configured to allow a user to select appropriate metadata or a virtual space generated through the metadata. The content generator 226 may select the content 720 according to the user's selection through the user interface. In this manner, the user may generate not only the image file photographed by the electronic device 200 but also the content 720 which reproduces the atmosphere of the image file photographed by another user. In other words, metadata encapsulated in an image file may be shared among users through the database 810. In addition, through the electronic device 200, the user may generate the content 720 by finely adjusting the dynamic effect or atmosphere to be applied.

As described above, the description of the technical features described above with reference to FIGS. 1 to 7 may be applied to FIG. 8 as it is, and thus redundant descriptions thereof will be omitted.

The apparatus described above may be implemented as a hardware component, a software component, and / or a combination of hardware components and software components. For example, the devices and components described in the embodiments may include a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable PLU (programmable). It can be implemented using one or more general purpose or special purpose computers, such as logic units, microprocessors, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to the execution of the software. For convenience of explanation, one processing device may be described as being used, but one of ordinary skill in the art will appreciate that the processing device includes a plurality of processing elements and / or a plurality of types of processing elements. It can be seen that it may include. For example, the processing device may include a plurality of processors or one processor and one controller. In addition, other processing configurations are possible, such as parallel processors.

The software may include a computer program, code, instructions, or a combination of one or more of the above, and configure the processing device to operate as desired, or process it independently or collectively. You can command the device. Software and / or data may be any type of machine, component, physical device, virtual equipment, computer storage medium or device in order to be interpreted by or to provide instructions or data to the processing device. It can be embodied in The software may be distributed over networked computer systems so that they may be stored or executed in a distributed manner. Software and data may be stored on one or more computer readable recording media.

The method according to the embodiment may be embodied in the form of program instructions that can be executed by various computer means and recorded in a computer readable medium. In this case, the medium may be to continuously store a program executable by the computer, or to temporarily store for execution or download. In addition, the medium may be a variety of recording means or storage means in the form of a single or several hardware combined, not limited to a medium directly connected to any computer system, it may be distributed on the network. Examples of the medium include magnetic media such as hard disks, floppy disks and magnetic tape, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, And ROM, RAM, flash memory, and the like, configured to store program instructions. In addition, examples of another medium may include a recording medium or a storage medium managed by an app store that distributes an application or a site or server that supplies or distributes various software.

Although the embodiments have been described by the limited embodiments and the drawings as described above, various modifications and variations are possible to those skilled in the art from the above description. For example, the described techniques may be performed in a different order than the described method, and / or components of the described systems, structures, devices, circuits, etc. may be combined or combined in a different form than the described method, or other components. Or even if replaced or substituted by equivalents, an appropriate result can be achieved.

Therefore, other implementations, other embodiments, and equivalents to the claims are within the scope of the claims that follow.

Claims

In the method for generating an image file in response to a photographing command for an object through a user terminal,

In response to the shooting command, obtaining sensor data associated with the object from at least one sensor, wherein the object is at least one of a subject and a background around the subject, wherein the sensor data is selected from among the subject and the background; Includes information required to generate a dynamic effect associated with at least one; And

Generating a desired image file by adding the sensor data as metadata to an image file generated by photographing the object by a camera in response to the photographing command

Including a method of generating an image file.
The method of claim 1,

The sensor data includes at least one of information about a positional relationship between the user terminal and the object and information about a color of the object.
The method of claim 1,

The sensor includes at least one of a position sensor, a gyro sensor, a distance sensor, an illuminance sensor, a depth sensor, a motion tracking sensor, and a shape identification unit,

The sensor data includes information about a viewpoint of the camera, information about a distance between the user terminal and the ground obtained from the position sensor, and an inclination of the user terminal when generating the desired image file measured by the gyro sensor. A distance from the user terminal measured by the distance sensor to at least one measurement point of the object, an illuminance value upon generation of the desired image file measured by the illuminance sensor, measured by the depth sensor Generating at least one of a depth value of the measurement point, motion tracking related data of the object obtained by the motion tracking sensor, and information about the shape of the object identified by the shape identification unit; Way.
The method of claim 1,

Wherein the dynamic effect is a Virtual Reality (VR) effect or an Augmented Reality (AR) effect, a three-dimensional effect, or a motion effect.
The method of claim 1,

Generating desired content including the dynamic effect based on the metadata included in the desired image file

Further comprising, the method of generating an image file.
The method of claim 5,

Generating the desired content,

Building a virtual space based on the metadata; And

Compositing the virtual space with a user-photographed or user-selected target object

Including a method of generating an image file.
The method of claim 6,

The virtual space corresponds to a target object background around the target object,

And the target object background has a spatial similarity to the background.
The method of claim 7, wherein

The target object background has spatial similarity to the background by including equally at least one environment element associated with the background,

The environmental element may comprise an image file including a color of at least a portion of the background, an object included in the background, a shape of the object, a type of the object, or a positional relationship of the subject to at least a portion of the background. Way.
The method of claim 6,

The step of building the virtual space,

Determining a location in the virtual space of at least one thing to be placed in the virtual space based on the metadata; And

Generating the virtual space by arranging or rendering the thing based on the determined position

Including a method of generating an image file.
The method of claim 6,

The step of building the virtual space,

Determining a shape of an object to be disposed in the virtual space based on the information about the shape of the object included in the metadata; And

Generating the virtual space by arranging or rendering the thing based on the determined shape of the thing;

Including a method of generating an image file.
The method of claim 10,

The object to be placed or rendered is the same thing as the object or another object having the same shape as the object.
The method of claim 1,

Constructing a database based on the metadata included in the desired image file

Further comprising, the method of generating an image file.
The method of claim 1,

And the metadata is stored in the desired image file in an Exchangable Image File format (Exif).
A computer-readable recording medium having recorded thereon a program for performing the method of any one of claims 1 to 13.
Extracting the metadata from an image file photographing the object, the metadata including sensor data related to an object obtained through at least one sensor, wherein the object is at least one of a subject and a background around the subject; The sensor data includes information required to generate a dynamic effect associated with at least one of the subject and the background; And

Generating desired content including the dynamic effect based on the metadata;

Comprising the content.
The method of claim 15,

Generating the desired content,

Building a virtual space based on the metadata; And

Compositing the virtual space with a user-photographed or user-selected target object

Comprising the content.
The method of claim 16,

The virtual space corresponds to a target object background around the target object,

And the target object background has spatial similarity to the background.
A computer-readable recording medium having recorded thereon a program for performing the method of any one of claims 15 to 17.
An electronic device for generating an image file in response to a photographing command for an object through a user terminal,

A sensor data acquisition unit for acquiring sensor data related to an object from at least one sensor in response to the photographing command, wherein the object is at least one of a subject and a background around the subject, and the sensor data is the subject and the Includes information required to generate a dynamic effect associated with at least one of the backgrounds; And

An image file generation unit for generating a desired image file by adding the sensor data as metadata to an image file generated by photographing the object by the camera in response to the photographing command;

Including, the electronic device.
The method of claim 19,

Content generation unit for generating a desired content including the dynamic effect, based on the metadata included in the desired image file

The electronic device further comprises.