WO2018135730A1 - Vr image generation method, vr image processing method, and vr image processing system - Google Patents

Abstract

The present invention relates to a VR image generation method, a VR image processing method, and a VR image processing system, and a VR image generation method according to an embodiment of the present invention comprises the steps of: matching a VR image to a three-dimensional coordinate system; driving and controlling the VR image, and specifying at least one object within the image on the basis of the matched three-dimensional coordinate system; setting object information including position information, expression time information, and output information on the basis of a coordinate value of the specified object; and generating VR image data to which the object information is added.

Description

VR video generation method, VR video processing method and VR video processing system

The present invention relates to a VR image generation method, a VR image processing method, and a VR image processing system based on a 3D coordinate system.

360-degree panoramic video of the VR (virtual reality) video can be viewed by selecting the direction or point that the user wants to view within the entire video. Such VR images can be viewed through a video display device such as a head mount display (HMD).

However, in the conventional VR image processing technology, a general display processing method in which a 360 degree video is unfolded in a virtual three-dimensional space in a circle and simply displays and displays a direction or a point that a user wants to see is applied.

Accordingly, by creating an enhanced interactive (interaction) environment system between the user and the image, which is a characteristic of VR (virtual reality) technology, the image generation technology and image processing that can provide a realistic and interesting VR image to the user Development of technology was required.

It is an object of the present invention to provide a VR image generation method, a VR image processing method, and a VR image processing system that can perform various image processing using at least one object information set in a VR image based on a 3D coordinate system. .

In order to achieve the above object, the VR image generating method according to an embodiment of the present invention, matching the VR image to the three-dimensional coordinate system; Specifying at least one object in the image based on the matched three-dimensional coordinate system while driving control of the VR image; Setting object information including location information, expression time information, and output information based on coordinate values for the specified object; And generating VR image data to which object information is added.

Herein, the step of specifying the object may include specifying an area formed from at least one specific coordinate value selected by the user during the time that the object is expressed in the VR image, or setting the direction based on the origin on the 3D coordinate system. The area | region made based on an excitation angle value can be specified.

The output information may include at least one of caption information, voice information, and additional information.

In addition, the additional information may include at least one of tag information, description information about an object, object related image information, object related advertisement information, and link information.

For example, in the step of specifying the object, when the object to be specified is moved in the VR image, the specified area may be moved along the object.

In addition, the VR image processing method according to another embodiment of the present invention, driving the VR image matched to the three-dimensional coordinate system; Detecting a line of sight of the user based on an origin of the VR image, and displaying a corresponding field of view of the VR image on the display unit according to the detected line of sight; And processing output information for each specified object based on at least one of object information set for at least one predetermined object in the VR image, gaze information of the user, and a user selection signal.

Here, the object information includes position information, expression time information, and output information based on coordinate values.

For example, in the processing of the output information on the specified object, when the specified object is located in an area outside the viewing area, at least one of direction information and output information indicating a position of the specified object during the time that the object is expressed. One can be output in the viewing area of the display unit.

The processing of the output information on the specified object may further include: measuring interest of each object according to the measured time length by operating a virtual timer for each object when the specified object is located in the viewing area of the display unit. It may include. At this time, in the process of processing the output information for the specified object, if the measured interest meets a set condition, at least one of the description information, object-related image information, object-related advertisement information, link information as the output information You can output

The processing of the output information on the specified object may include: updating interest for each user according to the measured interest for each object; And updating the output information according to the degree of interest.

In addition, in the processing of the output information on the specified object, when the pointer linked to the movement of the user's gaze is positioned for a predetermined time in any one specified object area, the description information about the object as the output information, the object At least one of related image information, object related advertisement information, and link information may be output.

On the other hand, the VR image processing system according to another embodiment of the present invention, a VR image server for providing the object information set for the VR image and at least one predetermined object in the VR image via a network; And driving the VR image by matching the 3D coordinate system, detecting a user's gaze based on the user's rotation or eye movement at the origin of the VR image, and a corresponding visual field of the VR image according to the detected gaze. Displaying an area, receiving a selection signal from a user, and processing output information for each specified object based on at least one of object information, gaze information of the user, and user selection signal; It may include an application stored in the storage medium.

For example, the VR image server may provide a VR image and object information in a real time streaming transmission method.

According to the present invention, various image processing can be performed using at least one object information set in a VR image based on a 3D coordinate system, thereby providing an improved interactive environment between the user and the image, thereby providing a more realistic and interesting VR image to the user. Can be provided.

Further effects by the present invention will be described according to the following examples.

1 is a conceptual diagram illustrating a VR image processing system according to an exemplary embodiment of the present invention.

2 is a block diagram illustrating a configuration of a user device according to an embodiment of the present invention.

3 is a flowchart illustrating a method of generating a VR image according to an embodiment of the present invention.

4 is an exemplary diagram for describing specifying an object based on a specific coordinate value in a VR image.

5 is an exemplary diagram for describing an angle value having directivity with respect to a user's gaze.

6 is an exemplary diagram for describing specifying a viewing area in a VR image.

7 is a flowchart illustrating a VR image processing method according to an embodiment of the present invention.

8 is a view for explaining movement of the viewing area according to the user's gaze movement.

9 is a diagram for explaining interest measurement.

10 is an exemplary diagram for describing output information output in a display screen of a viewing area.

11 is an exemplary diagram for describing output information output through a selection signal of a user device.

12 is a flowchart illustrating a method of outputting output information based on measured interests according to an embodiment of the present invention.

13 is a flowchart illustrating a method of outputting output information based on a user selection signal according to an embodiment of the present invention.

14 is an exemplary diagram illustrating a method of outputting object information and direction information based on location information of an object in a VR image.

FIG. 15 is a conceptual diagram for describing a method of expressing a position of an object based on position information of the object in a VR image.

Hereinafter, some embodiments of the present invention will be described in detail through exemplary drawings. In adding reference numerals to the components of each drawing, it should be noted that the same reference numerals are assigned to the same components as much as possible even though they are shown in different drawings. In addition, in describing the embodiments of the present invention, if it is determined that the detailed description of the related well-known configuration or function interferes with the understanding of the embodiments of the present invention, the detailed description thereof will be omitted.

First, Figure 1 is a conceptual diagram showing a VR image processing system according to an embodiment of the present invention.

As shown in FIG. 1, the VR image processing system according to the present invention may include a VR image server 100 and a user device 200.

First, the VR image server 100 is connected to the user device 200 through a network to the user device 200 for at least one predetermined object (eg, person or object) in the VR image and the VR image. Set object information can be provided. To this end, the VR image server 100 may be provided with a database for storing the VR image and object information. For example, the VR image server 100 may be a server computer of a VR image provider.

The user device 200 is a device that drives a VR image and outputs a part of the image in the corresponding VR image according to the user's eyes. For example, the user device 200 may be a head mount display (HMD), a computer, a smartphone, or the like. It may be a configuration consisting of a combination of.

For example, as illustrated in FIG. 2, the user device 200 may include a driving unit 210, a sensing unit 220, a display unit 230, an input unit 240, and a control unit 250.

The driver 210 is configured to drive the VR image by matching the 3D coordinate system.

The sensor 220 is a component that detects the gaze of the user based on the rotation of the user (ie, the rotation of the device) or the movement of the eye based on the origin of the VR image. The detector 220 may include an angular velocity sensor, a gyro sensor, and the like to detect the rotation of the device. In addition, the detector 220 may include an eye tracking sensor that detects the direction (eye) or movement of the eye. It can be provided.

The display unit 230 is configured to display a corresponding viewing area while driving the VR image according to the detected gaze.

The input unit 240 is configured to receive a selection signal, an operation signal, and the like from the user, and may include various operation buttons and selection buttons. For example, an input means such as a mouse or a keyboard may be used.

The controller 250 controls the driver 210, the detector 220, the display 230, and the inputter 240. The controller 250 controls at least one of object information, gaze information of the user, and a user selection signal while driving the VR image. Based on this, output information for each specified object can be processed.

For example, the user device 200 may be realized through an application stored in a storage medium that executes functions of the driver 210, the detector 220, the display 230, the input 240, and the controller 250. . Of course, it may be implemented as a combination of hardware for implementing each function.

In addition, the user device 200 may further include a communication unit 260 for communicating with the VR image server 100. Additionally, the apparatus may further include a speaker for outputting an audio signal of the VR image.

Meanwhile, the VR image server 100 may create and edit VR image data by using a VR image authoring tool. For example, when specifying an area for an object, an object area may be specified using an input device such as a mouse or a keyboard in an administrator computer connected to the VR video server 100, or may be connected to the manager computer as an input device. In addition to the HMD, the object area may be specified by using a pointer or the like that interworks with the user's gaze of the HMD.

Next, the VR image generation method according to the present invention will be described with reference to FIGS. 3 to 6. 3 is a flowchart illustrating a method of generating a VR image according to an embodiment of the present invention. FIG. 4 is an exemplary diagram for describing an object specification based on a specific coordinate value in a VR image. FIG. 6 is an exemplary diagram for describing an angular value having directionality based on a line of sight, and FIG. 6 is an exemplary diagram for describing specifying a viewing area in a VR image.

As shown in FIG. 3, in the VR image generating method according to the present invention, first, the VR image is matched to a 3D coordinate system (S10). Here, the VR image is a 360 degree video, for example, by arranging a plurality of (eg, six) photographing apparatuses at an angular interval and primarily generating images obtained through the photographing apparatuses through image processing such as stitching. A spherical 360 degree video. Since generating the primary VR image is well known, a detailed description thereof will be omitted.

Subsequently, while driving control of the VR image, at least one object in the VR image is specified based on the matched 3D coordinate system (S20).

Here, specifying the object (S20) may specify an area formed from at least one specific coordinate value selected by the user during the time that the object is expressed in the VR image, or may refer to the origin on the 3D coordinate system. The region made based on the angular value having directionality can be specified.

For example, when specifying an object, an object may be specified as shown in FIG. 4. 4 is a screen illustrating a part of a viewing area in a VR image. The user may select a specific coordinate value for the object to be specified in the VR image to specify an area formed by the specific coordinate value. In FIG. 4, regions are specified for two objects (people). For example, when specifying the right object in FIG. 4, a quadrangular area formed by selecting two or more specific points (for example, (x1, y1, z1) and (x4, y4, z4)) is specified. Can be. Similarly, when specifying the left object, it is possible to specify a quadrangular area formed by selecting two or more specific points (for example, (x6, y6, z6) and (x7, y7, z7)). Here, the area formed by the specific point is not limited to a quadrilateral, but may have a polygonal shape consisting of three or more specific points, or a circle having a predetermined distance from one specific point or about a specific point. A polygonal area of a predetermined size may be specified.

In the present embodiment, the specific point is specified in the rectangular coordinate system, but the present invention is not limited thereto. For example, the specific point may be specified in the spherical coordinate system.

For example, in the coordinates (r, θ, φ) of the spherical coordinate system, the value of θ (angle from the z-axis in the positive direction to the straight line formed by the specific point in the visual direction, that is, the angle of the vertical component) and φ value (positive The area | region made by specifying four specific points can be specified based on the angle to the straight line which the x-axis of a direction, the straight line which forms an origin and a specific point projected on the xy plane, ie, the angle of a horizontal component). Here, r is a constant value as the distance from the origin (that is, the eyes of the user) to the VR image. In this way, the region of the object to be specified can be specified using a specific point made based on an angle value having a directionality with respect to the origin in the spherical coordinate system.

Alternatively, as shown in FIG. 5, a region made based on an angular value having directivity may be specified. For example, as shown in (a) of FIG. 5, when the user's eyes (ie, a virtual camera) and the VR image are viewed from the lateral direction, the angle value θ1 having the directionality set based on the user's line of sight is upward and downward. Also, as shown in FIG. 5 (b), when the user's eye (ie, a virtual camera) and the VR image are viewed from the upper surface direction, the angle value φ 1 having the directionality set based on the user's gaze is Determined left and right. Accordingly, a rectangle is formed by two linear components in the horizontal direction in the VR image formed by the upper and lower angle values θ1 and two linear components in the vertical direction in the VR image formed by the left and right angle values φ1. Areas of shape may be specified.

Alternatively, when specifying an object, an area of the object may be specified through a function of tracking a specific image (face shape, object shape, etc.) in the VR image.

For example, in the step of specifying the object (S20), when the object to be specified moves in the VR image according to time, the object may be specified while moving the specified area along the corresponding object.

In addition, when specifying the viewing area (that is, the display screen area displayed on the display unit) in the VR image, the viewing area may be specified as shown in FIG. 6. FIG. 6A shows specifying a viewing area in a spherical VR image viewed by the user's eye (for example, a virtual camera), and FIG. 6B shows a specified viewing area. Similarly to the case of specifying the object area, the user may specify a view area of the VR image viewed by the user during viewing of the VR image as an area of view by specifying two or more coordinate values on the three-dimensional coordinate system. Alternatively, an area made based on an angular value having directivity may be specified as the viewing area.

Next, as shown in FIG. 3, location information based on coordinate values for the specified object, expression time information including start time and end time at which the object is expressed, and object information including additional output information about the object are set. (S30).

The output information may include at least one of caption information, voice information, and additional information. Here, the caption information may be a caption for the dialogue spoken by the person object or a caption for explaining the object of the object. The voice information may include additional sound information about the object. In addition, the additional information is set to variously use the specified object and may include at least one of tag information, description information about the object, image information related to the object, advertisement information related to the object, and link information.

The tag information may include type information or name information of an object for searching through a keyword or the like and categorizing by category. Descriptive information is information for describing an object. If the object is a person, it may include information such as occupation, age, body, company, education, and representative works. If the object is an object, the information may include information such as product specification and price. have. The object-related image information may include recommended image information related to the object, and the advertisement information may include advertisement image information in which the object appears. In addition, the link information may include related site link information such as Facebook, Twitter, etc. for the object.

In addition, the output information of each object may further include expression time information (ie, expression start time and expression end time) and position information to be expressed in the VR image based on the 3D coordinate system.

Subsequently, as shown in FIG. 3, VR image data to which the set object information is added is generated (S40). The generated VR image data may be stored on the VR image server 100. The VR image server 100 may store the VR image and the object information as one integrated data file and provide the same to the user device 200. Alternatively, the VR image server 100 may store the VR image and the object information as a data file separating the VR image and the object information. The object information may be selectively provided to the user device 200.

As such, VR image data to which object information generated by the VR image generation method is added may be used to provide various services to a user through image processing.

Next, the VR image processing method according to the present invention will be described with reference to FIG. 7. 7 is a flowchart illustrating a VR image processing method according to an embodiment of the present invention.

As shown in FIG. 7, the VR image processing method according to the present invention first drives a VR image matched to a 3D coordinate system through a driving unit of the user device (S110). Subsequently, the user's gaze is sensed based on the origin of the VR image while driving the VR image through the sensing unit, and a corresponding viewing area of the VR image is displayed on the display unit according to the detected gaze (S120). Subsequently, the controller processes the output information for each specified object based on at least one of preset object information, gaze information of the user, and a user selection signal through the input unit for at least one predetermined object in the VR image. (S130). As described above, the object information may include location information, expression time information, and output information based on coordinate values, and the output information may include at least one of caption information, voice information, and additional information. In addition, the additional information may include at least one of tag information, description information about an object, object related image information, object related advertisement information, and link information.

For example, in the process of processing the output information for the specified object (S130), if the specified object (now speaking person) is located in an area outside the viewing area, during the time that the object is expressed (for example, At least one of direction information (for example, an arrow) and output information (for example, a caption) indicating a position of the object may be output to a set position in the viewing area of the display unit.

For example, referring to FIG. 14, when the speaker is located outside the user's field of view as an object specified in the VR image, subtitles uttered by the speaker (for example, 'how was your today?') Are displayed as object information. 13 may be output along with direction information such as an arrow indicating a speaker's position (FIG. 13A), a gradually increasing point display (FIG. 13B), and a sequentially blinking dot display. In this case, when the subtitle is not set as the object information for the specified speaker, only the direction information may be displayed for the speaker to be uttered.

FIG. 15 is a conceptual diagram for describing a method of expressing a position of an object based on position information of the object in a VR image. As illustrated in FIG. 15, a method of displaying a speaker's position as a specified object based on an arrow is illustrated. Although there is no speaker in the field of view that utters the word 'hello ~', the user can know the position of the speaker based on the direction of the arrow, so that the user can be induced to identify the speaker by switching the field of view to the direction of the arrow. .

As another example, the position of the speaker may be expressed through visual image processing that changes the position / shape of the subtitle, which is object information, without directly displaying the direction information. That is, when the coordinates corresponding to the speaker's position information are not included in the screen range displayed to the user, the position of the caption, which is the object information, is gradually incremented in the direction close to the speaker's position at the set position such as the center, center, or bottom. By repeatedly moving to the designated position, the direction of the speaker can be indicated. Alternatively, the direction of the speaker may be indicated by using a gradual change such as subtitle color or density. For example, the direction of the speaker may be indicated by changing the color of the subtitle, such as blurring the subtitle when the speaker is far from the direction of the viewer, or changing to a set color (eg, green) as the speaker gets closer. . Alternatively, the direction of the speaker may be indicated by tilting the subtitle in the direction of the speaker or by gradually increasing the size of the subtitle.

Returning to FIG. 7 again, in operation S130 of processing output information on the specified object, when the specified object is located in the viewing area of the display unit, as shown in FIGS. 8 and 9, a virtual timer for each object Operate to measure the interest of each object according to the measured length of time. For example, as shown in FIG. 8, first, an object is included in the field of view when "user gaze-1". Then, when " user gaze-2 ", the gaze has moved but the object is still included in the field of view. Then, when "user gaze-3", the gaze moves so that the object is only partially included in the field of view. Then, when "user gaze -4", the gaze moves so that the object is located outside the viewing area. In this case, as shown in FIG. 9A, when it is determined that the object is included in the viewing area, a virtual timer is operated to measure the length of time that the object is located in the viewing area. In this case, as shown in FIG. 9B, the length of time may be measured until at least some of the objects are not included in the field of view, or may be measured until all of the objects are not included in the field of view.

If the measured interest meets a set condition (for example, a predetermined time), at least one of description information about the object, image information related to the object, advertisement information related to the object, and link information may be output as output information.

In addition, as shown in FIG. 10, the output information may display the recommendation image information as the object-related image information with high interest and the product of interest information as the object-related advertisement information on the display screen of the viewing area while driving the VR image. Alternatively, output information may be output for one object having the highest interest after the VR image driving ends. For example, for an object of high interest, we recommend an image whose name (product name, person's name, etc.) is included in the title (for example, tag information), or in an image classification category preset for the object. Related videos included may be recommended.

In addition, when processing the output information for the specified object, the interest of each user can be updated according to the measured interest of each object, and the output information can be updated according to the interest. To this end, in order to manage the user, the VR image server 100 may store user information through user-specific membership and then log in by accessing the VR image server 100 through the user device 200. have. If the non-login user is not registered, the output information may be output according to the measured interest of each object in the reproduced VR image.

Next, a method of outputting output information according to the measured interest level will be described with reference to FIG. 12. 12 is a flowchart illustrating a method of outputting output information based on measured interests according to an embodiment of the present invention.

As shown in FIG. 12, in the output information output method according to the measured interest level according to the present invention, first, the user device 200 connects to the VR image server 100, and among the object information about the VR image and the VR image. By requesting at least one, the requested VR image and the object information may be received respectively. Alternatively, the user device 200 may request VR image data including object information from the VR image server 100 and receive the requested VR image data. For example, the VR image server 100 may provide the VR image and the object information to the connected user apparatus 200 in a real time streaming manner, so that the user apparatus 200 may execute the VR image in real time. Alternatively, the VR image server 100 provides the VR image and the object information to the connected user device 200 in a download manner, and after the user device 200 downloads the VR image and the object information to the storage unit, the VR image and the object information are stored in the VR. You can also run an image.

Next, the user device 200 executes the corresponding VR image (S211). Subsequently, the user device 200 detects the user's gaze while executing the VR image to check the viewing unit range (ie, the display screen of the viewing area) (S212), and further, the object unit range (ie, the object) using the object information. The region specified by step S213). Next, it is determined whether the object part range in the view part is included (S214). If included (Yes) starts to operate the virtual timer (S215). If not included (No), the virtual timer stops operation (S216). Next, the interest of each object is measured according to the measurement time for each object measured by the virtual timer (S217). Subsequently, when execution of the corresponding VR image is ended (S218), output information on the object of interest with high interest may be output according to whether or not the user logs in (S219).

If the user is logged in, the user device 200 transmits the measured interest information and the user information to the VR image server 100, and based on the accumulated image interest object for each user as output information from the VR image server 100. Recommended video information and advertising information can be provided. For example, the user device 100 may add a score based on the interest information measured on a predetermined tag for each object. In this case, a classification keyword related to the VR image may be designated as a tag. Multiple tags per object may exist in the VR image. For example, the tag score for each object may be calculated as (timer time per object / total VR video time) × tag correction value. Here, the tag correction value is set to have a larger correction value as the degree of relevance for each tag increases. As such, the user device 200 may transmit the tag score for each object as the measured interest information to the VR image server 100 together with the user information. Meanwhile, the VR image server 100 may transmit output information to the user device 200 by using the accumulated tag information corrected by adding the tag value of the image previously viewed for each user. Accordingly, the user device 200 may output the recommendation image information and the advertisement information of the object of interest based on the accumulated interest of the user. On the other hand, if the user is not logged in, the user device 200 may output the recommendation image information and the advertisement information as output information of the object information for the object of high interest measured according to the execution of the VR image ( S220).

In addition, when processing the output information on the object specified in FIG. 7 (S130), the output information on the selected object may be output using a pointer linked to the user's eye movement. For example, when the pointer linked to the movement of the user's eyes is positioned for a predetermined time in any one specified object area, the output information includes description information about the object, object related image information, object related advertisement information, and link information. At least one of them may be output. For example, as shown in FIG. 11A, when a pointer is located in the object area, a selection signal is input through the input unit of the user device, and as shown in FIG. 11B, the object is output as the output information. Description information and link information may be displayed.

Next, a method of outputting output information of the selected object by using a pointer linked to the user's line of sight will be described with reference to FIG. 13. 13 is a flowchart illustrating a method of outputting output information based on a user selection signal according to an embodiment of the present invention.

As shown in FIG. 13, in the output information output method based on the user selection signal according to the present invention, first, the user device 200 accesses the VR image server 100 and requests object information on a VR image. Requested object information can be received. Alternatively, the user device 200 may request VR image data including object information from the VR image server 100 and receive the requested VR image data.

Next, the user device 200 executes the corresponding VR image (S231). Subsequently, the user device 200 checks the point coordinates of the device pointer linked to the user's line of sight (S232), and outputs the pointer together with the display screen of the viewing area. In this case, the user device 200 checks an object portion range (ie, an area specified as an object) (S233). Next, it is determined whether a pointer within the range of the object portion is located (S234). If the pointer within the range of the object part is not located, the determination step S234 is continuously repeated. If the pointer within the range of the object portion is located, the user device 200 determines whether a specific button is input as the selection signal (S235). If a specific button is input, output information on the corresponding object is output on the display screen (S236). If a specific button is not input, the process is repeated again from step S234. In this case, the user device includes a specific button as an input unit.

In this case, the specific button input step S235 may be replaced with a step of determining whether the pointer in the range of the object unit is positioned for a predetermined time. That is, when the pointer in the range of the object portion is located for a predetermined time, output information about the object is output, and when it is not located for the set time, the process may be repeated from step S234.

As described above, according to the present invention, various image processing can be performed using at least one object information set in the VR image based on the three-dimensional coordinate system, thereby providing a more interactive environment between the user and the image, thereby making the user more realistic and realistic. It can provide interesting VR video.

On the other hand, the method of the present invention as described above can be written in a computer program. And the code and code segments constituting the program can be easily inferred by a computer programmer in the art. In addition, the written program is stored in a computer-readable recording medium (information storage medium), and read and executed by a computer to implement the method of the present invention. The recording medium may include any type of computer readable recording medium.

The present invention described above is capable of various substitutions, modifications, and changes without departing from the technical spirit of the present invention for those skilled in the art to which the present invention pertains. It is not limited by the drawings.

[Description of the code]

100: VR video server 200: user device

210: driver 220: detector

230: display unit 240: input unit

250: control unit

Claims (14)

Matching the VR image to a three-dimensional coordinate system; Specifying at least one object in the image based on the matched three-dimensional coordinate system while driving control of the VR image; Setting object information including location information, expression time information, and output information based on coordinate values for the specified object; And Generating VR image data to which the object information is added; VR image generation method comprising a. The method according to claim 1, Specifying the object, During the time that the object is expressed in the VR image, an area made up of at least one specific coordinate value selected by the user is specified, or is made based on an angle value having a direction based on an origin point on the 3D coordinate system. VR video generation method that specifies the area. The method according to claim 1, The output information includes at least one of subtitle information, audio information, additional information. The method according to claim 3, The additional information includes at least one of tag information, description information about an object, object related image information, object related advertisement information, and link information. The method according to claim 2, In the step of specifying the object, And moving the specified area along the object when the object to be specified is moved within the VR image. Driving a VR image matched to a 3D coordinate system; Detecting a line of sight of the user based on an origin of the VR image, and displaying a corresponding field of view of the VR image on the display unit according to the detected line of sight; And And processing output information for each specified object based on at least one of object information set for at least one predetermined object in the VR image, gaze information of a user, and a user selection signal. Treatment method. The method according to claim 6, The object information is a VR image processing method including position information, expression time information and output information based on the coordinate value. The method according to claim 6, In the step of processing the output information for the specified object, When the specified object is located in an area outside the viewing area, a VR image outputting at least one of direction information and output information indicating a location of the specified object during the time that the object is expressed in the viewing area of the display unit. Treatment method. The method according to claim 6, Processing the output information for the specified object, And when the specified object is located within the viewing area of the display unit, operating the virtual timer for each object to measure the interest of each object according to the measured time length. The method according to claim 9, Processing the output information for the specified object, And outputting at least one of description information, object-related image information, object-related advertisement information, and link information as an output information when the measured interest is satisfied. The method according to claim 10, In the step of processing the output information for the specified object, Updating interest for each user according to the measured interest for each object; And And updating the output information according to the degree of interest. The method according to claim 6, In the step of processing the output information for the specified object, When the pointer linked to the eye movement of the user is located within a specified object area for a predetermined time, at least one of description information, object related video information, object related advertisement information, and link information about the object as output information. VR image processing method for outputting. A VR image server providing object information set for the VR image and at least one object specified in the VR image through a network; And Driving the VR image by matching the 3D coordinate system; detecting a user's gaze based on the user's rotation or eye movement at the origin of the VR image; and corresponding to the VR image according to the detected gaze. Displaying a viewing area, receiving a selection signal from a user, and processing output information for each specific object based on at least one of the object information, the user's gaze information, and the user selection signal An application stored in the storage medium for executing the; VR image processing system comprising a. The method according to claim 13, The VR image server is a VR image processing system for providing the VR image and object information in a real-time streaming transmission method.

Images