JP4933337B2 - Movie shooting device, movie playback device, movie distribution system, control method, control program, and recording medium - Google Patents

Description

  The present invention relates to a moving image distribution system for distributing a moving image shot by a shooting device to a moving image playback device via a communication network.

  2. Description of the Related Art Conventionally, with the development of information communication technology, various information providing services using systems capable of two-way communication have become widespread. Thereby, a viewer can easily obtain various information uploaded by an information provider using a communication terminal device (photographing device) from his / her communication terminal device (video playback device) via the Internet (for example, a patent). References 1 and 2).

  Furthermore, in recent years, high-speed and large-capacity data communication has become possible, and a moving image distribution system that distributes large-capacity digital content such as moving images can be realized. Then, a video distribution system in which a video (live video) being captured by the video capturing device is distributed in a streaming manner and the live video can be reproduced in real time by a viewer's video reproducing device has become widespread (for example, Patent Documents 3 to 5). reference).

In addition, portable communication terminal devices such as mobile phones are becoming more compact, multi-functional, and video communication devices that capture and distribute live video, and receive them. It has become possible to use as a moving image playback device for playback. For example, in the information distribution system described in Patent Document 1, it is described that a mobile phone is adopted as a communication terminal device on the information provider (photographer) side and the viewer side.
JP2003-036365 (released February 7, 2003) JP 2003-203035 (released July 18, 2003) JP2005-012393 (released on January 13, 2005) JP 2005-184536 (released on July 7, 2005) JP 2006-314073 (released on November 16, 2006)

  However, the above-described conventional configuration has the following problems.

  2. Description of the Related Art Conventionally, in an image distribution system that distributes live video, it is possible to control a photographing apparatus that captures live video from a communication terminal device of a viewer via a communication network such as the Internet or a mobile phone network. Specifically, the viewer can acquire a live video intended by the viewer by rotating the photographing apparatus according to the driving amount and the driving direction specified by the viewer.

  However, the above technique can be applied only to a system in which the photographing apparatus is fixed, and as described above, it cannot be applied when the photographer's photographing apparatus is a portable communication terminal device such as a mobile phone. There's a problem. Specifically, since the photographer operates by holding the photographing device built in the mobile phone, the photographing device is not fixed. For this reason, the viewer cannot directly control the photographing device by remote control, and the user cannot transmit the intention of the viewer to the photographing device used by the photographer.

  Such a problem is not limited to the case of adopting a mobile phone in the moving image distribution system, and similarly occurs in a moving image distribution system in which a shooting device for shooting a live image to be distributed is manually operated. is there.

  The present invention has been made in view of the above-described problems, and its purpose is to capture a viewer's intention via a photographing device operated by the photographer to the photographer, It is to realize a moving image reproducing apparatus, a moving image distribution system, a control method, a control program, and a recording medium.

  In order to solve the above-described problem, the moving image shooting apparatus according to the present invention shoots a moving image by manually checking the moving image displayed on the display unit and manually adjusting the position of the shooting unit. In the moving image shooting device that transmits the moving image to the moving image reproduction device, the moving image reproduction device displays the object designation information indicating the object of interest in the moving image by the viewer who views the moving image on the moving image reproduction device. And object operation information output means for emphasizing the object indicated by the object specification information acquired by the object specification information acquisition means in the moving image and displaying it on the display unit. It is characterized by being.

  According to the above configuration, first, the object designation information acquisition unit acquires the object designation information from the moving image playback device. The object designation information is information indicating an object that is viewed by the viewer who views the moving image with the moving image reproducing apparatus.

  Subsequently, when displaying the moving image, the operation instruction output means emphasizes the object indicated by the acquired object specifying information and displays it on the display unit.

  Thus, when the photographer manually adjusts the position of the photographing unit and performs photographing, the photographer can confirm the moving image being photographed with the object included in the moving image highlighted. .

  Therefore, the photographer can know which part of the moving image the viewer of the moving image is paying attention during the shooting of the moving image.

  As a result, it is possible to transmit the viewer's intention to the photographer via the photographing device operated by the photographer.

  If the photographer can know the target object that the viewer is paying attention to as described above, for example, it is possible to perform photographing by operating the photographing unit with reference to the target object as follows.

  The photographer can manually adjust the position of the photographing unit so that the object focused on by the viewer comes to a predetermined position on the screen (for example, the center of the screen). That is, the orientation of the photographing unit can be changed so as to capture the object in the center of the screen. Alternatively, the photographer can change the zoom ratio of the photographing unit so that the object is enlarged. Further, the photographer can change the orientation of the photographing unit so that the object is out of frame.

  The object specifying information includes coordinate information for specifying an object area that is an area including the object in the moving image, and the operation instruction output unit emphasizes the object area specified by the coordinate information. You may display on the said display part.

  As a result, the target object to be highlighted by the operation instruction output means is specified by the coordinate information in the moving image.

  The coordinate information includes, for example, information such as the XY coordinates of the start point and end point of the rectangle, or the XY coordinates and radius of the center of the circle.

  Alternatively, the object designation information is a still image obtained by adding information indicating an object area that is an area including the object to one frame of the moving image, and the operation instruction output unit adds the information to the still image. The object area specified based on the information indicating the included object area may be emphasized and displayed on the display unit.

  According to the above configuration, which object the operation instruction output means should highlight is specified by the information indicating the object area included in the still image.

  As a specific example of the method for adding information indicating the object area to the still image, for example, the following image processing is performed on the still image. That is, a frame surrounding the object area is superimposed, a color tone of the object area is changed, a color of the object area is reversed, and the object is indicated by an arrow.

  Alternatively, the object designation information is a trimmed image obtained by trimming an area including the object in the moving image, and the operation instruction output unit emphasizes an area that matches the trimmed image in the moving image, and displays the display unit. May be displayed.

  According to the configuration, the trimmed image specifies which object the operation instruction output means should highlight. More specifically, an area that matches the trimmed image is specified in the moving image being captured displayed on the display unit.

  The moving image capturing apparatus further includes an object tracking unit that tracks the position of the object specified by the object specifying information in the moving image, and the operation instruction output unit tracks the operation instruction output unit. It is preferable to perform highlighting in accordance with the movement of the object.

  According to the above configuration, the operation instruction output unit highlights the target specified by the target specifying information, and then the moving image displayed on the display unit is moved by the photographer moving the position of the shooting unit. Even if the object inside moves, highlighting is performed following the movement of the object.

  Thereby, even if the photographing unit is moved, the photographer does not lose sight of where the object that the viewer is paying attention to is reflected.

  In addition, when it is desired to capture at a predetermined position (for example, the center) of the object screen, it is possible to perform shooting while moving the shooting unit so that the highlighted portion is in the center.

  The moving image photographing apparatus further includes direction instruction means for determining a direction in which the photographing unit is moved in order to capture an object tracked by the object tracking means at a predetermined position of the moving image, and the operation instruction output Preferably, the means displays information indicating the direction determined by the direction indicating means on the display unit.

  According to the above configuration, the direction indicating means grasps the position of the object shown in the moving image being photographed and captures the object at a predetermined target position (for example, the center). The direction in which the imaging unit needs to be moved is determined and determined. Then, the operation instruction output unit displays information (for example, an arrow or the like) indicating the direction determined by the direction instruction unit on the display unit.

  Thus, the photographer can know in which direction the operation unit should be moved in order to capture the object designated by the viewer at a predetermined position (for example, the center). Therefore, it is possible to more easily capture a moving image intended by the viewer.

  The moving image photographing apparatus further includes a detecting unit that detects that the object tracked by the object tracking unit is captured at a predetermined position of the moving image, and the detecting unit moves the object to the predetermined position. It is preferable that a notification means for notifying the moving image playback device that the capture has been detected is provided.

  According to the above configuration, after the photographer moves the photographing unit and continues photographing, when the object designated by the viewer is captured at a predetermined position (for example, the center), the detection means Detect. And when the said detection means detects the said content, a notification means notifies a viewer's moving image reproducing device that the detection means detected that the said target object was capture | acquired by the screen center.

  As a result, the viewer's moving image playback device can know that the designated object has been captured at a predetermined position. Therefore, the viewer can recognize that the intended moving image has been obtained.

  Here, the viewer can recognize that the moving image is a live image by obtaining the moving image as specified by the viewer.

  The object specifying information includes a time code for specifying one frame of the moving image in addition to the coordinate information, and the moving image capturing device further includes the object specifying information acquiring unit of the moving image. Is provided with a change amount calculating means for calculating the degree of change between both frames by comparing the frame specified by the time code acquired by the latest frame and the object specifying information acquiring means is configured to calculate the change amount. When the degree of change calculated by the means is larger than a predetermined value, the moving image reproducing apparatus may be requested to retransmit the coordinate information of the object in the latest frame.

  According to the above configuration, the object specifying information acquiring unit acquires the coordinate information and a time code for specifying individual frames constituting the moving image as the object specifying information. Then, the change amount calculating means compares the frame specified by the time code with the latest frame in the moving image captured by the own apparatus, and calculates the degree of change between both frames. If the degree of change between both frames is greater than a predetermined value, the object designation information acquisition means requests the moving image reproduction apparatus to retransmit the coordinate information of the object in the latest frame.

  That the degree of change between both frames is large means that the video at the time when the viewer designates the target area, and the video at the time when the video shooting device specifies the target to which the viewer pays attention from the target area, Then, it shows that the contents are very different. Therefore, in such a case, even if the object area is designated by the coordinate information on the viewer side, there is a possibility that the coordinate information does not correctly capture the object when specified by the moving image photographing apparatus. .

  Therefore, when the degree of change between both frames is large, the object including the correct coordinate information in the latest frame can be requested by requesting again from the video shooting device to specify the object area in the latest frame. The item designation information can be acquired from the moving image reproducing device.

  From the above, it becomes possible to specify the target object more accurately and correctly convey the viewer's intention to the photographer.

  The predetermined value may be set in advance to the minimum value of the amount of change when the content of the frame changes to such an extent that the reliability of the coordinate information for capturing the object is lost.

  In order to solve the above-described problem, the moving image reproducing device according to the present invention is a moving image photographing device that shoots a moving image by confirming the moving image displayed on the display unit and manually adjusting the position of the photographing unit. In the video playback device that receives and plays back the video via the communication network, the designation of the target object that the viewer pays attention to in the video input by the viewer browsing the video Object receiving means for receiving object, object specifying information generating means for generating object specifying information for specifying the object received by the object receiving means, and object specifying information generated by the object specifying information generating means Is transmitted to the moving image photographing apparatus.

  According to the above configuration, the object receiving means receives the designation of the object that the viewer pays attention to in the video from the viewer who browses the video. Subsequently, the object designation information generating unit generates object designation information for specifying the received object. Finally, the transmission unit transmits the generated object designation information to the above-described moving image capturing apparatus.

  As a result, the viewer's intention can be transmitted to the photographer via the moving image photographing device operated by the photographer.

  The object specifying information generating means may generate coordinate information for specifying the area in accordance with the specification of the area including the object in the moving image input by the viewer through the operation unit.

  Thereby, in the moving image shooting apparatus, which object should be transmitted to the photographer is specified by the coordinate information in the moving image.

  The moving image reproduction apparatus further includes a detecting unit that detects that the target specified by the target specifying information generated by the target specifying information generating unit is captured at a predetermined position of the moving image, and the detecting unit It is preferable that the information processing apparatus further comprises notification means for displaying a predetermined message on the display unit of the apparatus when detecting that the object is captured at the predetermined position.

  Thereby, the viewer can know that the designated object has been captured at a predetermined position. Therefore, the viewer can recognize that the intended moving image has been obtained.

  Although the content of the predetermined message is not particularly limited, for example, a message indicating that it is a live video is displayed. Thus, the viewer can recognize that the moving image is a live image by obtaining the moving image as specified by the viewer.

  A moving image distribution system that transmits a moving image captured by the above moving image capturing device to the above-described moving image reproducing device also falls within the scope of the present invention.

  In order to solve the above-described problem, the moving image shooting apparatus according to the present invention shoots a moving image by manually checking the moving image displayed on the display unit and manually adjusting the position of the shooting unit. In the moving image shooting device that transmits the moving image to the moving image reproduction device via the moving image reproduction device, direction control information indicating the direction in which the moving part is moved is desired from the moving image reproduction device. Based on the direction control information acquisition means to be acquired, the direction control information acquired by the direction control information acquisition means, the direction specification means for specifying the direction in which the photographing unit should be moved, and the direction specification means specified in the video Operation instruction output means for displaying operation instruction information for instructing a photographer to move the imaging unit in a direction on the display unit is provided.

  According to the above configuration, first, the direction control information acquisition unit acquires the direction control information from the moving image playback device. The direction control information is information indicating a direction in which the image capturing unit desired by a viewer who views the moving image is moved.

  Next, the direction specifying means specifies the direction in which the photographing unit should be moved based on the direction control information. Subsequently, when displaying the moving image, the operation instruction output unit displays information (for example, an arrow or the like) indicating the specified direction on the display unit.

  Accordingly, the photographer can know in which direction the photographing unit should be moved (whether the viewer wants) when performing the photographing by manually adjusting the position of the photographing unit.

  As a result, it is possible to transmit the viewer's intention to the photographer via the photographing device operated by the photographer.

  In order to solve the above-described problem, the moving image reproducing device according to the present invention is a moving image photographing device that shoots a moving image by confirming the moving image displayed on the display unit and manually adjusting the position of the photographing unit. In a video playback device that receives and plays back the video via a communication network, the designation of the direction in which the viewer wants to move the photographing unit is input by the viewer browsing the video via the operation unit. Direction instruction receiving means for receiving, direction control information generating means for generating direction control information for specifying the direction received by the direction instruction receiving means, and direction control information generated by the direction control information generating means to the moving image photographing apparatus. And a transmission means for transmitting.

  According to the said structure, a direction instruction | indication reception means receives designation | designated of the direction which moves the said imaging | photography part which the said viewer desires from the viewer who browses the said moving image. Subsequently, the direction control information generation unit generates direction control information for specifying the accepted direction. Finally, the transmission unit transmits the generated direction control information to the above-described moving image capturing apparatus.

  As a result, the viewer's intention can be transmitted to the photographer via the moving image photographing device operated by the photographer.

  A moving image distribution system for transmitting a moving image captured by the moving image capturing device to the moving image reproducing device also falls within the scope of the present invention.

  In order to solve the above-described problem, the method for controlling the moving image shooting apparatus according to the present invention shoots a moving image by manually checking the moving image displayed on the display unit and manually adjusting the position of the shooting unit. A method for controlling a moving image photographing apparatus that transmits the moving image to a moving image reproducing apparatus via a communication network, wherein an object that indicates a target object that is viewed by the viewer viewing the moving image on the moving image reproducing apparatus An acquisition step of acquiring object designation information from the moving image reproduction device, and a display step of highlighting and displaying the object indicated by the object designation information acquired in the acquisition step in the moving image on the display unit. It is characterized by.

  According to the method, when the moving image is displayed in the display step, the object indicated by the acquired object designation information is highlighted and displayed on the display unit. When shooting by manually adjusting the position of the shooting unit, it is possible to check the moving image being shot in a state where the object included in the moving image is highlighted.

  Therefore, the photographer can know which part of the moving image the viewer of the moving image is paying attention during the shooting of the moving image. As a result, it is possible to transmit the viewer's intention to the photographer via the photographing device operated by the photographer.

  In order to solve the above-described problem, the method for controlling the moving image shooting apparatus according to the present invention shoots a moving image by manually checking the moving image displayed on the display unit and manually adjusting the position of the shooting unit. A method for controlling a moving image photographing apparatus that transmits the moving image to a moving image reproducing apparatus via a communication network, wherein a direction indicating a direction in which the image capturing unit is moved is desired by a viewer who views the moving image on the moving image reproducing apparatus. An acquisition step of acquiring control information from the moving image playback device, a direction specifying step of specifying a direction in which the photographing unit should be moved based on the direction control information acquired in the acquiring step, and the direction specifying step of the moving image And a display step of displaying operation instruction information for instructing the photographer to move the photographing unit in the direction specified in the display unit.

  Accordingly, the photographer can know in which direction the photographing unit should be moved (whether the viewer wants) when performing the photographing by manually adjusting the position of the photographing unit.

  As a result, it is possible to transmit the viewer's intention to the photographer via the photographing device operated by the photographer.

  In order to solve the above-described problem, a method for controlling a moving image playback apparatus according to the present invention is a moving image in which a photographer confirms a moving image displayed on a display unit and manually adjusts the position of the shooting unit. A method for controlling a moving image reproducing apparatus for receiving and reproducing the moving image from a photographing device via a communication network, wherein the viewer inputs the moving image of the moving image, which is input via an operation unit by a viewer who views the moving image. An accepting step for accepting designation of an object of interest, a generating step for generating object specifying information for identifying the object accepted at the accepting step, and an object designating information generated at the generating step Including a transmission step of transmitting to the moving image photographing apparatus.

  As a result, the viewer's intention can be transmitted to the photographer via the moving image photographing device operated by the photographer.

  In order to solve the above-described problem, a method for controlling a moving image playback apparatus according to the present invention is a moving image in which a photographer confirms a moving image displayed on a display unit and manually adjusts the position of the shooting unit. A method for controlling a moving image reproducing apparatus for receiving and reproducing the moving image from a photographing device via a communication network, wherein the viewer desires the viewer to input the moving image through the operation unit. An accepting step for accepting designation of a direction for moving the photographing unit, a generating step for generating direction control information for specifying the direction accepted in the accepting step, and the moving image photographing for the direction control information generated in the generating step. And a transmission step of transmitting to the apparatus.

  As a result, the viewer's intention can be transmitted to the photographer via the moving image photographing device operated by the photographer.

  Note that the moving image shooting device and the moving image playback device may be realized by a computer, and in this case, the moving image shooting device or the moving image playback device is realized by a computer by operating the computer as each of the means. The moving picture photographing apparatus or the moving picture reproducing apparatus control program to be executed and a computer-readable recording medium on which the control program is recorded also fall within the scope of the present invention.

  As described above, the moving image shooting apparatus according to the present invention obtains, from the moving image reproduction apparatus, object designation information indicating an object that is viewed by the viewer viewing the moving image on the moving image reproduction apparatus. And an operation instruction output means for emphasizing and displaying the object indicated by the object designation information acquired by the object designation information acquisition means in the moving image.

  Or the direction control information acquisition means which acquires the direction control information which shows the direction which the viewer who browses the said moving image with the said moving image reproducing device moves the said imaging | photography part, and the said direction control information acquisition Based on the direction control information acquired by the means, the direction specifying means for specifying the direction in which the photographing unit should be moved and the photographer are instructed to move the photographing unit in the direction specified by the direction specifying means in the moving image. And an operation instruction output means for displaying operation instruction information on the display unit.

  As described above, the moving image reproducing apparatus according to the present invention receives a designation of an object that the viewer pays attention to in the moving image, which is input through the operation unit by the viewer who views the moving image. The moving image photographing of the object specifying information generated by the receiving means, the object specifying information generating means for specifying the object specifying information for specifying the object received by the object receiving means, and the object specifying information generating means Transmitting means for transmitting to the apparatus.

  Alternatively, a direction instruction receiving unit that receives a designation of a direction in which the viewer wants to move the imaging unit, and a direction received by the direction instruction receiving unit, which are input via an operation unit by a viewer who views the moving image. Direction control information generation means for generating direction control information for specifying the direction control information, and transmission means for transmitting the direction control information generated by the direction control information generation means to the moving image photographing apparatus.

  Thereby, it is possible to transmit the viewer's intention to the photographer via the photographing device operated by the photographer.

Embodiment 1
An embodiment of the present invention is described below with reference to the drawings. In the present embodiment, as an example, the moving image shooting device and the moving image playback device of the present invention are each applied to a mobile phone, and a moving image taken by the photographer's mobile phone is transferred to the viewer's mobile phone via the moving image distribution server. A video distribution system that distributes to the telephone will be described.

[Outline of video distribution system]
FIG. 2 is a diagram showing a schematic configuration of the moving image distribution system 100 according to the embodiment of the present invention. As shown in FIG. 2, a moving image distribution system 100 of the present invention includes a mobile phone (moving image shooting device) 1 of a photographer S, a mobile phone (moving image playback device) 3 of a viewer R, a moving image distribution server 2, and these. And a communication network W to be communicably connected.

  The communication network W is realized as a mobile phone network including a base station, the Internet, etc., and each mobile phone can make calls and exchange mails, and can transmit and receive moving images to and from the moving image distribution server 2. ing.

  The video distribution server 2 is a video distribution service for transmitting and receiving moving images taken with a mobile phone in a streaming manner with each user's mobile phone with a mobile phone service contract, and distributing live video in real time. Is to provide. For example, when the photographer S at the shooting site operates his / her mobile phone 1 to acquire a live video, the mobile phone 1 transmits the live video to the moving image distribution server 2 via the communication network W.

  The moving image distribution server 2 transmits the live video received from the mobile phone 1 to the mobile phones 3. The destination of the live video may be specified from the mobile phone 1 of the photographer S, or may be preset in the video distribution server 2. The moving image distribution server 2 can simultaneously distribute the live video received from the mobile phone 3 to the mobile phones of many unspecified users, or distribute the live video only to the mobile phones of specific users.

  The mobile phone 3 receives the live video taken by the mobile phone 1 from the video distribution server 2 and plays it. As a result, the viewer R can browse in real time the moving image captured by the photographer S using the mobile phone 1.

  Furthermore, in the moving image distribution system 100 of the present invention, the viewer R can transmit the intention of the viewer R to the photographer S by designating an object to be a mark included in the live video. Specifically, the viewer R operates the mobile phone 3 and designates an object included in the live video being reproduced. Then, the cellular phone 3 transmits information indicating the designated object (hereinafter, object designation information) to the cellular phone 1 via the video distribution server 2. Thereby, it is possible to transmit to the photographer S who is an operator of the mobile phone 1 what the target object the viewer R is paying attention to in the live video. The photographer S may continue to shoot by operating the mobile phone 1 based on the received object designation information so that the video desired by the viewer R can be obtained.

  Hereinafter, each device of the moving image distribution system 100 capable of transmitting the viewer's intention to the photographer's mobile phone will be described in more detail.

[Schematic configuration of mobile phone]
First, a schematic configuration of a mobile phone used by the photographer S and the viewer R will be described. FIG. 3 is a diagram showing a schematic configuration of the mobile phone 1 and the mobile phone 3 according to the embodiment of the present invention. In the following description, the configuration of the mobile phone 1 will be described, but the mobile phone 3 has the same configuration, and the same description will not be repeated.

  As shown in FIG. 3, the mobile phone 1 includes an antenna unit 10, a radio processing unit 11, a voice processing unit 12, a voice input unit 13, a voice output unit 14, a data processing unit 15, an operation unit 16, a storage unit 17, and a display unit. 18, a moving image photographing unit 19, a power supply unit 20, and a control unit 21 (the mobile phone 3 is a control unit 24). Note that the mobile phone 3 functions exclusively as a moving image playback device and does not necessarily include the moving image shooting unit 19 when it does not function as a moving image shooting device.

  The antenna unit 10 is for sending radio waves to the outside and receiving radio waves from the outside.

  The wireless processing unit 11 converts the data received from the audio processing unit 12 or the data processing unit 15 into a format suitable for wireless transmission, and transmits the converted wireless signal to the outside via the antenna unit 10. The wireless processing unit 11 converts a wireless signal received from the outside via the antenna unit 10 into an original format, and transmits the converted data to the voice processing unit 12 or the data processing unit 15. Specifically, the wireless processing unit 11 performs channel codec processing, baseband signal processing, data modulation / demodulation processing, RF (Radio Frequency) processing, and the like.

  The audio processing unit 12 converts the audio signal from the audio input unit 13 into predetermined audio data and transmits it to the radio processing unit 11, and converts the audio data from the radio processing unit 11 into an audio signal and converts it into an audio output unit. 14 is transmitted. Specifically, the audio processing unit 12 includes an A / D converter, a D / A converter, an amplifier, and an audio codec circuit.

  The sound input unit 13 converts sound waves input from the outside into sound signals that are electrical signals and transmits the sound signals to the sound processing unit 12. Specifically, the voice input unit 13 includes a microphone.

  The audio output unit 14 converts the audio signal from the audio processing unit 12 into a sound wave and outputs it to the outside. Specifically, the audio output unit 14 includes a speaker, an earphone, an audio output connector, and the like.

  The data processing unit 15 encodes data from the control unit 21 into data of a predetermined format and transmits the data to the wireless processing unit 11, and also decodes data from the wireless processing unit 11 and transmits the data to the control unit 21. . Examples of data encoding / decoding (codec) systems performed by the data processing unit 15 include MPEG (Moving Picture Experts Group) -4 and ITU-T Recommendation H.264. H.263.

  The operation unit 16 creates operation data and transmits it to the control unit 21 when a user operates an input device such as a cross key or a numeric keypad provided on the surface of the mobile phone 1. Examples of the input device include a touch panel in addition to the button switch.

  The photographer S can operate the predetermined operation button of the operation unit 16 to activate the moving image shooting unit 19 to shoot a moving image or transmit the moving image.

  On the other hand, the viewer R can generate and transmit information for transmitting a shooting intention regarding a live video being played by operating a predetermined operation button of the operation unit 16.

  The storage unit 17 stores various data and programs. As an example of the storage unit 17, a ROM (Read Only Memory) that is a read-only semiconductor memory that stores fixed data such as a program and communication control data required when the control unit 21 operates, and barcode recognition and communication RAM (Random Access Memory) as a so-called working memory that temporarily stores data related to streaming, live video data distributed through streaming, data used for calculation, calculation results, and the like. Further, the storage unit 17 is realized by a rewritable nonvolatile memory such as a flash memory or an EEPROM. And you may memorize | store the video data imaged with the camera with which the mobile telephone 1 is equipped, the moving image image | photographed by the moving image imaging | photography part 19, or the video recording data which recorded the moving image delivered from the moving image delivery server 2. .

  The display unit 18 displays various data such as still images, texts, and moving images stored in the storage unit 17. For example, the display unit 18 includes a display panel including a display element of an LCD and a driver circuit that drives the display element based on data to be displayed.

  The moving image shooting unit 19 generates moving image data by shooting a subject. Specifically, the moving image photographing unit 19 converts an image sensor such as a CCD (Charge Coupled Device) and a CMOS (Complementary Metal-oxide Semiconductor) that converts light from a subject into an electrical signal, and an RGB electrical signal from the image sensor. A configuration provided with a video processing circuit for converting into digital data of each color is mentioned. The moving image photographing unit 19 transmits the generated data to the control unit 21.

  The power supply unit 20 supplies appropriate power to various components in the mobile phone 1. The power supply unit 20 includes a rechargeable secondary battery such as a lithium ion battery, a power supply circuit, and the like.

  The control unit 21 (control unit 24) controls various configurations in the mobile phone 1 (mobile phone 3) in an integrated manner. The functions of each control unit to be described later are described and illustrated as functional blocks. These functional blocks are executed by a CPU (Central Processing Unit) that executes a program stored in a storage device such as a RAM or a flash memory. It is realized by.

[Key phone configuration]
First, the configuration of the mobile phone 1 of the photographer S will be described. FIG. 1 is a diagram illustrating a main configuration of the mobile phone 1 of the photographer S according to the present embodiment. As shown in FIG. 1, the control unit 21 of the mobile phone 1 includes a target specifying information acquisition unit 30, a target specifying unit 31, an operation instruction output unit 32, and a moving image acquisition unit 34 as functional blocks. It has become. Furthermore, the object tracking part 33, the direction instruction | indication part 35, and the variation | change_quantity calculation part 36 may be provided. The storage unit 17 includes an object storage unit 40 and a moving image storage unit 41.

  The moving image acquisition unit 34 acquires a moving image obtained by the moving image shooting unit 19. The moving image acquisition unit 34 immediately outputs the acquired moving image to the display unit 18 so that the photographer S can confirm it as a live image. On the other hand, the moving image is also transmitted to the data processing unit 15, appropriately encoded by the data processing unit 15, and transmitted to the moving image distribution server 2.

  Note that when the captured moving image is recorded so that it can be reproduced later, the moving image acquisition unit 34 may store the recording data of the moving image together with the time code in the moving image storage unit 41.

  The object specifying information acquisition unit 30 acquires the object specifying information transmitted by the mobile phone 3 of the viewer R. Specifically, after the object specifying information is transmitted to the mobile phone 1 via the video distribution server 2 and the data processing unit 15 performs appropriate decoding, the object specifying information is acquired from the object specifying information acquiring unit 30. Get.

  The object designation information is information for designating a specific object as a mark in the live video, and is for transmitting the intention of the viewer R regarding the live video. The object specifying information includes information for specifying the object that the viewer R is paying attention to in the live video.

  In the present embodiment, coordinate information for specifying an object area that is an area including the object in the live video is used as the object specifying information.

  The object specifying unit 31 specifies an object in the live video based on the object specifying information acquired by the object specifying information acquiring unit 30.

  Specifically, in this embodiment, an area specified by the coordinate information in the live video is specified, and an image of the area is cut out from the latest frame and specified as an object. Accordingly, the mobile phone 1 can recognize which object (area) in the live video the viewer R is paying attention to.

  The object specified by the object specifying unit 31 is stored in the object storage unit 40.

  The operation instruction output unit 32 outputs information indicating the object (object area) specified by the object specifying unit 31 together with the live video, and highlights the object. The operation instruction output unit 32 outputs information indicating the object to the display unit 18 together with the live video so that the photographer S can visually confirm what the object is stored in the object storage unit 40. . Specific methods for emphasizing and displaying the object include displaying a frame surrounding the object area, blinking the object area, changing the color of the object area, and changing the color of the object area. Inverting or pointing an object with an arrow.

  Thereby, the photographer S can know which object in the live video the viewer R is paying attention to from the information displayed on the display unit 18, and the photographer S is paying attention. It is possible to shoot a moving image by operating the mobile phone 1 so that an image desired by the viewer R can be obtained with reference to the object. That is, the information indicating the target object is operation instruction information that instructs the photographer S how to operate the mobile phone 1 to perform photographing.

  Specifically, a rule that the orientation of the mobile phone 1 is changed in advance between the viewer R and the photographer S so that when the target is specified, the target is positioned in the center of the screen of the live video. And

  When the operation instruction output unit 32 outputs information indicating the object (operation instruction information) to the display unit 18, the photographer S confirms it and to which object in the live video the viewer R is paying attention. Can know. As a result, the photographer S can change the orientation of the mobile phone 1 so that the region in which the object appears is in the center of the screen.

  On the other hand, the viewer R can obtain a live video in which the object designated by himself / herself is reflected in the center of the screen from the mobile phone 1 via the video distribution server 2 and, as a result, directly operates the mobile phone 1. Even if it is not possible, it is possible to transmit the intention to the photographer S and obtain a desired live video.

  The object tracking unit 33 tracks the object specified by the object specifying unit 31 in the live video, and acquires the coordinate position of the object in the live video. Specifically, the target tracking unit 33 performs matching between the target image stored in the target storage unit 40 and the live video, and determines the coordinate position of the matching (high similarity) region. , To obtain the current position of the object.

  Thereby, the mobile phone 1 can always recognize in which position of the live video the specified object is reflected. Specifically, based on the coordinate position of the object output by the object tracking unit 33, the operation instruction output unit 32 displays a frame surrounding the object, an arrow pointing to the object, and the like as operation instruction information. Is mentioned. The operation instruction output unit 32 can track the object moving in the screen according to the orientation of the mobile phone 1 changed by the photographer S, and display the frame, the arrow, and the like on the display unit 18. . This prevents the photographer S from losing sight of the position of the designated object during photographing.

  Further, the object tracking unit 33 may transmit the acquired coordinate position of the object to the mobile phone 3 together with the moving image acquired by the moving image acquiring unit 34. Thereby, it becomes possible to always recognize the coordinate position of the object even on the mobile phone 3 of the viewer R.

[Coordinate information]
Next, coordinate information acquired by the object specifying information acquisition unit 30 will be described. FIG. 4 is a diagram illustrating an example of coordinate information as object designation information transmitted from the mobile phone 3. As shown in FIG. 4, in the present embodiment, the coordinate information includes information for specifying the display size of the live video displayed on the display unit 18 and information for specifying the object area in the live video.

  Specifically, when the object area is designated by a rectangle, coordinate information C1 including the display size and the XY coordinates of the start point of the rectangle (the upper left corner of the rectangle) and the end point (the lower right end of the rectangle). Is acquired by the object specifying information acquisition unit 30.

  Alternatively, when the target area is designated by a circle, the target designation information acquisition unit 30 acquires the coordinate information C2 including the display size, the XY coordinates of the center of the circle, and the radius r of the circle.

  When the coordinate information C1 (C2) illustrated in FIG. 4 is acquired, the object specifying unit 31 acquires the latest frame F (still image) from the moving images captured by the moving image acquiring unit 34, as illustrated in FIG. Thus, the area indicated by the coordinate information C1 (C2) in the frame F is specified as the object area A1 (A2). Then, the image of the object area A1 (A2) is stored in the object storage unit 40.

  Thereby, the target object which the browsing person R pays attention is specified.

  In the present embodiment, since the display size of the live video is included in the coordinate information, even if the mobile video 1 and the mobile phone 3 display live video at different resolutions, or each display unit Even if 18 has different resolutions, the object area is always obtained at the correct position from the ratio between the display size and the designated XY coordinates.

  Next, the configuration of the main part of the mobile phone 3 of the viewer R in this embodiment will be described based on FIG. In FIG. 13, blocks having the same reference numerals as those in FIGS. 1 and 3 are the same as those described with reference to FIGS. 1 and 3, and therefore description thereof will not be repeated here.

  In the mobile phone 3 according to the present embodiment, the control unit 24 includes a moving image reproduction unit 54 as a functional block and an object designation information generation unit 56.

  The moving image reproduction unit 54 acquires a moving image taken by the mobile phone 1 distributed from the moving image distribution server 2 via the data processing unit 15 and reproduces it as a live image in real time. When it is desired to record the acquired moving image, the moving image playback unit 54 may store the acquired moving image in the moving image storage unit 41 as recorded data.

  The object specifying information generating unit 56 generates object specifying information in response to an instruction from the viewer R input from the operation unit 16.

  More specifically, from one frame of the live video displayed on the display unit 18, the viewer R operates the operation unit 16 to designate a certain region so as to include an object of interest. The object designation information generation unit 56 generates coordinate information input from the operation unit 16 for specifying the area designated by the viewer R (for example, coordinate information C1 and C2 in FIG. 4). The target designation information generation unit 56 stores the generated target designation information in the target designation information storage unit 44 and transmits it to the video distribution server 2 and the mobile phone 1 via the data processing unit 15.

  Thereby, the viewer R can transmit the intention of the viewer R regarding the live video to the photographer S.

[Main components of video distribution server]
FIG. 6 is a block diagram showing a main configuration of the moving image distribution server 2 according to the present embodiment. As illustrated in FIG. 6, the moving image distribution server 2 includes a receiving unit 22, a transmitting unit 23, a control unit 28, an object designation information storage unit 42, and a moving image storage unit 43.

  The receiving unit 22 receives the moving image taken by the mobile phone 1 of the photographer S via the communication network W, and receives the object designation information transmitted from the mobile phone 3 of the viewer R.

  The transmission unit 23 transmits the moving image to a designated destination (for example, the mobile phone 3 of the viewer R) via the communication network W. In addition, the object designation information determined by the object designation information determination unit 51 is transmitted to the mobile phone 1.

  The control unit 28 comprehensively controls various configurations in the video distribution server 2, and controls the object designation information collection unit 50, the object designation information determination unit 51, and the video distribution unit 52 as functional blocks. I have. Further, a live authorization unit 53 may be provided.

  The object designation information collection unit 50 acquires each object designation information transmitted from the mobile phones 3 owned by each viewer R of the live video distributed from the mobile phone 1. When there are a plurality of viewers R of the live video (unspecified large number or several members of a specific viewer group), target designation information reflecting various intentions of different viewers R is stored in the video distribution server 2. Sent. The object designation information collection unit 50 aggregates these object designation information and stores them in the object designation information storage unit 42 for each mobile phone 3 that is the transmission source.

  The object designation information determination unit 51 decides one object designation information to be sent to the mobile phone 1 from the object designation information collected in the object designation information storage unit 42 based on a predetermined rule. is there.

  For example, when the rule that only a specific viewer R determines the object specifying information is determined, the object specifying information determining unit 51 is transmitted from the mobile phone 3 of the specific viewer R. The object designation information is determined as the object designation information to be transmitted to the mobile phone 1. The object designation information determination unit 51 transmits the object designation information finally determined to one to the mobile phone 1 via the transmission unit 23.

  Alternatively, in the case of a video distribution system that distributes live video to several people (for example, about 2 to 3 people) in a specific viewer group, the object specifying information specified for each viewer R is received, The plurality of object designation information may be determined as the object designation information to be transmitted to the mobile phone 1.

  If the determined plurality of target specifying information is transmitted to the mobile phone 1 in association with information for identifying each viewer R, the mobile phone 1 can specify the target area for each viewer R. Become. Thereby, the mobile phone 1 can display a plurality of object areas on the display unit 18 together with the information of the viewer R so that the viewer R can identify which object area is designated. .

  Alternatively, one piece of object designation information may be determined statistically based on each piece of object designation information from an unspecified number of viewers R. For example, the object designation information that designates the area having the largest overlap in each object designation information as the object area may be newly generated and transmitted to the mobile phone 1.

  Thereby, even when various object designation information is transmitted from each mobile phone 3 when there are an unspecified number of viewers R, one object designation information is appropriately determined and transmitted to the mobile phone 1. be able to. Therefore, the photographer S of the mobile phone 1 can prevent confusion as to how to perform shooting for any object designation information.

  The moving image distribution unit 52 acquires a moving image transmitted from the mobile phone 1 of the photographer S in real time, and distributes the moving image in a streaming manner to the specified destination mobile phone 3. Note that the acquired moving image may be stored in the moving image storage unit 43 in order to respond to a subsequent distribution request.

[Live certification]
As shown in FIG. 6, the video distribution server 2 preferably further includes a live authorization unit 53.

  The live authorization unit 53 determines whether or not the moving image provided from the mobile phone 1 is a moving image distributed in real time, that is, a live image, and notifies the mobile phone 3 of the viewer R. In the live authorization unit 53, a moving image that matches the instruction content of the operation instruction information (object specifying information) transmitted from the object specifying information determining unit 51 to the mobile phone 1 (= desired by the viewer R). By detecting that it is obtained from the mobile phone 1, the moving image is recognized as a live image.

  Specifically, it is as follows. When the object is determined by the video distribution server 2, a rule that the orientation of the mobile phone 1 is changed between the viewer R, the video distribution server 2, and the photographer S in advance so that the target is in the center of the live video. Suppose that it is stipulated.

  It is assumed that the moving image distribution server 2 has transmitted the object specifying information for specifying the object area A2 shown in FIG. The object tracking unit 33 of the mobile phone 1 captures the object area A2 as an object, and tracks the object moving in the live video by the operation of the moving image capturing unit 19 of the photographer S. The mobile phone 1 transmits the coordinate position of the object obtained from the object tracking unit 33 to the moving image distribution server 2 together with the live video.

  The live authorization unit 53 of the video distribution server 2 includes the coordinate position when the object region determined by the object specifying information determination unit 51 comes to the center (and near the center) and the object obtained from the object tracking unit 33. Compare the coordinate position of the object. If these coordinate positions match (or are similar within a predetermined range), the photographer S points the mobile phone 1 so that the object designated by the viewer R is at the center of the screen as described above. A live image intended by the viewer R is obtained by changing the shooting.

  Therefore, the live authorization unit 53 detects that the object determined by the object designation information determination unit 51 is captured in the center of the screen, and determines that the moving image being distributed is a live image. The live authorization unit 53 transmits a notification that the moving image is a live image to the mobile phone 3 together with the moving image to be distributed.

  Thereby, the mobile phone 3 can recognize that the moving image received and reproduced from the moving image distribution server 2 by the streaming method is a live image, and can output the notification to the viewer R.

  Conventionally, the viewer can control the direction etc. by direct remote control with respect to the fixed camera located far from the viewer. Was confirmed to be a live video.

  However, when the photographer S takes a picture of the mobile phone 1 having the moving image photographing unit 19 (camera) in his / her hand as in the present embodiment, the viewer cannot directly control like the fixed camera. For this reason, it has not been possible to confirm that the moving image currently being reproduced is a live image by the method described above.

  According to the above configuration, the object designation information reflecting the intention of the viewer R is transmitted to the mobile phone 1, and it is detected that the moving image corresponding to the intention of the viewer R is acquired from the mobile phone 1, and the live certification is performed. Can be performed.

  Note that the configuration for performing live certification is not limited to that described above. For example, the moving object distribution unit 2 may include the object tracking unit 33 of the mobile phone 1, or the mobile phone 3 may include the object tracking unit 33 and the live certification unit 53 of the moving image distribution server 2. . Alternatively, the mobile phone 1 may include the live authorization unit 53.

[Video distribution flow 1]
FIG. 7 is a sequence diagram illustrating a processing flow of each device in the moving image distribution system 100. FIGS. 8A to 8D are diagrams showing examples of screen display when a live video is displayed on the display unit 18 of the mobile phone 1 or the mobile phone 3. Each device refers to a mobile phone 1 (FIG. 1) as a moving image photographing device, a moving image distribution server 2 (FIG. 6), and a mobile phone 3 (FIG. 13) as a moving image reproducing device.

  First, when the photographer S activates the moving image shooting unit 19 and starts moving image shooting, the mobile phone 1 transmits the moving image acquired by the moving image acquisition unit 34 to the moving image distribution server 2 in real time (S1). The moving image distribution unit 52 of the moving image distribution server 2 distributes the moving image acquired from the mobile phone 1 to a predetermined destination (here, the mobile phone 3) (S2).

  The video playback unit 54 of the mobile phone 3 plays back the video distributed from the video distribution server 2 in real time. For example, as shown in FIG. 8A, the video playback unit 54 displays the live video to the viewer R by displaying it on the display unit 18. provide.

  The viewer R confirms the live video on the display unit 18 and designates an object that he / she pays attention to (for example, desires to copy in the center of the screen). Specifically, for example, the cross key and the enter key of the operation unit 16 are pressed to select an area including the target object or to specify the start point and end point of the area.

  In this way, when an object area is designated by the viewer R via the operation unit 16, the object designation information generation unit 56 generates coordinate information (FIG. 4) for specifying the designated object area. (S3). The mobile phone 3 transmits the generated coordinate information to the moving image distribution server 2.

  Subsequently, the object designation information collection unit 50 of the moving image distribution server 2 collects coordinate information as the object designation information from the mobile phone 3 (and another viewer's mobile phone) (S4). Then, the object designation information determination unit 51 determines one piece of coordinate information to be transmitted to the mobile phone 1 based on a predetermined rule (S5). Here, it is assumed that coordinate information is adopted from the mobile phone 3 of the viewer R, and the video distribution server 2 receives the object designation information (coordinate information) that reflects the intention regarding the live video of the viewer R received from the mobile phone 3. ) Is transmitted to the mobile phone 1 (S6).

  Subsequently, the object designation information acquisition unit 30 of the mobile phone 1 acquires the coordinate information transmitted from the moving image distribution server 2 (S7). The object specifying unit 31 acquires the area specified based on the coordinate information as an object area including the object to which the viewer R pays attention (S8).

  Then, the operation instruction output unit 32 outputs information indicating the object area specified in S8 to the display unit 18 (S9). Specifically, as shown in FIG. 8B, an object designation frame 90 surrounding the object area is displayed on the display unit 18 so as to be superimposed on the live video.

  Thereby, the photographer S can know which object the viewer R is paying attention to, and continues shooting by moving the direction of the mobile phone 1 so that the object is captured in the center of the screen. (S10).

  Here, in this embodiment, the object tracking unit 33 tracks the identified object area as the shooting continues (S11). Specifically, the target tracking unit 33 acquires the coordinate position of the target area in the live video.

  The mobile phone 1 returns the coordinate position of the object area acquired in S11 to the video distribution server 2 together with the video acquired in S10.

  And the live certification | authentication part 53 of the moving image delivery server 2 is based on the coordinate position of the said object area | region acquired from the mobile telephone 1, and whether the target object which the browsing person R designated was captured by the center (near). Determine (S12).

  While the target area is not captured near the center (NO in S12), the moving image distribution unit 52 distributes the moving image received from the mobile phone 1 to the mobile phone 3, similarly to S2.

  On the other hand, for example, as shown in FIG. 8C, when the live authorization unit 53 detects that the object area has been captured near the center (YES in S12), the live authorization unit 53 indicates that the moving image is the object. The live image reflecting the intention of the viewer R is recognized according to the designation information (S13). Then, the moving image distribution unit 52 distributes the moving image from the mobile phone 1 to the mobile phone 3, and the live authorization unit 53 notifies that the moving image being distributed is a live video.

  Subsequently, when the mobile phone 3 is notified that it is a live video, the moving image reproducing unit 54 displays an icon 91 indicating that it is a live video together with the live video, as shown in FIG. May be output. Thereby, the viewer R of the mobile phone 3 can obtain a moving image shot as intended, and can confirm that the moving image is a live image.

[Modification]
(Direction control)
The mobile phone 1 may include a direction instruction unit 35 (FIG. 1) in addition to the configuration described above.

  Based on the operation instruction information (object designation information), the direction instruction unit 35 should change the direction of the moving image photographing unit 19 in order to capture the object at a predetermined position (for example, the center) of the screen. Is to determine. Information on the direction determined by the direction instruction unit 35 is output to the display unit 18 by the operation instruction output unit 32 and transmitted to the photographer S. Thereby, the photographer S can know where the moving image photographing unit 19 should be photographed.

  A specific example will be described as follows. For example, a certain object area is specified by the object specifying information as shown in FIG. 4, and the direction of the moving image capturing unit 19 is changed so that the object area is captured in the center of the live video according to the rules described above. Suppose you need to.

  The direction instructing unit 35 compares the target coordinate position when the target region is captured in the center with the current coordinate position of the target region obtained by tracking by the target tracking unit 33, and determines the current coordinate position. The direction of the moving image photographing unit 19 for bringing the coordinate position close to the target coordinate position is obtained.

  The operation instruction output unit 32 outputs information (for example, an arrow or the like) indicating the direction obtained by the direction instruction unit 35 to the display unit 18 together with the live video.

  FIGS. 9A to 9D are diagrams showing screen display examples of operation instruction information regarding the shooting direction displayed on the display unit 18 of the mobile phone 1.

  As shown in FIG. 9 (a), when the object area is specified in the broken line frame on the upper right of the screen, the direction indicating unit 35 is directed in the right and upper directions in order to capture the object area in the center. It is determined that it is necessary to change the orientation of the moving image photographing unit 19. The direction instruction unit 35 transmits information on either the right direction or the upper direction (here, the right direction) to the operation instruction output unit 32.

  In response to an instruction from the direction instruction unit 35, the operation instruction output unit 32 displays an arrow 92 indicating the right direction on the display unit 18 as information indicating the right direction. In accordance with this, the photographer S can move the moving image shooting unit 19 to the right and obtain a moving image in which the object is captured at the center on the left and right as shown in FIG. 9B.

  Thereafter, the direction instruction unit 35 transmits the upward information to the operation instruction output unit 32. The operation instruction output unit 32 displays an arrow 92 indicating an upward direction on the display unit 18 in response to an instruction from the direction instruction unit 35. Accordingly, the photographer S can move the direction of the moving image shooting unit 19 upward to obtain a moving image in which the object is captured in the center.

  Further, when the direction of the moving image photographing unit 19 is appropriately adjusted according to the direction instructed by the direction instructing unit 35 (the object is captured in the center), the direction instructing unit 35 is connected via the operation instruction output unit 32. Thus, information indicating that it is not necessary to move in any direction may be output to the display unit 18. For example, as shown in FIG. 9C, an “OK!” Icon 93 is displayed on the display unit 18. Thereby, it is understood that the photographer S only needs to continue shooting while maintaining the current direction.

  In addition, since the timing at which the operation instruction output unit 32 outputs the “OK!” Icon 93 is a timing at which it can be recognized that it is a live video, the mobile phone 1 is provided with the live certification unit 53 and the live certification is performed at the above timing. May be notified to the moving image distribution server 2.

  Further, in the above-described example, the example in which the arrow 92 is displayed in the four directions of up, down, left, and right has been described. However, the present invention is not limited to this, and the shooting direction may be indicated in eight directions.

(Providing location information)
In the moving image distribution system 100, it is also possible to provide the mobile phone 3 of the viewer R of the live video with the position information of the shooting site where the photographer S is shooting.

  For example, when the mobile phone 1 has a GPS (Global Positioning System) function, the mobile phone 1 captures a moving image, acquires position information, associates the moving image with the position information, and stores a moving image distribution server. 2 may be transmitted.

  Further, the moving image distribution server 2 may acquire information on a base station through which a moving image is transmitted from the mobile phone 1 to the moving image distribution server 2, and distribute the moving image and the base station information to the mobile phone 3. .

[Embodiment 2]
In the above-described embodiment, an example has been described in which coordinate information for specifying a target area is used as target specifying information for transmitting the intention of the viewer R from the mobile phone 3 to the mobile phone 1. In the moving image distribution system of the present invention, the object designation information can further include a time code.

  The time code is information indicating the time when a moving image is shot, and is given for each frame forming the moving image. Therefore, by using the time code, it is possible to specify which frame of the above moving image, so that the object can be specified more accurately by comparing the latest frame and the past frame, or the object can be specified in the past. It is possible to specify retroactively.

  FIG. 10 is a diagram illustrating an example of the object specifying information according to the present embodiment. As shown in FIG. 10, the object designation information in the present embodiment includes a time code for specifying a moving image frame in addition to the coordinate information shown in FIG. 4.

  In the present embodiment, the mobile phone 1 of the photographer S further includes a change amount calculation unit 36 as shown in FIG.

  The change amount calculation unit 36, based on the time code included in the acquired object designation information, the frame specified by the time code in the moving image obtained by the mobile phone 1 and the latest frame in the moving image. And determining the magnitude of the amount of change.

  The amount of change is a quantification of how much the frame at the later time point has changed from the frame at the previous time point. Note that the comparison of the preceding and succeeding frames can be realized by determining matching / mismatching between frames by well-known image processing.

  When the change amount calculation unit 36 determines that there is a large change between the frame of the time code specified by the object specifying information and the latest frame, the area of the coordinate information included in the object specifying information However, there is a possibility that the object is not accurately captured as intended by the viewer R. In this case, the object specifying unit 31 cannot correctly specify the object, and therefore the operation instruction output unit 32 cannot output operation instruction information for capturing a moving image as intended by the viewer R to the display unit 18. The problem arises.

  Therefore, when the change amount calculation unit 36 determines that the amount of change between the frames is large, the object designation information acquisition unit 30 reacquires the latest object designation information. When the amount of change is small, the object specifying unit 31 specifies the object region based on the coordinate information included in the object specifying information, as in the first embodiment.

  Thereby, even when the object of the live video moves greatly in a short time, the object region can be specified more accurately.

[Video distribution flow 2]
FIG. 11 is a sequence diagram illustrating a processing flow of each device in the moving image distribution system 100. FIGS. 12A and 12B are diagrams illustrating a screen display example when a live video is displayed on the display unit 18 of the mobile phone 1 or the mobile phone 3.

  11 are the same as the processes of S1 to S7 shown in FIG. 7 of the first embodiment except for the following points.

  When the latest live video t (0) (FIG. 12 (a)) taken by the mobile phone 1 at S21 at a certain time t (0) is delivered to the mobile phone 3, the object designation of the mobile phone 3 is designated. The information generation unit 56 uses the time code t (0) of the frame specified by the viewer R and the coordinate information 0 of the target region A3 of the target 94 of interest of the viewer R in the frame as the target specifying information. And transmitted to the moving image distribution server 2 (S23).

  The object designation information acquisition unit 30 of the mobile phone 3 acquires the time code t (0) and the coordinate information 0 from the moving image distribution server 2 (S27).

  Subsequently, the change amount calculation unit 36 includes the latest live video frame t (1) (FIGS. 12B and 12C) at the current time t (1) and the time acquired by the object designation information acquisition unit 30. The amount of change is obtained by comparing with the frame t (0) (FIG. 12A) specified by the code t (0) (S28).

  When the amount of change between the frames is small (NO in S28), the object specifying unit 31 specifies the object 94 based on the object area A3 in FIG. 12A (S29), and outputs an operation instruction. The object specifying frame surrounding the object 94 by the unit 32 is output to the display unit 18 (S30).

  On the other hand, when the change amount calculation unit 36 determines that the change amount between the frames is large (YES in S28), the target designation information acquisition unit 30 requests the video distribution server 2 to retransmit the target designation information. (S31). At this time, the latest live video frame t (1) is transmitted to the moving image distribution server 2.

  By repeating S22 to S26 (or, if the video distribution server 2 has a function of tracking the target 94, the video distribution server 2), the target area A4 of the target 94 in the latest frame (FIG. 12 (c)) is specified, the object designation information determination unit 51 of the video distribution server 2 obtains the latest object designation information, that is, the time code t (1) and the coordinate information 1 of the object area A4. Resend to the mobile phone 1 (S32).

  Thereby, the mobile phone 1 can correctly specify the object 94 in the object area A4.

  When there is a large change between the frame at the time when the mobile phone 3 generates the object specifying information and the latest frame taken by the mobile phone 1 (for example, when the target 94 moves greatly in a short time) ) When the mobile phone 1 tries to specify an object without determining the amount of change, a state as shown in FIG.

  That is, the object specifying unit 31 of the mobile phone 1 specifies the object area A3 according to the coordinate information 0 of the object specifying information, but since the latest frame has changed greatly at the time of specifying, The object 94 that the person R actually focuses on cannot be captured in the object area A3. For this reason, an incorrect target object is specified, or the target object cannot be specified, and there is a problem that correct operation instruction information cannot be output to the photographer S.

  According to the above method and configuration, it is possible to determine whether or not the object designation information should be reacquired according to the amount of change, so even if the object of the live video moves greatly in a short time, more It becomes possible to specify the object region accurately.

[Modification]
As a configuration of the mobile phone 1 corresponding to the magnitude of the change of the live video, it is also possible to configure as follows instead of the configuration of the change amount calculation unit 36.

  The moving image acquisition unit 34 (FIG. 1) of the mobile phone 1 stores the moving image captured by the moving image capturing unit 19 in the moving image storage unit 41 in association with the time code.

  The object specifying unit 31 acquires a frame corresponding to the time code included in the object specifying information from the moving image storage unit 41 based on the object specifying information acquired by the object specifying information acquiring unit 30. . Then, the object specifying unit 31 specifies an object area from the coordinate information included in the object specifying information in the frame.

  Thus, the object region is identified from the frame corresponding to the designated time code, no matter how long the lime lag occurs before the object designation information is transmitted to the mobile phone 1 and the amount of change in the frame increases. Therefore, it is possible to always identify the correct object.

[Embodiment 3]
In the above-described embodiment, an example in which coordinate information (and time code) is used as the object specifying information has been described. In the moving image distribution system of the present invention, a still image including an object is attached to an e-mail instead of coordinate information (and time code), and is transmitted from the mobile phone 3 to the mobile phone 1 as object specifying information. Is possible.

  In the video distribution system according to the present embodiment, the viewer R uses the electronic mail function of the mobile phone 3 to attach a still image including an object of interest to the electronic mail to the mobile phone 1 and send it to the mobile phone 1. To do.

[Key phone configuration]
First, the configuration of the mobile phone 3 of the viewer R will be described. FIG. 13 is a diagram illustrating a main configuration of the mobile phone 3 of the viewer R according to the present embodiment. As illustrated in FIG. 13, the control unit 24 of the mobile phone 3 includes a moving image reproduction unit 54, an image processing unit 55, and an object designation information generation unit 56 as functional blocks. Further, a live authorization unit 53 may be provided. In the mobile phone 3 in FIG. 13, the blocks already described in the first embodiment and the blocks having the same reference numerals as the blocks in FIG. 1 are as described above, and thus description thereof will not be repeated here.

  The image processing unit 55 generates a target image that is a still image indicating the target specified by the viewer R in response to the instruction of the viewer R input from the operation unit 16.

  Specifically, the image processing unit 55 cuts out a frame (still image) from the moving image reproduced by the moving image reproducing unit 54, superimposes an object designation frame surrounding an area designated by the viewer R on the frame, An object image is generated.

  The object specifying information generating unit 56 generates object specifying information in response to an instruction from the viewer R input from the operation unit 16.

  In the present embodiment, the object designation information generation unit 56 generates an e-mail attached with the object image generated by the image processing unit 55 as the object designation information.

  The object designation information (e-mail) generated by the object designation information generation unit 56 is transmitted to the mobile phone 3 via the data processing unit 15.

  Next, the configuration of the cellular phone 1 of the photographer S will be described with reference to FIG.

  The object designation information acquisition unit 30 acquires the object image attached to the e-mail received via the data processing unit 15.

The object specifying unit 31 recognizes an object specifying frame from the object image acquired by the object specifying information acquiring unit 30, and specifies an area surrounded by the object specifying frame as the object area.
The image recognition function for detecting the object designation frame can be realized by a known image processing technique.

[Video distribution flow 3]
FIG. 14 is a sequence diagram showing a processing flow of each device in the moving image distribution system according to the present embodiment. Each device refers to a mobile phone 1 (FIG. 1) as a moving image photographing device and a mobile phone 3 (FIG. 13) as a moving image reproducing device capable of transmitting and receiving e-mails via a communication network W.

  When the photographer S activates the moving image shooting unit 19 to start moving image shooting, the moving image acquisition unit 34 transmits the acquired moving image to the mobile phone 3 (S41). The moving picture playback unit 54 of the mobile phone 3 plays back the moving picture received from the mobile phone 1 in real time (S42).

  Here, when the viewer R performs an operation of designating an object using the operation unit 16, the image processing unit 55 extracts one frame (still image) from the moving image in accordance with the operation (S43). ). Then, in accordance with the operation of the viewer R, a target object image is generated by adding an object specifying frame that surrounds an area specified by the viewer R in the frame (S44).

  The object designation information generating unit 56 generates an e-mail in which the object image generated in S44 is an attached file, and transmits it to the mobile phone 1 (S45).

  When the mobile phone 1 receives the electronic mail from the mobile phone 3, the target specifying information acquisition unit 30 acquires the target image attached to the electronic mail (S46).

  The object specifying unit 31 performs image recognition processing on the object image acquired by the object specifying information acquiring unit 30, detects the added object specifying frame, and specifies the object region (S47). ). Subsequently, the operation instruction output unit 32 superimposes an object designation frame 90 (FIG. 8B) surrounding the object area specified by the object specifying unit 31 on the live image being shot and outputs the superimposed image to the display unit 18. (S48).

  Thus, the photographer S can know the target object that the viewer R is paying attention to in the live video, and continue the photographing operation so that a video reflecting the intention of the viewer R can be obtained. Can do.

  The flow of the live certification process in S49 to S52 is the same as S10 to S13 in FIG. 7 except that the mobile phone 3 executes live certification.

[Modification]
In the above description, the example in which the image processing unit 55 of the mobile phone 3 performs the image processing for adding the object designation frame that surrounds the object to the designated frame has been described, but the present invention is not limited to this. The object specifying unit 31 of the mobile phone 1 may be any image as long as it has information that can specify the object.

  For example, the image processing unit 55 may trim the target area designated by the viewer R in the frame and use the trimmed image as the target image.

  In this case, the object specifying unit 31 executes a matching process and specifies a portion that matches the trimmed image from the moving images currently being photographed as the object region.

  Further, the operation instruction output unit 32 may superimpose the object image attached to the e-mail as the object specifying information on the live image so that the photographer S can check the object image on the display unit 18.

  FIGS. 15A to 15D are diagrams showing examples of display screens of object images displayed on the display unit 18 of the mobile phone 1.

  In the example shown in FIG. 15A, a trimmed image 95 indicating the object region is displayed superimposed on the live video. Furthermore, when the mobile phone 1 detects that a portion matching the trimming image 95 is captured at the center, the mobile phone 1 may display the icon 93 as in FIG. When the object image acquired by the object specifying information acquisition unit 30 is an image obtained by adding an object specifying frame to the frame, the object specifying unit 31 detects the object specifying frame and trims the inside of the frame do it.

  In the example shown in FIG. 15B, the object designation frame indicating the object area is displayed superimposed on the live video together with the frame 96. Furthermore, when the mobile phone 1 detects that the portion matching the object designation frame in the frame 96 is supplemented at the center, the mobile phone 1 may display the icon 93 as in FIG. When the object image acquired by the object specifying information acquisition unit 30 is a trimmed image, the object specifying unit 31 adds an object specifying frame to a portion that matches the trimmed image with respect to the latest frame. Then, it may be transmitted to the operation instruction output unit 32.

[Embodiment 4]
The moving image reproducing device and moving image photographing device of the present invention can also be applied to a video conference system. Hereinafter, a TV conference system using a mobile phone with a TV conference function as a video playback device and a video shooting device will be described as an example. The video conference system according to the present embodiment includes a mobile phone 4A with a video conference function of a conference participant A, a mobile phone 4B with a video conference function of a conference participant B (hereinafter simply referred to as mobile phones 4A and 4B), and The mobile phone 4 </ b> A and the mobile phone 4 </ b> B are configured to have a communication network W that connects the mobile phone 4 </ b> B so that communication is possible.

[Key phone configuration]
FIG. 16 is a diagram showing a main configuration of the mobile phone 4A / mobile phone 4B according to the embodiment of the present invention. In the following description, the configuration of mobile phone 4A will be described, but mobile phone 4B has the same configuration, and the same description will not be repeated.

  As shown in FIG. 16, the control unit 25 of the mobile phone 4 </ b> A includes an object designation information acquisition unit 30, an object identification unit 31, an operation instruction output unit 32, an object tracking unit 33, And a moving image obtaining unit 34, and a live authorization unit 53, a moving image reproducing unit 54, and an object designation information generating unit 56 as functional blocks of the moving image reproducing device.

  In FIG. 16, the blocks to which the same reference numerals as those in FIGS. 1, 3, 6, and 13 are given are the same as described above, and therefore description thereof will not be repeated here. Although not shown in FIG. 16, the voice input via the voice input unit 13 is transmitted to the mobile phone of the communication partner via the data processing unit 15 together with the video shot by the video shooting unit 19. The

[Video distribution flow 4]
FIG. 17 is a sequence diagram showing a processing flow of each device in the TV conference system according to the present embodiment. Each device refers to the mobile phone 4A and the mobile phone 4B (FIG. 16). However, the present invention is not limited to this, and the moving image capturing device and the moving image reproducing device of the present invention can be applied to a TV conference system in which two or more conference participants participate.

  When the mobile phone 4A of the conference participant A and the mobile phone 4B of the conference participant B start a session for executing the TV conference, each conference participant transmits a video of himself / herself to the other conference participant. Can be sent. First, the conference participant A takes a picture using the video shooting unit 19 (S61), and transmits a video (live video A) showing the conference participant A to the mobile phone 4B.

  The mobile phone 4B receives the live video A from the mobile phone 4A (S62), and the moving image playback unit 54 plays back the live video A and displays it on the display unit 18. The conference participant B confirms the live video A displayed on the display unit 18, and uses the operation unit 16 to specify an object of interest (for example, to move to the center of the screen).

  The object designation information generation unit 56 acquires the object area including the object designated by the conference participant B, and coordinates information (object designation information A) for specifying the object area in the live video A Is generated (S63).

  Subsequently, the object specifying information acquisition unit 30 of the mobile phone 4A acquires the object specifying information A from the mobile phone 4B (S64). The object specifying unit 31 specifies an object area in the live video A based on the object specifying information A (S65).

  Finally, the operation instruction output unit 32 outputs information indicating the object region (object specification frame or the like) to the display unit 18 as operation instruction information (S66).

  Thereby, the conference participant A can perform a shooting operation reflecting the intention of the conference participant B in accordance with the operation instruction information, and transmits the live video A desired by the conference participant B to the mobile phone 4B. Is possible.

  Contrary to the above, when the intention of the conference participant A is transmitted to the cellular phone 4B of the conference participant B (S71 to S71), the same processing as S61 to S66 is repeated.

  In each of the above-described embodiments, as an example of the operation to be performed on the identified target object, an operation for changing the shooting direction in order to capture the target object in the center is described, but the present invention is not limited to this. For example, a rule for performing a zoom operation for enlarging and photographing a specified object may be defined.

  In this case, when the object specifying unit 31 specifies the object, the operation instruction output unit 32 provides operation instruction information indicating that the object is focused and the zoom rate is increased so that the object is enlarged and displayed. Output. Then, the live authorization unit 53 performs live authorization by detecting whether or not the object area has been enlarged as intended by the viewer according to the ratio of the object area to the screen.

  Alternatively, a rule may be defined that the shooting direction is changed so that the specified object is out of frame. In this case, the viewer can transmit the intention to the photographer by designating a portion that the user does not want to copy as an object.

[Embodiment 5]
In each of the above-described embodiments, in order to transmit the intention (operation instruction information) of the viewer R about the live video to the mobile phone 1 of the photographer S, the target designation that designates a mark (target) in the live video. An example in which information is transmitted to the mobile phone 1 has been described. In the moving image distribution system of the present invention, it is also possible to transmit direction control information for instructing the photographer S in the photographing direction as operation instruction information for informing the photographer S instead of the object designation information. is there.

  In the video distribution system according to the present embodiment, the difference from FIG. 2 is that the mobile phone 7 of the viewer R is directed to the mobile phone 5 of the photographer S via the video distribution server 6 instead of the target designation information. It is a point which transmits control information.

[Main components of the photographer's mobile phone]
FIG. 18 is a diagram illustrating a main configuration of the mobile phone 5 according to the present embodiment. As shown in FIG. 18, the control unit 26 of the mobile phone 5 of the photographer S includes a direction control information acquisition unit 60, a direction specification unit 61, an operation instruction output unit 32, and a moving image acquisition unit 34 as functional blocks. It has become.

  The direction control information acquisition unit 60 acquires the direction control information transmitted by the mobile phone 7 of the viewer R. Specifically, after the direction control information is transmitted to the mobile phone 5 via the video distribution server 6 and the data processing unit 15 performs appropriate decoding, the direction control information acquisition unit 60 acquires the direction control information. .

  The direction control information is information for designating a shooting direction indicating which direction the moving image shooting unit 19 for shooting a live video should be changed, and for transmitting the intention of the viewer R regarding the live video. Is. The direction control information includes at least information indicating in which direction the moving image photographing unit 19 should be moved up, down, left and right. The direction control information may further include information for specifying eight directions including four oblique directions, and a direction indicating how much to move in the direction and the direction, such as a vector. May be included.

  The direction specifying unit 61 specifies a shooting direction to be changed by the photographer S based on the direction control information acquired by the direction control information acquiring unit 60.

  Specifically, for example, if the direction control information acquisition unit 60 acquires the direction control information illustrated in FIG. 20A, the direction specifying unit 61 includes the direction (here, the right direction) included in the direction control information. ) To identify the shooting direction as right.

  The operation instruction output unit 32 superimposes on the live video currently being shot, and displays information on the display unit 18 indicating that the shooting direction is right.

  According to the above configuration, when the viewer R wants to capture the right side of the current live video, the direction control information indicating that the moving image capturing unit 19 is moved in the right direction is transmitted from the mobile phone 7 to the mobile phone. 5 is transmitted.

  In the cellular phone 5 of the photographer S, the direction control information acquisition unit 60 acquires the direction control information including the right direction information, and the direction specifying unit 61 specifies the shooting direction as right. Then, the operation instruction output unit 32 displays operation instruction information indicating that the shooting direction is right on the display unit 18.

  The photographer S can move the moving image photographing unit 19 to the right in accordance with the operation instruction information and transmit a live video showing the right side to the mobile phone 7.

  Thereby, the intention about the live video of the viewer R is transmitted to the photographer S, and the viewer R can obtain a desired live video.

[Main components of the viewer's mobile phone]
FIG. 19 is a diagram illustrating a main configuration of the mobile phone 7 according to the present embodiment. As shown in FIG. 19, the control unit 27 of the mobile phone 7 of the viewer R has a configuration including a moving image playback unit 54, a direction instruction receiving unit 57, and a direction control information generating unit 58 as functional blocks. .

  The direction instruction receiving unit 57 receives a change instruction in a direction desired by the viewer R, which is input via the operation unit 16. For example, when the viewer R presses the right direction key among the cross keys of the operation unit 16, the direction instruction receiving unit 57 receives the right direction information and generates the direction control information. 58.

  The direction control information generation unit 58 generates direction control information in response to an instruction from the direction instruction reception unit 57. For example, when the direction instruction receiving unit 57 receives the right direction information, the direction control information including the right direction information is generated as shown in FIG. The direction control information generated by the direction instruction receiving unit 57 is transmitted to the mobile phone 5.

  As shown in FIG. 20B, the direction control information generation unit 58 may generate direction control information including information on the length indicating how much to move in the direction in addition to the direction. Good.

  The designation of the length from the viewer R is received by the direction instruction receiving unit 57 in accordance with, for example, the number of times the cross key is continuously pressed or the length of time for which the cross key is pressed.

  The direction control information generation unit 58 specifies the length of the shooting direction (for example, 300 pixels) according to the number of times the cross key is pressed received by the direction instruction reception unit 57, and the direction control information shown in FIG. Generate.

  The mobile phone 5 outputs, to the display unit 18, operation instruction information for instructing the photographer S to move the moving image capturing unit 19 to the right by 300 pixels based on the direction and length information of the direction control information. To do. As an example of the display of the operation instruction information, for example, the icon of the arrow in the right direction is continuously displayed until the live video moves to the right by 300 pixels.

  As a result, the photographer S can know how much and in what direction the moving image photographing unit 19 should be moved, and can photograph a live video as intended by the viewer R.

  Further, in the case where the mobile phone 7 includes the live authorization unit 53, the live authorization unit 53 detects that the moving video being played in the direction specified by the mobile phone 7 has moved the live video. It can be certified as a video.

[Video distribution flow 5]
FIG. 21 is a sequence diagram showing a processing flow of each device in the moving image distribution system according to the present embodiment. Each device refers to a mobile phone 5 (FIG. 18) as a moving image photographing device, a moving image distribution server 6, and a mobile phone 7 (FIG. 19) as a moving image reproducing device.

  First, when the photographer S starts moving image shooting, the mobile phone 5 transmits the moving image acquired by the moving image acquisition unit 34 to the moving image distribution server 6 in real time (S81). The moving image distribution unit 52 (FIG. 6) of the moving image distribution server 6 distributes the moving image acquired from the mobile phone 5 to the mobile phone 5 (S82).

  The moving image reproduction unit 54 of the mobile phone 7 reproduces the moving image distributed from the moving image distribution server 6. The viewer R confirms the live video on the display unit 18 and designates the direction in which the user wants to move and the length thereof. Specifically, for example, pressing any one of the direction keys of the cross key of the operation unit 16 and designating how much the user wants to move in that direction according to the number of times of pressing can be cited.

  In this way, when the shooting direction is designated by the viewer R, the direction control information generation unit 58 generates the direction control information (FIG. 20B) according to the content received by the direction instruction reception unit 57. (S83). The mobile phone 7 transmits the generated direction control information to the video distribution server 6, and the video distribution server 6 transmits the direction control information transmitted from the mobile phone 7 to the mobile phone 5 (S84).

  Subsequently, the direction control information acquisition unit 60 of the mobile phone 5 acquires the direction control information (S85). The direction specifying unit 61 specifies how much and in what direction the moving image shooting unit 19 should be moved based on the direction and length information included in the direction control information (S86). Then, the operation instruction output unit 32 outputs operation instruction information for instructing the photographer S to operate the moving image photographing unit 19 according to the content specified by the direction specifying unit 61 on the display unit 18.

  As a result, the photographer S can continue shooting by operating the moving image photographing unit 19 as intended by the viewer R, and the viewer R can obtain a desired live video.

  Further, when shooting is continued (S88) and a live video moved in a desired direction is distributed via the moving image distribution server 6 (S89), the live authorization unit 53 of the mobile phone 7 is designated by the viewer R. The live image can be detected by detecting the live video moved by the designated direction in the designated direction (S90).

  22A to 22C are diagrams showing examples of display screens of operation instruction information displayed on the display unit 18 of the photographer S's mobile phone 5.

  As shown in FIG. 22A, the operation instruction output unit 32 may display only the arrow in the shooting direction intended by the viewer R among the arrows displayed in the four directions on the screen. As shown in FIG. 22B, the operation instruction output unit 32 may superimpose icons indicating the four directions on the corners of the screen and display only one direction in color. As shown in FIG. 22C, the operation instruction output unit 32 may superimpose characters indicating any one of the four directions on the corners of the screen.

  The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope shown in the claims, and embodiments obtained by appropriately combining technical means disclosed in different embodiments. Is also included in the technical scope of the present invention.

  Finally, each block of the cellular phone 1 (3, 4A, 4B, 5, 7), in particular, the object designation information acquisition unit 30, the operation instruction output unit 32, the object designation information generation unit 56, and the direction control information acquisition unit 60. The direction specifying unit 61 and the direction control information generating unit 58 may be configured by hardware logic, or may be realized by software using a CPU as follows.

  That is, the mobile phone 1 (3, 4A, 4B, 5, 7) includes a CPU (central processing unit) that executes instructions of a control program that realizes each function, a ROM (read only memory) that stores the program, A RAM (random access memory) for expanding the program, a storage device (recording medium) such as a memory for storing the program and various data, and the like are provided. The object of the present invention is to obtain the program code (execution format program, intermediate code program, source program) of the control program of the mobile phone 1 (3, 4A, 4B, 5, 7), which is software that realizes the functions described above. A computer-readable recording medium is supplied to the mobile phone 1 (3, 4A, 4B, 5, 7), and the computer (or CPU or MPU) reads the program code recorded on the recording medium. It can also be achieved by executing.

  Examples of the recording medium include a tape system such as a magnetic tape and a cassette tape, a magnetic disk such as a floppy (registered trademark) disk / hard disk, and an optical disk such as a CD-ROM / MO / MD / DVD / CD-R. Card system such as IC card, IC card (including memory card) / optical card, or semiconductor memory system such as mask ROM / EPROM / EEPROM / flash ROM.

  The mobile phone 1 (3, 4A, 4B, 5, 7) may be configured to be connectable to a communication network, and the program code may be supplied via the communication network. The communication network is not particularly limited. For example, the Internet, intranet, extranet, LAN, ISDN, VAN, CATV communication network, virtual private network, telephone line network, mobile communication network, satellite communication. A net or the like is available. Further, the transmission medium constituting the communication network is not particularly limited. For example, even in the case of wired such as IEEE 1394, USB, power line carrier, cable TV line, telephone line, ADSL line, etc., infrared rays such as IrDA and remote control, Bluetooth ( (Registered trademark), 802.11 wireless, HDR, mobile phone network, satellite line, terrestrial digital network, and the like can also be used. The present invention can also be realized in the form of a computer data signal embedded in a carrier wave in which the program code is embodied by electronic transmission.

  The moving image shooting apparatus of the present invention takes a moving image by checking the moving image displayed on the display unit by a photographer and manually adjusting the position of the shooting unit, and the moving image is transferred to the moving image playback device via a communication network. It can use suitably with respect to the moving image imaging device to transmit.

It is a figure which shows the principal part structure of the photographer's S mobile phone concerning embodiment of this invention. It is a figure which shows schematic structure of the moving image delivery system concerning embodiment of this invention. It is a figure which shows schematic structure of the mobile telephone concerning embodiment of this invention. It is a figure which shows the example of the coordinate information as the object designation | designated information transmitted from the browsing person's R mobile telephone. It is a figure which shows the example of the target object area | region specified based on coordinate information in the photographer's S mobile phone. It is a block diagram which shows the principal part structure of the moving image delivery server concerning embodiment of this invention. It is a sequence diagram which shows the flow of a process of each apparatus in a moving image delivery system. (A)-(d) is a figure which shows the example of a screen display when displaying a live image | video on a display part. (A)-(d) is a figure which shows the example of a screen display of the operation instruction information regarding the imaging | photography direction displayed on the display part of the photographer's S mobile phone. It is a figure which shows an example of the object designation | designated information concerning other embodiment of this invention. It is a sequence diagram which shows the flow of a process of each apparatus in the moving image delivery system concerning other embodiment of this invention. (A)-(b) is a figure which shows the other example of a screen display when displaying a live image | video on a display part. It is a figure which shows the principal part structure of the mobile telephone of the browsing person R concerning other embodiment of this invention. It is a sequence diagram which shows the flow of a process of each apparatus in the moving image delivery system concerning other embodiment of this invention. (A)-(d) is a figure which shows the example of a display screen of the target object image displayed on the display part of the photographer's S mobile phone. It is a figure which shows the principal part structure of the mobile telephone concerning other embodiment of this invention. It is a sequence diagram which shows the flow of a process of each apparatus in the video conference system concerning other embodiment of this invention. It is a figure which shows the principal part structure of the mobile phone of the photographer S concerning other embodiment of this invention. It is a figure which shows the principal part structure of the mobile telephone of the browsing person R concerning other embodiment of this invention. (A) And (b) is a figure which shows an example of the direction control information concerning other embodiment of this invention. It is a sequence diagram which shows the flow of a process of each apparatus in the moving image delivery system concerning other embodiment of this invention. (A)-(c) is a figure which shows the example of a display screen of the operation instruction information displayed on the display part of the photographer's S mobile phone.

Explanation of symbols

1 Mobile phone (video camera)
2 Video distribution server 3 Mobile phone (video playback device)
4A Mobile phone with video conferencing function (video camera / video player)
Mobile phone with 4B video conferencing function (video camera / video player)
5 Mobile phone (video camera)
6 Video distribution server 7 Mobile phone (video playback device)
10 Antenna section (transmission means / notification means)
11 Wireless processing unit (transmission means / notification means)
12 audio processing unit 13 audio input unit 14 audio output unit 15 data processing unit (transmission means / notification means)
16 Operation unit 17 Storage unit 18 Display unit 19 Movie shooting unit (shooting unit)
DESCRIPTION OF SYMBOLS 20 Power supply part 21 Control part 22 Reception part 23 Transmission part 24 Control part 25 Control part 26 Control part 27 Control part 28 Control part 30 Object designation | designated information acquisition part (object specification information acquisition means)
31 object specifying unit 32 operation instruction output unit (operation instruction output means)
33 Object tracking unit (object tracking means)
34 Video acquisition unit 35 Direction indication unit (direction indication means)
36 Change amount calculation unit (change amount calculation means)
40 Object storage unit 41 Movie storage unit 42 Object specification information storage unit 43 Movie storage unit 44 Object specification information storage unit 50 Object specification information collection unit 51 Object specification information determination unit 52 Movie distribution unit 53 Live authorization unit ( Detection means / notification means)
54 video playback unit 55 image processing unit (object receiving means)
56 Object designation information generation unit (object designation information generation means)
57 Direction instruction receiving unit (direction instruction receiving means)
58 Direction control information generation unit (direction control information generation means)
60 Direction control information acquisition unit (direction control information acquisition means)
61 Direction specifying part (Direction specifying means)
100 video distribution system

Claims (14)

In a video shooting device in which a photographer confirms a video displayed on a display unit and shoots a video by manually adjusting the position of the shooting unit, and transmits the video to a video playback device via a communication network.
Object designation information acquisition means for acquiring from the video reproduction apparatus object designation information indicating an object to which a viewer who views the video in the video reproduction apparatus focuses attention in the video;
Operation instruction output means for emphasizing the object indicated by the object specifying information acquired by the object specifying information acquiring means in the moving image and displaying it on the display unit ,
The object specifying information includes coordinate information for specifying an object area that is an area including the object in the video, and is associated with information for identifying the viewer,
The operation instruction output means emphasizes the object area specified by the coordinate information and combines the display unit with information for identifying the viewer associated with the specified object area. A moving image photographing device characterized by being displayed on the screen .   The object designation information is a still image obtained by adding information indicating an object area that is an area including the object to one frame of the moving image.
  The moving image according to claim 1, wherein the operation instruction output unit emphasizes and displays the target area specified based on information indicating the target area included in the still image on the display unit. Shooting device.   The object designation information is a trimmed image obtained by trimming an area including the object in the video,
  2. The moving image photographing apparatus according to claim 1, wherein the operation instruction output means emphasizes an area that matches the trimmed image in the moving image and displays it on the display unit.   An object tracking means for tracking the position of the object specified by the object specifying information in the video,
  The moving image photographing apparatus according to claim 1, wherein the operation instruction output unit performs highlight display in accordance with the movement of the object tracked by the object tracking unit.   Direction indicating means for determining a direction in which the photographing unit is moved in order to capture the object tracked by the object tracking means at a predetermined position of the moving image;
  The moving image photographing apparatus according to claim 4, wherein the operation instruction output unit displays information indicating the direction determined by the direction instruction unit on the display unit.   Detecting means for detecting that the object tracked by the object tracking means is captured at a predetermined position of the moving image;
  6. The moving image photographing apparatus according to claim 5, further comprising notification means for notifying the moving image reproducing apparatus that the detecting means has detected that the object has been captured at the predetermined position. .   The object specifying information includes a time code for specifying one frame of the moving image in addition to the coordinate information,
  Among the moving images, comprising a change amount calculating means for comparing the frame specified by the time code acquired by the object specifying information acquiring means and the latest frame, and calculating the degree of change between both frames,
  The object designation information acquisition means requests the moving image reproduction apparatus to retransmit the coordinate information of the object in the latest frame when the degree of change calculated by the change amount calculation means is larger than a predetermined value. The moving image photographing apparatus according to claim 1, wherein:   In a video playback apparatus that receives and plays back a video via a communication network from a video shooting apparatus that shoots a video by manually checking the video displayed on the display unit and manually adjusting the position of the shooting unit ,
  An object receiving means for receiving designation of an object to which the viewer pays attention in the video, which is input via an operation unit by a viewer who views the video;
  Object specifying information generating means for generating object specifying information for specifying the object received by the object receiving means;
  The object designation information generated by the object designation information generation means is associated with information for identifying the viewer, and includes a transmission means for transmitting to the video shooting device,
  The object designation information includes coordinate information that identifies an object area that is an area including the object in the video,
  The object designation information generating means is configured to obtain the coordinate information for specifying the object area according to the designation of the object area including the object in the moving image, which is input by the viewer through the operation unit. An apparatus for reproducing a moving image, comprising:   Detecting means for detecting that the object specified by the object specifying information generated by the object specifying information generating means is captured at a predetermined position of the moving image;
  9. A notification means for displaying a predetermined message on the display unit of the apparatus when the detecting means detects that the object is captured at the predetermined position. The moving image reproducing apparatus according to 1.   A moving picture distribution system comprising: the moving picture photographing apparatus according to claim 1; and the moving picture reproducing apparatus according to claim 8.   A method for controlling a moving image photographing apparatus in which a photographer confirms a moving image displayed on a display unit and manually captures a moving image by adjusting a position of the photographing unit and transmits the moving image to a moving image reproducing apparatus via a communication network. Because
  An acquisition step of acquiring from the video playback device object designation information indicating a target of interest in the video by a viewer who browses the video on the video playback device;
  A display step of emphasizing the object indicated by the object specifying information acquired in the acquisition step in the moving image and displaying it on the display unit,
  The object specifying information includes coordinate information for specifying an object area that is an area including the object in the video, and is associated with information for identifying the viewer,
  The display step emphasizes the object area specified by the coordinate information and displays the information on the display unit together with information for identifying the viewer associated with the specified object area. A method for controlling a moving image photographing apparatus.   A video playback device that receives and plays back a video via a communication network from a video shooting device that shoots the video by manually checking the video displayed on the display unit and manually adjusting the position of the shooting unit. A control method,
  An accepting step for receiving designation of an object that the viewer pays attention to in the video, which is input via an operation unit by a viewer who browses the video;
  A generating step for generating object specifying information for specifying the object received in the receiving step;
  A step of transmitting the object specifying information generated in the generating step in association with the information for identifying the viewer, and transmitting the information to the moving image photographing device,
  The object designation information includes coordinate information that identifies an object area that is an area including the object in the video,
  The generation step includes generating the coordinate information specifying the target area in accordance with the specification of the target area including the target object in the moving image, which is input through the operation unit by the viewer. A method for controlling a moving image playback apparatus.   A control program for causing a computer to function as each means of the moving picture photographing apparatus according to any one of claims 1 to 7 or the moving picture reproducing apparatus according to claim 8 or 9.   A computer-readable recording medium on which the control program according to claim 13 is recorded.

Images