JP2005064729A - CONTROL DEVICE, CONTROLLED DEVICE, REMOTE CONTROL SYSTEM, CONTROL DEVICE CONTROL METHOD, CONTROLLED DEVICE CONTROL METHOD, PROGRAM, AND RECORDING MEDIUM - Google Patents

Abstract

PROBLEM TO BE SOLVED: To realize a moving display of a focus on a control device of a remote control system for reducing user operation procedures and improving convenience.
A control device receives operation information including one or more operation elements including display image information from a controlled device, and displays an image corresponding to each operation element based on display image information of each operation element. Display, and the focus is automatically moved and displayed from the operation element to another predetermined operation element for the operation element.
[Selection] Figure 1

Description

  The present invention relates to a remote control system in which a control device can control a controlled device via a network.

  There has been an increasing demand for connecting home audio / visual devices and home appliances that have been operated independently by themselves, or that have been operated by a unique connection for each type of device, through a single network. Network communication that enables high-speed digital communication technology to transfer data that requires real-time performance such as moving image data and audio data, and data that requires communication reliability such as still image data and control data. Has realized this.

  The device that provides the user interface as the center of the home network is a television, and the remote operation screen of the connected device is displayed on the television screen, and the device is controlled by inputting the user's operation with the remote control device of the television. A so-called remote control service is provided, but a program for executing device control in accordance with display of an operation screen or user operation is required for each device. Since there are a wide variety of devices connected to the network and models with new functions appear over time, it is difficult to prepare a control program for all devices that may be connected on the TV side.

  One approach to solving these problems is to send the image data of the operation panel from the controlled device connected to the network, and the remote control device equipped with a display typified by a television receives it to operate it. A system has been proposed in which an operation panel image is displayed on a display device for a user and at the same time an operation input of the operator is transmitted to the controlled device to control the controlled device. For example, Patent Document 1 “Network Control System and Network Terminal and Control Terminal”.

  FIG. 45 shows such a system. In FIG. 45, reference numeral 530 denotes a controlled apparatus that transmits remote control display data and receives remote control via a network. 533 is an image generation unit that generates display data for remote control (remote control display data), 531 is a bidirectional communication unit connected to the network, 532 transmits remote control display data, and receives user operation information. A processing unit 534 for controlling the controlled main body unit 534 is a controlled main body unit that provides services to the user by operating under control. 520 is a remote control device having a display function and performing remote control, 524 is a display unit for displaying remote control display data, 525 is an input control unit for receiving user input such as a remote control, and 523 is for receiving An information storage unit for storing and holding the remote control display data, 522 stores the received remote control display data in the information storage unit, and displays a remote operation screen on the display unit 524 based on this data, and an input control unit A processing unit 521 for outputting user operation information based on an operation input from a user received from 525, and a bidirectional communication unit 521 for transmitting / receiving such data to / from the network.

  The remote control display data sent from the controlled device via the network 500 is displayed on the display unit 524 of the remote control device 520, and the user operates the remote controller or the like while viewing this. Information relating to the user's operation is sent to the controlled device via the network 500, and the controlled main body is controlled based on this information, and the service for the user is performed.

  FIG. 46 is a diagram showing a display state (operation panel) of remote control display data displayed on the display unit 524 of the remote controlled apparatus 530. 540 is a display area of the display unit 524, 542 is an operation panel displayed based on remote display data, 543 to 547 are operation buttons on the display representing individual services of the remote controlled device, and 548 is on the operation panel 541 is a display showing a point of interest (focus) or a selection candidate, and 541 is a video as a result of the service provided by the remote controlled device 530.

  The user operates the operation panel with a remote controller as shown in FIG. That is, by pressing the direction keys 552, 553, 554, and 555 provided on the remote controller 550, the focus 548 on the operation panel is moved to the target operation button display, and the desired operation is performed by pressing the enter key 556. Let it be an operation to select and execute.

  Remote operation based on remote control display data sent from a remote controlled device has the great feature that it can operate any unique function of the remote controlled device by displaying it as remote display data. However, the remote control device only notifies the controlled device which operation element the user finally selected from the given operation elements, and moves the selection candidates on the operation elements. Do not notify the controlled device of the operation. That is, when the user selects an operation target with the operation element displayed on the remote control device, the movement of the focus must be managed by the remote control device.

  For example, in FIG. 46, when the button 544 as the display element of the operation panel 542 has the focus, when the user presses the direction key 554 of the remote controller 550 to move the focus downward, the processing of the remote control device The unit 522 moves the focus to the operation button 540 and displays it. However, if the user presses the direction key 552 of the remote controller 550 and moves the focus upward when the button 540 has focus, the focus destination by the processing unit 552 of the remote control device is the button 543 or the button 544, and may be a different destination depending on the connected remote control device, which may cause operational confusion. Further, there is a possibility that the selected one destination is not necessarily a destination that satisfies the convenience of the user.

  Focusing on these points, for example, Patent Document 2 proposes a method in which control information is provided for each display element and an adjacent display element at the time of movement is designated from the controlled device. FIG. 48 is a table schematically showing the contents of display element information including operation information for each display element. That is, the display element indicating the button 540 has an element ID = 4, and designates that the focus is moved to the display element with the element ID = 2 when the upward focus movement is performed from this element. Considering the meaning of the operation on the controlled device, it is only possible to know the focus movement considering the convenience of the user, and it is effective to have such information in the remote control display data. It is.

Japanese Patent Laid-Open No. 11-3314 Japanese Patent Laid-Open No. 11-328081

  However, the remote control system requires an indirect control structure in which the operation panel represented on the screen is further operated by a remote controller, and the remote control system requires a remote controller directly connected to the operation or function of the apparatus by a conventional remote controller for the controlled device. Compared to the environment where the user directly controls the buttons, the procedure required for the operation is increased. In such an environment, there is a desire to further improve the user's operational convenience. Regarding the movement of the focus, it is also possible to determine the movement destination of the focus according to the user's operation on the display element on the operation panel. There has been a need for a configuration of a remote control system that realizes, for example, designating a focus movement destination from a controlled device that can have knowledge for improving contents and convenience of operation.

  Accordingly, an object of the present invention is to realize a focus movement display on a control device of a remote control system for reducing user operation procedures and improving convenience.

  In order to achieve such an object, the control device of the present invention includes communication means for receiving operation information including one or more operation elements including display image information from the controlled device, and display image information of each operation element. Display means for displaying an image corresponding to each operation element based on the control information, and the communication means receives control information indicating an operation element selected and determined from the one or more operation elements by a user operation input. The information is transmitted to the control device, and the display means moves and displays the selection candidate information from the operation element to another operation element predetermined for the operation element.

  Further, the controlled device of the present invention includes communication means for transmitting operation information including one or more operation elements including display image information to the control device, and the controlled device based on the control information transmitted from the control device. Control means for controlling the control device, wherein the communication means displays an image corresponding to each operation element on the control device based on display image information of each operation element, and by an operation input by a user The control information indicating an operation element selected and determined from the one or more operation elements is received from the control device.

  The remote control system of the present invention is a remote control system in which at least a control device and a controlled device are connected via a network, and the controlled device is an operation composed of one or more operation elements including display image information. Information is transmitted to the control device, the control device receives the operation information from the controlled device, and displays an image corresponding to each operation element based on display image information of each operation element. The control information indicating the operation element selected and determined from the one or more operation elements by the user's operation input is transmitted to the controlled device, and another operation predetermined for the operation element from the operation element is transmitted. Selection candidate information is moved and displayed in an element, and the controlled device controls the controlled device based on the control information received from the control device. And butterflies.

  Further, the control method of the control device according to the present invention includes a communication unit that receives operation information including one or more operation elements including display image information from the controlled device, and each of the operation elements based on the display image information of each operation element. A control method for a control device having display means for displaying an image corresponding to an operation element, wherein control information indicating an operation element selected and determined from the one or more operation elements by a user operation input is transmitted by the communication means. The information is transmitted to the controlled device, and the display means moves and displays selection candidate information from the operation element to another operation element predetermined for the operation element.

  The control method for a controlled device of the present invention is based on communication means for transmitting operation information including one or more operation elements including display image information to the control device, and control information transmitted from the control device. And a control means for controlling the controlled device, wherein an image corresponding to each operation element is displayed on the control device based on display image information of each operation element. The control information indicating the operation element selected and determined from the one or more operation elements by the user's operation input is received from the control device by the communication means, and the control means is controlled based on the control information. The apparatus is controlled.

  The program of the present invention causes a computer to execute the control method of the control device. The program of the present invention causes a computer to execute the control method of the controlled device. Furthermore, the computer-readable recording medium of the present invention is characterized in that the program is recorded.

  According to the present invention, the selection candidate information is automatically moved and displayed from the operation element selected and determined on the display unit of the control device to another predetermined operation element. It is possible to achieve reduction, and convenience can be further improved if the destination of the selection candidate information is determined in consideration of the operational characteristics of the controlled device.

DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, preferred embodiments to which the invention is applied will be described in detail with reference to the accompanying drawings.
<First Embodiment>
FIG. 1 is a block diagram showing the configuration of the remote control system according to the first embodiment of the present invention. This system is connected to the network 1 by a high-speed serial interface, and is remotely controlled based on the remote control input of the digital television 110 by performing display on the digital television 110 having a user operation input by the remote controller 120 and the digital television 110 for operation. It consists of a digital video camera (DVC) 100.

  Reference numeral 100 in FIG. 1 denotes a DVC that is remotely controlled via a digital television and serves as another controlled device. 104 is an imaging device (not shown), or a DVC main unit for recording stream data input through a network or an input / output terminal (not shown), and reproducing the recorded data to output stream data including video and audio data. 103 is a remote A display data generator for storing or generating display data (GUI data) and operation information to be displayed on the digital television 110 for control. 102 communicates via a network to control the DVC main unit 104 and the display data generator 103. It is a control part. Reference numeral 101 denotes a 1394 I / F unit that is an interface of a 1394 high-speed serial bus, and is connected to the 1394 I / F unit of the digital television 110 or the 1394 I / F unit of another device via the network 1.

  Reference numeral 110 in FIG. 1 denotes a digital television having a user operation input by a remote controller. Reference numeral 111 denotes a 1394 I / F unit which is an interface of a 1394 high-speed serial bus. Reference numeral 113 denotes a decoder including a function for separating stream data received by the 1394 I / F unit 111 into a video stream and an audio stream and decoding each stream data. 119 is an audio output unit including D / A conversion and amplification of the decoded audio data and a speaker.

  114 is a display data memory for storing operation data and display data (GUI data) for remote control received by the 1394 I / F unit 111, and 116 is a remote control for performing user operation input using infrared rays or wireless communication. , 115 is a drawing control unit that generates a display for remote control of a remotely controlled device connected based on display data for remote control, and 117 is a video data output from the decoder 113 and a drawing control unit 115. An image composition unit for compositing the display output from the image display unit 118 and displaying it on the display device 118, 112 is a 1394 I / F unit 111, decoder 113, display data memory 114, remote controller control unit 116, network I / F 111 and drawing This is a control unit that controls the control unit 115, receives stream data transmitted from the DVC 100 connected to the network and outputs it to the display device 118 and the audio output unit 119. The operation of the entire digital television 110 that controls the DVC 100 is controlled in an integrated manner by the display for remote control displayed on the display device 118 and the remote control operation input.

  The 1394 I / F (IEEE 1394 high-speed serial bus) is a network bus that supports a daisy chain method, a node branching method, and a combination thereof, and enables a highly flexible connection. In a data transfer system that performs serial transfer at 100 Mbps, 200 Mbps, and 400 Mbps, devices having higher transfer rates support lower data transfer rates, so devices having different transfer rates can be mixed. Asynchronous transfer for the transfer of asynchronously transferred data such as control signals such as commands and file data, and transfer of data that requires temporal continuity such as video data and audio signals (so-called stream data) Has two transfer modes of isochronous transfer in which the above is sequentially performed every predetermined period (125 us).

  AV / C (Audio Video Control) commands for controlling AV devices are constructed on the Asynchronous transfer, and at the same time, models of AV devices controlled by these commands are defined as AV / C units or subunits. In addition, for data transfer that requires high reliability and high speed, such as still image file transfer and print data transfer, Asynchronous Serial Bus Connection, which is a protocol based on Asynchronous transfer, is connected to or eliminated from this connection. Procedures and control commands are defined and used.

  Stream data transfer by isochronous transfer includes image / audio data transfer in DV format and image / audio stream transfer in MPEG2 format. The procedure and control command for establishing or canceling the connection for transferring these stream data are also defined as AV / C commands by Asynchronous transfer.

  FIG. 2 is a diagram for explaining the transmission and display of display data (GUI data) for remote control of the remote control system and the procedure of remote control, focusing on communication on the 1394 network. In the explanation of this figure, the procedure with the DVC100, which is one of the controlled devices on the 1394 network, is described, but the same communication is performed when another controlled device is connected to the network, Further, similar control can be performed in parallel with the DVC 100.

  In FIG. 2, the control unit 112 of the digital television first establishes an isochronous connection for transmitting the DVC 100 stream output to the digital television 112 by a connection establishment procedure. That is, after securing a channel and a band for isochronous transfer from an IRM (Isochronous Resource Manager; in this embodiment, the digital television 110) of the 1394 network, the digital television 110 is connected from the isochronous plug of the 1394 I / F 101 of the DVC 100. A peer-to-peer connection to the isochronous plug of the 1394 I / F 111 is established (201).

  Next, the control unit 112 of the digital television issues a Gui_update (open) command of an AV / C command to start a session using the DVC 100 as a remote control device (202). FIG. 3 shows a command frame of the Gui_update command. An instruction to the remote control device by this command is specified by the value of subfunction (FIG. 4) stored in operand [3]. open indicates the start of a remote control session, and close indicates the end of the session. start indicates the transfer of GUI data for remote control.

  The remote control apparatus transmits GUI data to the control apparatus according to the procedure described later when the GUI data is changed due to the state of the apparatus or the operation result until the next stop instruction is given. “restore” is specified when the Asynchronous serial bus connection for GUI data transfer is restored when a 1394 network bus reset occurs. change is specified when specifying the change of the reporting range of GUI data changes.

  Upon receiving the Gui_update (open) command, the control unit 102 of the digital video camera stores the plug number to be used for Asynchronous serial bus connection in response to the command and returns it in operand [0]. Using the plug number, the control unit 112 of the digital television establishes a necessary internal connection of the DVC 100 and a connection between the DVC 100 and the DTV 110 by issuing a Connect command (203) and an Allocate_attach command (204).

  Subsequently, the control unit 112 of the digital television issues a Gui_update (start) command to instruct the start of GUI data transfer (205), and issues a Push_gui_data command to instruct the transfer of GUI data of the DVC 100 (206). ).

  FIG. 5 shows a command frame of the Push_gui_data command. In operand [0], specify the plug number acquired by Gui_update (open). The indicator of operand [5] stores the level and with_data designation shown in FIG. The level specifies the transfer range of the GUI data of the remotely controlled device that is transferred in accordance with the instruction by the remotely controlled Push_gui_data command according to the GUI data tree structure described later. The specified value is a 2-bit 3-value shown in FIG. 7. The element specified by an element ID (described later) is 00 (itself), and the element specified by the element ID is 01 (itself and next level) and belongs to the element. The element is specified in the range of 01 (itself and next level) in 11 (all level) and up to the next element belonging to the elements included in this range. With_data specifies actual data such as text, image, and sound of the element specified by element ID and level when this bit is 1. In the 4 bytes from operand [6], an element ID for identifying the GUI data element described later is stored, and the GUI data for instructing the transfer through Asynchronous serial bus connection is specified, but the panel element as the root is specified. Only in some cases, it is possible to substitute the element ID with FFFFh.

  The control unit 102 of the digital video camera that has received the Push_gui_data command returns a response to the command and transfers display data for remote control from the panel data generator 103 of the digital video camera to the digital television 110 through the asynchronous serial bus connection. (207). The transferred display data is stored in the display data memory 114, then read out element by element in the control unit 112, sent to the drawing control unit 115, displayed on the display device 118 through the image composition unit 117, and operation information for the user. Provided as (operation panel).

  The display data is a set of elements displayed as operation information and has a tree-structured data structure with a panel element as a root. FIG. 9 is a table showing these elements and essential data or selection data subordinate to each element, and FIG. 10 is a table showing user actions that are operations that the user can take on each element.

  The panel element indicates a set of operation elements, and holds a string of IDs that each element has as a link. In addition, it has a link to a label, which is character string data serving as a title of the panel element, and position and size data on the display screen displayed by the panel. Also, it is possible to retain options for specifying the background color of the panel, linking to bitmap data displayed on the background, and selecting fonts for the character string to be displayed. In addition, isochronous transfer channel data for transferring video and audio streams can be stored as an option. When this option is specified, audio video data decoded from the transferred stream is stored in the entire panel element. indicate.

  The button element is an element that the user can press or release with the remote control, and holds a link to two bitmap data to be displayed on the key top when pressed and when released. . The size of the bitmap is indicated by size data, and the display position is indicated by position data. The position data is obtained by designating the relative position in the panel element to which this element belongs in pixel units. In addition, a link to a label serving as a title (or functional description) of a button element can be held, and font options for a character string to be displayed can be held as an option. Focus navigation that can be held as an option will be described later.

  The text element is an element that displays the title and description as a character string, and holds the position, size, and the link to the text data that becomes the label, and the background color specification, font choices, and focus navigation as options. I can do things.

  The icon element is an element that displays a bitmap and can be selected by the user, holds a link to the bitmap data, a label, a position, a size and a link to the text data that becomes the label, a font option, Audio visual designation and focus navigation can be held as options.

  The animation element is an element that can hold a plurality of bitmaps, switch them in order at regular time intervals, and display them so that the user can select them. It holds the position, size, and the link to the text data that is the label, and can hold font options and focus navigation as options.

  11 to 22 are diagrams showing the data structure of elements constituting GUI data. FIG. 11 shows the data structure of element data such as panel elements and button elements. Every element holds an element ID for identification in the display data. The element type is a code indicating the type of element, and the element type shown in FIG. 23 is stored. The element data stores the data held by this element in the order of essential data and selection data.

  FIG. 14 shows a data structure of actual data such as text, bitmap, and sound linked from each element as essential data or selection data. All actual data holds an element ID for identification as well as an element. The element type is a code indicating the type of actual data, and stores the data type shown in FIG. If the actual data is text data, character string data is stored, if bitmap data is stored, a bitmap data string is stored, and if sound is stored, a sound data string is stored.

  FIGS. 13 to 22 show link data structures of elements or data to be essential data or selection data held by the element data. FIG. 13 shows a link to the actual text data, and the element ID of the actual text data at the link destination is stored in the element ID. FIG. 14 is a link to bitmap actual data, and the element ID of bitmap actual data is stored in the element ID. FIG. 15 is a link to an element. The element ID stores element IDs of elements such as button elements and text elements belonging to the panel element, and the element type stores the element type shown in FIG. FIG. 16 shows a data structure of color data designated as a background color, in which color data represented by 8 bits for each RGB is stored.

  FIG. 17 shows a data structure for designating the display position of an element. The position of each element is expressed in a two-dimensional coordinate system on the display device as a relative position with the upper left corner of the upper element to which the element belongs as the origin. That is, the position on the display device is specified in units of pixels when the element is the root panel element, and the position on the panel element is specified if the element belongs to the panel element.

  FIG. 18 shows a data structure for designating a size for displaying an element, and “width” and “height” store values representing the size of the element in units of pixels. FIG. 19 shows a data structure for designating the font size of the text displayed by the element, and the font code shown in FIG. 26 is stored.

  FIG. 20 shows a data structure when selection data is stored in element data of an element. The selection data type shown in FIG. 25 is stored in “selection data type”, and background color, font, etc. are selective in “selection data”. The data structure added to is stored.

  FIG. 21 shows a data structure for designating an isochronous channel for transferring audio / video data displayed in the background or icon element of the panel element. The isochronous channel is stored in the lower 6 bits and 0B is set in the upper 2 bits. .

  FIG. 22 shows a data structure for designating an element to which the focus is to be moved when there is a focus movement in each of the upper, lower, left, and right directions when the element has a focus and when there is a selection input for this element. As shown in FIG. 22, the focus movement destination when there is an operation input for focus movement in the four directions of up, down, left, and right is the element ID, and the movement destination of the focus when there is an operation input for selecting an element is Stored as an element ID. When FFFFh is stored as the element ID of the focus movement destination, or when its own element ID is designated, the focus is not moved even if there is a corresponding operation input.

  FIG. 27 is a diagram showing an example of a data structure in display data for remote control in a tree structure. FIG. 28 is a diagram showing the data structure of the panel element in the data structure. Such data is transferred to the digital television 110 through the asynchronous serial bus connection.

  FIG. 29 shows a display example (operation panel) of display data for remote control. The control unit 112 displays such a display on the display device 118 through the drawing control unit 115, and accepts a user operation input through the remote control unit 116. For example, in FIG. 29, the highlight display 313 of the button indicates that the PLAY button 304 indicating DVC playback has a focus as a user selection candidate, and the focus position is changed in response to a remote control operation. This operation is called “farcus navigation”. In addition, in response to a remote control operation to determine and select the focused button, the button display is switched from the released bitmap display to the depressed bitmap display.

  When detecting a user action allowed for each display element, such as pressing a button, the control unit 112 of the digital television issues a User_Action command to notify the control unit 102 of the DVC 100 (208 in FIG. 2). The control unit 102 performs control corresponding to a user action predetermined for each notified display element on the DVC main body unit 104. The display data generator 103 reflects the change in the operating state of the DVC main body 104. When the display data changes, the display data having changed is transferred to the digital television 110 by the asynchronous serial bus connection (FIG. 2). 209). Depending on the control to the DVC main body, transfer of new stream data is started through an isochronous connection (210). A series of operations for performing focus navigation and notifying the remote control device of user operation input as a user action is called user interaction.

  FIG. 30 is a diagram showing a remote controller 120 used for user interaction. The remote controller 120 includes direction keys 312, 313, 314, and 315 for instructing movement of focus on display data and a determination key 316 for instructing selection of a display element with focus. When the display element is a button element, the selection decision instruction generates a user action of depressing (and releasing) the button, and when it is an icon and an animation element, a selection user action is generated.

  30 also includes a power on / off key 317, a volume up key 318, a volume down key 319, and a menu key 320. Operations on these keys are always operations on the DTV 110 and are notified to the control unit 112 through the remote control unit, and are passed to power on / off of the DTV 110, volume adjustment, and control of the DTV menu call.

  FIG. 31 is a table that schematically summarizes the focus navigator included in the display elements of the display data sent from the DVC 100 to the DTV 110. Bit map data representing button elements and element IDs 1 to 5 are designated for each direction. The element ID to be the focus movement destination and the element ID to which the focus is moved after the button element is selected and the button is pressed are shown.

  For example, when the rewind button 303 with the element ID = 1 has the focus, when the user presses the direction key 313 of the remote controller 120 to move the focus downward, the processing unit 112 of the DTV 110 uses the element according to the focus navigation. The focus is moved to the stop button 306 with ID = 4 and displayed.

  Next, when the user presses the direction key 315 of the remote controller 120 to move the focus upward, the processing unit 112 of the DTV 110 moves the focus to the playback button 304 of element ID = 2 according to the focus navigation. This is based on the idea that the possibility of operating the playback button is higher than the rewind button when considering the possibility of the user operating after the stop operation. Such considerations are necessary for each controlled device or application running there, but it is possible to consider that the feeling of operation will not be confused by the focus destination being different depending on the remote control device to be connected, user operability, The effect of improving convenience can be expected.

  When the fast-forward button 305 with the element ID = 3 is in focus and the user performs a fast-forward operation by pressing the selection / determination key 316 of the remote controller 120, the processing unit 112 of the DTV 110 switches the display image of the button element 305 down. Change and notify the DVC 100 of the user action of pressing the button. Next, when the selection determination key 316 is released, the display image of the button element 305 is returned to the open image, and the user action of releasing the button is notified to the DVC 100. At the same time, the focus is moved to the stop button 306 according to the focus navigation. This is because the operation of the DVC100 is very likely to operate the stop button for the fast-forward stop operation after the fast-forward operation. Moving the focus to the stop button in advance is the user's operability. This is because the effect of improving the resistance is expected to be extremely high.

  Considering the case where there is no automatic focus movement at the time of selection decision, after fast forward operation, it is necessary to move the focus to the stop button in order to press the stop button to stop the fast forward operation. Direction, or at least two key operations in the downward direction and the left direction are necessary, and it is difficult to perform a smooth operation, and there is a possibility that the fast-forward operation cannot be stopped at a timing intended for the user. If the focus is automatically moved to the stop button after starting the fast-forwarding operation as in the present embodiment and the next operation is ready, the user can concentrate on timing the stop operation. It is possible to perform the intended operation smoothly and the operability is remarkably improved.

  The same effect can be expected when the rewind button 303 is operated. When the selection determination key 316 of the remote controller 120 is pressed, the focus is moved to the stop button 306 according to the focus navigation. Such operational characteristics and user convenience depend on the design of the controlled device and its operation panel, and are information that can be known only on the controlled device side. Therefore, it is possible to improve the operability of the user by being implemented as focus navigation that is sent from the controlled device to the remote control device together with display data for remote control.

  Further, not only the focus movement but also the focus movement at the time of selection decision can be combined to realize more effective operability.

  In the above description, the case where the focus is moved by the direction key or the case where the focus is automatically moved by the selection decision operation has been described. However, when the selection of the playback button 304 is decided or when the fast-forward button 305 is moved to the right It is also possible to specify that the focus is not moved as the focus navigation specification, as in the case of trying to move the focus to. In addition, the movement destination of the focus does not have to be a display element adjacent in position, and a more natural and rational operation or a highly convenient operation can be realized for the user operation of focus and selection determination. If there is, the display element can be freely specified.

<Second Embodiment>
The second embodiment of the present invention has different features in focus navigation in the network remote control system according to the first embodiment. In the following description of the second embodiment, only differences from the first embodiment will be described, and components having the same configuration and the same operation as those of the first embodiment will be given the same reference numerals. This will be shown and will not be described again.

  FIG. 32 is a block diagram showing a remote control system in the second embodiment. In the remote control system in the present embodiment, the digital television 160 connected to the network by the high-speed serial interface and having a user operation input by the remote controller 120, displays for operation on the digital television 160, and based on the remote control input of the digital television 160 A digital video camera (DVC) 150 that is remotely controlled and a digital video recorder (VCR) 130 that performs display on the digital television 160 and is remotely controlled based on a remote control input of the digital television 160 are configured.

  Reference numeral 160 in FIG. 32 denotes a digital television having a user operation input by a remote controller. The digital TV 160 integrally controls the drawing control unit 162 that generates a display for remote control of a remotely controlled device connected based on display data for remote control, and the operation of the entire digital TV 160. The control unit 161 performs the same operation as that of the digital television 110 described in the first embodiment except that the control unit 161 has a different feature in the handling of focus navigation.

  Reference numeral 150 in FIG. 32 denotes a DVC that is remotely controlled via a digital television and serves as another controlled device. The DVC 150 communicates with a display data generator 152 that stores or generates display data (GUI data) and operation information to be displayed on the digital television 160 for remote control via a network, and generates display data with the DVC main unit 104. The control unit 151 that controls the device 152 performs the same operation as the DVC 100 described in the first embodiment, except that it has different features in the generation and handling of focus navigation.

  Reference numeral 130 in FIG. 32 denotes a VCR that is remotely controlled via a digital television and serves as another controlled device. 134 is a VCR main unit for recording stream data input through a network or an input / output terminal (not shown), and reproducing recorded data to output stream data including video and audio data. 133 is a digital television for remote control. A display data generator 132 stores or generates display data (GUI data) to be displayed on the display 160 and operation information, and 132 is a control unit that controls the VCR main unit 134 and the display data generator 133 through communication via a network. Reference numeral 131 denotes a 1394 I / F unit that is an interface of a 1394 high-speed serial bus, and is connected to the 1394 I / F unit of the digital television 160 or the 1394 I / F unit of another device via the network 1.

  FIG. 33 is a table showing display elements constituting display data for remote operation in the present embodiment, and essential data or selection data subordinate to each element. It differs from the first embodiment in that the focus navigation of the selected data has a different configuration and each element can have related panel information and function information.

  FIG. 34 shows the data structure of these selection data. FIG. 35 shows the data structure of focus navigation. The focus navigation is a data structure that specifies an element to which a focus is to be moved when there is a focus movement in each direction and a selection input for this element when the element to which this data is designated has a focus. The movement destination designation information has a specific function included in the element ID that designates one of the elements included in the display data as the movement destination, or the display data (panel) designated by the related panel information described below. One of the function codes for designating the element as the movement destination is stored, and which one is stored is indicated by the flg bit. That is, when the flg bit is 0, the element ID is stored as the movement destination designation, and when the flg bit is 1, the function code is stored as the movement destination designation. FIG. 37 shows the data structure of the destination designation information.

  FIG. 35 is a diagram illustrating a data structure of related panel information. The related panel information is display data (panel) to be searched with the element having the function code as the movement destination when the function code is stored in any of the movement destinations in the focus navigation of the element for which the related panel information is designated. Is also specified. The node ID is an identifier that designates one of the devices connected to the network. The panel specifier is an identifier that logically designates display data (panel) for remote control or a panel data generator that outputs this display data. It is.

  FIG. 36 shows the data structure of function information. The function information indicates a function on the controlled apparatus that operates when the user operates a display element included in the display data. For example, in the display data 332 displayed by the DVC described in FIG. 39, when the playback button 334 is operated, when the playback operation of the tape medium with the DVC of the controlled apparatus is performed, the display that displays the playback button 334 The function information of the element is “reproduction”, and is designated as 01h using the function code shown in FIG.

  When a focus is moved or selected for an element in user interaction, if a function code is specified as the focus navigation destination, the focus destination is the specified related panel information and function code. Based on the search, it is determined. First, the panel displayed on the digital television of the device specified by the related panel information is searched. Next, the function code of the display element in the searched panel is sequentially checked, the display element that matches the function code specified by the focus navigation is searched, and the focus is moved to this display element. That is, the user's operation target moves to the display element having the function code specified by the focus navigation in the operation panel specified by the related panel information.

  FIG. 39 shows a display example (operation panel) of display data for remote control, and the operation in response to user operation input will be described. In FIG. 39, an operation panel 332 for remote control of the DVC 150 and an operation panel 350 for remote operation of the VCR 130 are displayed on the display device 300 of the digital television 160. DVC operation panel 332 displays icons 338 to 340 for selecting a content source device when performing dubbing recording, and allows a user to perform a simple dubbing operation.

  The panel data generator 152 of the DVC 150 searches for devices connected on the network, and configures panel data to select a device that can be a dubbing source and display it as an icon. In FIG. 39, a VCR 130 on the network is detected, and an icon 338 of VCR1 is displayed as a source device. Since there is no other device that can be a source, the icons 339 and 340 are displayed in a state where they cannot be selected. FIG. 41 shows the focus navigation and function information contents of the display elements operable on the operation panel 332 at this time, and FIG. 43 shows the focus navigation and function information contents on the operation panel 350. In any operation panel, the destination of focus navigation designates the display element included in each operation panel by the element ID.

  Hereinafter, the operation until the user starts dubbing recording from the VCR to the DVC and the operations of the digital television 160, DVC150, and VCR130 through the displayed two operation panels will be described with reference to FIG.

  When the source device selection icon 338 on the operation panel 332 is selected by the user's operation, a user action command is issued from the digital television 160 to the DVC 150, and the user's icon selection operation is notified (401). The DVC 150 detects that the user is going to perform dubbing recording from the VCR 130 to the DVC 150, and updates the focus navigation and related panel information of the operation panel 332 as an operation that can easily perform this operation (402). That is, when the user operates the recording button 337 and the DVC 150 enters the recording standby state, the focus navigation recording button 337 can be moved so that the focus can be moved to the operation button for performing the operation for starting the stream data output of the source device. Is changed to the function code, and at the same time, the related panel information is changed to designate the operation panel 350 of the VCR 130. FIG. 42 shows the focus navigation of the display elements that can be operated on the operation panel 332 and the contents of the function information at this time. In FIG. 42, the numerical value following “f =” represents a function code.

  When the user moves the focus on the operation panel 332 and selects the record button 337, a user action command is issued from the digital television 160 to the DVC 150 (403), and the DVC 150 shifts the DVC main unit 104 to the recording standby state at the same time. Establish an isochronous connection from VCR130 to DVC150 and wait for recording to start. On the other hand, the VCR 130 detects that the dubbing recording is performed because the isochronous connection is established, and updates the focus navigation and related panel information of the operation panel 350 as an operation that can easily perform this operation (405). That is, when the user operates the playback button 357 and the playback operation of the VCR is started, the focus can be moved to the operation button for performing the operation for releasing the standby state of the recording device and starting the recording operation next. The destination designation information of the navigation playback button 357 is changed to the function code, and the related panel information is changed to designate the operation panel 332 of the DVC 150 at the same time. FIG. 43 shows the focus navigation of the display elements that can be operated on the operation panel 350 and the contents of the function information at this time.

  On the other hand, in the digital TV 160, the display element with the function code = 1 is searched from the operation panel 350 in accordance with the focus navigation of the operation panel 332 and the related panel information, and the focus is moved to the found play button 357. .

  When the user selects the operation button 357 according to the automatically moved focus, a user action command is issued from the digital television 160 to the VCR 130, and the user's icon selection operation is notified (406). In accordance with the contents of the focus navigation of the operation panel 332 and the related panel information, the display element of the function code = 4 is searched from the operation panel 332, and the focus is moved to the found recording button 337.

  The VCR 130 starts outputting stream data to the established isochronous connection, and the user can immediately select and operate the recording start button 337 of the DVC 150 on the operation panel 332 according to the moved focus. By the user action (408), recording of the DVC 150 is started and a dubbing operation is executed.

  In the above description, the focus is moved to the operation panel of another device by focus navigation in the case of the display element selection decision. However, not only the display element selection decision but also other movement in accordance with the focus movement operation by the user. It is also possible to move the focus to the display element of the operation panel.

  Further, in the present embodiment, it has been described that related panel information for designating another operation panel can be added for each display element. However, the data structure is configured so that a different operation panel is designated for each movement designation destination of focus navigation. It is also possible to do. In such a case, it is possible to specify a focus movement destination with a wider degree of freedom than in the example described in the present embodiment. It is also possible to configure the system so that each operation panel has one related panel information.

  In the above embodiment, the operation panel specified by the related panel information has been described on the premise that it is displayed on the digital television in advance. However, when the focus is moved to the operation panel specified by the related panel information, the operation panel is not yet displayed. When not displayed on the digital television, it is also possible to configure the function of the control unit or the display control unit of the digital television to start displaying the designated operation panel and then move the focus there.

  According to the above-described embodiment, an indirect control structure is required in which an operation panel expressed on the screen is further operated by a remote controller, and the remote controller directly connected to the operation or function of the apparatus by the conventional remote controller for the controlled device is necessary. Even in a remote control system with the aspect that the procedure required for operation increases compared to the environment where the user directly controls the button, the focus is automatically moved to the next operation element to be operated according to the operation result of the user. It becomes possible to make it. Accordingly, it is possible to reduce the user's operation procedure, and to designate such a movement destination of the focus as a movement destination that can further improve convenience in consideration of operational characteristics of the controlled device. This makes it possible to provide remote operation information that is easier to operate and easier to use.

  Also, the point at which the display data of the operation panel is mounted on the controlled device by using a means to move the focus by specifying the function destination and searching for the corresponding operation element from the specified operation panel. Even if the display data is not linked in advance, it is possible to move the focus to the intended operation element, increase the opportunity with a device that can provide the user with remote control operation linked on the network, remote operation is easier and easier to use Operation information can be easily provided.

  Another object of the present invention is to supply a storage medium storing software program codes for realizing the functions of the above-described embodiments to a system or apparatus, and the computer (or CPU or MPU) of the system or apparatus stores the storage medium. Needless to say, this can also be achieved by reading and executing the program code stored in.

  In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments, and the program code itself and the storage medium storing the program code constitute the present invention.

  As a storage medium for supplying the program code, for example, a flexible disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM, a CD-R, a magnetic tape, a nonvolatile memory card, a ROM, or the like can be used.

  Further, by executing the program code read by the computer, not only the functions of the above-described embodiments are realized, but also an OS (basic system or operating system) running on the computer based on the instruction of the program code. Needless to say, a case where the functions of the above-described embodiment are realized by performing part or all of the actual processing and the processing is included.

  Further, after the program code read from the storage medium is written in a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer, the function is determined based on the instruction of the program code. It goes without saying that the CPU or the like provided in the expansion board or function expansion unit performs part or all of the actual processing and the functions of the above-described embodiments are realized by the processing.

1 is a block diagram showing a configuration of a remote control system according to a first embodiment of the present invention. It is a figure explaining transfer and display of display data (GUI data) for remote control of a remote control system, and the procedure of remote control paying attention to communication on a 1394 network. It is a figure for demonstrating the control command for remote control. It is a figure for demonstrating the control command for remote control. It is a figure for demonstrating the control command for remote control. It is a figure for demonstrating the control command for remote control. It is a figure for demonstrating the control command for remote control. It is a figure for demonstrating the control command for remote control. It is a figure for demonstrating the element of operation information, its essential data, and selection data. It is a figure for demonstrating the user action which the element of operation information can take. It is the figure which showed the data structure of display data. It is the figure which showed the data structure of display data. It is the figure which showed the data structure of display data. It is the figure which showed the data structure of display data. It is the figure which showed the data structure of display data. It is the figure which showed the data structure of display data. It is the figure which showed the data structure of display data. It is the figure which showed the data structure of display data. It is the figure which showed the data structure of display data. It is the figure which showed the data structure of display data. It is the figure which showed the data structure of display data. It is the figure which showed the data structure of display data. It is a figure for demonstrating the type and code which are stored in display data. It is a figure for demonstrating the type and code which are stored in display data. It is a figure for demonstrating the type and code which are stored in display data. It is a figure for demonstrating the type and code which are stored in display data. It is the figure which showed the example of the tree structure of display data. It is the figure which showed the actual data example of the panel element among display data. It is the figure which showed the example of a display of the display data for remote control. It is an external view of the remote control of a remote control device. It is a figure for demonstrating the designation content of the focus navigation of display data. It is the block diagram which showed the structure of the remote control system which concerns on the 2nd Embodiment of this invention. It is a figure for demonstrating the essential data and selection data of a display element. It is the figure which showed the data structure of the selection data of a display element. It is the figure which showed the data structure of the selection data of a display element. It is the figure which showed the data structure of the selection data of a display element. It is the figure which showed the data structure of the movement destination designation | designated information of focus navigation. It is a figure for demonstrating a function code. It is the figure which showed the example of a display of display data. It is a figure for demonstrating the procedure of remote control. It is a figure for demonstrating the designation | designated content of focus navigation. It is a figure for demonstrating the designation | designated content of focus navigation. It is a figure for demonstrating the designation | designated content of focus navigation. It is a figure for demonstrating the designation | designated content of focus navigation. It is the block diagram which showed the structure of the remote control system in a prior art example. It is the figure which showed the example of a screen display of the remote control system in a prior art example. It is the figure which showed the remote control in a prior art example. It is a figure for demonstrating the designation | designated content of the focus navigation of the remote control system in a prior art example.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Network 100 Digital television 101, 111 1394 I / F part 102, 112 Control part 103 Panel data generator 104 DVC main-body part 110 Digital television 113 Decoder 114 Display data memory 115 Drawing control part 116 Remote control part 117 Image composition part 118 Display Device 119 Audio output unit 120 Remote control

Claims (16)

Communication means for receiving, from the controlled device, operation information including one or more operation elements including display image information;
Display means for displaying an image corresponding to each operation element based on display image information of each operation element;
The communication unit transmits control information indicating an operation element selected and determined from the one or more operation elements by a user operation input to the controlled device, and the display unit transmits the operation element from the operation element to the operation element. A control device characterized in that selection candidate information is moved and displayed on another predetermined operation element.   The control apparatus according to claim 1, wherein the operation information includes movement destination information that defines the other operation element as a movement destination from the operation element.   The operation information defines the other operation element as a movement destination from the operation element by associating an identifier of the operation element with an identifier of the other operation element. The control device described in 1.   The identifier of the other operation element includes an identifier corresponding to a user operation content from a state in which the selected and determined operation element is selected, and the operation information includes the other information corresponding to the user operation content. The control device according to claim 3, wherein an identifier of the operation element is associated with an identifier of the operation element selected and determined.   5. The control device according to claim 3, wherein the operation element identification information and the other operation element identification information are included in the operation element. Communication means for transmitting operation information including one or more operation elements including display image information to the control device;
Control means for controlling the controlled device based on control information transmitted from the control device,
The communication means displays an image corresponding to each operation element based on display image information of each operation element on the control device, and an operation selected and determined from the one or more operation elements by a user operation input A controlled device that receives the control information indicating an element from the control device.   The controlled apparatus according to claim 6, wherein the operation information includes movement destination information that defines the other operation element as a movement destination of the selection candidate information from the operation element on the control apparatus.   8. The operation information defines the other operation element as a movement destination from the operation element by associating an identifier of the operation element with an identifier of the other operation element. The controlled device according to 1.   The identifier of the other operation element includes an identifier corresponding to a user operation content from a state in which the selected and determined operation element is selected, and the operation information includes the other information corresponding to the user operation content. The controlled device according to claim 8, wherein an identifier of the operation element is associated with an identifier of the operation element selected and determined.   The controlled device according to claim 8 or 9, wherein the operation element identification information and the other operation element identification information are included in the operation element. A remote control system in which at least a control device and a controlled device are connected via a network,
The controlled device is:
Transmitting operation information including one or more operation elements including display image information to the control device;
The control device includes:
The operation information is received from the controlled device, an image corresponding to each operation element is displayed based on display image information of each operation element, and is selected and determined from the one or more operation elements by a user operation input. Control information indicating the operated element is transmitted to the controlled device, and the selection candidate information is moved and displayed from the operating element to another predetermined operating element for the operating element,
The controlled device is:
A remote control system that controls the controlled device based on the control information received from the control device. Communication means for receiving operation information including one or more operation elements including display image information from a controlled device; display means for displaying an image corresponding to each operation element based on display image information of each operation element; A control method for a control device comprising:
Control information indicating an operation element selected and determined from the one or more operation elements by a user's operation input is transmitted to the controlled device by the communication unit, and the operation element transmits the control element from the operation element to the control element. A control method for a control apparatus, wherein selection candidate information is moved and displayed on another predetermined operation element. Communication means for transmitting operation information including one or more operation elements including display image information to the control apparatus, and control means for controlling the controlled apparatus based on the control information transmitted from the control apparatus. A method for controlling a controlled device, comprising:
The control indicating an operation element selected and determined from the one or more operation elements by a user's operation input, wherein an image corresponding to each operation element is displayed on the control device based on display image information of each operation element. Information is received from the control device by the communication means, and the control means controls the controlled device based on the control information.   The program for making a computer perform the control method of the control apparatus of Claim 12.   A program for causing a computer to execute the method for controlling a controlled device according to claim 13.   The computer-readable recording medium which recorded the program of Claim 14 or 15.

Images