CN101019090A - User interface for multi-device control - Google Patents

Abstract

A control server or similar central processor manages the distribution of data (including audio and video), voice, and control signals among a plurality of system components (102-120) connected via a wired and/or wireless communication network (180). The system components (102-120) include audio/visual components (such as televisions, monitors, PDAs, notebooks, MP3, portable stereo, etc.), as well as household appliances (such as lighting, ovens, alarm clocks, etc.). The portable controller (108) allows a user to access and control the system components (102-120) from any location within the controlled residential and/or non-residential environment (100), including its perimeter areas. The portable controller (108) includes a user and management interface (e.g., 200, 600) containing customized control screens (e.g., 204, 700) for system components. The control screens are stored in a hierarchical architecture that is selectable by region (e.g., 200) or component (e.g., 600) type.

Description

User interface for multi-device control

technical field

The present invention relates generally to user interfaces, and more particularly to user interfaces for use with remote control units comprising a plurality of consumer electronic devices.

Background technique

The user interface provides increased flexibility and versatility for the remote control unit, and can be used to control consumer electronics (CE) devices, such as televisions, digital video equipment (DVD) players, and compact disc (CD) players. Most remote control units include a fixed "hard key" format that cannot be easily reconfigured to support multiple CE devices.

Especially when multiple devices must be programmed to support a single output, even a remote control unit that can be programmed to support multiple devices can be intimidating, confusing and difficult for the average user to operate. For example, a device such as a video output device (eg, a television) may accept input from different sources. However, the available input sources may not be readily apparent to the user. Additionally, when configuring a multifunction remote control unit to instruct multiple devices, users often find it difficult to identify the correct code for teaching the control unit.

The issue of device management and control begins to present additional complexities in environments that include not only multiple devices, but multiple device system groups.

What is needed are techniques for allowing a user interface to manage multiple devices in a clear, intuitive and simple manner.

Contents of the invention

Methods, systems, and computer program products are provided for managing multiple devices and/or applications within a controlled environment, such as a home, business, school, etc., and its surrounding area. A control center includes one or more servers or other data processing devices and allows centralized command and control of devices and/or applications.

In an embodiment of the invention, a portable controller (such as a personal digital assistant, wireless laptop, etc.) allows the user to interact with the control center. Such interactions include changing the configuration and performance of other devices and/or applications. Thus, the portable controller provides remote access to other devices and/or applications, as well as allowing a user to control their function and/or operation from anywhere within the environment.

In one embodiment, the portable controller is a handheld platform with a graphical display that communicates with other devices within the controlled environment via a wireless network (such as specified by IEEE Standards 802.11a, 802.11b, 802.11g, etc.). Wireless connectivity for control centers of devices and/or applications. In another embodiment, the control center is built into the portable controller. In another embodiment, the portable controller communicates directly with other devices and/or applications via infrared (IR) encoded signals.

According to an embodiment of the present invention, a portable controller facilitates control of a computer having an output component (eg, television, monitor, etc.) with multiple input components (eg, DVD, VCR, satellite tuner, digital video recorder (DVR), etc.) system. According to the present invention, the user first selects an output component via a graphical user interface (GUI) presented on the portable controller. He or she is then presented with a control screen providing the user with the ability to select specific input components using a "marked" interface. When that input is selected, a control screen for that system component is presented.

Description of drawings

The accompanying drawings, which are incorporated in and form a part of this specification, illustrate the invention and, together with the description, serve to further explain the principles of the invention and to allow those skilled in the art to make and use the invention. In the drawings, generally, identical symbols indicate identical or functionally or structurally similar components. Also, generally, the left-most digit of a reference number identifies the figure in which the reference number first appears.

FIG. 1 illustrates a home controlled system according to an embodiment of the present invention.

Figure 2 illustrates a hierarchical control interface according to an embodiment of the present invention.

Figure 3 illustrates a room control screen according to an embodiment of the present invention.

Figure 4 illustrates a component control screen according to an embodiment of the present invention.

Figure 5 illustrates a component control screen according to another embodiment of the present invention.

Figure 6 illustrates a component-type hierarchical control interface according to an embodiment of the present invention.

Figure 7 illustrates a lighting control screen according to an embodiment of the present invention.

FIG. 8 illustrates a flowchart for displaying a control interface based on a room hierarchy according to an embodiment of the invention.

FIG. 9 illustrates a flowchart for displaying a control interface based on a component type hierarchy according to an embodiment of the present invention.

Figure 10 illustrates a definition control screen according to an embodiment of the present invention.

Figure 11 illustrates a definition control screen according to another embodiment of the present invention.

Figure 12 illustrates a tuner definition control screen according to an embodiment of the present invention.

Figure 13 illustrates an audio-video component definition control screen according to an embodiment of the present invention.

Figure 14 is an exemplary computer system for implementing the present invention.

Detailed ways

The present invention relates to centralized command and control of multiple devices and/or applications within a controlled environment, such as a residence, business, school, and the like. A residential controlled environment includes the confines of a home, room, mobile home, yacht, or other type of dwelling. In an embodiment, however, the residential environment includes the surrounding area of the residence, as well as any shelters, buildings, improvements, etc. within the specified perimeter.

In other embodiments, the invention is implemented in a non-residential environment. Non-residential environments include, but are not limited to, office complexes, small offices, studios, warehouses, entertainment venues, schools or universities, health care facilities, hotels, resorts, airplanes, ships, automobiles, and the like. In an embodiment, the controlled environment for non-residential embodiments includes not only the physical boundaries of the aforementioned structures, but also their environment within a designated perimeter.

Examples of controlled embodiments are found in applications entitled "Method, System, and Computer Program Product for Managing Controlled Residential or Non-Residential Environments" (U.S. Patent Application Serial No. 10/382,897), and applications entitled "Method, System, and Computer Program Produce for Managing Controlled Residential or Non-Residential Environment" (U.S. Patent Application Serial No. 10/180,500), which is hereby incorporated by reference in its entirety. As described in these applications, various methods and systems can be provided to manage the distribution of information (including video, audio, voice, text, graphics, control messages, etc.) to other devices and/or applications within a controlled environment. These devices and/or applications include, but are not limited to, communication devices (such as telephones, walkie-talkies, etc.), entertainment systems (such as televisions, CD/DVD players, gaming applications, stereos, etc.), monitoring systems (such as security cameras, baby monitoring appliances, etc.), security/security systems (such as fire alarms, automatic sprinklers, locks on doors or windows, etc.), personal computers (such as desktops, laptops, personal digital assistants, etc.), cooking appliances (such as ovens, coffee kettles, electric food/beverage heaters, etc.), comfort equipment (such as heating and air conditioning (HVAC), humidifiers, dehumidifiers, air purifiers, light switches, dimmers, etc.), power outlets, power supplies, etc. wait.

Examples of these controlled environments are shown in Figure 1, illustrating a home control system 100 according to an embodiment of the present invention. System 100 includes a communication network 180 that interconnects various system components. The system components include a positioning unit 102, two televisions 104 (shown as televisions 104(a) and 104(b)), two computer clients (shown as computer clients 106(a) and 106(b)), a portable control 108, lighting 110, thermostat 112 for the HVAC system, tuner 114, media player 116, cable box 118, and DSS box 120. Other devices and/or applications can also be included as system components.

The location unit 102 specifies a spatial location within a home serving as the host environment for the system 100 . The positioning unit 102 is coupled to other system components (eg, the portable controller 108 ) via wired and/or wireless interfaces. The location unit 102 is used to specify a floor or a room in a residence. The location unit 102 is also used to designate a specific location or area within a floor or room. In addition, the location unit 102 can also be located outside the home, thereby specifying an area outside the home.

Computer clients 106 include wired and/or wireless personal computers, personal digital assistants (PDAs), enhanced telephones, personal televisions, or other data processing devices linked to communication network 180 . As a personal computer, personal client 106 may be a desktop, notebook, laptop, or the like.

The portable controller 108 is a wired and/or wireless data controller that allows a user to interact with, send control messages to, and/or manage the distribution of information (including audio, video, voice, and other data) among other system components. Handle equipment. Portable controller 108 may be a portable type of device listed as computer client 106 . For example, portable controller 108 may be a personal notebook or notebook computer, PDA, enhanced telephone, or other device linked to communication network 180 and including a display with the ability to interact with other system components. Thus, the portable controller 108 allows a user to operate the various components of the system 100 remotely. In an embodiment, the display for the portable controller 108 can receive video and/or audio from other system components. In one embodiment, the portable controller 108 includes a flash ROM that allows wireless downloads and/or uploads.

Television 104 is a conventional television. In an embodiment, television 104 is enhanced to support interactive and/or personal services. Personal services include virtual recording, programming, pause/rewind live broadcasts, and more. For example, television 104 may be enhanced to support personal televisions provided by WebTV Networks, Inc. (Mountain View, CA) that support the MSN(R) TV service available from Microsoft Corporation (Redmond, WA) WEBTV(R) service. As shown, the television 104 can be connected to a cable set-top box 118, a DSS set-top box 120, and/or a media player 116 (eg, a PVR, VCR, or DVD player).

The system components described above are not intended to be exhaustive. Other devices (including appliances), applications, and the like can be implemented, including but not limited to refrigerators, stoves, microwaves, ovens, coffee makers, alarm clocks, humidifiers, sprinkler systems, lighting, dimmers, and the like. In an embodiment, the portable controller 108 controls the operation and/or functions of these components, such as on/off, timers, modulation (eg, oven temperature, etc.), pause, snooze, and the like.

As noted, communication network 180 provides the transmission medium for communicating among system components. Communication network 180 is a wired and/or wireless local area network (LAN). Accordingly, communication network 180 includes wired, wireless, or both transmission media, including satellite, terrestrial (e.g., fiber optic, copper, UTP, STP, coaxial, hybrid fiber coaxial (HFC), etc.), radio, microwave, free space optical system and/or other form or method of transmission.

In an embodiment, the server controls all communication traffic among other system components. Likewise, the exchange of information among system components is routed or controlled via the server. In another embodiment, communication network 180 supports peer-to-peer communication. Likewise, system components exchange audio, video, voice, other data and/or control messages directly with each other without central management by a server.

In an embodiment using a wired transmission medium, the communication network 180 is an Ethernet LAN with CAT-5 cables or the like distributed to each system component, such as the television 104, media players, and the like. The system component includes an audio/video (AV) connector responsive to the receiving cable. In an embodiment, the communication network 180 includes telephone lines and/or power lines (such as communication technologies available from the Home Phone Networking Alliance (HomePNA) or the like).

In embodiments using a wireless transmission medium, communication network 180 supports IEEE Standard 802.11(a), which specifies a wireless Ethernet protocol for large video. In another wireless embodiment, the communication network 180 supports IEEE 802.11(b), which specifies a wireless Ethernet protocol for small video. In another wireless embodiment, the communication network 180 supports IEEE standard 802.11(g). In another embodiment, short-range wireless interfacing with system 100 is supported using Bluetooth ^™ wireless technology (developed by Bluetooth SIG, Inc.).

As shown in FIG. 1, communication network 180 includes a wireless network access point 182, such as available from Linksys Group Inc. (Irvine, CA) or Cisco Systems, Inc. (San Jose, CA), as a wireless network access point 182 for system components. platform. Wireless access point 182 provides a central point for connectivity in the wireless network and long-range connectivity needed to track the status of system components. Additionally, wireless access point 182 can provide a connection point between wired and wireless networks. In embodiments using a server, as described above, the wireless access point 182 serves as the platform for the server. In another server-based embodiment of the invention, in addition to a wireless access point (such as wireless access point 182 shown at 180), the server can actually reside on a number of different device platforms, including bridge devices such as bridges 130(a)-(e)), personal digital assistants (such as 108), personal computers (such as 106), and the like.

System 100 also includes a plurality of infrared/serial bridges 130(a)-(d), which conform to the IEEE 802.11(b) standard for wireless communications. Each infrared/serial bridge 130 interacts with one or more components. As shown, infrared/serial bridge 130( a ) interfaces with television 104( a ), cable box 118 , and media player 116 . The infrared/serial bridge 130(b) interfaces with the tuner 114 or any type of dedicated device that relies on an infrared/serial communication protocol as will be apparent to those skilled in the relevant art. Infrared/serial bridge 130(c) interfaces with thermostat 112. Infrared/serial bridge 130 ( d ) interacts with lighting device 110 . Wireless Ethernet bridge 130(e) interacts with television 104(b) and DSS box 120. Wireless Ethernet bridge 130(e) can support any IP addressable device. Likewise, television 104(b) and DSS box 120 are "next generation" UPnP devices with IP addresses.

Thus, the present invention can incorporate legacy devices such as consumer electronics (CE) devices relying on infrared/serial communication protocols, as well as UPNP ^™ devices and applications defined by the Universal Plug and Play (UPnP) Forum as system components. Implementations of IEEE802.11(b) infrared/serial bridges are described in an application entitled "Legacy DeviceBridge for Residential or Non-Residential Networks" (US Patent Application Serial No. 10/387,590, filed March 14, 2003) An example of a controlled environment is incorporated herein by reference.

As noted above, a portable controller 108 (such as a digital personal assistant, wireless notebook, etc.) allows a user to remotely alter the configuration and performance of other devices and/or applications from anywhere within the controlled environment. In an embodiment, the portable controller 108 is a handheld platform with a graphical display that communicates via a wireless communication network 180 (such as specified by IEEE Standard 802.11b) to control other devices and/or applications within the controlled environment. wireless connection to the central server. In another embodiment, the server is built into the portable controller 108 . In another embodiment, the portable controller 108 communicates directly with other devices and/or applications via infrared (IR) encoded signals.

In an embodiment, the present invention facilitates the control of a computer that includes output system components (such as televisions, monitors, speakers, etc.) with multiple input system components (such as DVDs, VCRs, satellite tuners, digital video recorders, stereos, etc.) system. In accordance with the present invention, the user first selects an output component via a GUI displayed on the portable controller 108 . The user is then presented with a control screen, providing the user with the ability to select specific input components using a "marked" interface. When that input is selected, the control screen for that component is presented.

FIG. 2 illustrates a hierarchical control interface 200 according to an embodiment of the invention. The control interface 200 is based on a hierarchical representation of the controlled environment (eg, a user's home), with basic interface sub-levels corresponding to the rooms in the controlled environment, and then the system components in each room. As suggested by interface level identification field 202, control interface 200 allows a user to select a room to control system components located within that room.

Control interface 200 includes several control objects related to a series of executable commands. Control objects include a home object 210, an inter-level navigation object 212, intra-level navigation objects 208a-208b, and a number of room control objects 206a-206n.

When activated, the family object 210 provides the user with a global or "family-wide" control screen, such as the control interface 600 shown and described below with reference to FIG. 6 . Referring again to FIG. 2, the control screen 204 is the main control screen presented to the user and allows the user to view a plurality of room control objects 206a-206n corresponding to rooms in the controlled environment. Each room control object 206a-206n corresponds to a room and is associated with an executable command to retrieve a base interface child to display a screen identifying controllable system components located within that room. By activating one of the room control objects 206a-206n, the user can thereby select a particular room in order to control the system components contained in that room.

Intra-level navigation objects 208a-208b are activated to scroll screen 204 horizontally. If not all room control objects 206a-206n can be seen in the current view, the navigation objects 208a-208b scroll the screen 204 so that the invisible control objects 206a-206n can be seen.

Interlevel navigation objects 212 allow a user to move from one interface sublevel to the next. For example, if a user activates room control object 206 to access a lower level view, the user may activate interlevel navigation object 212 to return to the next higher level.

As described above, when a room control object 206a-206n is activated, a control screen for the corresponding room is presented. FIG. 3 illustrates an embodiment of a room control screen 300 presented by the portable controller 108 upon activation of the room control objects 206a-206n. The room control screen 300 provides control options for controlling the family room, as shown in the interface level identification field 302 .

The room control screen 300 includes a video control object 304a, an audio control object 304b, a lighting control object 304c, and an environmental control object 304d. Video control objects 304a relate to executable commands for controlling the video system in the room. Audio control objects 304b relate to executable commands for controlling the audio system in the room. Lighting control objects 304c relate to executable commands for controlling room lighting. Environmental control objects 304d relate to executable commands for controlling the HVAC system in the room. Additional control objects can be included to control other types of devices and/or applications within a particular room.

As mentioned above, the inter-level navigation object 212 allows the user to switch to the next higher interface level. For example, the next higher level for control screen 300 is top level control screen 200 .

Switch objects 306 relate to executable commands that turn on or off all system components in the room. Thus, all system components in the room can be turned on or off by clicking, or more specifically, by activating one control object (ie, switch object 306).

As described above, when a component control object, such as control objects 304a-304d, is activated, a control screen for the corresponding system component is presented. FIG. 4 illustrates a component control screen 400 according to an embodiment of the invention. Specifically, when the user activates video control object 304a from FIG. 3, component control screen 400 is presented. Component control screen 400 displays a control panel for controlling a television that is a parent or primary system component for a video system in a room. The parent icon 402 is highlighted to designate the system component as the parent system component. The parent icon 402 can be highlighted by being displayed with a brighter intensity or a specified color, such as green. If the system component is a child or subordinate system component, the parent icon 402 is not highlighted.

The control panel for component control screen 400 includes numeric channel keypad 416 , keypad entry activator 420 , on-screen menu display activator 406 and on-screen menu navigator/selector 414 . Control screen 400 also includes several specialized control objects, such as favorite channel object 418 , display object 408 , electronic guide object 410 , and previous channel object 412 .

A switch object 404 is associated with an executable command to turn on or off a TV/parent system component. Interlevel navigation object 212 allows the user to switch to the next higher interface level. For example, the next higher level for component control screen 400 is room control screen 300 .

The component control screen 400 also includes a side tab 422 in the form of a scrollable field that diagrammatically represents a subsystem component connected to a parent system component. Subsystem components are represented by subcomponent control objects 426a-426n. Figure 4 shows three subcomponent control objects, a DVD control object 426a for controlling a DVD player, a VHS control object 426b for controlling a VHS player, and a VHS control object 426b for controlling a PVR player, such as those available from TiVo Inc. (USA ) obtained TIVO(R) digital video recorder (DVR) PVR control object 426c. By activating the upper navigation object 424a and the lower navigation object 424b, and scrolling the side tab 422 up or down, other sub-component control objects 426a-426n can be displayed.

In one embodiment, component control screen 400 is customized to fit portable controller 108 . For example, the screen shown in Figure 4 can be used with the PhilipsIPRONTO(R) available from Philips Consumer Electronics which has hard buttons (not shown) for some basic functions, such as volume. Likewise, the control screen of the present invention may include a combination of soft and hard keys for specifying functions, such as volume.

As described above, when a subcomponent control object 426a-426n is activated, a control screen for the corresponding system component is presented. FIG. 5 illustrates a component control screen 500 resulting from activating subcomponent control objects 426a-426n, in accordance with an embodiment of the present invention. The component control screen 500 displays control options for the DVD player. For example, if the user wants to watch a movie that is playing on a DVD player, the user can activate the DVD player object 426a, and the DVD component control screen 500 is displayed. The DVD control object 426a is highlighted to inform the user that a control screen for the DVD player has been selected. Other subcomponent control objects 426a-426n are not highlighted.

The DVD component control screen 500 displays options for controlling functions of the DVD player. The options include rewind object 510 , skip back object 512 , skip forward object 522 , fast forward object 524 , play object 528 , slow play object 518 , pause object 516 , and stop object 520 . Chapter navigator 508 allows a user to select a particular chapter in a digital recording. The DVD component control screen 500 also includes an on-screen menu display activator 506 and an on-screen menu navigator/selector 514 .

The parent icon 402 is not highlighted on the control screen 500 because the system components controlled by the control screen 500 are subsystem components. Switch object 504 is associated with an executable command to turn on or off the DVD player. Interlevel navigation object 212 allows the user to switch to the next higher interface level. For example, the next higher level for the child component control screen 500 is the parent component control screen 400 .

As noted above, the user interface for the portable controller 108 can be organized in a hierarchical layout. However, the layout is not limited to the house-room hierarchy. In one embodiment, the user interface organizes system components by component type for ease of control. FIG. 6 illustrates an embodiment of a control interface 600 that controls hierarchies based on component types. The control interface 600 provides options for aggregating controls for all system components that match the particular component type shown in the interface-level identification field 602 . Base interface sub-levels correspond to the types of system components located throughout the controlled environment. Control interface 600 includes six component type control objects 604a-604f that represent six types of system components within the controlled environment. In particular, all available lighting systems are linked and composed into a lighting component type control object 604a. All environment/comfort systems are linked and composed into an environment component type control object 604b. Available audio components are grouped and linked to audio component type control objects 604c-604d. Alarm components for a security/security system are linked and constitute a security/security component type control object 604e. Finally, a video component for a security/security system is assembled and linked to a security/security component type control object 604f.

Switch objects 606 relate to executable commands that turn on or off all system components that match a particular component type. For example, if a lighting component type control object 604a is selected, a switch object 606 can be activated to turn on or off the lighting system linked to the control object 604a. Control object 604a can be "selected" by placing a cursor over control object 604a using a mouse or similar input means, and right-clicking or left-clicking the mouse once. A double left click can be used to "activate" the control object 604a. If using a touch screen, a single tap can be used to select a control object and a double tap can be used to activate an object. It is also possible to use the pull-down menu to distinguish between "selection" and "activation" of control objects.

Activation of a component type control object 604a-604f sends a command to present the next base interface sub-level, which identifies all available system components that match the corresponding component type. Figure 7 illustrates an embodiment of a lighting control screen 700 displaying all controllable lighting in a controlled environment home. A number of lighting control objects 706a-706n are associated with lighting systems in each room of the controlled environment. If more than one lighting system (eg, lamp, dome light, etc.) is located in the room, activation of the lighting control objects 706a-706n invokes a sub-screen displaying the various lighting systems in the corresponding room. In an embodiment, activating a lighting object 706a-706n (eg, by double-tapping the object or double-tapping) invokes a pre-screen displaying available control options for the lighting system. Control options include dimming, timing controls, and more.

The switch object 708 is associated with an executable command to turn on or off the lighting system associated with the lighting control objects 706a-706n. In an embodiment, the user will operate the input components to select one or more lighting control objects 706a-706n, and then select the switch object 708 to operate the associated lighting system. In an embodiment, a user can turn on or off, for example, all lights in a house through a switch object 708 that operates as a house-wide "all lights off" or "all lights on" switch.

Activating the in-level navigation objects 208a-208b scrolls the screen 704 to make invisible lighting control objects 706a-706n viewable. Interlevel navigation object 212 allows the user to switch to the next higher interface level. For example, the next higher level for lighting control screen 700 is component type hierarchical control interface 600 .

In an embodiment, a lighting control object 706a-706n (eg, intensity, color, etc.) is highlighted to indicate that the lighting system is on or off. As such, the control interface of the present invention provides the user with a convenient visual representation of which system components (eg, lights, televisions, security cameras, ovens, etc.) are on/off in a controlled environment. In addition, depending on the number of control objects, visual representations can be provided on a single screen or a single scrollable screen. This provides the user with the ability to control these components using one home screen, rather than requiring the user to navigate through several room control screens to control lighting on a room-by-room basis. For example, in Figure 7, the lights in the family room and the garage lights are shown as "on". The user can turn off the lights in those rooms by deactivating the corresponding lighting control object 706a-706n, or turn on the lights in any other room of the house by activating the lighting control object 706a-706n corresponding to that room.

The present invention supports various protocols for displaying a hierarchical control interface for controlling multiple system components within a controlled environment. Referring to FIG. 8, a flowchart 800 represents the general operational flow of an embodiment for displaying a control interface. More specifically, flowchart 800 represents an example of a control flow for presenting a room hierarchy based hierarchical control interface.

Referring to FIG. 8 , the control flow of flowchart 800 begins at step 801 and passes to step 803 immediately. At step 803, the control screen is accessed and presented for display. The control screen displays all available rooms within the controlled environment. An example of a room-based hierarchical control screen was described above with reference to the control screen 200 in FIG. 2 . The control screen 200 displays available rooms as control objects 206a-206n. However, available rooms may also be presented as a hypertext-based list. Each room specified in the list or associated with a control object 206a-206n is linked to a base screen that identifies controllable system components within the room.

At step 806, a room control screen is accessed and presented to display one or more system components located within the selected room. In response to activating the corresponding control object 206a-206n, etc., the room control screen is accessed, as described in step 803. An example of a room control screen is described above with reference to the room control screen 300 of FIG. 3 . The room control screen displays all available system components as component control objects 304a-304d or in a hypertext based list, both linked to the base screen. The basic screen identifies the control options for the relevant system components.

At step 809, a component control screen is accessed and presented to display control options for the relevant system component. As noted, the component control screens are accessed in response to activating the corresponding component control objects 304a-304d, etc. FIG. The component control screen allows the user to select various control options to manage the operation and/or function of the system components. Examples of component control screens are described above with reference to component control screens 400 and 500 in FIGS. 4 and 5 .

At step 812, the system component associated with the component control screen is checked to determine if it is a parent system component. If so, control passes to step 815. Otherwise, control passes to step 821.

At step 815, all subordinate or subsystem components are displayed on the same component control screen to provide the user with the opportunity to control additional system components. For example, if the system component selected and controlled at step 809 is a parent system component (eg, a television), then at step 812, the user can control and/or set a subordinate system component (eg, a DVD player).

At step 818, if the affiliation system component is available and desired to be controlled, control flow returns to step 809, where a component control screen for the selected affiliation system component is presented for display. The subcomponent control screen is accessed in response to activating the corresponding subordinate component control object 426a-426n, etc. The operation and/or function of the subordinate system components are managed via the control screen. As described above with reference to FIGS. 4 and 5 , the component control screen 500 presents a control interface for controlling subordinate system components of a parent system component controlled by the component control screen 400 .

On the other hand, if no subordinate system component is selected for control, control flow passes to step 821 .

In step 821, specific control options are implemented to control the associated system components. For example, if the parent and child system components have been programmed to play DVD recordings, the associated commands are executed to send control signals to the various system components, such as TV/parent and DVD player/child. After the control interface has been set up and executed, as shown in step 895, control flow ends.

Referring to FIG. 9, a flowchart 900 represents the general operational flow of another embodiment for displaying a control interface. More specifically, flowchart 900 represents an example of a control flow for rendering a hierarchical control interface based on a component-type control hierarchy.

Referring to FIG. 9 , the control flow of flowchart 900 begins at step 901 and immediately passes to step 903 . At step 903, the control screen is accessed and presented for display. The control screen displays all available component types available within the controlled environment. An example of a hierarchical control screen based on component type is described above with reference to control interface 600 in FIG. 6 . Control interface 600 displays available component types as control objects 604a-604f. However, it is also possible to present the available component types as a hypertext-based list. Each component type (specified in a list, control object, etc.) is linked to a base screen that identifies the controllable system components associated with each component type.

At step 906, a component type dashboard is accessed and presented to display system components matching the selected component type by location (eg, room). In response to activating the corresponding control object 604a-604f, etc., the component type control screen is accessed, as described in step 903. An example of the part type control screen is described above with reference to the part type control screen 700 in FIG. 7 . The component type control screen displays all available system components as component control objects 706a-706n or in a hypertext based list, both linked to the base screen. The basic screen identifies the control options for the relevant system components.

At step 909, a component control screen is accessed and presented to display control options for the relevant system component. As noted, the component control screen is accessed in response to activating the corresponding component control object 706a-706n, etc. The component control screen allows the user to select various control options to manage the operation and/or function of the system components. The component control screens are described above with reference to the component control screens 400 and 500 in FIGS. 4 and 5 .

At step 912, the system component associated with the component control screen is checked to determine if it is a parent system component. If so, control passes to step 915. Otherwise, control passes to step 921.

At step 915, all subordinate system components are displayed on the same component control screen to provide the user with the opportunity to control additional system components. For example, if the system component selected and controlled at step 909 is a parent system component (eg, a television), then at step 912, the user can control and/or set subordinate or subsystem components (eg, a DVD player).

At step 918, if the subordinate system component is available and desired to be controlled, control flow returns to step 909, where a component control screen for the selected sub or subordinate system component is presented for display. In response to activating the corresponding subcomponent control object 426a-426n, etc., the subcomponent control screen is accessed. The operation and/or functions of the subsystem components are managed via the control screen. As described above with reference to FIGS. 4 and 5 , the component control screen 500 presents a control interface for controlling subordinate system components of a parent system component controlled by the component control screen 400 .

On the other hand, if no subordinate system component is selected for control, the flow of control passes to step 921 .

At step 921, specific control options are implemented to control the associated system components. For example, if the parent and child system components have been programmed to play DVD recordings, the associated commands are executed to send control signals to the various system components, such as TV/parent and DVD player/child. After the control interface has been set up and executed, as shown in step 995, the control flow ends.

The present invention includes various control screens that allow a user to add and/or configure system components for a controlled environment. In an embodiment, a setup wizard is provided to allow a user to add or configure system components. Figure 10 illustrates a definition control screen 1000 according to an embodiment of the invention. The definition control screen 100 may be included as part of a setup wizard for adding and/or configuring system components, as shown in definition field 1002 . Control screen 1000 includes a number of control fields that are populated by the user. As shown, the control fields include a type field 1004 , manufacturer field 1006 , model field 1008 , room field 1010 , network extender field 1012 , display field 1014 , exclude field 1016 , and name field 1018 .

Type field 1004 specifies the component type. As described above, component type hierarchical control displays such as control interface 600 and control screen 700 organize system components within a controlled environment by component type, allowing a user to control all system components that match a particular component type from the same screen . Type field 1004 allows the user to specify component types that support the above-described hierarchical control user interface based on component type. An exemplary component type listing is shown in FIG. 11 illustrating another embodiment of a control screen 1000 . In FIG. 11 , the Type field 1004 indicates that the component type can be Amplifier, Audio, Cable Box, DSS Box, DVD Player, Lighting System, Thermostat, PVR Device, or Television. The list in type field 1004 is not exhaustive. Other system component types can be specified and included in the present invention, including but not limited to video servers, messaging systems, security or perimeter cameras, residential equipment, and the like.

Referring again to FIG. 10, the manufacturer field 1006 allows the user to specify the manufacturer of the system component, and the model field 1008 allows the user to specify the model number of the system component. In one embodiment, one or more software modules are associated with the control interface of the present invention, and look up in a lookup table, database, etc., the IR codes that are sent to control the system components. The information specified in the manufacturer field 1006 and the model field 1008 is used by the software module to query a database, etc.

Room field 1010 specifies the room or area of the controlled environment in which the system components are placed. The information in room field 1010 is used to support the room-based hierarchical control displays (eg, control interface 200 and control screen 300 ) of the present invention. Although this information is initially specified by the user via the setting wizard, the content of the room field 1010 can also be initially set or dynamically updated by the positioning unit 102 . In applications entitled "Method, System, and Computer Program Product for Managing Controlled Residential or Non-Residential Environments" (U.S. Patent Application Serial No. 10/382,897), and entitled "Method, System, and Computer Program Produce for Managing Controlled Residential or Non-Residential Environments" An example of dynamically tracking the location of system components within a controlled environment is described in more detail in "Residential Environment" (U.S. Patent Application Serial No. 10/180,500).

A name field 1018 is included to allow the user to customize the name of the system component. Users can specify personalized descriptors for easy recall.

The definition control screen 1000 also includes a network extender field 1012 to allow the user to associate system components with the appropriate bridges. As described with reference to FIG. 1 , an infrared/serial bridge 130 can be coupled to system components to support wireless communication with other system components. As noted, system 100 may have multiple infrared/serial bridges 130(a)-130(e) (eg, one per floor or one per room).

Display field 1014 allows a user to include and/or configure one or more "hidden" subordinate system components that are linked to a parent system component. Hidden system components (such as amplifiers) are configured into system 100, but the control screens of the present invention (e.g., control screen 400) can be configured to not present hidden affiliations to the user because hidden affiliations are invisible to the user. operate invisibly. However, link hidden subordinate system parts to one or more other parent system parts. For example, the amplifier can be linked to a TV, DVD player and CD player. When any of these parent system components is selected via screen 100, the amplifier will be automatically activated. Automatically redirect eg volume controls established for these parent system components to hidden amplifiers. Accordingly, display field 1014 provides the user with the ability to select whether to view the affiliated system components on the component control screen (eg, side tab 422 on control screen 400). In one embodiment, if the value of the display field 1014 is "true", the affiliation system component is displayed, and if the value of the display field 1014 is "false", the affiliation system component remains hidden from being displayed.

The definition control screen 1000 also includes an exclusion field 1016 that allows the user to exclude specific system components from the "full close" or "full open" (room or home) commands. As described above, for example, switch objects 306, 606, and 708 can To be executed to turn off or turn on all system components (or all system components matching a particular component type) within a specific area, or within the entire controlled environment, such as system 100. However, the exclusion field 1016 allows the user Exclude specific system components, specific system component types, or system components within a specified area from these global commands.For example, a user can exclude turning off the television in the living room when the switch object 306 is activated.

The control panel 1000 covers the concept of a chain of devices. When configuring system components according to the invention, chains of dependent or subordinate system components (system components with input/output dependencies) are established. For example, one such chain could be a DVD player, amplifier and television. Another chain could be a CD player and amplifier. A third chain could be a TV and a tuner. The present invention includes tools for turning on or off all system components in a chain (e.g. when a CD player is started, the amplifier is on), and proxy control of the appropriate subordinate system components (e.g. volume requests go to the amplifier, or chapter selection goes to the DVD player) methods and/or techniques.

The control screen 1000 also includes a next object 1020, which invokes the next control screen in the setup wizard. The inter-level navigation object 212 recalls the previous screen within the setup wizard.

The control interface of the present invention provides a convenient and efficient way for controlling parent system components and their subordinate (including hidden) system components from a common platform. Referring again to FIG. 4, side tab 422 is a scrollable field that graphically represents pre-controlled objects 426a-426n associated with subsystem components. As mentioned, a subsystem component is subordinate to a parent system component that is controlled by the component control screen 400 . In an embodiment, each individual subsystem component that makes up side tab 422 is configured by the user using a setup wizard. The setup wizard allows the user to specify the system components connected to the TV/parent system components and forwards this information to the software modules supporting the control interface of the present invention.

FIG. 12 illustrates a definition control screen 1200 that allows a user to specify subordinate system components of a parent system component, according to an embodiment of the present invention. The definition control screen 1200 allows establishing specifications for the tuner device, as shown in the definition field 1202 . The definition control screen 1200 includes a parent field 1204 , an input field 1206 , an audio device field 1208 , an audio input field 1210 , and a number input field 1212 .

Parent field 1204 specifies whether the tuner device is a parent system component or a subsystem component. If the system component is a parent system component, the parent field 1204 contains the input "none". If the system component is a subsystem component, the parent field 1204 lists the name of the parent system component.

Input field 1206 specifies a channel input. Channel inputs are used to tune system components to receive and decode video signals from the associated channel.

Audio field 1208 identifies whether an audio output system component is affiliated with a tuner device. For example, if the user intended the audio output to be fed to an auxiliary speaker system or audio amplifier, then the system component name for the speaker system is entered into the audio field 1208 . In this case the speaker system would be a subsystem component of the tuner/parent system component.

Audio input field 1210 identifies the appropriate input channel on the speaker system specified in audio field 1208 .

Numeric entry field 1212 specifies the number of digits required to change the channel on the tuner. For example, some tuners only accept two digits (ie, limited to channels 01 to 99). Other tuners can accept up to four digits. The number of channel numbers depends on the make and model of the system component (eg, tuner) being configured. Thus, the numeric entry field 1212 identifies metadata that describes the unique control behavior of the system component. In one embodiment, the portable controller 108 is programmable to extract channel number metadata for the tuner from a metadata database. An example of a system and method for extracting channel digital metadata from a metadata database is described in the application entitled "Device Control Database" (U.S. Patent Application Serial No. TBD; Attorney Docket No. 2100.0070000), hereby All are incorporated by reference. However, if the metadata database does not contain this information, the channel number metadata is read from the number entry field 1212 . Alternatively, numeric entry fields 1212 can be used in place of data stored in the metadata database.

The next object 1020 invokes the next control screen in the setup wizard. The previous object 1222 invokes the previous control screen. The inter-level navigation object 212 allows the user to switch to the next higher interface level through the setup wizard. For example, if the user has configured a parent system component on one level, and is currently configuring a second, next level subordinate system component, the user can return to the first level for the parent system component by activating the navigation object 212, Or the user, by activating the previous object 1222, returns to the previous screen for defining subordinate system components of the second level.

FIG. 13 illustrates another embodiment of a definition control screen 1300 that allows a user to link subordinate system components to parent system components. The definition control screen 1300 allows establishing specifications for components of the audio-video system, as shown in the definition field 1302 . Control screen 1300 can be used to slave audio-video system components to tuners configured in control screen 1200 . As such, control screen 1300 will represent a sub-level of control screen 1200 . Thus, activation of the interlevel navigation object 212 will invoke the next higher interface level, screen 1300 . However, activating the previous object 1222 invokes the previous control screen in a series of control screens for configuring the audio-visual system components.

1-13 are schematic illustrations that allow to explain the principles of the invention. It should be understood that embodiments of the present invention can be implemented in hardware, firmware, software or a combination thereof. In this embodiment, the respective components and steps will be implemented in hardware, firmware and/or software in order to perform the functions of the present invention. That is, the same hardware, firmware or software module can perform one or more exemplary blocks (ie components or steps).

Additionally, the invention can be implemented in one or more computer systems or other processing systems that can perform the functionality described herein. Referring to FIG. 14, an exemplary computer system 1400 for use in implementing the present invention is shown. In terms of this exemplary computer system 1400, various embodiments are described. After reading this specification, it will become apparent to a person skilled in the relevant art how to implement the invention using other computer systems and/or computer architectures.

Computer system 1400 includes one or more processors, such as processor 1404 . Processor 1404 may be a special purpose or general purpose digital signal processor. The processor 1404 is connected to a communication infrastructure 1406 (eg, a communication bus, crossbar, or network).

Computer system 1400 may include display interface 1402 that forwards graphics, text, and other data from communications infrastructure 1406 (or frame buffer, not shown) for display on display unit 1430 .

Computer system 1400 also includes main memory 1408 , preferably random access memory (RAM), and can also include secondary memory 1410 . Secondary storage 1410 may include, for example, hard disk drive 1412 and/or removable storage drive 1414, representing a floppy disk drive, tape drive, optical disk drive, and so forth. Removable storage drive 1414 reads from and/or writes to removable storage unit 1418 in a well known manner. Removable storage unit 1418 represents a floppy disk, magnetic tape, optical disk, etc., which is read from and written to by removable storage drive 1414 . As will be appreciated, removable storage unit 1418 includes computer usable storage media in which computer software (eg, programs or other instructions) and/or data are stored.

In other embodiments, secondary storage 1410 includes other similar devices for allowing software and/or data to be loaded into computer system 1400 . These devices include, for example, a removable storage unit 1422 and an interface 1420 . Examples of these devices include program cartridges and cartridge interfaces (such as found in video game equipment), removable memory chips (such as EPROM or PROM) and associated sockets, and allow transfer of software and data from the removable storage unit 1422 Other removable storage unit 1422 and interface 1420 to computer system 1400 .

Computer system 1400 may also include a communication interface 1424 . Communication interface 1424 allows software and/or data to be transferred between computer system 1400 and external devices. Examples of communications interface 1424 include a modem, a network interface (such as an Ethernet card), a communications port, PCMCIA slots and cards, and the like. Software and data communicated via communications interface 1424 may be in the form of signals 1428 that are electronic, electromagnetic, optical, or other signals capable of being received by communications interface 1424 . These signals 1428 are provided to a communication interface 1424 via a communication path (ie, channel) 1426 . Communications path 1426 carries signals 1428 and can be implemented using wire or cable, fiber optics, telephone lines, cellular telephone links, RF links, free-space optics, and other communication channels.

In this application, the terms “computer program medium” and “computer usable medium” are used to refer generally to media such as removable storage unit 1418 , removable storage unit 1422 , hard disk installed in hard drive 1412 , and signal 1428 . These computer program products are means for providing software to the computer system 1400. In an embodiment, the invention is directed to these computer program products.

Computer programs (also referred to as computer control logic or computer readable program code) are stored in main memory 1408 and/or secondary memory 1410 . Computer programs can also be received via communication interface 1424 . These computer programs, when executed, allow computer system 1400 to implement the invention as described herein. In particular, the computer programs, when executed, allow processor 1404 to implement the processes of the present invention, such as using, for example, portable controller 108, control panel 200, control panel 400, control panel 700, control panel 1000, and/or system 100 described above. Methods implemented by other system components, such as methods 800 and/or 900 . Accordingly, these computer programs represent the controllers of the computer system 1400 .

In embodiments where the invention is implemented using software, the software can be stored in a computer program product and loaded into computer system 1400 using removable storage drive 1414 , hard drive 1412 or communication interface 1424 . The control logic (software), when executed by the processor 1404, causes the processor 1404 to perform the inventive functions described herein.

In another embodiment, the invention is implemented primarily in hardware using, for example, hardware components such as application specific integrated circuits (ASICs). Implementing a hardware state machine to perform the functions described herein will be apparent to those skilled in the relevant art.

In another embodiment, the invention is implemented using a combination of hardware and software.

While various embodiments of the invention have been described above, it will be understood that they have been presented by way of example, and not limitation. It will be apparent to those skilled in the relevant art that various changes in form and detail can be made without departing from the spirit and scope of the invention. Furthermore, as previously stated, it should be understood that the methods, systems and computer program products of the present invention should not be limited to residential environments. The present invention can be implemented in various environments having a central processing system for distributing media and sending command and/or control signals to multiple devices and/or applications distributed over a designated area. In addition to residences, designated areas include, but are not limited to, office complexes, a small office set, studios, warehouses, entertainment venues, schools or universities, health care facilities, hotels, resorts, airplanes, ships, automobiles, etc. Accordingly, the invention should not be limited by any of the above-described exemplary embodiments, but only in accordance with the following claims and their equivalents.

Claims (30)

1. a graphic user interface is used for a plurality of system units in the managing controlled environment, comprising:

First controlling object set, wherein, the interior system unit of described controlling object and controlled environment is associated; And

The unit control interface, be used to present the control option that is used for selected system unit, wherein, activation is indicated described selected system unit and is filled user interface with described control option from the controlling object of described first set, wherein, each control option is associated with command sequence, when carrying out described command sequence, sends operation or function that described selected system unit is controlled in instruction.

2. graphic user interface as claimed in claim 1 further comprises:

The set of second controlling object, expression can provide the affiliate system component of input to described selected system unit, wherein, activation uses the control option of the affiliate system component that is associated with the controlling object that is activated to fill user interface from the controlling object of described second set, wherein, each the control option that is used for described affiliate system component is associated with command sequence, when carrying out described command sequence, sends operation or function that described affiliate system component is controlled in instruction.

3. graphic user interface as claimed in claim 1, wherein, a plurality of zones in described first controlling object set expression controlled environment, wherein, each zone comprises one or more system units.

4. graphic user interface as claimed in claim 3 further comprises:

The set of second controlling object, represent the available system components in the selected zone, wherein, activation is indicated described selected zone and is filled user interface with described second set from the controlling object of described first set, wherein, activation is indicated described selected system unit and is filled user interface with described control option from the controlling object of second set.

5. graphic user interface as claimed in claim 1, wherein, a plurality of unit types in described first controlling object set expression controlled environment.

6. graphic user interface as claimed in claim 5 further comprises:

The set of second controlling object, Free Region in the expression controlled environment, wherein, each zone comprises one or more system units of selected unit type, wherein, activation is indicated described selected unit type and is filled user interface with described second set from the controlling object of described first set, wherein, activates from the controlling object of described second set and indicates described selected system unit and fill user interface with described control option.

7. graphic user interface as claimed in claim 1 further comprises:

With global command's associated switch object, be used for when carrying out described global command, sending the ON-OFF that the appointing system parts are changed in instruction, wherein, from described first set, select one or more controlling object to indicate described appointing system parts.

8. graphic user interface as claimed in claim 7, wherein, described switch object is associated with global command, when carrying out described global command, send instruction and change the ON-OFF of the one or more system units that are complementary with the specified parts type, wherein, from described first set, select controlling object to indicate described specified parts type.

9. graphic user interface as claimed in claim 7, wherein, described switch object is associated with global command, when carrying out described global command, send instruction and change the ON-OFF of at least one system unit of the appointed area that is positioned at controlled environment, wherein, from described first set, select controlling object to indicate described appointed area.

10. graphic user interface as claimed in claim 7 further comprises:

Remover, be used for from described global command get rid of one or more appointing system parts, one or more system units of one or more system units of being complementary with the specified parts type and the specific region that is positioned at controlled environment at least one, wherein, carry out the ON-OFF that described global command does not send one or more system units that instruction is excluded with change.

11. graphic user interface as claimed in claim 1 further comprises:

The device definition device, being used to specify inputing or outputing in one or more affiliate system component and the main system parts links or correlativity, thereby set up the system unit chain that comprises described main system parts and described one or more affiliate system component, wherein, described main system parts with from described first the set the main control object be associated.

12. graphic user interface as claimed in claim 11, wherein, the activation of described main control object is filled user interface with the control option, and described control option is used for fill order, so that send operation or function that described system unit chain is controlled in instruction.

13. a method that is used for a plurality of system units in the managing controlled environment comprises:

Present first controlling object set on user interface, the interior one or more system units of each object and controlled environment are associated;

Response receives and the activation signal that is associated from described first controlling object of gathering, selective system parts;

Fill described user interface by the control option that is used for selected system unit; And

Each control option is associated with the executable command sequence, and described executable command sequence sends instruction so that control the operation or the function of selected system unit.

14. method as claimed in claim 13 further comprises:

Presenting expression on described user interface can provide second controlling object set of the affiliate system component of input to selected system unit.

15. method as claimed in claim 14 further comprises:

Response activates the controlling object from described second set, fills described user interface by the control option that is used for affiliate system component; And

Each control option that will be used for described affiliate system component with send to instruct so that control the operation of described affiliate system component or the executable command sequence of function is associated.

16. method as claimed in claim 14 further comprises:

Fill described user interface by the control option,, and hide described affiliate system component and on described user interface, further do not checked so that affiliate system component is linked to selected system unit.

17. method as claimed in claim 13, wherein, a plurality of zones of described first controlling object set expression, each zone comprises the one or more system units in the controlled environment.

18. method as claimed in claim 17, wherein, described selection step comprises:

Response receives and the activation signal that is associated from described first controlling object of gathering, and selects the zone;

Gather by second controlling object and to fill described user interface, to represent the available system components in the selected zone; And

Response receives and the activation signal that is associated from described second controlling object of gathering, selective system parts.

19. method as claimed in claim 13, wherein, a plurality of unit types in described first controlling object set expression controlled environment.

20. method as claimed in claim 19, wherein, described selection step comprises:

Response receives the activation signal that is associated with described first set, alternative pack type;

Gather with second controlling object and to fill user interface representing Free Region, each zone comprises one or more system units of the selected unit type in the controlled environment; And

Response receives the activation signal that is associated with described second set, selective system parts.

21. method as claimed in claim 13 further comprises:

From described first set, select one or more controlling object to come the appointing system parts;

On described user interface, present switch object, when described switch object is activated, carry out the global command that is used to specify system unit; And

Carry out described global command so that send the instruction of the ON-OFF of change appointing system parts.

22. method as claimed in claim 13 further comprises:

From described first set, select one or more controlling object to come the specified parts type;

On described user interface, present switch object, when described switch object is activated, carry out the global command be used for one or more system units of being complementary with described unit type; And

Carry out described global command, to send the instruction of the ON-OFF of changing the described one or more system units that are complementary with described unit type.

23. method as claimed in claim 13 further comprises:

The one or more controlling object of selection are specified the zone in the controlled environment from described first set;

On described user interface, present switch object, when described switch object is activated, carry out the global command of the one or more system units be used to be positioned at described zone; And

Carry out described global command, to send the instruction of the ON-OFF of changing the described one or more system units that are positioned at described zone.

24. method as claimed in claim 13 further comprises:

From described first set, select one or more controlling object to come the appointing system parts;

On described user interface, present switch object, when described switch object is activated, carry out the global command that is used to specify system unit;

At least one of one or more system units of the one or more system units from described global command, getting rid of one or more appointing system parts, be complementary with the specified parts type and the appointed area that is positioned at controlled environment; And

Carry out described global command, to send the instruction of the ON-OFF of changing all the appointing system parts except one or more system units of being got rid of.

25. method as claimed in claim 13 further comprises:

Specify inputing or outputing between one or more affiliate system component and main system parts to link or correlativity, thereby set up the system unit chain that comprises described main system parts and described one or more affiliate system component, wherein, described main system parts with from described first the set the main control object be associated.

26. method as claimed in claim 25, wherein, the activation of described main control object is filled described user interface with the control option, and described control option is used for fill order, with the operation of the described system unit chain of transmission control or the instruction of function.

27. a computer program that comprises computer usable medium has the computer readable program code unit that embeds in described medium, be used to make a plurality of system units in the computer management controlled environment, described computer program comprises:

The first computer readable program code unit is used for presenting first controlling object set on user interface, the interior one or more system units of each object and controlled environment are associated;

The second computer readable program code unit is used to respond and receives and from the activation signal that is associated of controlling object of described first set, selective system parts;

The 3rd computer readable program code unit is used for filling described user interface by the control option that is used for selected system unit; And

The 4th computer readable program code unit is used for each control option is associated with the operation of controlling selected system unit or the executable command sequence of function with send instructing.

28. computer program as claimed in claim 27 further comprises:

The 5th computer readable program code unit is used for presenting the set of second controlling object on described user interface, its expression can provide the affiliate system component of input to selected system unit.

29. computer program as claimed in claim 27, wherein, a plurality of zones of described first controlling object set expression, each zone comprises the one or more system units in the controlled environment, described computer program further comprises:

The 5th computer readable program code unit is used to respond and receives and from the activation signal that is associated of controlling object of described first set, select the zone; And

The 6th computer readable program code unit, be used for gathering and fill described user interface to represent to select the available system components in the zone with second controlling object, wherein, response receives and the activation signal that is associated from described second controlling object of gathering, and selects selected system unit.

30. computer program as claimed in claim 27, wherein, a plurality of unit types in described first controlling object set expression controlled environment, described computer program further comprises:

The 5th computer readable program code unit is used to respond receive with described first and gathers the activation signal that is associated, alternative pack type; And

The 6th computer readable program code unit, be used for gathering and fill user interface to represent Free Region with second controlling object, each zone comprises one or more system units of the selected unit type in the controlled environment, wherein, response receives the activation signal that is associated with described second set, selects selected system unit.

Images