AU743455B2 - Environment adaptive multimedia presentation - Google Patents

Description

S&F Ref: 482492

AUSTRALIA

PATENTS ACT 1990 COMPLETE SPECIFICATION FOR A STANDARD PATENT

ORIGINAL

Name and Address of Applicant: Actual Inventor(s): Address for Service: Invention Title: Canon Kabushiki Kaisha 30-2, Shimomaruko 3-chome Ohta-ku Tokyo Japan Josiah Poon Spruson Ferguson St Martins Tower 31 Market Street Sydney NSW 2000 Environment Adaptive Multimedia Presentation ASSOCIATED PROVISIONAL APPLICATION DETAILS [33] Country [31] Applic. No(s) AU PP7672 [32] Application Date 11 Dec 1998 The following statement is a full description of this invention, including the best method of performing it known to me/us:- 5815c [R:\LIBW] 14908.doc:gmm -1- ENVIRONMENT ADAPTIVE MULTIMEDIA PRESENTATION Field of the Invention The present invention relates to the presentation of multimedia and, in particular, to the presentation of multimedia in response to environmental stimuli.

Background Art There are a few traditional methods for controlling the flow of multimedia presentations. The most common approach is to have a preauthored sequence of multimedia content (eg. audio, video, still images, etc.) such that a predetermined media object or segment is selected on completion of the current object. Such occurs, for example, in the production of broadcast of radio programmes. At the other extreme, the presentation flow may be randomised without any external control. This may occur in a musical jukebox operating continually on random selection. Generally however, a middle-ground between these two extremes is provided. This middle-ground typically involves some limited form of control being granted to the user so that one can fast 15 forward or skip to the next object in the sequence or to rewind (backtrack) to the previous object. Such facilities can easily be found in the control panels of compact disc (CD), audio cassette and video cassette players. The user can also programme the flow of the *aa.a.

S"presentation such that the sequence is different from the authored or produced sequence of the media objects. This is exemplified in the PROGRAM function commonly found in multi-discs CD players.

However, it remains that the presentation sequence is largely determined by the producer of the media object, or the user who provides explicit instruction to determine the flow.

It is an object of the present invention to provide an alternative mechanism for determining the flow or sequencing of multimedia presentations.

Summary of the Invention In accordance with one aspect of the invention there is provided a media presentation device incorporating sensors arranged to detect local environmental 484492 MMedia32 CFP1531AU 4 e[\ELEC\CISRA\MMEDIA\MMEDIA32]482492.doc:idp conditions and to provide sensory input that is used to provide cues to select the next appropriate multimedia object(s) for presentation.

In accordance with another aspect of the invention there is provided a media presentation system comprising: a media presentation unit for reproducing a media object; at least one environmental sensor associated with said media presentation unit and configured to provide a corresponding indication regarding an environment about said media presentation unit; a source of a plurality of media objects able to be reproduced by said media presentation unit; *a control unit with which a user of said media presentation system can establish media selection criteria; and.

a processing unit for determining, from at least said one environmental indication Sand said established selection criteria, one of said media objects for reproduction by said media presentation unit and enabling reproduction of said one media object.

In accordance with another aspect of the invention there is provided a method of presenting plural media objects in an environment, said method comprising the steps of: determining at least one of a plurality of environmental conditions associated with said environment; analysing said environmental conditions to identify an analysed state of said environment; establishing user-desired selection criteria; examining a plurality of available media objects to determine those of said media objects determinably related to said analysed state and said established selection criteria; and selecting from said related media objects at least one media object for reproduction within said environment.

482492.doc -2a Brief Description of the Drawings A number of embodiments of the present invention will now be described with reference to the drawings in which: Fig. 1 is a schematic block diagram representation of a multimedia presentation system according to the preferred embodiment; Fig. 2 is a flow chart of the operation of the preferred embodiment; *o* 482492.doc Fig. 3 schematically illustrates one example of implementation; and Fig. 4 schematically illustrates another example of implementation.

Detailed Description Fig. 1 shows an multi-media presentation system 100 which incorporates a presentation unit 101 for the reproduction of one or more multimedia objects. Typically, the presentation unit 101 may take the form of a display device for displaying either static or moving images or an audio device for sound reproduction. In some situations, the presentation unit 101 may include a display device and an audio device arranged to be used to simultaneously reproduce visual and audio matter from a common multimedia source (eg. a motion picture film). The system 100 also includes a storage unit 102 for storing multimedia objects and a control unit 103 which permits user input and control of the operation of the presentation system 100.

g The system 100 also includes a number of sensors 104 which are configured to monitor environmental conditions in and about the location where the presentation unit 15 101 is situated. The sensors 104 may include a variety of instruments for measuring environmental conditions such as temperature, humidity, atmospheric pressure, i background noise level, luminance (light) level and motion.

Each of the storage unit 102, control unit 103 and sensors 104 provide a respective inputs 106, 107 and 108 to a processing unit 105 which is configured to assess 20 the state of the various inputs and to determine therefrom a next object or a sequence of multimedia objects to be accessed from the storage unit 102 for multimedia presentation using the presentation unit 101. As a consequence, an output 109 of the processing unit typically incorporates multimedia data corresponding to an object currently being reproduced.

The operation of the system 100 is depicted in the flow chart of Fig. 2 as a series of method steps 200. Operation commences at step 202 where the user, via the control unit 103 selects one of a number of templates in order to specify a desired behaviour of the multimedia presentation. Each template incorporates a number of decision rules that draw upon inputs from one or more of the sensors 104 and which together determine an 484492 MMedia32 CFP1531AU [I:\ELEC\CISRA\MMEDIA\MMEDIA32]482492.doc:ldp appropriate type of media object which is suitable for the next presentation. For example, the templates may be based upon specific desires of the user, for example some relating only to audio media presentation, some relating to audio/visual media presentation. Other templates may accommodate the individual tastes of the user such as "party mode", "dinner mode" or "background mode".

In step 204, inputs from the sensors 104 are applied to the selected template to determine retrieval conditions and criteria for media objects. Application of the rules from different templates may produce divergent behaviour even though the same readings are obtained from each of the sensors 104. For example, selection of the "party mode" by the user may cause loud rock and roll music to be reproduced even though temperature and movement sensors for example indicate there is no one in the vicinity of the :presentation unit 101 which, under the "background mode" might otherwise invoke the presentation of soft contemporary music.

Step 206 examines the storage unit 102 to retrieve candidate multimedia objects S 15 which fit within the determined retrieval conditions obtained from step 204. Typically, if the multimedia objects are stored in a digital format in the storage unit 102, those objects may be associated with meta-data which provides information related to the content of the multimedia object. The retrieved candidate multimedia objects thus form a list which the processing unit 105 is able to identify.

20 At step 208, the processing unit 105 examines the list of candidate media object to identify a best candidate suitable for the next media object presentation. This selection may be based on a number of criteria, a first one of which is typically presentation history which attempts to avoid replaying the same media object in close proximity to a previous presentation of the same media object. However, where the number of media objects that fit within a specific category are limited, it may be that a significant time between reproducing the same media object cannot be avoided. Further information can be used to assess which is the best candidate media object. This can include applying a reasoning tree, which typically assesses probability functions of specific outcomes, as compared to the templates which apply absolute rules.

484492 MMedia32 CFP1531AU [I:\ELEC\CISRA\MMEDIA\MMEDIA32]482492.doc:ldp At step 210, a determination is made as to whether or not it is the time to change the media object being presented. Typically, media objects have some finite time in which they are presented (eg. three to five minutes for a musical piece) and thus it is appropriate for the presentation of the current best selected media object to be deferred until the presentation of the currently presented object is completed. When this occurs, the current best candidate is presented by rendering via the output 109 to the presentation unit 101. Control then returns to step 204 to again assess retrieval conditions and to therefrom select the best candidate for presentation.

The preferred embodiment utilises a plurality of the sensors thus giving increased capacity to determine the state of the local environment in a vicinity of the presentation unit 101. For example, temperature sensors sense the ambient temperature within the presentation area which, for example, may be moderate and constant when there is little or no activity in the area, but alternatively can rise significantly where the area is well populated. Accordingly, for example where a number of persons congregate in one room, a rise in temperature may cause the "party mode" to be considered. If a movement sensor were also being used, an indication of a large amount of movement may also substantiate the "party mode". However, if there was little or no movement, but a increase in S"temperature, this could be construed as merely a high ambient atmospheric temperature thus suggesting for example the presentation of calm, relaxation media objects.

Specific implementations of the preferred embodiment will now be provided to exemplify how the principles of the present invention may be best used.

Example 1: Digital Photo Presentation Fig. 3 shows an arrangement 300 in which digital photo frame 302 is configured as a counterpart to traditional picture frames where paper-based photographs are traditionally displayed. The digital photo frame 302 may be designed in such a manner so that it appears as a conventional photo frame and as such may be hung upon a wall or placed upon a surface such as a table. The digital photo frame 302 incorporates a display area 304 for the presentation of digital images. Typically, a new image may be displayed at a predetermined interval (eg. the image displayed is changed every hour, every day or 484492 MMedia32 CFP1531AU ([I:\ELEC\CISRA\MMEDIA\M EDIA32]482492.doc:ldp -6every week). According to this specific embodiment of the present invention, the decision to display a particular image is determined by a selected template and current environmental conditions associated with the arrangement 300.

As seen in Fig. 3, the user may insert a compact disc 312 into an appropriate drive (not illustrated) of the digital photo frame 302 to thus provide a source of a plurality of digital images. Each digital photograph encoded on the disc 312 incorporates metadata such as date and time which is able to be read for the purposes of selection of individual images. The digital photo frame 302 incorporates a user control panel 306 and a number of sensory inputs 308 to which sensors 310 can be coupled.

Using the control panel 306, the user is able to select a specific time interval for displaying any one image and thus determining the rate at which images are cycled through the display area 304. The user may also be offered, using the display area 304, a o number of template options for determining selection of the next photograph for display.

With the control panel 306, the user may press a play button 318 to commence the presentation and display of photographs using the display 304 and for which the display of photographs continues until such time as a stop button 314 is depressed. If the user is not appreciative of the current photograph on display, the user may press a forward button 316 to skip to the next picture. If the user prefers a previous picture, the user may depress the backward button 320 to revert to the previous picture.

20 In this embodiment, each template provided to the user incorporates its own algorithm to impose conditions upon the types of pictures chosen for display. Each template may also be supplied with necessary graphics, images and even appropriate music in order to enhance the presentation. Examples of such templates may include "my children", "now and then", and "outdoors scenes", "birthday parties" to name but a few.

As seen in Fig. 3, an outdoors-family scene is illustrated.

A reasoning process is carried out within the digital photo frame 302 using knowledge obtained from the template. Meta-data of the current photograph, and system information such as date and time and presentation history, together with a sensory input are captured in real-time and used by the decision rules in the template to retrieve 484492 MMedia32 CFP1531AU [I:\ELEC\CISRAWMEDIA\M M EDIA32]482492.doc:ldp candidate photographs for the next display. In Fig. 3, the sensors 310 include a temperature sensor 322, a light sensor 324 and a sound sensor (eg. microphone) 326.

For example, if the presentation device is mounted with a thermometer, an infrared sensor and a microphone, the decision rules within the template can use inputs (ie.

temperature, movement, and noise level) from these sensors to determine the environmental conditions of the room in which the device is located and which in turn enables the making of decision as to the type of photograph that is best displayed with the current environment. If, for example, the room is very warm, much movement is detected and there exists much background noise, one embodiment may incorporate a rule which has the following conclusion If temperature is very warm And movement a lot And noise a_lot Then 15 activity is strong If the current date is for example December 25, then a further rule may determine that a Christmas party is occurring at the location of the presentation unit. hi that case, the system may retrieve similar images from the disc 312 for display. If there are more •than one digital photographs which satisfy this requirement, priority may be given to, say, 20 pictures from Christmas parties from previous years than to, for example, other parties such as birthday parties, graduation and wedding parties.

The selected images may be rendered with enhancement graphics, such as text, where appropriate for display. Further, the user, if he so chooses, can provide feedback to the system 300 by pressing a "dislike" button 328 if the displayed image is considered inappropriate. If the image is considered to be appropriate, feedback may be provided by the user depressing a "like" button 330. In some embodiments, where the "like" button 330 is depressed and/or the "dislike" button 328 is depressed, these can be interpreted further by the processing unit 105 in the assimilation of images to be displayed. When the 484492 MMedia32 CF;P1531AU [I:\ELEC\CISRA\IMEDIA\MM EDIA32482492.doc:idp time period for the current presentation is ended, the next image to be presented is displayed.

Example 2: Music Presentation Fig. 4 shows an arrangement of a digital-DJ (disc jockey) system 400 which is used to select and broadcast music depending on conditions in a local environment.

The arrangement 400 includes a DJ control module 402 which is coupled to a music reproduction system 404 incorporating amplifier and loud speakers. The control unit 402 includes a control pad 406 and receives input from a number of sensor inputs 408 to which sensors 410 are coupled. A number of music sources 412 are provided to the control unit 402 to provide a selection to be output via the reproduction system 404.

In one particular implementation, the musical inputs 412 are derived from a server network to which a number of music providers 432, 434 and 436 are connected. In this fashion, selections made via the control panel 402 enable music to be sourced from any one of the music providers 432-436 via the network 438. Because the musical sources are provided on a computer network, such as the Internet, such may be associated with meta-data able to be stored locally within the control unit 402 or alternatively downloaded from the network 438.

As with the previous embodiments, a user may be offered a list of templates such that each template relates to a particular type of music to be selected. The user may depress a play button 418 on the control panel 406 to commence music listening and music will continue to be reproduced until a stop button 414 is depressed. As with the previous example, skipping, replaying, or an indication of like or dislike can be provided using buttons 416, 420, 428 and 430 respectively.

As with the previous example, the sensors 410 incorporate a temperature sensor 422, light sensor 424 and a sound sensor 426 and each template performs a reasoning process based upon sensor inputs and meta-data associated with available music together with system information such as date, time and presentation history to determine candidate music for listening.

484492 MMediM3 CFP153AU [I:\ELEC\CISRA\MMEDIA\M M EDIA32482492.doc:ldp For example, if the room is only dimly lit, the light sensor 424 returns a low reading and such, when coupled with a low background noise being sensed from the sound sensor 426, may result in quiet easy-listening music to be selected for reproduction.

If there is more than one musical piece with satisfies the requirement at any particular time, priority may be given to music which derives from a previous source known to the user or has a stronger resemblance to music that has previously been appreciated.

It will be apparent from the foregoing that an arrangement is provided whereby a plurality of environmental conditions are sent and used to determine, according to generic 10 user input, the particular type of media presentation suited to the environment. The oooo system also provides for changing the reproduction based upon changing environmental conditions. For example, in the embodiment of Fig. 4 if low temperature, low light and o low noise were present and easy listening music playing, entry of a number of persons to the room, enabling of the light and an increase in ambient atmospheric temperature could •I .15 well result in louder, more contemporary music to be played, thus reflecting the change in environment from the "easy listening" environment.

~Although the described embodiments only refer to static images and music, other S•media presentations can be made and these may include slide shows having associated background music, video motion presentation (eg. video clips) which may well be suitable for party environments and the like. Further, combinations of the two examples described can be used in other environments. For example, the easy listening/relaxation mode may provide for wild life or rain forest images to be displayed on a display wall in concert with rain forest sounds that may be sourced from a compact disc.

Further, whilst the example of Fig. 4 refers to music being sourced from a computer network, other sources such as CD's, tapes or broadcasts may be used. In cases where broadcasts may be used, the selection may take the form of selecting a broadcast channel, known for the presentation of media of a particular style (eg. classical music, c.f.

rock music). In such situations, selection of individual media objects (eg. musical pieces, 484492 MMedia32 CFP'1531AU [I:\ELEC\CISRA\MM EDIA\MM EDIA32]482492.doc:dp songs) need not be performed as such programming is controlled by the radio station for example.

The foregoing describes only a number of embodiments of the present invention and modifications, can be made thereto without departing from the scope of the present invention. For example, the sensors may include a real-time clock which given similar environmental conditions may provide for a different selection depending on the time of day.

In the context of this specification, the word "comprising" means "including principally but not necessarily solely" or "having" or "including" and not "consisting only of". Variations of the word comprising, such as "comprise" and "comprises" have corresponding meanings.

484492 MMedia32 CFP1531AU [I:\ELEC\CISRA\MMEDIA\MMEDIA321482492.doc:idp

Claims (4)

1. A media presentation system comprising: a media presentation unit for reproducing a media object; at least one environmental sensor associated with said media presentation unit and configured to provide a corresponding indication regarding an environment about said media presentation unit; a source of a plurality of media objects able to be reproduced by said media presentation unit; a control unit with which a user of said media presentation system can establish media selection criteria; and. a processing unit for determining, from at least said one environmental indication and said established selection criteria, one of said media objects for reproduction by said Smedia presentation unit and enabling reproduction of said one media object. S

2. A system according to claim 1, further comprising a plurality of selection templates each of which being associated with a set of rules that define characteristics of said corresponding template relating to media objects, a selection of one said template by the user thereby establishing said media selection criteria.

3. A system according to claim 2 wherein for each said set of rules, at least one rule thereof is determinable according to one said corresponding indication.

4. A system according to claim 2 or 3 wherein said rules and said templates are editable via said control unit by the user in respect of media characteristics desired for reproduction, said processing unit being configured to analyse said characteristics according to a selected one of said templates into which the characteristics are entered and S to modify a selection of a media object according to said analysed characteristics as acted upon by said environmental indications.

482492.doc -12- A system according to any one of claims 1 to 4, wherein on conclusion of reproduction of said one media object, said processing unit enables reproduction of a further said media object. 6. A system according to any one of claims 1 to 5, said system comprising a plurality of said environmental sensors wherein said processing unit processes said corresponding indications to determine a set of media objects available from said source for reproduction, said set of media objects having characteristics related to said template, said processing unit further determining from said set of media objects, a media object for reproduction. 7. A system according to claim 6, wherein said set is reproducible as a sequence of oeeeo media objects. 8. A system according to any one of claims I to 7, wherein said sensors are selected from the group consisting of: U S° a light sensor; a movement sensor; a sound sensor; a temperature sensor; and a humidity sensor. 9. A system according to any one of the preceding claims, wherein said media object is selected from the group consisting of: audio information; image information; audio-motion video information; and S audio-still image information. 482492.doc -13- A system according to any one of the preceding claims, wherein said media object includes media data reproducible by said media presentation unit and meta-data indicative of said media data and interpretable by said processing unit in comparison with said environmental indications. 11. A system according to any one of the preceding claims, wherein said source is selected from the group consisting of: an optical disk device; a magnetic memory device; a magnetic tape device; a computer network; and a broadcast transmission. 12. A method of presenting plural media objects in an environment, said method o comprising the steps of: CC.. determining at least one of a plurality of environmental conditions associated o S°with said environment; analysing said environmental conditions to identify an analysed state of said environment; establishing user-desired selection criteria; examining a plurality of available media objects to determine those of said media objects determinably related to said analysed state and said established selection criteria; and selecting from said related media objects at least one media object for reproduction within said environment. 482492.doc -14- 13. A method according to claim 12, wherein said determining comprises receiving inputs from a plurality of environmental sensors to determine a plurality of said environmental conditions. 14. A method according to claim 12 or 13, wherein said analysing comprises applying said environmental conditions to a user selected template to determine said any analysed state. A media presentation system substantially as described herein with reference to S 10 any one of the embodiments as illustrated in the drawings, or any one of the Examples. ;16. A method of presenting plural media objects in an environment substantially as described herein with reference to any one of the embodiments as illustrated in the drawings, or any one of the Examples. DATED this twelfth Day of September, 2001 Canon Kabushiki Kaisha o Patent Attorneys for the Applicant SPRUSON FERGUSON 482492.doc