HK1103540A - Multicast and broadcast streaming method and system - Google Patents

Description

Multicast and broadcast streaming method and system

Technical Field

The present invention relates generally to communicating data to client devices over a communication network, and more particularly, but not exclusively, to a system and method for streaming data to client devices over wired and/or wireless networks in a multicast and/or broadcast environment.

Background

There are a variety of techniques that can be used to deliver data to client devices over communication networks (multicast and broadcast are two examples). In at least some types of multicast environments, a plurality of multicast groups are provided in which subscribing members (e.g., client devices) receive multicast session data, services, content, or other data that is available to subscribing members through the multicast groups. Each of the multicast groups may provide the same primary content (e.g., a particular video program), but there may be several differences between the signals available in each of the multicast groups. For example, each multicast group may transmit a video program at a different bit rate than the other multicast groups.

In such a multicast environment, client devices or other recipients desiring to receive the video program may subscribe to or otherwise join a particular multicast group in order to receive the transmission. Typically, each client device selects the appropriate multicast group to join based on the provided bit rate or based on some other compatible characteristic. Thus, for example, multicast group 1 may provide video programs to client devices 1-10 at bit rate A, multicast group 2 may provide the same video programs to client devices 11-14 at bit rate B, multicast group 3 may provide the same video programs to client devices 15-21 at bit rate C, and so on.

This is typically the case: certain conditions may make the initial bit rate undesirable for a particular client device. For example, network bandwidth conditions or client device characteristics that dynamically change during a session may instruct a particular client device to switch to a lower (or higher) bit rate. However, frequently changing from one multicast group to another may lead to adverse results. One result is cost. Switching from one multicast group to another (e.g., connecting and disconnecting, and vice versa) is expensive for client devices, as cost or program requirements may need to be met in order to subscribe to a new multicast group. Another result is a service interruption. That is, switching from one multicast group to another during the transmission of a video program may result in the loss of some portion of the video program during the transition-the change between multicast groups is typically not a seamless process.

With the development of media compression and wireless network infrastructure, media streaming has become a promising area of technology for end users, content providers, wireless operators, and other entities. While wireless technologies, such as 2.5G or 3G, will have more bandwidth and despite the fact that some advanced compression techniques enable very low bit rate streaming, there are several inherent problems when they relate to the wireless environment. The multicast environment described above further makes such a problem more difficult, where it may not be optimal for a client device to switch to another multicast group in order to get a more favorable signal — the client device may not have other better choices than remaining in the same multicast group and "fully utilizing" the situation.

Areas of wireless streaming applications that encounter such problems include real-time media applications (including both audio and video streaming), real-time audio applications (such as live music or sports broadcasts), offline media applications, and offline audio applications. Unlike wired networks, wireless networks suffer from high rates of effective packet loss and intermittent packet delay, among other problems. Packet loss and delay can be caused by several factors, in addition to the effects of fading, which is an inherent feature of wireless networks, such as network congestion, bit error rates, or data overflow at the user device.

In addition to packet loss, there are other factors that adversely affect the media received by the end user. The impact of any of these factors on the user experience may vary significantly depending on communication channel conditions, user device characteristics, environmental conditions, intentional or unintentional events occurring during communication, or other impacts.

All of the above and other factors ultimately adversely affect end user quality of experience (QoE) in mobile wireless multicast environments in media delivery and consumption scenarios, where streaming is just one example of media delivery. These same or other factors may also adversely affect end user multicast sessions in a hardwired communication network. These same or other factors may also adversely affect the broadcast session in a hardwired or wireless communication network, especially since all users typically receive the same signal in a broadcast environment and thus the ability to switch to a different signal is limited (if possible).

Disclosure of Invention

According to one aspect, a method for delivering streaming content to a plurality of client devices is provided. The method includes associating a client device with a group. Multiple simultaneous unique streams are communicated to each group to allow client devices in each group to receive one of the unique streams, respectively. Metric data is received and evaluated with respect to delivering the stream to the client device. Causing at least one of the client devices to switch to a different stream in a same set based at least in part on the estimated metric data, wherein the different stream is more favorable than a current stream in the same set. If a suitable flow is not available in a current set, causing the at least one of the client devices to switch to a different flow in a different set based at least in part on the estimated metric data, wherein the different flow is more suitable than a current flow in the current set.

Drawings

Non-limiting and non-exhaustive embodiments of the present invention are described with reference to the following figures, wherein like reference numerals refer to like parts throughout the various views unless otherwise specified.

FIG. 1 is a block diagram of an exemplary system in which one embodiment of the invention may be implemented.

Fig. 2 is a block diagram illustrating delivery of content to groups of client devices using multiple streams, according to one embodiment.

Figure 3 is a flow diagram of an embodiment of a technique to deliver content to sets of client devices, including switching between streams and sets, according to one embodiment.

Detailed Description

Several embodiments of techniques for providing streaming data to client devices in a multicast and/or broadcast environment are described herein. In the following description, certain specific details are set forth in order to provide a thorough understanding of various embodiments. However, it will be understood by those skilled in the art that the present systems and methods may be practiced without these details. In other instances, well-known structures and protocols associated with communication devices and protocols have not been shown or described in detail to avoid unnecessarily obscuring descriptions of the embodiments.

Reference throughout this specification to "one embodiment" or "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment. Thus, the appearances of the phrases "in one embodiment" or "in an embodiment" in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

The headings provided herein are for convenience only and do not interpret the scope or meaning of the claimed invention.

In general, an embodiment provides a technique for delivering streamed data from at least one server to a plurality of client devices in a multicast and/or broadcast environment. Streaming data may be communicated over one or both of wireless and hardwired communication networks. Streaming data may include, but is not limited to, video, audio, files, live or recorded programs, multimedia content, services, other types of data, or combinations thereof. Client devices may include portable wireless devices, such as cellular telephones, laptop computers, and the like. Client devices may also include wired devices (e.g., personal computers), workstations, and the like.

In an embodiment, a client device may subscribe to or otherwise join a multicast group to receive data being delivered by a server. The server may have multiple signals of, for example, multiple different bit rates for each multicast group stream. Thus, each multicast group may be provided with a different plurality of bit rates. The client devices in the various multicast groups may tune to particular signals corresponding to, for example, their appropriate bit rate, resolution, format, etc.

In one embodiment, if the client device characteristics and/or channel conditions indicate that the switch will provide a more suitable service, the server may instruct the client device to change/switch to a different signal (e.g., having a different bit rate) provided in the same multicast group. In one exemplary embodiment, the server has multiple flows available for a group, but the server streams the most appropriate (e.g., most appropriate for the entire group) flow (one flow at a time) at one particular time and debugs seamlessly for the group. In this embodiment, the server (or client device) exchanges signals only when it is involved in changing groups. In another embodiment, multiple streams are provided by a server to a set, and each of the streams is customized for a particular client device in the set, and further, the client devices (single, plural, or all) may change to different streams in the same set or different streams in another set.

In an embodiment, the server may also instruct the client device to switch to a different signal available in some other multicast group (i.e., instruct the client device to switch to another multicast group) in order to obtain a more appropriate service. This switch from one multicast group to another may be performed if the present multicast group does not provide the client device with the required bit rate and/or the end user experience is generally more appropriate for some other multicast group.

Various techniques may be used to gather and use information before, during, and after a streaming session to determine whether a client device should switch to another signal in the same multicast group and/or switch to another signal in a different multicast group. For example, one embodiment uses Dynamic Bandwidth Adaptation (DBA) and/or quality of experience (QoE) techniques to assist in making the determination.

FIG. 1 is a block diagram of a system in which an embodiment of the invention may be implemented. At least one server 100 receives content from one or more content providers 102. The content provided by the content provider 102 may include, but is not limited to, audio, video, movies, music, live or recorded programs, files, games, internet content, multimedia, instant messaging, on-demand content, or any other type of data that may be transmitted by the server 100 to the plurality of client devices 104. In one embodiment, the server 100 may transmit/deliver such content by using streaming technology. Alternatively or additionally, the server 100 may use download or other delivery techniques to deliver this content or other data.

One embodiment of the server 100 incorporates a Dynamic Bandwidth Adaptation (DBA) module 106 and a QoE server module 108. Various other embodiments of the server 100 may also include a quality of service (QoS) module 112, a transcoder module 110, and/or other modules and components. The various modules may include separate modules in certain embodiments, while in other embodiments the various different functionalities may be exhibited or otherwise combined into a single module. Each of these modules and their associated functionality will be described next.

With respect to DBAs, one embodiment of the invention allows streaming of audio or video (a/V) data, for example, to precisely match the bandwidth of the channel to the client device 104. This dynamic bandwidth adaptation consists of two components: (1) monitoring a fluctuating bandwidth of a channel; and (2) the DBA module 106 of the server 100 is able to change the streaming bit rate in real-time to match or otherwise compensate for changes in channel bandwidth. By using this bandwidth monitoring and bit rate adaptation technique, the user is able to receive relatively smooth video and clear audio.

According to an embodiment of the present invention, a DBA algorithm is provided that achieves stable streaming quality and smooth transitions during periods of congestion and network resource fluctuation. The end-user experience is optimized by adapting the rich media stream to the client bandwidth, which characteristics are adapted to the characteristics or conditions of the wireless network (e.g., shared packet network). The DBA algorithm automatically adjusts the audio and video bit rates being served by the server 100 based on the available bandwidth of the end users in the network. Thus, the end user can receive the most appropriate bit rate stream.

The DBA algorithm provides congestion avoidance and rate control throughout the streaming time by monitoring the channel to each client for statistically significant and sustained variations in bandwidth associated with packet loss, delay, or delay variations. When these changes occur and when there are existing closely matching, but lower bit rate streams, the DBA module 106 switches to the lower stream for a particular client device 104. Switching to a higher bit rate stream may also be performed if bandwidth conditions are improved to allow switching to an appropriate higher bit rate transmission.

Examples of parameters that may be used to determine whether to increase or decrease the bit rate may include, but are not limited to, maximum delay, delay variables, maximum packet loss, instantaneous and previous packet loss, and other parameters. The packet loss or other parameters may also be based on any one or more of average, cumulative, or consecutive packet losses, for example, in order to obtain the most informed and accurate determination of bandwidth conditions. In one embodiment, the parameters of the base delay or delay variable (which is calculated by determining the base delay and subtracting the base delay from the instantaneous delay to obtain the value of the delay variable) add tracking information and can be used to further determine, improve and optimize bandwidth adaptation according to client device or network/operator requirements.

In a streaming session, an embodiment of the DBA module 106 selects the appropriate track at the beginning of the session. Then, if the bandwidth drops or increases during the stream, the DBA algorithm will suggest a bit rate adjustment. In generating this recommendation, an embodiment of the DBA algorithm multiplies the statistical and sustained behavior of the channel by a factor, but does not react to transient spikes. As a result, the media quality does not change abruptly, and the bit rate change occurs smoothly and gradually.

It should be noted that an embodiment of the DBA module 106 may work for a group of client devices. Thus, the DBA module 106 may analyze data from all or a subset of client devices and make decisions for the entire group (and/or a subset of client devices in the group). This decision may be optimal for some clients in the set, but not optimal for other clients. For example, if 8 of 10 client devices (the majority of the client devices in a set) may receive up to 64kbps, the remaining two client devices that wish to receive 128kbps may not reach that bit rate, since that may not be the best decision. In that case, it is preferable for these "two client devices" to switch to the other set or hold 64 kbps. It should be noted that each group may further have a switching range. For the set in the example, it may be 32 to 128 kbps.

An exemplary embodiment of the DBA module 106 is described in more detail in U.S. application No. 10/452,035 entitled METHOD AND APPARATUS FOR dynamics band with advance number "filed on 30/5/2003, which is assigned to the same assignee as the present application AND is incorporated herein by reference in its entirety.

With respect to QoE, the QoE framework upon which an embodiment of the QoE server module 108 is based provides a technique to monitor and handle QoE issues that may arise during communications between network components. For example, when media is being communicated from the server 100 to one of the client devices 104, QoE issues may arise during communication between the server and one of the client devices 104. The components of the QoE framework of an embodiment include: start and stop procedures, which define the start and end of a session, respectively; a negotiation process in which the server 100 negotiates with the client device 104 which QoE metrics to use during a session; one or more QoE metrics defined and implemented (e.g., collection/measurement of metric values); communicating during a session of metric values, the metric values being relative to metrics at a predetermined frequency and for a predetermined range of the session, all metric values having been accepted during the negotiation; and analysis/application of the metric values to estimate QoE and adjust conditions so that QoE can be improved (if necessary).

Thus, the QoE framework of an embodiment evaluates end-user experience in a communication environment (e.g., a wireless communication environment). QoE uses a combination of deterministic and subjective methods for analyzing and providing satisfactory data delivery, reliability, availability, scalability, speed, accuracy, efficiency, etc. An embodiment of the QoE framework on which the QoE server module 108 is based is a cross-protocol layer concept involving networks, transport, operating systems, players, codecs, client features, and/or any application layer specific issues, as well as other issues that may span different communication protocols. Also, in one embodiment, QoE parameters (e.g., QoE metric data) may be separated for media and session level details. That is, the QoE framework of an embodiment examines metric data from various combinations of sources that may affect the end user experience and provides this metric data to the server 100, such that the QoE server module 108 can determine whether and in what manner QoE can be improved for any one or more of the client devices 104.

As mentioned above, an embodiment of a QoE framework may involve a negotiation process, which may be performed between QoE server module 108 and any one or more of the client devices 104. An embodiment of the negotiation protocol may be summarized as follows: initiating a session between the server 100 and one of the client devices 104; some QoE metrics may or may not be supported by one or both of the server 100 and the client device 104; likewise, client device 104 may select a subset that includes the QoE metrics, which supports a particular session; client device 104 and server 100 may thus participate in a negotiation process, which may involve several round-trip exchanges to determine which QoE metrics should be sent by client device 104, how often the supported/accepted QoE metrics should be sent, how to activate and/or deactivate QoE metrics, the content or values that the accepted QoE metrics will include, and other QoE metric related factors; QoE metric value measurement and collection by client device 104; communicating the QoE metric values from the client device 104 to the server 100; and termination of the session. The communicated metric values may be evaluated to determine whether the QoE can or should be improved during the streaming session and/or for subsequent sessions.

In an embodiment, the client device 104 measures QoE metrics at the transport layer, but may also measure at the application layer for better accuracy. The reporting period of QoE metrics may be the period over which a set of metrics is calculated. The maximum value of the reporting period may be negotiated through a QoE negotiation protocol. In other embodiments, one or more QoE metrics may be measured by elements in addition to client device 104 or in place of client device 104 and then communicated to server 100 and/or to client device 104.

In an embodiment, at least some of the metrics are indicative of a characteristic affecting quality in the communication environment or some other indication or outcome of the communication channel. These QoE metrics may be measured at a protocol stack of the client device 104, an application of the client device 104, a buffer of the client device 104, a codec of the client device 104, or other client features that may be related to QoE, or any combination thereof. The metrics may be used to adjust behavior at any of these layers at the server 100 and/or at the client device 104.

The following exemplary QoE metrics may be derived by the client device 104 implementing the QoE: corruption duration, rebuffering duration, initial buffering duration, consecutive packet loss, frame rate deviation, jitter duration, and/or any other parameter (alone or in combination) that may affect QoE. It should be appreciated that these QoE metrics are not the only metrics that can be used for QoE purposes. These QoE metrics may be supplemented by, replaced by, modified by, combined with, etc. The QoE metrics may be applied to, for example, audio, video, voice, and timed text media types, as well as other media types.

The QoE server module 108 of one embodiment is responsible for quantifying the impact of several factors on the media being delivered, including network conditions, client characteristics, etc. The QoE server module 108 does this by gathering feedback from the client device 104. Features and characteristics of various embodiments of the QoE server module 108 may be described as follows:

the QoE server module 108 may reside on a streaming server, such as the server 100.

The QoE server module 108 may reside on an RTSP proxy or any other suitable network device.

The QoE server module 108 may accept input from various protocols.

The QoE server module 108 configuration may be stored in an SDP file or generated by a server/proxy.

QoE server module 108 may interact with DBA module 106:

. Decision to influence a statistically based QoE result to increase bit rate

. Influencing decisions to increase bit rate based on subjective QoE results

. Decision to influence statistics-based QoE results to reduce bit rate

. Influencing decisions to reduce bit rate based on subjective QoE results

. The following features may also be increased/decreased or affected/changed based on subjective and/or statistical QoE results: frame rate, refresh interval and behavior, error resilience (err resilience), buffering behavior, maximum frame size, peak bit rate, fragmentation, retransmission, and/or other characteristics.

. If the DBA module 106 is turned on:

■ the QoE may affect rate adaptation (configurable).

■ in one embodiment, the reporting is controlled by the DBA module 106.

. If the DBA module 106 is disconnected, then:

■ in one embodiment, the QoE server module 108 has no impact on rate adaptation, but may impact rate adaptation in another embodiment.

■ are controlled by the QoE server module 108 in one embodiment, but are controlled by other modules or components in another embodiment.

. In an embodiment, if both the DBA and QoE modules 104 and 108 are disconnected, reporting may be controlled by the QoS module 112.

The QoE server module 108 may operate in one or both of the following modes:

. Statistical model

. Subjective mode

. Details are as follows: the metrics returned from the client device 104 to the server 100 may be used/organized within the QoE server module 108 in a variety of ways. One way is "statistical mode". Here, the QoE server module 108 organizes the statistics of metrics in the form of min, max, etc. The second way is "subjective mode". Here, the QoE server module 108 organizes the statistics by mapping the metrics it receives to quality of service levels. Thus, for example, after reviewing the metrics, QoE server module 108 may determine that a particular metric belongs to an intermediate quality level. Thus, this information can be used for confirmation purposes. For example, if client device 104 subscribes to a high quality level, but for this particular session, it is determined that this session only belongs to an intermediate quality level based on metrics received by server 100, then this information may be used for many purposes. There may be many other analyses of the metrics received by the QoE server module 108.

QoE statistics pattern:

. Performing calculations at the media or session level

. Making measurements during a single period or an entire session

. Calculating minimum, maximum, mean and standard deviations of at least:

■ corruption duration

■ rebuffering duration

■ initial buffer duration

■ continuous loss

QoE subjective mode:

. Performing calculations at the media or session level

. Measurements are taken during the entire session (without a single periodic report)

. Providing mapping to predetermined QoS classes

■ best effort or streaming level,

■ low, medium, or high QoE level.

. Providing isolation of potential problem locations:

. Link layer

. Network protocol stack

. Codec stack problem

. Client application problem

. Clipping problem

. Others

An exemplary embodiment of the QoE server module 108 is described in more detail in published PCT application serial No. PCT/US2004/027618 entitled "quality of EXPERIENCE (QoE) METHOD AND APPARATUS FOR wireless communication network," filed on 23/8/2004, AND also described in more detail in its related priority application, which are assigned to the same assignee as the present application AND are incorporated herein by reference in their entirety.

With respect to QoS, QoS module 112 of one embodiment balances the negotiated maximum bit rate, guaranteed bit rate, and maximum transfer delay parameter between client device 104 and the network. QoS module 112 also weighs any additional network layer data such as losses, delays, and other metrics associated with higher order device and network metrics. The QoS framework upon which one embodiment of QoS module 112 is based specifies a guaranteed level of service, and is typically implemented as network QoS, by using deterministic and subjective methods with respect to analyzing and providing consistent and predictable data delivery services.

With respect to the transcoder and other modules 110, an embodiment receives a single input (e.g., raw video) from the content provider 102 and converts the input into a plurality of simultaneous outputs (e.g., output video streams) having unique characteristics. Thus, a one-to-many technique is provided for converting input data to output data in a single transcoding session. For example, the outputs may have different bitrates, frame rates, resolutions, encoding formats, color schemes, or other characteristics that are customized based on one or both of client device characteristics or bandwidth conditions. The client device 104 may thus have an appropriate output provided to it by the server 100, and may switch to some other output under the direction of circumstances, which may be, for example, circumstances resulting from determinations made by the QoE module server 108, the DBA module 106, and/or the QoS module 112. Exemplary embodiments of these transcoding techniques are described in more detail in U.S. patent application serial No. 09/502,390 entitled "COMPUTER PROGRAM PRODUCT formatting STREAMING VIDEO DATA," filed on 10/2/2000, assigned to the same assignee as the present application and incorporated herein by reference in its entirety.

All of these modules cooperatively ensure that the user experience in a multicast and/or broadcast environment is as desired, and even monitor the entire streaming session under network conditions that can vary drastically. For example, if the DBA module 106 determines that a particular client device 104 needs to change to a lower bit rate signal, the server 100 may command or otherwise cause that client device 104 to switch to the lower bit rate signal generated by the transcoder 110. Similar handovers and/or signal adaptations to other signals may be performed based on determinations made by the QoE server module 108 and the QoS module 112.

Although the QoE server module 108 and other modules in fig. 1 are shown as residing in the server 100, it should be appreciated that the QoE server module 108 (or any of the other modules) may be located elsewhere, as appropriate, in a wireless or hardwired network. For example, the QoE server module 108 may be located at a proxy device, router, switch, or other network component, including at the client device 104 in some embodiments.

Continuing with the description of FIG. 1, the server 100 and the client device 104 may communicate with each other by communicating 114 using one or more protocols. Multimedia broadcast/multicast service (MBMS) configuration and its associated protocols may be used in one embodiment to allow communication between the server 100 and the client devices 104. MBMS further utilizes other communication protocols and services such as Packet Switched Streaming (PSS), real-time transport protocol (RTP), Session Description Protocol (SDP), User Datagram Protocol (UDP), hypertext transfer protocol (HTTP), Internet Protocol (IP), real-time transport control protocol (RTCP), file delivery over unidirectional transport (FLUTE), and others. These and/or other protocols and services and methods may be used in other embodiments not involving MBMS.

In one embodiment, the protocol communication may be implemented in a multicast and/or broadcast manner over the wireless and/or wired network 118. Examples of wireless networks include cellular or other RF, satellite, and optical networks. Examples of wired networks include the Public Switched Telephone Network (PSTN), coaxial cable networks, and other types of networks. In one embodiment, a combination of wired and wireless networks may be implemented in the network 118. In yet another embodiment, the network 118 may include or be coupled to the Internet.

In one embodiment, one or more back channels or other channels 116 may be provided for additional communications between the server 100 and each of the client devices 104. The channels 116 may also include the same channels that are themselves used to deliver content (e.g., multicast/broadcast streaming content) from the server 100 to each client device 104. One purpose of channel 116 is to transmit QoE, DBA, and/or QoS metric data from client device 104 to various modules of server 100. For example, RTCP Receiver Report (RR) packets that convey stream statistics may be communicated by the client device 104 to the server 100.

Another purpose of the channel 116 may be to communicate control signals from the server 100 to any of the client devices 104. For example, the channel 116 may be used to communicate instructions from the server 100 to a particular client device 104 to switch to a different stream in the same multicast group or a different stream in some other multicast group.

Fig. 2 illustrates delivery of content to a client device, for example, streaming in a multicast environment, according to one embodiment. Client devices subscribe to or otherwise join a multicast group A, B, C, etc. (labeled 200, 206, and 208, respectively). For example, client devices 202, 204, etc. are subscribers to multicast group a (200). Each of the sets 200, 206, and 208 may include at least one to any suitable number of client devices.

Different criteria may be used to define a "group". For example, the set may be defined according to some signal characteristic (e.g., set 200 is associated with signals having a bit rate of W bps to X bps, set 200 is associated with signals having a bit rate of Y bps to Z bps, set 200 is associated with signals having W bps and Z bps instead of X bps, etc.). Other types of signal characteristics that may be used to define a group include resolution, encoding format, frame rate, color scheme, and other aspects. In some cases, similar signal characteristics may exist between groups, while other signal characteristics are different.

Other criteria or combinations thereof may be used instead of or in addition to signal characteristics to define the sets 200, 206, and 208. For example, the groups may be defined based on the geographic location and/or IP address of the client device and/or server 100. Groups may also be defined based on content type (e.g., streaming video versus streaming audio versus graphical messaging, etc.).

Other possibilities include business rules, such as defining groups based on user fees for subscriptions (e.g., "basic," "medium," or "premium" subscription packages, where more or better service is made available to the customer depending on the subscription package selected by the customer). A subscription package may be associated with a particular flow. In these or other types of packet definitions, supplemental content may be added to the same base content provided by the stream. For example, all streams may provide the same video program, while some streams may not include commercials or advertisements if the user has selected a "premium" subscription package that excludes advertisements and commercials. Thus, users can join a "premium" multicast group that streams a continuous signal, and the signal is free of advertisements and commercials. In other embodiments, the type of supplemental content provided along with the base content may be customized for a single or a plurality of customers based at least in part on user profile data specifying the subscription level of the user, as well as other possible data about the user (e.g., interests, demographics, hobbies, etc.).

In an embodiment, another module 110 may perform evaluation and enforcement associated with the business rule. For example, another module 100 may evaluate user profile data and its subscription level to determine which stream should be delivered to a particular group or user. This evaluation may include determining whether different streams should be delivered to a user or group (e.g., when a user changes subscription packages to a higher or lower level), and may further perform other operations associated with enforcing business rules. In an embodiment, the transcoder of module 110 may also perform transcoding, which is based at least in part on business rules. For example, if a user is currently subscribed to receive a low resolution stream and then subscribes to receive a higher resolution stream, the transcoder may convert the stream of the set being provided to the user so that the user is receiving the higher resolution stream.

Other possibilities include defining groups based on the subject of their content, e.g. the kind of content they provide (drama, comedy, opera, etc.), and/or based on the time of transmission. Briefly, embodiments may define a multicast group using any one or more of the possible criteria.

In an embodiment, the sets 200, 206, and 208 each receive a plurality of streams 210, 212, and 216, respectively, such that their subscribing client devices can tune to receive the content carried by the streams. For example, the set 200 may receive multiple streams 200 from the server 100 over the network 118, where each stream has one or more of different bit rates, resolutions, frame rates, or other signal characteristics. Thus, the subscribing client devices in the set 200 can each tune to the most appropriate one of the multiple streams 200, e.g., a stream having a bit rate compatible with the capabilities of the respective subscribing client device.

In another embodiment, multiple streams 210, 212, and 216 are available, but only a single stream per group is active at a time. Thus, all client devices in the group receive the same flow, which may be updated based on DBA, QoE, etc., metric data. In such embodiments, if the conditions indicate, all client devices seamlessly receive updated/different streams available in the same group. If there are some client devices in the same set where the updated stream is not appropriate, such client devices may remain in the same set and receive the updated stream, or switch to some other set to receive the more appropriate stream.

In an embodiment, the server 100 is capable of providing server signals 218, 222, and 226 (or other data or control commands) to any client device in any of the groups 200, 206, and 208, respectively. For example, such server signals 218, 222, and 226 may be communicated over channel 116 (shown in fig. 1), and may include instructions to cause one or more client devices to tune to particular signals available in their multicast group, to cause one or more client devices to tune to particular signals available in another multicast group, requests for QoE/DBA/QoS metric data (including stream statistics and QoE reports, which may be sent by client devices before, during, and/or after a streaming session), instructions to cause one or more client devices to connect or disconnect from the group (or a confirmation that a client device requested connection or disconnection), QoE negotiation data, or other information or commands that may be associated with a streaming session.

The return data and/or requests provided by the client devices in the groups 200, 206, and 208 may be provided back to the server 100 as client signals 220, 224, and 228, respectively, over the channel 116. Examples of such return data and/or requests include, but are not limited to, QoE/DBA/QoS metric data (including QoE reports mentioned above), requests to connect to or disconnect from a multicast group, requests for a server to identify flows and/or groups that are appropriate for a particular client device, QoE negotiation data, or other commands and data.

As described above, an embodiment allows at least one client device to switch to a different flow if, for example, the server 100 determines that the QoE or DBA results indicate that the different flow may provide a more suitable service. According to one embodiment, the server 100 may signal a single client device (e.g., client device 202 in the group 200) to switch and/or may signal multiple client devices in the same group (e.g., client devices 202 and 204 in the group 200) to switch.

In yet another embodiment described above, the client devices in a group all receive the same stream (e.g., the same bit rate) rather than individually customized streams, even though the client devices may be identified as having different needs. Thus, in this embodiment, the client device may consistently have an updated stream, and/or the client device (for which the updated stream is not appropriate) may have the option to switch to another set.

A server signal that switches to a different stream (e.g., a stream having a different bit rate) may include a signal that switches to a different stream in the same group. That is, because server 100 provides multiple different streams to each group 200, 206, and 208, the client devices therein have more flexibility in selecting different streams available in the same group rather than having to switch to another group. Maintaining group "membership" of client devices (i.e., maintaining client devices in the same group) minimizes the potential adverse effects of "group switching" previously discussed.

In an embodiment, switching between groups is possible, for example, if no suitable stream is available in an existing group for a client device that needs to change streams. In this case, the server 100 may signal these client devices to change to a new group, and/or the server 100 may provide information to these client devices to enable them to decide when and to which group to switch. Also, QoE metric data, DBA metric data, or other data indicative of client device characteristics and/or bandwidth conditions, all of which may be dynamically varied, may be used to determine whether, when, and where a client device should switch.

In one embodiment, the plurality of streams 210, 212, 216, etc. may be provided using layer techniques. For example, a single base layer may be provided to each group 200, 206, and 208, or multiple base layers may be provided to each group 200, 206, and 208 via separate multiple streams. The base layers may be completely different between the groups, or some of the base layers may be the same.

In addition to the base layer, the multiple streams to each group 200, 206, and 208 may also provide various enhancement layers. The various enhancement layers may, for example, have different bitrates, and then the client device and/or server 100 may select a particular enhancement layer for use by each client device. In other levels of granularity, each enhancement layer may itself be allocated a plurality of different bit rates.

Layering may also be replaced or otherwise performed based on other features, such as client device features, codec type, frame rate, resolution, color design, and other criteria. As before, the client device may switch (if needed) to different layers available in the same group, or to different layers available in other groups.

In a broadcast environment, signals similar to those illustrated in fig. 2 may be provided. That is, multiple streams with different characteristics may be available to a client device. Initially, the server 100 will transmit (or the client device will tune to) a particular one of the streams that would result in the most appropriate result, such as a stream with the appropriate bit rate. QoE, DBA, QoS, and/or other metric data may be gathered before, during, or after a streaming session to estimate whether a handoff to a different flow should be performed. If the estimate of metric data suggests that a change may be needed, the client device may switch to a more appropriate flow.

Since broadcasting involves sending the same (or similar) content to all client devices, it is not necessary to define groups in an embodiment (i.e., have only "one" broadcast group). It should be understood, however, that multiple broadcast groups may actually be defined by using similar criteria as described above for defining multicast groups.

Fig. 3 is a flow diagram of an embodiment of a technique 300 to deliver content to a client device, such as by streaming, in a multicast or broadcast environment. In an embodiment, at least some of the elements of the technique 300 may be implemented in software or other machine-readable instructions stored on a machine-readable medium, such as a memory in the server 100, having code stored thereon that is executable by one or more processors. It should be understood that the various operations shown in the technique 300 need not be performed in the exact order shown, and that the various operations may be added, removed, modified, combined, or processed in any combination thereof, as appropriate.

At block 302, a group (e.g., a multicast group) is configured and client devices are subscribed into the group. For example, client devices may join a group to receive a broadcast of a particular streaming video program. At block 304, content is received from the content provider 102 and transcoded or otherwise converted by the transcoder 110 of the server 100 into a plurality of unique simultaneous output streams, such as simultaneous video streams having different bit rates.

At block 306, the plurality of output streams are passed to the client device through the groups in a manner such that the plurality of different output streams are available for each group. In one embodiment, the client devices in each group may then tune to receive the most appropriate stream. In another embodiment as described above, client devices in the same group receive the same stream. In one embodiment, the server 100 instructs the client device to which stream to tune, and in another embodiment, the client device determines which stream it wishes to receive.

At block 308, metric data is transmitted from the client device to the server 100. This metric data may include the QoE metric data, DBA metric data, and/or QoS metric data described above or other relevant metric data that provides an indication as to the quality of experience/service. Metric data may be gathered and communicated before, during, or after a streaming session. The various modules of the server 100 shown in fig. 1 estimate the metric data.

At block 310, the server 100 determines whether one or more of the client devices need to switch streams. If no switching of streams is required, the server 100 continues to receive and evaluate metric data at block 308. If the estimate of metric data suggests a change, the server 100 determines at block 312 whether a suitable stream can be used in the same multicast group or otherwise determines whether a handoff within the group is possible.

If a switch within the group is possible, the client device is switched to a new flow at block 314. Otherwise, the server 100 looks for a suitable stream in another group. If the server 100 locates this flow, the client device is switched to the flow (in a different set) at block 316. The process is repeated to continuously monitor the metric data to determine if a stream switch needs to be performed subsequently.

All of the above U.S. patents, U.S. patent application publications, U.S. patent applications, foreign patents, foreign patent applications and non-patent publications referred to in this specification and/or listed in the application data sheet, are incorporated herein by reference, in their entirety.

The above description of illustrated embodiments, including what is described in the Abstract of the disclosure, is not intended to be exhaustive or to limit the invention to the precise forms disclosed. While specific embodiments of, and examples for, the invention are described herein for illustrative purposes, various equivalent modifications are possible within the scope of the invention, and various equivalent modifications may be made without departing from the spirit and scope of the invention.

These and other modifications can be made to the invention in light of the above detailed description. The terms used in the following claims should not be construed to limit the invention to the specific embodiments disclosed in the specification and the claims. Rather, the scope of the invention is to be determined entirely by the following claims, which are to be construed in accordance with established doctrines of claim interpretation.

Claims (22)

1. A method for delivering streaming content to a plurality of client devices, the method comprising:

associating the client device with the group;

passing a plurality of simultaneous unique streams to each group to allow client devices in each group to receive one of the unique streams, respectively;

receiving and evaluating metric data regarding delivery of the stream to the client device;

causing at least one of the client devices to switch to a different stream in a same set based at least in part on the estimated metric data, the different stream being more favorable than a current stream in the same set; and

if a suitable flow is not available in a current set, switching the at least one of the client devices to a different flow in a different set that is more suitable than a current flow in the current set based at least in part on the estimated metric data.

2. The method of claim 1, wherein associating the client devices with groups comprises associating different client devices with respective different multicast groups.

3. The method of claim 1, wherein associating the client device with a group comprises associating the client device with at least one broadcast group.

4. The method of claim 1, wherein causing the at least one client device to switch to a different stream comprises causing the client device to switch to a stream having a different characteristic based on client device capabilities and channel conditions, wherein the characteristic may comprise bit rate, frame rate, resolution, encoding format, color scheme, and user profile information, wherein at least one of these characteristics may be the same for the unique stream while some other characteristic is different in the stream.

5. The method of claim 1, wherein receiving and estimating metric data comprises receiving and estimating one or both of:

quality of experience (QoE) metric data; and

dynamic Bandwidth Adaptation (DBA) metric data regarding client device characteristics and channel conditions.

6. The method of claim 1, wherein causing the at least one of the client devices to switch to a different stream comprises sending server-generated instructions to a plurality of client devices.

7. An article of manufacture, comprising:

a machine-readable medium having instructions stored thereon that are executable by a processor to deliver streaming content to a plurality of client devices associated with a multicast group by:

making multiple unique streams available to each group to allow most client devices in each multicast group to receive one of the unique streams that is appropriate for the entire group;

receiving and evaluating metric data regarding delivery of the stream to the client device;

causing a majority of the client devices to switch to a different stream in a same multicast group that is more favorable than a current stream in the same multicast group based at least in part on the estimated metric data; and

if a suitable stream is not available in a current multicast group, at least other of the client devices are switched to a different stream in a different multicast group that is more suitable than a current stream in the current multicast group based at least in part on the estimated metric data, and otherwise the client devices remain in the current multicast group.

8. The article of manufacture of claim 7, wherein the instructions that cause the at least one client device to switch to a different stream comprise instructions to cause the client device to switch to a stream having a different bit rate.

9. The article of manufacture of claim 7 wherein the machine-readable medium further comprises instructions stored thereon to convert, including transcoding, a single input having content into the plurality of simultaneously unique streams having the content.

10. The article of manufacture of claim 7, wherein the instructions to receive and estimate metrology data comprise instructions to receive and estimate one or both of:

quality of experience (QoE) metric data; and

dynamic Bandwidth Adaptation (DBA) metric data with respect to client device characteristics and channel conditions, wherein the QoE and the DBA metric data may be with respect to certain ones of the client devices in a group and used to determine a different stream to provide to all client devices in a group.

11. The article of manufacture of claim 7, wherein the instructions to communicate the plurality of simultaneous unique streams comprise instructions to communicate the base layer and the enhancement layer as a plurality of unique streams.

12. A system for delivering streaming content to a plurality of client devices, the system comprising:

means for associating a client device with a group;

means for communicating a plurality of simultaneous unique streams to each group to allow client devices in each group to receive one of the unique streams, respectively;

means for receiving and for estimating metric data regarding delivery of the stream to the client device;

means for switching at least one of the client devices to a different flow in a same set based at least in part on the estimated metric data, wherein the different flow is more favorable than a current flow in the same set; and

means for switching the at least one of the client devices to a different stream in a different set based at least in part on the estimated metric data if a suitable stream is not available in a current set, wherein the different stream is more suitable than a current stream in the current set.

13. The system of claim 12, wherein the means for receiving and for estimating the metric data comprises means for receiving and for estimating quality of experience (QoE) metric data, the system further comprising:

means for negotiating QoE metric data between a server and a client device; and

means for estimating the QoE metric data in conjunction with rate adaptation data to determine whether to switch to a different flow.

14. The system of claim 12, further comprising means for instructing one or both of a single client device and a plurality of sets of client devices to switch to a different stream.

15. An apparatus for delivering streaming content to a plurality of client devices associated with a multicast group, the apparatus comprising:

a server that passes a plurality of unique streams to each group to allow client devices in each multicast group to receive one of the unique streams; and

at least one module in communication with the server to receive and evaluate metric data regarding delivery of the stream to the client device;

wherein, based at least in part on the estimated metric data, the server can cause at least one of the client devices to switch to a different stream in a same multicast group, wherein the different stream is more favorable than a current stream in the same multicast group; and is

Wherein, if a suitable stream is not available in a current multicast group, the server may cause the at least one of the client devices to switch to a different stream in a different multicast group based at least in part on the estimated metric data, wherein the different stream is more suitable than a current stream in the current multicast group.

16. The apparatus of claim 15, wherein the at least one module comprises:

a first module that receives and estimates quality of experience (QoE) metric data; and

a second module that receives and estimates Dynamic Bandwidth Adaptation (DBA) metric data regarding client device characteristics and channel conditions, at least some of one or both of the QoE metric data and DBA metric data being usable to determine whether to switch to a different flow.

17. The apparatus of claim 15, further comprising:

a third module that receives and estimates metrics data other than the QoE and DBA metrics data; and

at least one additional module including a fourth module to convert a single input code into the plurality of simultaneously unique streams.

18. The apparatus of claim 15, further comprising:

means for delivering a plurality of unique streams to client devices in a broadcast environment; and

means to switch at least some of the client devices from one stream to another stream in the broadcast environment based at least in part on metric data.

19. The apparatus of claim 15, wherein the stream is unique in that it includes at least one characteristic that is based on client device capabilities and channel conditions, wherein such characteristic may include bit rate, frame rate, resolution, encoding format, and color scheme, wherein at least one of these characteristics may be the same for the unique stream while some other characteristic is different.

20. An apparatus for delivering streaming content to a plurality of client devices associated with a multicast group, the apparatus comprising:

a server that delivers a plurality of unique streams to each group to allow client devices in each multicast group to receive one of the unique streams, wherein at least some of the groups may be defined based on business rules, which may include subscription packages; and

at least one module in communication with the server to evaluate and enforce the business rules to determine the subscription package and associated streams to deliver to the client device; and is

Wherein, based at least in part on the estimated business rules, the server can cause at least one of the client devices to switch to a different stream in a same multicast group or a different multicast group if the subscription package of the client device changes.

21. The apparatus of claim 20, wherein the subscription package is associable with a stream having supplemental content, wherein the supplemental content is customized for at least some users in the client devices based at least in part on configuration data.

22. The apparatus of claim 20, further comprising a transcoder to convert an input stream to the server into a plurality of different output streams, wherein the output streams have been customized for the set based at least in part on the business rules.