WO2008018015A1

WO2008018015A1 - Image rate increasing

Info

Publication number: WO2008018015A1
Application number: PCT/IB2007/053102
Authority: WO
Inventors: Franciscus H. Van Heesch; Michiel A. Klompenhouwer; Maurice P. H. Penners; Wim Michiels; Erwin B. Bellers; Johan G. W. M. Janssen
Original assignee: Koninklijke Philips Electronics N.V.
Priority date: 2006-08-09
Filing date: 2007-08-07
Publication date: 2008-02-14

Abstract

An image rate increasing device for increasing an image rate in dependence on whether incoming images originate from film, comprises a first image rate increasing circuit (MC, DFI) for generating film output images at a film output rate from incoming images (F) that originate from film, and a second image rate increasing circuit (DFI) for generating video output images at a video output rate from incoming images (V) that do not originate from film. If an original film rate (e.g. 24 Hz) at which the film was picked up, differs from half an incoming image rate (e.g. 60 Hz) at which the incoming images are received, the film output rate (e.g. 96 Hz) differs from the video output rate (e.g. 120 Hz).

Description

Image rate increasing

FIELD OF THE INVENTION

The invention relates to image rate increasing.

BACKGROUND OF THE INVENTION High-end LCD-TV systems typically discriminate film content from video content and process both input types differently. In case of film, preferably, film judder reduction is applied in order to improve the motion portrayal by means of motion compensated frame rate conversion. Because LCDs typically suffer from motion blur, the motion portrayal of both video and film can be further improved by applying motion blur reduction. Although this can be achieved by applying additional frame rate conversion, often this is too computational intensive and more cost effective techniques are used, such as black frame insertion, grey frame insertion and dynamic frame insertion. The latter one (DFI) has been disclosed by JP-A-2002351382. See also Han-Feng Chen, et. al., Smooth Frame Insertion Method for Motion-Blur Reduction in LCDs, Samsung Electronics Co., Ltd., EuroDisplay 2005. The proposed method described here combines motion compensated frame rate conversion with Dynamic Frame Insertion to improve the motion portrayal of an LCD-TV system. Dynamic Frame Insertion is a motion blur reduction technique that doubles the frame rate, alternating a peaked and blurred version of each input picture that are obtained by spatial filtering. This effectively reduces the hold time of spatial details by a factor of two, while large areas have a hold time of the input frame rate preventing large area flicker.

The motion portrayal of TV signals on LCD-TVs is limited by both source and display, resulting in motion judder if the intrinsic frame rate of the TV signal is lower than approximately 30 φs and motion blur caused by the sample and hold addressing of the LCD. Film judder reduction requires the detection of the film type, and motion compensated frame rate conversion to a frame rate typically above 40 Hz. At this and higher frame rates remaining judder is observed as motion blur. An additional frame rate increase by means of motion compensated interpolation would further remove this motion blur, but this is often too computationally expensive. SUMMARY OF THE INVENTION

It is, inter alia, an object of the invention to provide an improved image rate increasing. The invention is defined by the independent claims. Advantageous embodiments are defined in the dependent claims. In one embodiment of the invention, by combining motion compensated frame rate conversion with Dynamic Fame Insertion a more cost effective implementation is obtained. For optimal motion portrayal on high-end LCD-TV systems, video and film content require a different kind of processing. When a TV-signal is detected as film, motion judder is removed prior to motion blur reduction, while in case of video applying motion blur reduction suffices.

These and other aspects of the invention will be apparent from and elucidated with reference to the embodiments described hereinafter.

BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 shows a basic implementation of the present invention;

Fig. 2 shows a schematic implementation of a general embodiment; Fig. 3 shows a third embodiment of the present invention; Fig. 4 shows a fourth embodiment of the present invention; and Fig. 5 shows a fifth embodiment of the present invention.

DESCRIPTION OF EMBODIMENTS

In accordance with an embodiment of the present invention, motion portrayal enhancement for an LCD-TV system should at least contain film detection, motion judder cancellation, and motion blur reduction, as illustrated in Fig. 1. This leaves the system architect with the options to decide on the output frame rate for both film and video and to decide on the motion blur reduction method to use for an optimal cost/quality tradeoff. Incoming images I are applied to a film detector FD to determine whether the images originated from film. There are several known techniques for film detection; a suitable technique has been described in US 5365280. Images F that have been determined to originate from film are first applied to a motion judder cancellation circuit MJC, and thereafter to a motion blur reduction circuit MBR, while images V that have not been determined to originate from film are directly applied to the motion blur reduction circuit MBR, which furnishes output images O. From a picture quality point of view, motion compensated frame rate up- conversion to the maximum obtainable refresh rate is optimal for both film and video. For modern displays and content this typically entails frame rate up-conversion of HD input material that has an intrinsic frame rate of 25 Hz or 50 Hz (24 Hz or 60 Hz in the US) to a display frame rate of 100 ( 120) Hz .

If cost effectiveness is also taken into account as a design parameter, a hybrid method that uses both motion compensated up-conversion and Smooth Frame Insertion (SFI) might be preferred, since high quality motion compensated frame rate up-conversion is very computationally intensive and therefore expensive. In this case, a distinction between film and video content is made, where motion compensated frame rate up-conversion is applied only to film to remove motion judder.

To optimally reduce motion blur, one could argue that this is then achieved by up-converting the frame rate to exactly half the maximum possible frame rate using motion compensation, such that after applying SFI the shortest possible hold time is obtained. As such, this typically yields a system that performs an up-conversion with a factor of two (two and a half) in case the input material is film to obtain a frame rate above the judder limit, i.e. from 25 Hz to 50 Hz (24 Hz to 60 Hz). A frame doubling using SFI then follows resulting in a motion blur reduced TV-signal at 100 Hz (or 120 Hz in the US). However, in a world where the number of input formats is quickly expanding mainly due to the convergence of the PC and TV market, this design tradeoff has to be re-evaluated for each use case, requiring a video processing system that can dynamically switch between the different up-conversion methods.

Motion portrayal improvement of TV-signals on LCD displays requires motion judder removal and motion blur reduction. Both require a frame rate increase, but where there are multiple options for motion blur reduction, motion judder reduction can only be reduced using motion compensated interpolation. System design considerations determine the optimal method for motion blur reduction and the extent to which motion judder is reduced. To produce an optimal picture quality at any time a video processing system should, therefore, be capable of dynamically switching between different up-conversion methods depending on the input format.

Fig. 2 shows a schematic implementation of a general embodiment. A motion judder detector (e.g. a detector FD for detecting whether the source material was film) determines if the intrinsic frame rate of the input material is high enough to be judder free or not. In case of judder, the input pictures F are temporally up-converted to a frame-rate higher than the judder limit, using motion-compensated frame rate conversion ME+MC, followed by a further frame rate doubling by means of DFI. It should be noted that in this step, the image rate is not adjusted, as both the input images and the output image are at a rate matching that of the TV system, i.e. 50 Hz is Europe, and 60 Hz (or more accurately 59.94 Hz) in the USA. However, image repetition resulting from a fϊlm-to-video conversion is replaced by a smoother motion portrayal. The motion-compensated frames are spatially low- pass filtered (LPF) to serve as a second input to the DFI processing block. In case the input material V does not suffer from motion judder, only Dynamic Frame Insertion (DFI) is applied to obtain a frame rate of twice the input frame rate. In a known manner, DFI processing alternately outputs sharpened and low-pass filtered (LPF) images, the average of the sharpened and low-pass filtered images corresponding to the input images.

Alternatively, in case of input material F impaired with judder, the low-pass filtering LPF and motion-compensated frame rate conversion MC order can be swapped, as illustrated in Fig. 3. This embodiment has the advantage that the low-pass filtering LPF is applied on the input frame rate, which is beneficial in terms of computational costs and/or complexity on certain system architectures. However, now two motion-compensation blocks are present, viz. MC A for the film-originated images F that have not been low-pass filtered and MC B for the film-originated images F that have been low-pass filtered. Both motion- compensated blocks MC A and MC B receive their motion vectors from a motion vector estimation block ME. For the remainder, the embodiment of Fig. 3 corresponds to the embodiment of Fig. 2.

Fig. 4 shows a more detailed embodiment of a video system that discriminates between video (judder free) input material V and film (judder impaired) input material F. Film detector FD determines if the input material I is video or film. In case of film (F) the input picture rate is 24 Hz (or 25 Hz in Europe). The frame rate is doubled to 48 Hz (or 50 Hz in Europe) by means of motion compensated frame rate conversion, using a motion estimator ME to determine the motion in the video sequence and a conversion unit MC to interpolate the moving objects along the motion vector, e.g. by using the techniques described in G. de Haan, "Video Processing for Multimedia Systems", University Press, Eindhoven, The Netherlands, ISBN-90-014015-8, 2000. As usual, a frame memory FM is used for the motion-compensated processing ME / MC. So, in case of 60 Hz input images I, if the film detector FD detects that they originate from film images picked up at 24 Hz and then transformed into TV images at 60 Hz by e.g. a 2-3 (or 3-2) pull down conversion, the motion- compensation algorithm goes back to the original images at 24 Hz (which may involve combining 2 interlaced TV images into 1 de-interlaced 24 Hz image) and produces 48 Hz images by motion-compensated image rate doubling in a known manner. Another factor of two in frame rate increase is obtained using Dynamic Frame Insertion (DFI) by 2D low-pass filtering (LPF) a previous image and the motion compensated image, followed by outputting both the blurred and the derived peaked images. This results in an output frame rate of 96 Hz (or 100 Hz in Europe). In case of video (V) the input picture rate is 60 Hz (or 50 Hz in Europe). This signal is deinterlaced (Deint) and its frame rate is doubled by using Dynamic Frame Insertion (DFI). This results in an output frame rate of 120 Hz (or 100 Hz in Europe).

Alternatively, again motion compensation can be applied on the low-pass filtered image instead of low-pass filtering the motion compensated picture, as depicted in Fig. 5. Here, the low-pass filtered picture is motion compensated (MC LPF) and the input picture is motion compensated (MC IN). Both outputs are used for the Dynamic Frame Insertion processing block DFI to generate a motion compensated peaked and blurred frame. Together with the output frames of the DFI processing block that operates on the previous picture that has been stored in a frame memory the output picture rate is upconverted with a factor of 4.

The input frames preferably are encoded in the linear light RGB domain, but an alternative embodiment applies this processing scheme in the YUV domain, by either processing on the Y-component only or applying similar processing to Y and UV components separately.

In the embodiments of Figs. 4 and 5 a motion-compensated up-conversion of a factor of 2 has been chosen, because this is the minimum integer number to eliminate film judder. Other (non-integer) factors could have been chosen as well.

The embodiments in Figs. 4 and 5 show that motion estimation and motion compensation is performed using a current frame and a stored frame. Other upconversion techniques that use a different number of frames are also possible.

The low-pass filtering that serves as an input to the DFI algorithm in a video system can be applied to either the interlaced or de- interlaced input picture. Improved versions of the DFI algorithm make it motion vector dependent. The DFI block could have its own motion estimation or, more advantageously, use the motion estimation block ME already present in the circuit.

In sum, one aspect of the invention covers a video processing system for both video and film that combines motion judder reduction with motion blur reduction. The invention can be applied in video processing pipelines for TV systems, e.g. backend-scaler ICs on the small-signal board of an LCD-TV or TCON ICs on the LCD panel itself.

It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design many alternative embodiments without departing from the scope of the appended claims. As shown in Fig. 1, the same circuitry may be used for the actual image rate increase (motion blur reduction) both when the incoming images have been determined to originate from film and when the incoming images have been determined to not originate from film. While several embodiments show a distinction between the motion estimation / compensation, and the DFI, an integrated approach is possible in which in a single step both motion compensation and DFI are carried out. While several embodiments include a film detector, no such film detector is necessary if the incoming image signal includes a control signal indicating whether the incoming images originate from film. The scope of the present invention is not limited to DFI processing in relation to LCD devices; the invention may be used in connection with other image rate increasing techniques and in relation with other types of displays. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps other than those listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention may be implemented by means of hardware comprising several distinct elements, and/or by means of a suitably programmed processor. In the device claim enumerating several means, several of these means may be embodied by one and the same item of hardware. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

Claims

CLAIMS:

1. An image rate increasing device for increasing an image rate in dependence on whether incoming images originate from film, the image rate increasing device comprising: a first image rate increasing circuit (MC, DFI) for generating film output images from incoming images (F) that have been determined to have originated from film, at a film output rate that is a first multiple of an original film rate at which the film was picked up, a second image rate increasing circuit (DFI) for generating video output images from incoming images (V) that have not been determined to have originated from film, at a video output rate that is a second multiple of an incoming image rate at which the incoming images are received, wherein if the original film rate differs from half the incoming image rate, the film output rate differs from the video output rate.

2. An image rate increasing device as claimed in claim 1, wherein the film output rate is 96 Hz.

3. An image rate increasing device as claimed in claim 1, wherein both first and second image rate increasing circuits include a unit (DFI) for alternatingly generating sharpened and blurred images.

4. An image rate increasing device as claimed in claim 1, wherein the first image rate increasing circuit comprises a motion compensation unit (MC).

5. A display device, comprising: an image rate increasing device as claimed in any of the preceding claims, and a display unit.