SYSTEM FOR SIGNAL PROCESSING THE DYNAMIC RANGE
AND CONTRAST OF ELECTRONIC IMAGE BRIGHTNESS
WITH RELATED COLOR MODIFICATIONS
FIELD OF THE INVENTION
The present invention generally relates to video or electronic image signal processing, and more particularly systems and techniques for modifying the dynamic range and contrast of video signals.
BACKGROUND OF THE INVENTION
Many electronic images such as news and sporting events are taken under uncontrolled lighting conditions. Thus image dynamic range and contrast may be less than aesthetically pleasing. On the other hand, medical images such as fluoroscopic videos may contain such poor contrast and dynamic range so as to limit their value. These and other electronic imaging conditions need to be addressed in a timely and cost effective manner.
A wide variety of techniques and systems have been devised for the processing of electronic image signals with less than ideal dynamic range and/or contrast. A large number of systems are based first upon the analysis of the image using a histogram. Then based upon this analysis the image is modified under operator control. Some techniques consist primarily of simple gray scale modification. On the other hand, some techniques are more sophisticated such as the system set forth in U.S. Patent No. 5,012,333, to Lee et al. Most of these techniques depend upon a relatively slow computer to perform signal processing.
In general these techniques are effective in image improvement, but they can be relatively time consuming especially in the derivation of the histogram. When many frames of electronic images such as live sports, news events, or fluoroscopic images are required to be processed at real time or close to real time most techniques are too time consuming. Faster computers could be employed for signal processing, but this usually increases the cost substantially.
The text Digital Signal Processing by Alan N. Oppenheim and Ronald W. Schafer, Prentice-Hall, 1975, in the chapter titled Homomorphic Signal Processing teaches that some classes of images can be improved with signal processing without extensive analysis of a histogram. The author states that observed images are formed by a multiplying of pattern illumination by pattern reflectance to produce the brightness image. The author also states that these components of the brightness image can be separated as a function of frequency content. That is to say, in general the illumination component is in a different frequency range than the reflectance component. The author concludes that in general within an electronic image illumination is a low frequency signal and reflectance is a high frequency signal.
The author's signal processing gave image improvement, but addresses only a very limited number of image problems. Therefore, if one could implement a system that could be used to improve a larger variety of image dynamic range and contrast type problems in a timely manner using adjustable hardware, this would be desirable.
SUMMARY OF THE INVENTION
The present invention involves a system that first separates image luminance into high and low frequency components. The low frequency component is processed through mathematical operators, processing elements, whose variable values can be selected manually by an observer of the modified and unmodified images, or automatically by a computer that contains an edit list or other criteria for adjustments. The high frequency component may then be separated into positive and negative going excursion components, or processed as a single signal.
The high frequency component is then modified with proportionalities that are related to the instantaneous changes to the low frequency component with variable values being determined by either an observer or a computer as above. Then the high frequency component is processed through mathematical operators
whose variables are also to be selected as above. Then high and low frequency components are then recombined. If the image to be modified is in color, then the color image portions red, green, blue, or chrominance can be modified directly proportional to the instantaneous ratio of magnitude changes to the luminance image with respect to the low frequency portion of the luminance input.
An object of the present invention is to provide dynamic range and contrast modifications to video images. It is a further object of the present invention to provide a stand alone system wherein an operator interactively adjusts dynamic range and contrast. Another object of the present invention is to provide a system wherein dynamic range and contrast can be accomplished under computer control. A further object of the present invention is to provide a system wherein the low frequency portion of the image can be adjusted automatically under the control of automatic gain control, AGC, circuitry.
These and other objects and advantages of the present invention will become more readily apparent to those ordinarily skilled in the art when taken in conjunction with the following description and accompanying drawings which form a part of this application.
BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is an overall functional block diagram of the luminance dynamic range and contrast image processing in accordance with the present invention.
FIG. 2 is a block diagram of the specific variable control signals that are used for the control of signal processing in FIG. 1.
FIG. 3 is a block diagram of the processing elements within the low frequency processing block of FIG. 1.
FIG. 4 is a block diagram of the processing elements within the high frequency, positive, processing block of FIG. 1.
FIG. 5 is a block diagram of the processing elements within the high frequency, negative, processing block of FIG. 1.
FIG. 6 is a block diagram to show how the luminance processor of FIG. 1 can be used to modify image luminance and chrominance components.
FIG 7 is a block diagram to show how the luminance processor of FIG. 1 can be used to modify image RGB components. FIG. 8 is an overall functional block diagram of the luminance dynamic range and contrast image homomorphic processing for an analogue embodiment in accordance with the present invention.
FIG. 9 is a block diagram of the specific variable control signals that are used for the control of signal processing in FIG. 8. FIG. 10 is a block diagram of the linear analogue processing elements within the low frequency processing block of FIG. 8.
FIG. 1 1 is a block diagram of the linear analogue processing elements within the high frequency, positive, processing block of FIG. 8.
FIG. 12 is a block diagram of the linear analogue functions processing elements within the high frequency, negative, processing block of FIG. 8.
FIG. 13 is a simplified schematic to show how a logarithmic voltage is converted to an antilogarithmic volts for an analogue embodiment of the present invention.
FIG. 14 is a simplified schematic to show how a logarithmic voltage is multiplied by either a positive or negative value for an analogue embodiment of the present invention.
FIG. 15 is a simplified schematic to show how a logarithmic voltage is multiplied within a range.
FIG. 16 is a block diagram to show how an automatic gain control, AGC, can be incorporated into the primary path for low frequency processing.
FIG. 17 is an overall functional block diagram of the analogue luminance dynamic range and contrast image processing in accordance with the present invention.
FIG. 18 is a block diagram of the specific variable control signals that are used for the control of signal processing in FIG. 17.
FIG. 19A-C is a block diagram of the processing elements within the low frequency processor of FIG. 17 (some simplified schematics are shown).
FIG. 20 is a block diagram of the processing elements within the high frequency, positive, processing block of FIG. 17. FIG. 21 is a block diagram of the processing elements within the high frequency, negative, processing block of FIG 17.
FIG. 22 is a simplified schematic to show how an analogue multiplier can produce output voltage signals that vary from a gain of 'one' to the square of the input voltage signal. FIG. 23 is a simplified schematic to show how an analogue multiplier can produce output voltage signals that vary from a gain of 'one' to the square root of the input voltage signal.
DETAD ED DESCRIPTION OF THE PREFERRED EMBODIMENT An improved dynamic range and contrast electronic image processing system is disclosed. In the following description for purposes of explanation, numerous details are set forth to provide a thorough understanding of the present invention. However, it will be apparent to one ordinarily skilled in the art that these details are not required in order to practice the invention. FIG. 1 depicts the top level blocks for modification of dynamic range and contrast of image luminance component. Digital luminance YIN 105 is received by a separator 100. Within the separator 100 the luminance is separated into both a high and low frequency components. The high frequency component is then separated into a positive voltage and negative voltage components. Further the polarity of the high frequency negative voltage component is changed to a positive voltage signal, high frequency negative + component.
Frequency separation can be of any standard forms: For example, the lowpass filter could be either one or two dimensional. The high frequency component could be derived by the subtraction of the low frequency component from the unfiltered signal; or by the division of the low frequency component into the
unfiltered signal. A two-dimensional low-pass filter is preferred. The subtraction method for deriving the high frequency component is preferred. The separator 100 low frequency output YL1 110 is sent to a low frequency processor 200 that sends its processed digital signal YL3 220 to a combiner 500. Processing is performed based upon the value of controlling signals from a control block 800.
The low frequency processor 200 generates process control signals SI 230 and S2 240 that are sent to a high frequency, positive, processor 300, and process control signals S3 250 and S4 260 that are sent to a high frequency, negative, processor 400. The separator 100 high frequency positive component output YHPl 120 is sent to the high frequency, positive, processor 300 that sends its processed digital signal YHP5 340 to the combiner 500. The separator 100 high frequency negative + component output YHNl 130 is sent to the high frequency, negative, processor 400 that sends its processed digital signal YHN5 440 to the combiner 500. The combiner 500 subtracts YHN5 440 from the arithmetic sum of the other two inputs, YL3 220 and YHP5 340 when the preferred (subtraction) separation method is employed.
FIG. 3 depicts the processing elements within the low frequency processing block 200 of FIG. 1. Separator 100 sends the low frequency component YL1 110 to the low frequency processor 200. The low frequency component YL1 110 is received by a digital mapping PROM, programmable read only memory, 205. The PROM 205 has sixteen address bits. Ten address bit of the PROM 205 are used for YL1 110 and six bits are used for control Kl 810. The PROM 205 contains look-up tables that are used to map YL1 110 input to a YL2 210 output based upon Kl 810 being used to select maps that contain predetermined mathematical exponent modifiers. The sixty four separate mappings that can be addressed correspond in 205 to exponent kl values of 0.30 through 1.90 in steps of 0.025. Thus if Kl 810 had the value of forty two this would correspond to a kl value of 1.35, and the PROM 205 would perform the following mapping: YL2 = YL1 1 35 (1)
It should be noted that in 205 kl values can be other than those illustrated. For instance negative values of kl could be used for special effects purposes. Thus in this case and elsewhere herein specific values are for the given embodiment and should not be construed so as to limit the use of other values for other embodiments. The PROM 205 output YL2210, expressed in sixteen digital data bits, is sent to a digital multiplier 215 and is multiplied by a variable control K2 820 to produce an output product YL3 220 that is rounded to the ten most significant digital data bits. In FIG. 1 it can be seen that this output product YL3 220 is sent to the combiner 500. In order to maintain resolution during extreme amplitude compression PROM 205 output is expressed in sixteen digital data bits.
After multiplication in 215 the low frequency output YL3 220 can be rounded to ten digital data bits.
Oftentimes bright portions of an image contain very low contrast. Thus in order to improve the appearance of that image it may be desirable to compress the bright portion of the image while at the same time providing extra enhancement for the contrast in that same portion of the image. In FIG. 1, output controls SI 230,
S2240, S3 250, and S4 260 from the low frequency processor 200 can perform the desired selective enhancement and other functions. These controls SI 230, etc. can be used to modify the contrast represented by the high frequency components YHPl 120 YHNl 130 based upon changes to the general lighting represented by the low frequency component YL1 110. FIG. 3 depicts the generation of these controls. A block 225 consists of a two stage process to generate the control SI 230. First the instantaneous transfer ratio of block 205 output YL2 110 to input YL1 120 is determined using a digital divider. Then the quotient is modified through a mapping PROM in a similar manner as the mapping PROM in block 205. The mapping in this block 225 differs in that the exponent values, K3, vary between -3.1 to +3.2 in steps of 0.1. Thus, if K3 830 were to have a value of eleven then 225 would execute the following equation: S7 = (YL2 ÷ YLi 20 (2)
S3 250 is generated by a block 245 that functions the same as block 225 except it is controlled by K5 850.
Blocks 235 and 255 consist of mapping PROMs that operate on the linear transfer ratio of block 215. This transfer ratio is equal to K2 820. K4 840 and K6 860 consist of six bits, and are used to vary the mapped exponent between -
3.1 and +3.2 in steps of 0.1 as explained for block 225 above.
FIG. 4 depicts the processing elements within the high frequency, positive, processing block 300 of FIG 1. The high frequency positive voltage YHPl 120 from separator 100 is multiplied by SI 230 from block 225 to produce a product YHP2 310. YHP2 310 is then multiplied by S2 240 from block 235 to produce a product YHP3 320. YHP3 320 is sent to a mapping PROM block 325 that is controlled by K7 870 from block 800. This PROM is identical to and functions the same the PROM in block 205 as explained above. Block 325 sends its output YHP4 330 to a digital multiplier 335. YHP4 330 is multiplied by control K8 880 to produce a product YHP5 340 that is seen in FIG. 1 to be one of the inputs to the combiner 500.
FIG. 5 depicts the processing that occurs to the high frequency negative + component YHNl 130 in FIG. 1 high frequency, negative, processing block 400. Processing block 400 functions in the same manner as block 300 as explained above. Block 400 sends its output YHN5 440 to the combiner 500 as can be seen in FIG. 1. Combiner 500 signal output YOUT 505 is the modified luminance.
FIGS. 6 and 7 are novel examples as to how the luminance processors 950 may be used for modification of the electronic image, color components. FIG. 6 depicts how chrominance components CHIN 905 and the unmodified luminance YIN 105 can be modified by same ratio 915 as the change in the luminance processor 950, with respect to the low frequency input YL1 110.
The luminance processing 950 is depicted in FIG. 1 and has been explained above. The instantaneous transfer ratio of the luminance processor 950 output YOUT 505, with respect to YL1 100 is calculated in block 915. The ratio is
determined by a digital divider that divides YOUT 505 by YL1 110. CHIN 905 is used to instantaneously limit the output of 915. Consider the case where the instantaneous image element contains only blue and no other color; YIN 105 then has an instantaneous amplitude of approximately 10% of the blue signal. With this low amplitude of YIN 105 there exist some ratios from 915 that could cause
CHOUT 910 to exceed standard limits. Therefore, CHIN 905 is used to instantaneously limit the ratio from 915 so that CHOUT 910 will not exceed standard limits. If the unlimited ratio from 915 does not cause CHOUT 910 to exceed standard limits then no limiting occurs (standard limits could be for Betacam SP, S-VHS, or etc.). The ratio from 915 is used in a digital multiplier
920 to scale the chrominance components CHIN 905 to produce modified chrominance components CHOUT 910. The chrominance components CHIN 905 are composed of standard chrominance terms Cr and Cb. Oftentimes the high frequency content of an image is acceptable and only a modification to the light content, low frequency portion, is needed. In order for the image to look natural, it is desirable for both the high and low frequency portions of the image (including both luminance and color components) be changed by the same ratio. Thus by modifying all unmodified image elements, YIN 105 and CHIN 905, in FIG. 6 by the ratio of the processed luminance YOUT 505 divided by the unmodified luminance low frequency portion YL1 110, it can be seen that YIN 105 and
CHIN 905 will only be changed by the instantaneous ratio of the light content, low frequency, change to produce YMOD 999 and CHOUT 910 respectively if the high frequency terms YHP5 340 and YHPN5 440 are kept to zero value. This is important, because both the high and low frequency portions contained in the unmodified signals YIN 105 and CHIN 905 are modified only by the ratio of the change in light content, low frequency content. In other words, instantaneously they, YIN 105 and CHIN 905, were re-lighted to produce YMOD 999 and CHOUT 910 respectively. Processor 950 high frequency signals YHP5 340 and YHN5 440 will only have non-zero values for special high frequency emphasis or when the original image contains poor high frequency
portions. One ordinarily skilled in the art can appreciate the fact that for reasons of simplicity zero processing delays are assumed herein. This assumption allows for the explanation of the essence of the present invention without the inclusion of numerous compensating delays that do not enhance the understanding of the patent. Thus when the ratio is calculated in 915, the low frequency components within YOUT 505 corresponds to the same image element as YL1 110 except it may have been modified in amplitude by the luminance processor 950.
In like manner, FIG. 7 depicts how color components are modified by the ratio of the luminance processor 950 output YOUT 505 with respect to low frequency portion YL1 110 of the input. First the input signal RGB IN 955, expressed in R,G,B, red, green, blue format is transformed within 960 to luminance signal YIN 105. Processors 950,915, and 920 function as explained above. The R,G,B signals RGBIN 955 are scaled by multiplier 920. It should be apparent that other color components can be scaled with the same ratio as the output from 915.
One example of how the luminance processor 950 could be used is in the case where portions of the image are aesthetically satisfactory, however other portions are too bright and have very low contrast. The goal is to make significant changes to the too bright portion while at the same time making much smaller changes to the other portions of the image. In 205 kl would be set to 0.60 causing the brighter portions of the image to be compressed proportionately more in amplitude than the other portions. The digital multiplier would be set to 10.0 to add a gain to compensate for the amplitude compression of the acceptable portion of the image. This causes the average brightness of the image to be raised. In 225, k3 is set to -0.6 causing the greatest contrast enhancement to occur in the area of the image that had the greatest range compression. In 245, k5 is set to - 0.1 to cause some enhancement in the negative direction but not so much as to drive the image deep into black. In 235, k4 is set to 0.0; and in 255 k6 is set to 0.0. Controls K7 870, K8 880, K9 890, and K10 900 are set to taste. The
forgoing and the following are based upon the maximum normalized image amplitude having the value 1020.
Another example of how the luminance processor 950 could be used is in the case of a sporting event where the contrast is good, but part of the playing area is in sunlight and part of the playing area is in shadow. This can be annoying to the viewer. On the surface this appears to be the much the same as the previous example. The big difference is that contrast is acceptable. Once again the brighter portion of the low frequency component YIN 105 is compressed into region of the darker portion of the image, and then linearly expanded. This corresponds to kl in 205 being set to 0.4 and K2 820 being set to 50. Because contrasts are acceptable, k3 in 225, k4 in 235, k5 in 250, and k6 in 255 should be maintained at or around 1.0. Contrast can be modified to taste using K7 870, K8 880, K9 890, and Kl 0 900.
One can see that a whole host of dynamic range expansion and compression, and contrast enhancement and suppression combinations can be performed using the present invention.
DESCRIPTION OF ALTERNATE EMBODIMENTS (1) First Alternative Embodiment An analogue embodiment using homomorphic signal processing to is disclosed. Homomorphic signal processing allows mathematical exponent operations on signals using linear circuitry. By converting signal voltages to their logarithmic equivalent voltages one can accomplish all of the mathematic operations that were required for the preferred embodiment using linear circuitry to implement the well known mathematical rules associated with the use of logarithms. For example, multiplication operations are accomplished by the summation of two logarithmic values. After linear processing the antilogarithmic operation is performed to convert to standard signal voltage form. From the following detailed description it will become apparent that the digital homomorphic embodiments can be implemented using look-up tables to perform
the logarithmic and antilogarithmic operations, and standard circuitry for the performance of the linear operations.
FIG. 8 shows the top level block diagram for analogue luminance signal processing. Analogue luminance AIN 106 is sent to a separator 101 where high and low frequency components are separated. The high frequency component is further separated into positive and negative voltage components with the negative voltage component being converted to a positive voltage for processing. These three components, low frequency, high frequency positive, and high frequency negative, are each converted to logarithmic equivalent voltages using Analog Devices' logarithmic amplifier AD640.
The separator 101 low frequency output LL1 111 is sent to a low frequency processor 201 that sends its processed logarithmic signal LL3 221 to a combiner 501. The low frequency processor 201 generates process control signals Rl 231 and R2 241 that are sent to a high frequency, positive, processor 301, and process control signals R3 251 and R4 261 that are sent to a high frequency, negative, processor 401. The separator 101 high frequency positive component output LHPl 121 is sent to the high frequency, positive, processor 301 that sends its processed logarithmic signal LHP5 341 to the combiner 501. The separator 101 high frequency negative component output LHNl 131 is sent to the high frequency, negative, processor 401 that sends its processed logarithmic signal LHN5 441 to the combiner 501.
The combiner 501 performs the antilogarithmic operation on each of the three inputs, LL3 221, LHP5 341, and LHN5 441. The antilogarithmic signal derived from LHN5 441 is subtracted from the sum of the two other antilogarithmic signals derived from LL3 221 and LHP5 341 to produce output
AOUT 506. Discounting biasing, scaling, d.c. blocking, etc. in its simplified form combiner 501 performs the following mathematical operation:
AOUT= \0LU + lO^ - tf™5 (3)
FIG. 13 depicts how a typical logarithmic voltage 511 is converted to an antilogarithmic voltage 541 using negative feedback circuitry. The positive
logarithmic voltage 511 is sent through a resistor 521 to an operational amplifier 526's noninverting input. This voltage 511 is compared to analogue ground 536 to produce a positive voltage out of the operational amplifier 526. This positive voltage is passed through a forward biased diode 531 to a logarithmic amplifier 516. The logarithmic amplifier 516 is the same type part, AD640, that was used to produce the logarithmic voltages as described for the separator 101. This positive input voltage causes the logarithmic amplifier 516 to produce a negative current proportional to the logarithm of its input voltage 541. This negative current enters resistor 521 to produce a negative voltage to offset the positive logarithmic input voltage 511. Diode 531 is used to block negative voltages from the operational amplifier 526 from entering the logarithmic amplifier 516 in order to prevent the circuitry from functioning in the positive feedback mode. The logarithmic amplifier 516 produces a negative current based upon the absolute magnitude of its input voltage 541. Thus if the blocking diode were not present positive feedback could occur.
A control block 801 is used to control the processing within the three processors, 201, 301, and 401. FIG. 9 depicts the particular controls Ml 811, N2 821, etc. from the control block 801. The M controls Ml 811, M3 831, are used to control variable potentiometer. The potentiometer could be an electronic type under computer control, but for this embodiment they are passive components and the M controls are shaft adjustments by an operator. The N controls N2 821, N8 881, etc. are adjustable d.c. voltages that are proportional to the desired logarithmic values for process control.
FIG. 10 depicts the processing elements for the modification of the logarithmic voltage signal within the low frequency processor 201. The logarithmic analogue low frequency signal LL1 111 is sent to a multiplier 206. Multiplier 206 is implemented with a twenty turn cermet potentiometer 295 as shown in FIG. 15. An amplifier 293 with a gain of 2 for this implementation is used to set the upper limit for the multiplier 206; and a resistor 297 is used to set the lower limit for the multiplier 206. The potentiometer 293 varies the multiplier
206 between the value of 0.5 and 2.0. This is analogous to modifying the antilogarithmic value of the signal with an exponent as was shown in block 205 above. The output LL2 211 from multiplier 206 is sent to a summer 216 to be added to the d.c. control voltage N2 821. N2 821 control voltage can be either positive or negative in polarity, and is proportional to the logarithm of the control value by which the low frequency component is to be multiplied. (It should be noted that this d.c. value N2 821, other d.c. values, and low frequency terms such as Rl 231, FIG.11, used herein are used for mathematical purposes, but must be removed from the normal image component after the antilogarithmic signals are derived.)
A summer 227 subtracts multiplier 206 input LL1 111 from multiplier 206 output LL2 211, and sends the result to multiplier 226. FIG. 14 depicts how the multiplier 226 is implemented. The logarithmic voltage enters on line 271 and is amplified by a noninverting amplifier 276 and an inverting amplifier 281. Each amplifier has a fixed gain equal to the maximum value expected for M3 831 , M3 max. Thus by adjusting the twenty turn potentiometer 286 continuous gain values between +M3 max. and - M3 max. are obtained at the potentiometer wiper which in this case corresponds to the multiplier 226 output Rl 231. Blocks 247 and 246 have a different control, M3, but function the same as blocks 227 and 226 respectively as explained above. Block 246 produces output R3 251. Multipliers
236 and 256 receive N2 821 as input and function the same as was described for 226 above.
FIG. 11 depicts the processing elements within the high frequency, positive, processor 301. Processor input LHP 1 121 from separator 101 is added in a summer 306 to Rl 231 from multiplier 226 to produce output LHP2 311.
LHP2 311 is added in a summer 316 to R2 241 from multiplier 236 to produce output LHP3 321. These two summers 306 and 316 are used to modify the analogue logarithmic high frequency component with logarithmic voltages derived from the modifications that were made to the analogue logarithmic low frequency component. LHP3 321 is sent to a multiplier 326 that functions the same as
multiplier 206 to produce output LHP4 331. LHP4 331 is sent to a summer that functions the same as summer 216. FIG. 12 depicts the processing elements within high frequency, negative, processor 401. The processing in FIG. 12 is identical to the processing in FIG. 11 except it has different process controls, R3 251, R4261, M9 891, and N10 901. Combiner 501 puts out signal AOUT 506.
FIG. 15 depicts how color components are modified proportionately to the transfer ratio of the luminance processor 951. First the input signal RGB1NA 956, expressed in analogue RG,B, red, green, blue, format is transformed within 961 to luminance signal AIN 106. Luminance processor 951 functions as explained above. Processor 966 calculates the luminance processor transfer ratio by dividing AOUT 506 by AIN 106 with the result being sent to multiplier 971. Then the R,G,B signals RGBINA 956 are scaled by multiplier 971 to the same transfer ratio as the luminance processor 951. Processor blocks 966 and 971 use Analog Devices' part AD734. FIG. 16 depicts how an AGC 802 circuit can be connected to automatically control the low frequency processor, e.g. 200 or 201, to vary the dynamic range of the electronic image. Two processing elements PE1 207 and PE2 217 correspond to 205 and 215 respectively, or 206 and 216 respectively. Feedback is taken from the second processing element PE2 217, but could be taken from the first processing element PE1 207. This feedback is sent to the
AGC 802. A peak type AGC 802 is preferred over an average or other type of AGC. The AGC 802 output is sent to the variable control input of PE1 207.
By setting the variable value of PE2 217 to a high value, range equalization can occur. That is to say both nonlinear compression and linear expansion can occur to compensate for unbalanced lighting conditions in the original image. By setting the variable value of PE2 217 to a very small value nonlinear expansion can occur to improve the appearance of an image of a flat field scene. From the foregoing examples one ordinarily skilled in the art can see that with the AGC 802 present, the present invention could be incorporated
directly into a video camera with very few manual adjustments required by an operator for a given scene environment. (2) Second Alternative Embodiment
When custom or semi-custom integrated circuits are used it is less difficult to control noise sources when performing analogue logarithmic operations. On the other hand logarithmic voltages on an ordinary printed circuit board can have noise sources that are significantly multiplied when an antilogarithmic operation is performed upon the voltages that contain the noise sources. Therefore, it is sometimes more desirable to not use logarithmic voltages directly from part to part on some printed circuit boards. An analogue embodiment that employs analogue multipliers is disclosed (analogue multipliers oftentimes use logarithmic operations internally, but they are able to better control noise sources within a single device).
In the above embodiments, the high frequency elements are modified with values related to the changes in the low frequency signal. Oftentimes it may be desirable to make low frequency related changes to the high frequency elements (HF+ & HF- elements) without regard to whether the low frequency signal has undergone changes. Also, if the low frequency signal undergoes little or no change, insufficient signal is available to make changes to the high frequency elements. The following embodiment contains circuitry to overcome these problems.
FIG. 17 depicts the top level blocks for modification of dynamic range and contrast of the image luminance component. Analogue luminance YAIN 107 is received by separator 102. Within the separator 102 the luminance is separated into both high and low frequency components. The high frequency component is then separated into positive voltage and negative voltage components. Further the polarity of the high frequency negative voltage component is changed to a positive voltage signal, high frequency negative + component.
The separator 102 low frequency output YALl 112 is sent to a low frequency processor 202 that sends its processed signal YAL3 222 to a combiner
502. The low frequency processor 202 generates process control signals Tl 232 and T2 242 that are sent to a high frequency, positive, processor 302; and processor control signals T3 252 and T4 262 that are sent to a high frequency, negative, processor 402. The separator 102 high frequency positive component output YAHPl 122 is sent to the high frequency, positive, processor 302 that sends its processed signal YAHP5 342 to combiner 502. The separator 102 high frequency negative component output YAHNl 132 is sent to the high frequency, negative, processor 402 that sends its processed signal YAHN5 442 to combiner 502. The combiner 502 converts YAHN5 442 to a negative voltage signal and the add it to a summing junction with the other two inputs YAL3 222 and
YAHP5 342 to produce output YAOUT 507.
A control block 805 is used to control the processing within the three processors, 202, 302, and 402. FIG. 18 depicts the particular controls LI 812, J2 822, L3 832, etc. from the control block 805. The 'L' controls LI 812, L3 832, etc. are shaft adjustments that are used to control variable potentiometers.
The 'J' controls J2 822, J8 882, etc. are adjustable d.c. voltages for the control of analogue multipliers.
FIG. 19A shows the processing elements for the modification of the low frequency signal within the low frequency processor 202. The low frequency signal YALl 112 enters an analogue multiplier 218 and is multiplied by a value proportional to the d.c. value of J2 822 to produce signal YAL 223; YAL 223 is then sent a mathematical operations block 208 where its amplitude is modified by an exponent operator LI 812. The resulting output YAL2 212 is then sent to a multiplier block 219 which functions the same as multiplier block 218 and is controlled by the same d.c. voltage J2 822.
The intention of the three processing blocks 218, 208, and 219 is not to make great changes to the peak value of the input YALl 112; the intention is to make significant changes to the dynamic range of this signal, YALl 112. Therefore, YALl 112 was scaled to a peak value of one (one relates to the denominator or U value in FIG. 22 and FIG. 23 that will be explained later). For
this embodiment one was chosen to be 1 volt; thus the peak value of YALl 112 is 1 volt. This is significant, because for high power such as the seventh power, and also, low powers such as the seventh root, the peak value of the output from processor block deviates little for a value of one. Now it will become apparent why the pre- and post- multipliers 218 and 219 are around processor block 208.
Example 1: multiplier 218 in no way has the range to keep the output of processor block 208 going to one if the say the sixth root is taken; yet on the other hand multiplier 219 can easily modify the output of block 208. Example 2: if the peak amplitude of the input to this processor is not very close to one, then the output from this processor would diverge greatly (very large or very small) from one for large powers.
For exponent operator values LI 812 of greater than or equal to one, multiple stages of a simplified circuit in FIG. 22 are used. An input SIGIN 1100 is sent to an analogue multiplier 1108 such as an AD734, which according to the data sheets for the AD734, has an open loop transfer function of
(XI - X2)(Y1 - Y2)
WOUT = A0 - (ZJ - 12)
U
where A0 is the open loop gain of the output op-amp. When a negative feedback path is provided, the circuit will force the quantity inside the brackets essentially to zero, resulting in the equation (XI - X2)(Y1 - Y2) = U(ZJ - Z2) .
The absence of the output, WOUT, in this equation only reflects the fact that we have not yet specified which of the inputs is to be connected to the op- amp output. The function of Fig. 22 is realized with Zl connected to WOUT. So, substituting WOUT in place of Zl in the above equation results in an output WOUT = (X] ~ )(77 " YT) + Z2.
U
In Fig. 22, Y2 and Z2 are both connected to ground and thus have a value of zero in the above equation. Therefore, the circuit of Fig. 22 produces an output PWROUT 1110 based upon the following equation:
WOUT - [(X1-X2) Y1] / U. The value of U was chosen to be one volt for this embodiment.
Potentiometers 1102 and 1104 are controlled by a common adjustment shaft 1106. Therefore, if the shaft 1106 were to be rotated fully clockwise one could see that the output PWROUT 1110, WOUT, would be the square of the input SIGIN 1100 as follows:
10 WOUT = [(SIGIN - 0) SIGIN] / 1 = SIGIN2
On the other hand, if the shaft 1106 were to be rotated fully counterclockwise then the following would result:
WOUT = [(0 - (-1)) SIGIN] / 1 = SIGIN It now should now be apparent that the rotation of the shaft 1106 from fully counterclockwise to fully clockwise produces a continuum of exponent values between 1 and 2. By cascading two stages of the circuitry in FIG. 22 while using the same common shaft 1106 for both circuits a continuum of exponent values between 1 and 4 can be obtained. The present embodiment employs three cascaded stages of the simplified circuitry in FIG. 22 for exponent values of greater than or equal to 1. When three stages of the circuitry in FIG. 22 are employed, then SIGIN 1100 in the first stage corresponds to YAL 223 and PWROUT 1110 of the third stage corresponds to YAL2 212, the processed signal.
When exponent values of less than 1 are required in block 208 then the simplified circuitry in FIG. 23 is used. An input signal SGIN 1150 is sent to a multiplier 1158, AD734 with the modified signal being sent to RTOUT 1160. A common adjustment shaft 1156 controls two potentiometers 1152 and 1154. When the shaft 1156 is in the fully counterclockwise position then the multiplier 1158 performs a square root function on the input signal SGIN 1150. When the shaft 1156 is in the full clockwise position then the multiplier 1158 has unity gain
for the input signal SGIN 1150. A blocking diode 1162 is used to prevent positive feedback into multiplier 1158. A diode 1164 is used to keep the multiplier 1158 well behave if a negative input, SGIN 1150, voltage is present; otherwise the WOUT of multiplier 1158 would slam to the negative voltage rail. In this case the rotation of the shaft 1156 from fully clockwise to fully counterclockwise produces a continuum of exponent values between 1 and 0.5. In like manner the cascading of multiple stages of the circuitry in FIG. 23 produces even smaller exponent values. For the present embodiment three cascaded stages were employed. It can be seen that the circuit of Fig. 23 illustrates analog computational apparatus that includes an arithmetic functional unit 1158 which produces an output value RTOUT which is (a) a linear function of (b) the value on said input raised to a power Kl; and a control mechanism 1152, 1 154 and 1156 which is coupled to the arithmetic functional unit 1158 to vary Kl over a continuous range which includes the range from 0.5 to 1. More specifically, the arithmetic functional unit 1158 has inputs XI, X2 and Z2 and an output RTOUT, and generates a signal on the output ROUT as a predetermined function of at least such three inputs. The input Z2 is coupled to receive the input analog signal SGIN of the circuit of Fig. 23. In addition, the input XI of the arithmetic functional unit is coupled to an output tap of a potentiometer 1 154, which thereby provides a first signal source for input XI which is variable between ground and the output value RTOUT of the arithmetic functional unit. Furthermore, the input X2 of the arithmetic functional unit is coupled to an output tap of another potentiometer 1152, which thereby provides a second signal source for input X2 which is variable between -IV and ground. Both the potentiometers 1152 and
1154 are ganged on a common control shaft such that they operate in tandem. It can be seen further that the arithmetic functional unit 1158 produces the value on output RTOUT by producing thereon a linear function of (a) an average of (b) (1) a first component being a predetermined factor times the value on the analog signal input SGIN and (2) a second component being the square root of
the value on the analog signal input SGIN, the first and second components in such average being weighted in response to the position of the common potentiometer control shaft.
In the present embodiment the low frequency processing element 208 is implemented with three each cascaded stages of the simplified circuitry in FIG.
22 and FIG. 23. Each set of these stages are in parallel with each other with a switch to select the output from the desired set; control LI 812 is implemented with two separate shafts, 1106 and 1158, with adjustments being made to the set that has been selected with the switch. It should be noted that the circuits in FIG. 22 and FIG. 23 have potential applications outside of the present invention.
The low frequency processor block 202 input YALl 112, also, goes to brightness detector block 267; the brightness detector block 267 output BL 272 goes math processor blocks 228 and 248. The math processor block 228 sends its output Tl 232 to multiplier 307 in FIG. 20. The other math processor block 248 sends its output T3 252 to multiplier 407 in FIG. 21. A simplified schematic of the brightness detector 267 is shown in FIG. 19 A. A reference d.c. voltage VREF1 113 with a value of 0.65 percent of the peak value of input YALl 112 is subtracted from input YALl 112; this sum is then sent to an operational amplifier 1204; a diode 1206 is used at the output of operational amplifier 1204 to block any negative voltage; a resistor 1208 is supplied to add a load to the diode 1206 output BL 272. BL 272 goes to the math processor 228 where its amplitude is multiplied by the value represented by the d.c. voltage Jl 1; then the product of this multiplication is modified by the exponent L3 832. The exponent L3 832 functions the same as the exponent LI 812 as explained above, except L3 832 has a more limited range. Therefore, L3 832 consists only of one circuit of
FIG 23 in series with one circuit of FIG. 22; shafts 1106 and 1156 for this case are commoned into one single shaft. The math processor block 248 functions the same as block 228 except it has different control inputs, J13 932 and L5 852.
A simplified circuit of the darkness detector 277 is shown in FIG. 19C. This circuit, FIG. 19C, with the exceptions, that its reference voltage VREF2 114
is a d.c. voltage equal to 0.35 percent of the peak voltage of YALl 112, and it is used to measure the amplitude of YALl 112 below VREF2 114.
FIG 20 shows the math processing blocks that comprise the high frequency, positive, processor 302. Input signal YAHPl 122 is processed in three different parallel paths. The first path consists of a math processor block
327 that sends its output YAHP2 312 to multiplier 337; multiplier 337 output is sent to mathematical summer 347 which creates the high frequency, positive, processor output YAHP5 342. The second path consists only of multiplier 307 that sends its output YAHP4 332 to summer 347. The third path consists of multiplier 317 that sends its output YAHP6 352 into summer 347. The processing block 327 functions the same as the L3 832 portion of the block 228 in FIG. 19A. The multipliers, 337, 307, and 317, function the same as multiplier 217 in FIG. 19 A. FIG 21 shows the processing blocks that comprise the high frequency, negative, processor 402. The high frequency, negative, processor functions the same as the high frequency, positive, processor except that it has different input controls, L9 892, J10 902, T3 252, and T4 262. In reducing this embodiment to practice, the signals YAHPl 122 and YAHNl 132 were amplified by a fixed gain of two in order to maintain a good signal to noise ratio on the board; this type of information is not necessary in understanding this embodiment. (3) Third Alternative Embodiment
It is desirable that most digital data that represent a magnitude are preprocessed so as to have their magnitude within fixed bounds. These bounds can be obtained with straight gain/attenuation circuits or through AGC circuitry or the like. For example video signals are modified (using AGC) to acceptable levels. It may be desirable to modify the dynamic range and contrast of the video image, but the basic amplitudes are acceptable. Therefore, any signal processing that may be performed would have to be scaled to fit within the amplitude range of the original video image.
This is especially important when root or power operations are performed on the amplitude of a signal. As an example, consider that 10 data bits
(magnitude of the video) were to address a look-up table that could take roots or powers of the magnitude. As an example, assume that the maximum magnitude is 943. The maximum value (943) would produce a wide range of outputs if the look-up table were to have selectable ranges between its cube power (839,000,000) and cube root (9.81). On the other hand, by normalizing the maximum usable value to T and performing the appropriate root or power operation and then multiplying by the maximum usable value, then 10 output data bits would be adequate in most cases.
One way to normalize an instantaneous video amplitude (AMP^) is to use the following formula:
AMPMAχ ( AM- AMPj^ )X ; where AMPMAχ is the maximum input video amplitude and x is a real value.
Thus with pre- or post-linear scaling circuitry it would take very little to maintain the lookup table output at the desired value of the observer.
It is possible to include a scale factor in the above formula. The formula is then changed to: f(x) AMPMAχ ( AMP / AMPMAX )X ;
The invention has been described with reference to specific exemplary embodiments thereof Various modification and changes may be made thereunto without departing from the broad spirit and scope of the invention. The specification and drawings are, accordingly, to be regarded in an illustrative rather than a restrictive sense; the invention is limited only by the provided claims.