JP2018526765A - Synchronous light source for rolling shutter image sensor - Google Patents

Abstract

Illumination system (30) adapted for rolling shutter imaging devices. The system uniquely provides synchronized operation of the light source (114) in the context of an image sensor (103) having a “rolling shutter” type exposure architecture that operates on a line-by-line basis rather than a frame-by-frame basis. A preferred embodiment includes a lamp, such as an LED (114), a drive circuit (112) coupled to the lamp and capable of energizing the lamp to switch the lamp on and off, and a drive circuit coupled to the imaging system. A lamp control circuit (110) having means for receiving a timing signal (201) from (20). The lamp control circuit energizes the lamp through the drive circuit in synchronization with the timing signal. The timing signal is a line timing signal for a PWM system based on multiple lines. The signal indicates the readout pause of the image sensor, or the signal indicates the portion of the image sensor readout time at which the lamp is safely turned on without compromising the required active image sensor line.

Description

Related Applications None Background of the Invention TECHNICAL FIELD The present invention relates generally to the field of medical video devices. More specifically, the present invention includes a control system for a synchronous light source for a rolling shutter image sensor, ie, an LED light source for an endoscopic camera system having a CMOS type image sensor or a similar camera system.

2. Description of Related Art There are many types of light sources and associated control systems for endoscopic video. Many of these use high-power lamps or incandescent bulbs such as metal halide, quartz halogen, or xenon types. These lamps are typically used in a mode where the light intensity is constant, and the intensity of the light emitted from the device is controlled using a movable and variable mechanical aperture. An example is an aperture that blocks some or all of the light emitted by the lamp toward the receiving optical fiber light guide.

  Controlling this light intensity is important for various functional and safety reasons. Inappropriate light levels can cause underexposure or overexposure, and the camera system may have to overcompensate to reduce or limit image quality and camera performance. Safety concerns associated with high light transmission can include skin and cell burns and the possibility of flammable combustibles.

  Employing high power light emitting diode (LED) technology is becoming common in many fields and industries, including medical endoscopy. This reduces overall power and cost, while improving product reliability and service life. The LED light output intensity can be controlled by changing the amount of current that drives the LED device. This method has several drawbacks, such as inefficiency, practical limitations due to the lower limit of light output, and the tendency of the LED lamp color or output wavelength to shift with intensity. As a solid-state device, it is also a general technique to control the entire LED light output by a switch type pulse width modulation (PWM) method. For the human eye, film camera, and some motion picture cameras, this pulsed light is substantially averaged to “average”. This “average” is virtually indistinguishable from a uniform light source when used at the appropriate frequency. For human afterimage effects, this frequency is typically about 30 pulses per second.

  If the light switching frequency is equal to or higher than the frame capture rate of the camera, typically 60 exposures per second with a video camera, the PWM light intensity control is performed by a frame such as a charge-coupled device (CCD). Works well with video cameras with transfer imagers. It is convenient to use an integer multiple of the frame rate, for example twice, for the PWM switching frequency. It is also beneficial to synchronize the light source with the camera frame rate to avoid frequency mismatches that can lead to image beating, flickering, or “strobing”.

  However, with the recent adoption of CMOS (Complementary Metal Oxide Semiconductor) image sensors for medical endoscopy, PWM controlled LED light sources present challenges. Specifically, the problem relates to the CMOS imaging device having a “rolling shutter” type exposure architecture, which is the most common type. This is because they are not frame transfer devices. Instead, each line of the raster image is exposed in a cascading overlapped sequence that exposes the other lines while reading the line. This prevents the exposure of one line from starting and stopping at the same time as another line, even though the resulting duration is the same, and the exposure of those lines will overlap in timing.

  In this way, individual lines or collections of lines are subjected to light exposure that is significantly different from other lines, resulting in undesirable areas of different exposure in the image, so that conventional frame-rate based (frame-rate based) ) PWM control is not suitable. The number of different exposure areas is equal to twice the relative PWM frequency, and the size of the complementary light and dark areas is directly proportional to the PWM duty ratio. If the PWM system does not synchronize with the image sensor frame rate, the “roll” of this action, that is, that these areas in the output video will be in different positions in the current image frame and the next image frame, Will cause more.

  The rolling shutter image sensor can control exposure via an internal shutter mechanism. This is done by setting a control register inside the image sensor that defines the number of line times that the image sensor line is exposed, as represented by the HSYNC signal. The granularity of this exposure control thus increases by one line and can vary from full frame duration exposure to only one line duration exposure. For full frame exposure time, this is typically the total number of lines in the image sensor minus one. This is because the line typically cannot be exposed during readout. If the image sensor is operating at 60 frames per second, the exposure time will be approximately 1 / 60th of a second, or 16.67 milliseconds. On the other hand, if the image sensor has 1000 image sensor lines, the shortest exposure duration achievable by the image sensor shutter when capturing images at a rate of 60 frames per second is 1/60/1000 of 1 second, that is, 16. 67 microseconds. This is a particularly serious limitation. This is because in the absence of any other control of the brightness of the light source, exposures of 1 / 100,000 (10 microseconds) per second or less are often necessary in medical imaging situations.

  Therefore, a CMOS image sensor that can cause image sensor exposure corresponding to a time shorter than a typical readout time of one image sensor line and does not cause an undesirable exposure effect on a moving image. What is needed is a method for variable pulse width and PWM control of an LED light source.

SUMMARY OF THE INVENTION In one aspect, the present invention is an illumination system adapted for a rolling shutter imager having a horizontal line rate. The illumination system is coupled to a lamp, a drive circuit coupled to the lamp and capable of energizing the lamp to switch the lamp on and off, and coupled to the drive circuit for receiving a line timing signal from the imaging system A lamp control circuit having means. The lamp control circuit energizes the lamp in synchronization with the line timing signal via the drive circuit. The line timing signal is based on the horizontal line rate of the image sensor.

  In another aspect, the present invention resides in an illumination control system for transmitting timing information from an imaging system using a rolling shutter imaging device operating at a horizontal line rate to an illumination system. The irradiation control system includes an electronic connector and a digital electronic signal transmitted by the electronic connector. In the digital electronic signal, the digital pulses are timed to synchronize with the horizontal line rate of the image sensor.

  In yet another aspect, the present invention resides in an illumination control system for transmitting timing information from an imaging system using a rolling shutter imaging device that operates at a horizontal line rate and a pixel clock rate to an illumination system. The irradiation control system includes an electronic connector and a digital electronic signal transmitted by the connector. The signal is a clock obtained from the pixel clock rate, and the illumination system obtains the horizontal line rate from the clock signal using information obtained in advance about the timing of the horizontal line rate of the image sensor.

  In yet another aspect, the present invention resides in an illumination system adapted for an imaging system using a rolling shutter image sensor that operates at a horizontal line rate and a frame rate. The illumination system is coupled to a lamp, a drive circuit coupled to the lamp and capable of energizing the lamp to switch the lamp on and off, and coupled to the drive circuit for receiving a line timing signal from the imaging system A lamp control circuit having means. The lamp control circuit energizes the lamp in synchronization with the line timing signal via the drive circuit, and the line timing signal is based on the horizontal line rate of the image sensor. The lamp control circuit outputs a pulse width modulation (PWM) signal, and the drive circuit energizes the lamp by the pulse width modulation method in synchronization with the line timing signal based on the PWM signal. The period of the PWM signal is an integral multiple of the horizontal line period of the image sensor, but is shorter than the frame period. The PWM duty ratio is variable to control the amount of light output.

  In yet another aspect, the present invention resides in an illumination system adapted for an imaging system using a rolling shutter imaging device that can be temporarily paused during readout. The illumination system includes a lamp, a drive circuit coupled to the lamp and capable of energizing the lamp to switch the lamp on and off, and a means coupled to the drive circuit for receiving a timing signal from the imaging system A lamp control circuit. The lamp control circuit energizes the lamp in accordance with the timing signal via the drive circuit. The lamp control circuit outputs a lamp driving signal, and the driving circuit in response to the lamp driving signal energizes the lamp while reading of the image sensor is stopped, and cuts off the power of the lamp during reading of the image sensor.

  In yet another aspect, the present invention resides in an illumination system adapted for an imaging system using a rolling shutter imager having a necessary portion of a line of active pixel data and an unnecessary portion of the line. The illumination system includes a lamp, a drive circuit coupled to the lamp and capable of energizing the lamp to switch the lamp on and off, and a means coupled to the drive circuit for receiving a timing signal from the imaging system A lamp control circuit. The lamp control circuit energizes the lamp in accordance with the timing signal via the drive circuit. The lamp control circuit outputs a lamp driving signal, and the driving circuit in response to the lamp driving signal energizes the lamp during reading of an unnecessary part of the image sensor, and cuts off the power supply of the lamp during reading of the necessary part of the image sensor.

FIG. 2 is a block diagram of a presently preferred embodiment of the present invention. FIG. 2 is a timing diagram illustrating various timing relationships of signals in the embodiment of FIG. 1, wherein one line read time is used as a time base for pulse width modulation. In the context of an exemplary endoscopic video camera system, timing information may be integrated from the imaging system into the illumination system (in a common housing or on a common circuit board, or may not be integrated). In the endoscope video camera system, the imaging system includes a rolling shutter imaging device that operates at a horizontal line rate, and the illumination system includes a rolling shutter imaging device. A digital electronic signal having a digital pulse synchronized with the horizontal line rate is received. FIG. 4 is a block diagram of an illumination control system for transmitting timing information from the imaging system to the illumination system, similar to FIG. 3, in which the imaging system includes a horizontal line rate and, more specifically, a pixel clock. The lighting system receives a digital electronic signal corresponding to a clock signal that oscillates at a pixel clock rate, and the lighting system receives the horizontal image sensor from the clock signal and other known parameters. Includes suitable processing functions for obtaining line rates. FIG. 2 is a timing diagram showing various timing relationships of signals in the embodiment of FIG. 1, and the read time of a plurality (here, two) lines is used as a time base for pulse width modulation.

  The invention and its various embodiments can be better understood with reference to the following detailed description of the preferred embodiments, presented as examples of the invention as defined in the claims. . It is expressly understood that the invention as defined by the claims can be broader than the embodiments described below.

Detailed Description of the Preferred Embodiment FIG. 1 shows a block diagram of a presently preferred embodiment of the present invention. The illustrated embodiment and other embodiments are based on the basic view that a “rolling shutter” image sensor can be said to have a “horizontal line rate” in operation. The rate is a rate at which the entire horizontal line of raster data is read. Further, the image sensor has a “pixel clock rate”. The rate is the rate at which each pixel in the line is individually read out or “clocked out” from the sensor (imaging device). This pixel clock rate is typically several hundred to several thousand times faster than the line rate. The line rate is directly related to the shutter mechanism of the sensor and thus the exposure. This is because the exposure time of a line is changed by increasing or decreasing the number of lines between when a line is read and when the line starts to be exposed further or finishes clearing. Because it does.

  As shown, the preferred embodiment relates to an endoscopic or similar video system comprising a camera having a rolling shutter image sensor 103 and a light source 114. The light source 114 is turned on and off in a pulsed manner by the PWM signal 111 and the associated drive circuit 112 in order to strategically irradiate an object focused on the rolling shutter image sensor 103. In a preferred embodiment, the rolling shutter image sensor 103 is realized by a CMOS image sensor. However, the rolling shutter technique for exposure may be used for other image sensor technologies. Furthermore, although the preferred light source 114 is comprised of one or more LEDs, preferred and / or alternative embodiments can be realized with any suitable light source that currently exists or will be developed in the future. Thus, when referring to an LED, it should be understood that it is a reference to any suitable light source.

  More specifically, the camera timing logic unit 101 provides a timing signal 102 to a rolling shutter image sensor 103 (for example, a CMOS image sensor), and the rolling shutter image sensor 103 generates an image sensor moving image signal 104. The image processor 105 processes the image sensor moving image signal 104 to generate a moving image output signal 106 and an intensity signal 109. The moving image output signal 106 drives the moving image output device 107. The intensity signal 109 is fed back to the lamp control logic 110 to indicate whether more or less light is required for the LED 114 to achieve the desired exposure. The camera timing logic unit 101 also provides a line timing signal 108. This line timing signal 108 is used by the lamp control logic unit 101 to synchronize the PWM signal 111 with the drive circuit 112. Then, the drive circuit 112 outputs a drive current pulse 113 linearly correlated with the PWM signal 111 to the light source 114 (for example, LED).

  FIG. 2 shows a typical timing relationship of several signals in the preferred embodiment. Line timing signal 201 (eg, HSYNC) generated by camera timing logic 101 is applied to lamp control logic 110 to create the time base and synchrony of drive current pulse 113. An intensity signal 202 is applied, which can be an analog signal (shown) or a digital signal. This intensity signal 202 is interpreted by the lamp control logic 110 to determine the relative duty ratio or percent intensity. Finally, the lamp control logic unit 110 outputs a PWM signal 203 that is used to drive the light source 113. Driving the light source 113 is, for example, generating an LED emitter current having an LED driving current pulse having a timing based on the timing of HSYNC and a duty ratio based on the intensity signal 202. The higher the intensity indicated by the intensity signal 202, the longer the active state of the PWM signal 111, that is, the current drive state.

  As a further context, the preferred embodiments of FIGS. 1 and 2 can be implemented in the context of a medical imaging device. For example, an endoscopic video camera with a CMOS imager with a rolling shutter, a light source using LEDs to illuminate the scene to be imaged by the camera, and typically a transistor or similar current switching device A LED driving circuit, a control circuit having means for controlling the LED driving circuit by controlling the average LED light output intensity using a PWM method, and line reading of a CMOS image sensor output from the camera to the control circuit And a signal that is synchronized with the rate. The control circuit blinks the LEDs on and off at a rate based on the line frequency of the CMOS imager exposure system. In a CMOS imager exposure system, one line of CMOS imagers is read out and one pulse (or multiple pulses per unit time elapsed when resuming exposure of that line before moving to the next line). Pulse). For example, if a standard HD imaging device having 1920 horizontal pixels × 1080 vertical lines is exposed and read out at 60 frames per second, the line frequency is 1080 (number of lines per frame) × 60 (number of frames per second). It will be. This is equal to 64,800 lines per second, or approximately 1 line per 15.432 microseconds. The timing circuit will generate a pulse that is synchronized with this line rate of the CMOS imager. This pulse can typically be referred to as a digital horizontal sync (sync) pulse and is known as HSYNC. Note that this HSYNC pulse is typically not the same as the HSYNC signal or timing element of the camera output video. This is because a rolling shutter imaging device typically does not operate in line synchronization with a conventional video transfer standard, and often cannot operate in line synchronization with a conventional video transfer standard. An example of a conventional video transfer standard is the 1080p video format as defined by SMPTE-274M. The control circuit will typically receive this image sensor HSYNC digital pulse from the imaging system using an electrical cable and electrical connector and utilize the position and frequency tailored to the PWM time base. Using the drive circuit, the control circuit is positioned to the duty ratio of each drive pulse corresponding to the desired percent intensity desired for the LED, as determined by direct input by the user or as calculated by camera processing. And pulse the LED current at frequency. One possible timing is shown in FIG.

  Such line-based PWM systems operate at a much higher speed than conventional frame-based PWM systems, thus requiring LEDs with appropriate on and off response times, There is a need for a drive circuit that can drive a large LED current with the pulse width duration as short as possible at this rate. The ratio between the line-based pulse width and the conventional frame-based pulse width is approximated by the number of lines in the image sensor. For example, assuming 1 pulse per frame, a frame-based system operating at 60 frames per second would have a 1% PWM pulse duration of 0.01 × 1/60 = 160 microseconds. On the other hand, for the line-based system described above, the corresponding 1% PWM pulse duration would be 0.01 × 1 / 64,800 = 150 nanoseconds. That is, a ratio of 1: 1080. With increasing frame rate and sensor resolution, special LEDs and drive circuitry may be required to achieve the desired results. In addition, the drive circuitry of this line-based system will typically consume more power and dissipate more heat as opposed to the frame-based system. This is because the drive circuit of a line-based system needs to make much more off-to-on transitions and on-to-off transitions per frame, and in each of these transitions the state is analogized. This is because the heat in the apparatus is dissipated when switching. For the above example, this line-based system would have 2 × 1080 = 2160 transitions per frame, whereas in a frame-based system there would be 2 or 4 transitions per frame. Thus, to minimize cost and dissipative power, which is a mechanical and electrical condition, it is advantageous to have the LED only one pulse per line rather than multiple pulses per line. Will.

  FIG. 3 shows a typical medical endoscopy situation where the imaging system 20 and the illumination system 30 are separate units, but that the two systems may be integrated within the same housing. Should be noted. Between these two systems is an illumination control system that transmits a horizontal line rate signal between the two systems. This will typically be physically realized by the electrical connectors present on both systems and the electrical cables in between. In this system, the signal transmitted between the systems is a digital pulse representation of the horizontal line rate of the image sensor in the imaging system. Based on this signal, the illumination system 30 energizes the associated light source (eg, an LED or other lamp) in synchronization with the horizontal line rate of the imaging system 20.

  FIG. 4 shows a system that is very similar to the system of FIG. 3, with the difference that the signal transmitted from the imaging system to the illumination system is a clock signal based on the pixel clock rate of the imaging device. As shown, the illumination system 30 includes suitable means for obtaining a horizontal line rate from the clock signal. In such a case, the lighting system will need some pre-obtained information about the imaging device and the timing of the imaging system. This information can be programmed into the illumination system or communicated from the imaging system to the illumination system using another electrical interface, such as a serial communication port.

  Another embodiment of the invention would be a general form of the system described above. Here, the signal from the camera to the control circuit is the moving image output of the camera itself, and the control circuit extracts the PWM time base from the line interval of the moving image signal. This can be in accordance with moving picture standards such as SMPTE 274M. This embodiment allows a more general interface between the camera and the control circuit, and these two devices can use a standard interface to achieve proper synchronization. This can be done without a direct interface to the image sensor timing logic, and therefore has the potential advantage of being a non-special and non-dedicated interface. This embodiment is advantageous when the light source element and camera element of the system are not contained within the same unit or enclosure. For best results, this method requires the same time relationship between the image sensor line rate and the output video line rate. These are not universal solutions because they are not necessarily the same duration or simultaneous in all implementations of video cameras. However, in such cases, this interface has the additional advantage of also conveying image level information. Because it ’s the video itself. Thus, the lighting system will have the necessary information to automatically adjust the brightness of the light as needed.

  Another embodiment of the invention would be where there is a duration that the lines of the CMOS imager are not read and all the lines of the image are exposed. This is a common approach in multi-frame rate imaging systems, such as a system that operates at both 50 frames per second and 60 frames per second, depending on the national moving picture standard in which the imaging system is used. For example, the duration that all lines are exposed may be 16% of the total frame duration, but may be longer or shorter depending on the design. During these “idle” exposure times, the LEDs are pulsed at the same frequency that would otherwise correspond to the horizontal line frequency, as if the line were being read at that rate. May continue to be done. The LED may also be turned off or on during this entire idle time. The LED may also be turned on at a fixed or variable percentage of this idle time in a pulsed or constant manner during a portion of this time. This ratio may or may not correspond to the duty ratio or frequency used during the non-idle period of the frame.

  For example, a CMOS sensor for a typical 1080p (1920 pixels × 1080 lines, progressive scan) resolution camera application is the Panasonic MN34041, which is 1944 × 1092 of the entire 2010 × 1108 pixel array. Active pixel array. Inactive pixels of the entire array are optically black pixels (active pixels that are covered so that they are not exposed to light and form a relative black level or noise floor) or have no useful data. It can be a “dummy” pixel or an invalid pixel. The SMPTE 274 standard specifying 1080p video timing requires a total of 1125 lines (including 1080 active picture lines and 45 blanking lines) per frame. Since MN34041 has only a total of 1108 lines to be read out per frame, if the sensor lines are read out relatively synchronously with the output moving image timing, there are 17 shortages. During the additional time of this 17-output video line, the sensor may pause reading in order to blink the lamp.

  For some image sensors, camera readout may be paused to expose the image sensor so that there is no line that is prevented from being exposed by the readout process being performed. For other image sensors, there may be no lines other than the readout mode (there is no possibility of exposure in this mode). For such an image sensor, the pause and blinking of the lamp may be timed to coincide with a predetermined line that is unnecessary for the output image. This predetermined line may be the active line selected for truncation, such as the top or bottom line of the image, may be an extra active line not used in the output image, and optical Alternatively, it may be a black line or a dummy pixel line. Some rolling shutter imaging devices may be stopped for a short period of time. In that case, the method can only be used if a relatively short exposure is desired. For example, the imaging device may not allow reading pause of the imaging device for a single duration of 17 output video lines. In this case, there may be a plurality of pauses during readout of a plurality of lines, preferably during readout of an optically black line or dummy line. This is to reduce the amount of pause for any one line. The lamp may be turned on at any part of any or all of these pauses to achieve the desired exposure.

  In this embodiment, the image sensor shutter is set by the camera control logic to expose the line for the full frame duration. After the image sensor control logic reads all lines from the frame, the image sensor control logic pauses the readout for a short time to flash the lamp on and off, creating a new exposure. The image sensor control logic then proceeds to read all lines in the frame based on the exposure and starts the cycle again. In a typical rolling shutter image sensor, the lines being read from the image sensor cannot be exposed simultaneously. Thus, if a single blink of the lamp occurs during the active line readout, the line will not be exposed in the next output frame and the line will be dark. In many camera applications, the extra active line is used for features such as image stabilization, so the most advantageous location of the ramp pulse is during the readout time of the optical black pixel line and the dummy pixel line. Only during read time.

  The advantage of this embodiment is that the duration of exposure of the image sensor is shorter than a typical line period (reciprocal of line frequency) while using a lamp driving circuit having the same speed as a conventional frame-based PWM system. It is a point that can be achieved. This embodiment also allows for a duration of exposure longer than one line (eg, 17 lines). As a result, in this embodiment, the proportion of time used as the operation mode of the image sensor and the lamp is increased, and the automatic image sensor control logic unit performs smooth automatic exposure without continuously changing the image sensor shutter operation. Operation is obtained. Another advantage of this embodiment is that it eliminates the motion distortion often seen in rolling shutter imaging devices. This is because all lines are exposed to light simultaneously, as is the case with a full frame shutter imaging device such as a CCD. The trade-off in this embodiment is limited in the length of time that the exposure can be performed, so to achieve longer exposures in the same system, it is necessary to use another method accordingly. That would be the point. This may be another embodiment of the present invention. That is, with a fixed lamp output, use a longer image sensor shutter exposure time (relative to the lamp pulse time when using this embodiment).

  Another embodiment, closely related to the previous embodiment, further exploits the fact that many image sensors have more output lines than may be necessary for display in the resulting image. About these image sensors, it can be said that an image sensor has two parts. That is, a “desired” portion that includes lines that are actually used in the output image and an extra or including any remaining lines consisting of optically black pixels, dummy pixels, and any unused active pixels. It is the “undesired” part. In this embodiment, the lamp may be blinked so as to expose the necessary lines equally using the time during the reading of the lines in the unnecessary part. Further, the lamp may be blinked using the entire unnecessary portion of the image sensor so that the lamp blinking time spans a plurality of unused line times. Furthermore, this may be an extension of the previous embodiment. That is, the image sensor may be paused during readout so that the lamp is energized both during readout of unnecessary portions and during any readout pause. Similar to the previous embodiment, if the lamp is set to flash during the time that the image sensor is outputting the unwanted portion of the pixel array, the image sensor shutter is for the entire frame period, i.e. all The line time should be set to expose the line.

  In this embodiment, when the image sensor cannot be paused for a long time, when it is not desirable to pause the image sensor, and / or the output of the image sensor can be frame-synchronized with the output video, but it is not This is especially useful when not. Similar to the previous embodiment, while using a lamp drive circuit at the same speed as the frame-based PWM system, it is possible to achieve a shorter exposure duration than a typical line period, and the motion distortion due to the rolling shutter is reduced. Disappear. In addition, in this embodiment, when the ratio of the time used as the operation mode of the image sensor and the lamp is larger, the exposure duration can be longer than that in the above-described embodiment, and the image sensor control logic unit A smooth automatic exposure operation can be obtained without continuously changing the image sensor shutter operation. This embodiment has the same tradeoff as the previous embodiment. Here, in order to achieve an exposure time longer than the time for reading out the unnecessary portion, the present embodiment requires the use of the second exposure mode.

  Another embodiment of the invention would be where the lamp is modulated on and off on a line-by-line basis. In other words, the lamp will be turned on for one or more lines and alternately turned off for the next one or more lines. The advantage of this type of system is that it employs slower, and thus less expensive drive circuits that are inherently longer pulses or that the lamp itself cannot be turned on and off at a faster rate as in the previous embodiment. Is that it can be done. This further has the advantage that lower frequency radiated and conducted emissions are produced by high power switching. However, the disadvantage of this type of line modulation is that the imaging element can only be driven by factors such as the number of on-lines and the number of off-lines in order to achieve a perfectly uniform exposure across all lines or regions of the image. The overall light exposure time cannot be increased or decreased. For example, if 75% power is required for the lamp, the lamp may blink so that it is on for three lines and off for one line. As a result, the rolling shutter must be set so that the exposure time increment is 4 lines to ensure that all lines receive a 3: 1 ratio of lamp on time to lamp off time. I will have to. Therefore, the fewer lines to create the required output ratio, the greater the increment of shutter available for a given number of horizontal image sensor lines. The larger the exposure increase unit, the smaller the number of exposure values that can be taken, and thus the exposure control becomes rougher. This can be disadvantageous when a smooth shutter operation is desired in an automatic exposure system. The minimum exposure increase unit to achieve 50% light output from the lamp is two lines, one alternating on line and one off line. Most sensors have an even number of horizontal image lines, but an odd number of shutter exposure increments may be used if they are divided into a total number of even imager lines. While 1080 line systems are common in HD, the number 1080 is a factor of 3 and 5, so these exposure increments would be feasible.

  The following table shows the most realistic ratio for this method up to 5 line exposure increments.

  Note the 2: 2 line ratio in the table above. This is because a 2: 2 line ratio yields the same lamp output result as a 1: 1 line ratio, but flashes the lamp at half the frequency. Thus, slower driving circuits or lamps are available with slower on-time and off-time at the cost of higher exposure increments.

  Each of the above embodiments including lamp line-based PWM control provides significant advantages over the prior art. However, all of the above embodiments have the disadvantage of requiring a very fast LED driver, or the coarse adjustment of the shutter is complex, limited in flexibility, and visually “smooth” with respect to automatic brightness control. It has the disadvantage that it is not. Therefore, it is desirable to employ a PWM system that can use both a low-speed LED driver and a variable PWM duty ratio for visual “smoothness”.

  In an embodiment in which this is achieved, a timing is used where the PWM period (the reciprocal of the frequency) is a multiple of the horizontal line period (the reciprocal of the horizontal line rate), eg, two, three, or more horizontal line periods. The PWM duty ratio is typically applied throughout this multiple line PWM period. For optimal visual characteristics, the frame period (the reciprocal of the frame rate) should be divisible by the PWM period. In this embodiment, the image sensor r line exposure time is variable, but for optimal visual characteristics, the line exposure time in the image sensor is the same as the PWM period or an integer multiple of the PWM period. Should be either one.

  For example, a system with 1080 lines uses a smooth 0% to 100% duty cycle with two horizontal line period PWM periods, and all functionally effective line exposure durations are multiples of two lines, i.e. 2, 4, 6, ... 1076, 1078, 1080. FIG. 5 shows an example, where the base of the PWM period is two horizontal line periods. In this example, this is shown by generating one PWM (203) pulse for every two HYSNC (201) pulses. As shown typically in rolling shutter imagers, three arbitrary consecutive exposure shutter periods exposure lines X, X + 1, and X + 2 (duration is two horizontal line periods) are shown staggered and overlapped Has been. Line X and line X + 2 are both exposed to light of a single PWM pulse, and each has an equal duty ratio. However, line X + 1 receives light from a portion of the first PWM pulse and a portion of the illustrated second PWM pulse. Since the shutter exposure time and the PWM period are the same, line X + 2 will be exposed to the same amount of light as line X and line X + 1.

  The advantage of this embodiment is that the PWM frequency ranges from a horizontal line rate (˜65 kHz in a 1080 line 60 frame per second system) to half line rate (˜32 kHz), a quarter line rate (˜16 kHz), or It can be reduced to a much lower line rate. As a result of these PWM frequency reductions, the PWM pulse width becomes longer while keeping the duty ratio the same, and it becomes possible to use a slower LED driver and an LED with a longer minimum switching time. They are the actual limiting factors in the system.

  A mathematical comparison of this embodiment to a single line based PWM reveals advantages. A typical 1080 line 60 frame per second imaging system has a horizontal line period of approximately 15 microseconds. A typical target exposure for an imaged frame may be 1 / 10,000th of a second, or 100 microseconds, which means that the shutter is open for a time when the lamp is on or longer ( In other words, it is the time that the lamp is desired to be on. For the one-line PWM based system of the previous embodiment, this 100 microsecond period would be evenly distributed over all 1080 line periods in the frame. Thus, each pulse of the LED will be 92.6 nanoseconds. This is because the exposure of the image sensor to the light is cumulative. Using this embodiment, the interval will be halved at the 2-line cycle timing (960 instead of 1080). Thus, the pulse will be twice as long (100 microseconds / 960 = 185 nanoseconds). In addition, at 3 line cycle timing, the pulse duration will be 277 nanoseconds. In the 108 line period PWM timing example, the pulse duration is 10 microseconds, which is 108 times longer than the single line PWM timing, and its advantages become apparent. Since it is only an order of magnitude faster than a frame-based PWM system, the 108-line PWM timing LED driver is much more realistic to implement than what is required in a single-line PWM system (3 digits faster). .

  This embodiment allows for a lower cost and simple LED drive circuit, slower LED lamps, and a potential reduction of conducted and radiated electromagnetic radiation. Although the granularity of the line shutter control in the image sensor is reduced by reducing the PWM frequency, the aspect of shutter exposure control is essentially replaced by the variable PWM duty ratio of the lamp, and the simplification of the control is achieved. And a visually “smooth” exposure change. According to this embodiment, it is possible to make the distortion of the motion of the rolling shutter inconspicuous. This is because if the image sensor shutter is exposed for a relatively large number of lines and there are multiple PWM periods per line exposure time, the movement of the acquired image is virtually blurred, resulting in a relatively low duty cycle of the lamp. This is because even if the exposure is low and the overall exposure is short, the distortion is not so noticeable to the observer.

  Many other embodiments are possible without departing from the spirit and scope of the invention. Accordingly, it is to be understood that the illustrated embodiments have been described for purposes of illustration only and are not to be construed as limiting the invention as defined by the following claims. For example, even if there is the fact that the elements of a claim are described below in a certain combination, the present invention includes fewer elements, more elements, or other combinations of different elements, It should be expressly understood that it is disclosed above, even if it was not initially claimed in such a combination.

  The terms used herein to describe the present invention and its various embodiments are not only in a generally defined meaning, but also in a book beyond the scope of a generally defined meaning. It should be understood to include structures, materials, or operations according to special definitions in the specification. Thus, when an element can be understood as including more than one meaning in the context of this specification, its use in the claims is intended for all possible meanings supported by this specification and the term itself. It should be understood as comprehensive.

  Accordingly, the following claims terms or element definitions perform substantially the same function in substantially the same way to obtain substantially the same result, not just the literally described combination of elements. As defined herein to include all equivalent structures, materials, or operations for. Thus, in this sense, any element in the following claims can be equivalently replaced by two or more elements, or two or more elements in a claim can be replaced by a single element It is contemplated to obtain. Although elements are described above as operating in a certain combination and originally claimed as such, one or more elements of the claimed combination may be deleted from the combination in some cases It should be expressly understood that the claimed combinations may be directed to sub-combinations or variations of sub-combinations.

  It is expressly contemplated that non-essential changes from the claimed subject matter, now known or devised in the future, as contemplated by those skilled in the art, are equally within the scope of the claims. Accordingly, obvious alternatives now or future known to those skilled in the art are defined within the scope of the defined elements.

  Thus, the claims include what has been specifically illustrated and described above, what is conceptually equivalent, what can be clearly substituted, and what essentially incorporates the essential idea of the invention. Should be understood.

Claims (22)

An illumination system adapted for an imaging system using a rolling shutter imager operating at a horizontal line rate and a frame rate,
A lamp,
A drive circuit coupled to the lamp and capable of energizing the lamp to switch the lamp on and off;
A lamp control circuit coupled to the drive circuit and receiving a line timing signal from the imaging system, the lamp control circuit energizing the lamp in synchronization with the line timing signal via the drive circuit; The line timing signal is based on the horizontal line rate of the image sensor,
The lamp control circuit outputs a pulse width modulation (PWM) signal, and the PWM signal causes the driving circuit to energize the lamp in a pulse width modulation method in synchronization with the line timing signal,
The period of the PWM signal is an integral multiple of the horizontal line period of the image sensor but shorter than the frame period,
The illumination system, wherein the PWM duty ratio is variable to control the amount of light output.   The lighting system according to claim 1, wherein the frame period is divisible by the PWM period.   The illumination system of claim 1, wherein a line exposure time is equal to the PWM period.   The illumination system according to claim 1, wherein a line exposure time is an integral multiple of the PWM period but shorter than the frame period.   The illumination system according to claim 1, wherein the rolling shutter image sensor is a CMOS image sensor.   The lighting system of claim 1, wherein the lamp is a light emitting diode.   The illumination system according to claim 1, wherein the imaging system and the illumination system are integrated so as to constitute the same unit. An illumination system adapted for an imaging system using a rolling shutter imaging device that can be temporarily paused during readout,
A lamp,
A drive circuit coupled to the lamp and capable of energizing the lamp to switch the lamp on and off;
A lamp control circuit coupled to the drive circuit and receiving a timing signal from the imaging system, wherein the lamp control circuit energizes the lamp in accordance with the timing signal via the drive circuit;
The lamp control circuit outputs a lamp driving signal, and the driving circuit energizes the lamp while reading of the imaging device is stopped by the lamp driving signal, and cuts off the power of the lamp during reading of the imaging device. , Lighting system.   9. The illumination system of claim 8, wherein the lamp drive signal is variable in duration to change the amount of light output and the resulting image sensor exposure.   The illumination system according to claim 8, wherein the shutter of the image sensor is set so that all active image sensor lines are exposed during readout pause of the image sensor.   The illumination system according to claim 8, wherein the shutter of the image sensor is set to expose each image sensor line for a duration of readout of all other image sensor lines.   The illumination system according to claim 8, wherein the rolling shutter image sensor is a CMOS image sensor.   The lighting system of claim 8, wherein the lamp is a light emitting diode.   The illumination system according to claim 8, wherein the imaging system and the illumination system are integrated so as to constitute the same unit. An illumination system adapted for an imaging system using a rolling shutter imaging device having a necessary part of a line of active pixel data and an unnecessary part of the line,
A lamp,
A drive circuit coupled to the lamp and capable of energizing the lamp to switch the lamp on and off;
A lamp control circuit coupled to the drive circuit and receiving a timing signal from the imaging system, wherein the lamp control circuit energizes the lamp in accordance with the timing signal via the drive circuit;
The lamp control circuit outputs a lamp driving signal, and the driving circuit energizes the lamp during reading of the unnecessary portion of the image sensor by the lamp driving signal and during reading of the necessary portion of the image sensor. A lighting system that cuts off the power of the lamp.   16. The illumination system of claim 15, wherein the lamp drive signal is variable in duration to vary the amount of light output and the resulting image sensor exposure.   The illumination system according to claim 15, wherein the lamp is energized during part or all of a readout pause of the imaging device.   The illumination system according to claim 15, wherein the shutter of the image sensor is set so that all image sensor lines in the necessary part are exposed during reading of the image sensor line in the unnecessary part.   The illumination system according to claim 15, wherein the shutter of the image sensor is set to expose each image sensor line for a duration of readout of all other image sensor lines.   The illumination system according to claim 15, wherein the rolling shutter image sensor is a CMOS image sensor.   The lighting system of claim 15, wherein the lamp is a light emitting diode.   The illumination system according to claim 15, wherein the imaging system and the illumination system are integrated so as to constitute the same unit.

Images