CN107018339A - Image sensor, image processing method, image processing device and electronic device - Google Patents

Abstract

The invention discloses an image sensor. Most image sensors include an array of light-sensitive pixel cells including a first light-sensitive pixel and a second light-sensitive pixel. The first photosensitive pixel has a first sensitivity, and the second photosensitive pixel has a second sensitivity that is less than the first sensitivity. In addition, the invention also discloses an image processing method, an image processing device and an electronic device for acquiring the ambient brightness. The image sensor, the image processing method for acquiring the ambient brightness, the image processing device and the electronic device of the invention measure the ambient brightness by using the second photosensitive pixel with lower sensitivity, thereby obtaining more accurate measurement effect in a high-brightness environment.

Description

Image sensor, image processing method, image processing device and electronic device

technical field

The invention relates to imaging technology, in particular to an image sensor, an image processing method, an image processing device and an electronic device.

Background technique

In order to ensure the image acquisition capability, the sensitivity of the photosensitive pixels of the image sensor is generally relatively high, and signal saturation is prone to occur in a bright environment, resulting in the inability to correctly obtain ambient brightness information.

Contents of the invention

The present invention aims to solve at least one of the technical problems existing in the prior art. To this end, embodiments of the present invention provide an image sensor, an image processing method, an image processing device, and an electronic device.

An image sensor, the image sensor includes a photosensitive pixel unit array, and the photosensitive pixel unit includes a first photosensitive pixel and a second photosensitive pixel.

The first photosensitive pixel has a first sensitivity.

The second photosensitive pixel has a second sensitivity, and the second sensitivity is smaller than the first sensitivity.

An image processing method for obtaining ambient brightness, used for processing scene images collected by an imaging device including the image sensor, the image processing method includes the following steps:

processing the scene image to obtain a first brightness value corresponding to the first photosensitive pixel;

judging whether the first brightness value is greater than a brightness threshold;

processing the scene image to obtain a second brightness value corresponding to the second photosensitive pixel when the first brightness value is greater than the brightness threshold; and

The second brightness value is used as the ambient brightness.

An image processing device for acquiring ambient brightness, used for processing scene images collected by an imaging device including the image sensor, the image processing device includes a first processing module, a judging module, a second processing module and a third processing module.

The first processing module is configured to process the scene image to obtain a first brightness value corresponding to the first photosensitive pixel.

The judging module is used to judge whether the first brightness value is greater than a brightness threshold.

The second processing module is configured to process the scene image to obtain a second brightness value corresponding to the second photosensitive pixel when the first brightness value is greater than the brightness threshold.

The third processing module is configured to use the second brightness value as the ambient brightness.

An electronic device includes an imaging device and the image processing device.

The image sensor, the image processing method for obtaining ambient brightness, the image processing device and the electronic device of the present invention use the second photosensitive pixel with low sensitivity to measure the ambient brightness, so that a more accurate measurement effect can be obtained in a bright environment.

Additional aspects and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention.

Description of drawings

The above and/or additional aspects and advantages of the present invention will become apparent and comprehensible from the description of the embodiments in conjunction with the following drawings, wherein:

FIG. 1 is a schematic structural diagram of an image sensor according to an embodiment of the present invention;

2 is a schematic flow diagram of an image processing method according to an embodiment of the present invention;

3 is a schematic plan view of an electronic device according to an embodiment of the present invention;

4 is a schematic flow diagram of an image processing method in some embodiments of the present invention;

Fig. 5 is a schematic diagram of functional modules of an image processing device in some embodiments of the present invention;

Fig. 6 is a schematic flowchart of an image processing method in some embodiments of the present invention;

Fig. 7 is a schematic diagram of another functional module of the image processing device in some embodiments of the present invention.

Description of main component symbols:

Electronic device 100, image processing device 10, first processing module 11, judgment module 12, second processing module 13, third processing module 14, fourth processing module 15, acquisition module 16, imaging device 20, image sensor 21, photosensitive The pixel unit array 212 , the photosensitive pixel unit 212 a , the first photosensitive pixel 2122 , and the second photosensitive pixel 2124 .

detailed description

Embodiments of the present invention are described in detail below, embodiments of which are illustrated in the drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the figures are exemplary only for explaining the present invention and should not be construed as limiting the present invention.

Referring to FIG. 1 , an image sensor 21 according to an embodiment of the present invention includes a light-sensing pixel unit array 212 , and the light-sensing pixel unit 212 a includes first light-sensing pixels 2122 and second light-sensing pixels 2124 . The first photosensitive pixel 2122 has a first sensitivity. The second photosensitive pixel 2124 has a second sensitivity, which is smaller than the first sensitivity.

Please refer to FIG. 2 and FIG. 3 together. The image processing method for acquiring ambient brightness according to the embodiment of the present invention may be used to process scene images collected by the imaging device 20 including the image sensor 21 . The image processing method includes the following steps:

S11: Process the scene image to obtain a first brightness value corresponding to the first photosensitive pixel 2122;

S12: Determine whether the first brightness value is greater than a brightness threshold;

S13: Process the scene image to obtain a second brightness value corresponding to the second photosensitive pixel 2124 when the first brightness value is greater than the brightness threshold; and

S14: Adopt the second brightness value as the ambient brightness.

Please refer to FIG. 3 again, the image processing device 10 for acquiring ambient brightness according to the embodiment of the present invention may be used to process scene images collected by the imaging device 20 including the image sensor 21 . The image processing device 10 includes a first processing module 11 , a judging module 12 , a second processing module 13 and a third processing module 14 . The first processing module 11 is used for processing the scene image to obtain the first brightness value corresponding to the first photosensitive pixel 2122 . The judging module 12 is used for judging whether the first brightness value is greater than a brightness threshold. The second processing module 13 is configured to process the scene image to obtain a second brightness value corresponding to the second photosensitive pixel 2124 when the first brightness value is greater than the brightness threshold. The third processing module 14 is used for adopting the second brightness value as the ambient brightness.

That is to say, the image processing method of the embodiment of the present invention can be implemented by the image processing device 10 of the embodiment of the present invention, wherein, step S11 can be implemented by the first processing module 11, step S12 can be implemented by the judging module 12, and step S13 can be It is realized by the second processing module 13 , and step S14 may be realized by the third processing module 14 .

In some implementations, the image processing device 10 of the embodiment of the present invention can be applied to the electronic device 100 of the embodiment of the present invention, or the electronic device 100 of the embodiment of the present invention can include the image processing device 10 of the embodiment of the present invention. In addition, the electronic device 100 in the embodiment of the present invention further includes an imaging device 20 , and the imaging device 20 is electrically connected to the image processing device 10 . The imaging device 20 includes the image sensor 21 of the embodiment of the present invention.

The image sensor 21, the image processing method for obtaining ambient brightness, the image processing device 10, and the electronic device 100 of the present invention use the second photosensitive pixel 2124 with low sensitivity to measure the ambient brightness, so that more accurate measurement can be obtained in a bright environment Effect.

It can be understood that in order to ensure the image acquisition capability of the image sensor 21, the light-sensitive pixels generally used are high-sensitivity light-sensitive pixels, and the high-sensitivity light-sensitive pixels are easily overexposed in a bright environment, so that the brightness value of the real environment cannot be obtained, so Brightness detection in high-brightness environments can be realized by adding low-sensitivity photosensitive pixels.

In some embodiments, the electronic device 100 includes a mobile phone, a tablet computer, a smart watch, a laptop computer, a smart bracelet, smart glasses or a smart helmet. In the embodiment of the present invention, the electronic device 100 is a mobile phone.

In some implementations, the imaging device 20 includes a front camera and/or a rear camera, without any limitation here. In an embodiment of the present invention, the imaging device 20 is a front camera.

In some embodiments, the ratio of the number of the first light-sensing pixels 2122 to the number of the second light-sensing pixels 2124 is between 7-63.

In this way, the image sensor 21 can not only ensure a good image acquisition ability, but also ensure the brightness acquisition ability in a bright environment.

It can be understood that the ability of the image sensor 21 to acquire images is realized by the first photosensitive pixel 2122 , and the ability to acquire brightness in a bright environment is realized by the second photosensitive pixel 2124 . When the ratio of the number of the first light-sensitive pixels 2122 to the second light-sensitive pixels 2124 is less than 7, it means that the number of the first light-sensitive pixels 2122 is too small, and the image acquisition capability of the image sensor 21 is greatly weakened at this time, thereby affecting the imaging quality; When the ratio of the number of the first photosensitive pixel 2122 to the second photosensitive pixel 2124 is greater than 63, it indicates that the number of the second photosensitive pixel 2124 is too small at this time, and at this time, the image sensor 21 acquires too little data for the brightness of the high-brightness environment, and it is easy to appear. big error.

In some embodiments, the ratio of the number of the first photosensitive pixels 2122 to the second photosensitive pixels 2124 may be 7, 15, 31 or 63, without any limitation here.

Please refer to FIG. 1 again. In one embodiment, the ratio of the number of the first photosensitive pixels 2122 to the second photosensitive pixels 2124 is 15, that is, in one photosensitive pixel unit 212a, the number of the first photosensitive pixels 2122 can be 15 The number of second photosensitive pixels 2124 may be one.

In some embodiments, the dynamic range of the second photosensitive pixel 2124 is greater than the dynamic range of the first photosensitive pixel 2122 .

In this way, the second light-sensitive pixel 2124 is less likely to be over-exposed in a bright environment.

In some embodiments, the second photosensitive pixel 2124 has the same sensitivity as the first photosensitive pixel 2122 , and the input upper limit of the second photosensitive pixel 2124 is higher than that of the first photosensitive pixel 2122 . In this way, in a high-brightness environment, since the input upper limit of the second photosensitive pixel 2124 is relatively high, more charges can be obtained, and signal saturation is not easy to occur, which ensures the accuracy of ambient brightness acquisition.

Referring to Fig. 4, in some embodiments, the image processing method includes the following steps:

S15: Adopt the first brightness value as the ambient brightness when the first brightness value is less than or equal to the brightness threshold.

Referring to FIG. 5 , in some implementations, the image processing device 10 includes a fourth processing module 15 . The fourth processing module 15 is configured to adopt the first brightness value as the ambient brightness when the first brightness value is less than or equal to the brightness threshold.

That is to say, step S15 can be implemented by the fourth processing module 15 .

In this way, when the ambient brightness is relatively low, the first brightness value may be used as the ambient brightness.

It can be understood that the average value of the luminance values of all the first photosensitive pixels 2122 of a photosensitive pixel unit 212a can be obtained as the first luminance value, since the first luminance value is obtained by calculating a plurality of first photosensitive pixels 2122, the first Accuracy and reliability of brightness values. When the first brightness value is less than or equal to the brightness threshold, it means that the ambient brightness is relatively low at this time, and the first brightness value may be used as the ambient brightness. In addition, utilizing the high sensitivity of the first photosensitive pixel 2122 can enable the image sensor 21 to acquire ambient brightness more accurately in a low-light environment.

Referring to Fig. 6, in some embodiments, the image processing method includes the following steps:

S16: Obtain the average value of the first brightness values of the four first photosensitive pixels 2122 adjacent to the second photosensitive pixel 2124 as the third brightness value of the second photosensitive pixel 2124 .

Referring to FIG. 7 , in some implementations, the image processing device 10 includes an acquisition module 16 . The obtaining module 16 is used to obtain the average value of the first brightness values of four first photosensitive pixels 2122 adjacent to the second photosensitive pixel 2124 as the third brightness value of the second photosensitive pixel 2124 .

That is to say, step S16 can be realized by the obtaining module 16 .

In this way, a relatively accurate third brightness value can be obtained as an imaging brightness value of the image through the first brightness value.

It can be understood that due to the low sensitivity of the second photosensitive pixel 2124, the ability of the second photosensitive pixel 2124 to acquire images is poor, so when performing image processing, the second photosensitive pixel 2124 can be regarded as a dead pixel, and the dead pixel compensation can be used method to obtain the brightness value of the second photosensitive pixel 2124.

In some embodiments, the dead point compensation method may be other methods besides the method in step S16 above. For example, the average value of the first brightness values of the eight first photosensitive pixels 2122 corresponding to the 8 neighborhoods of the second photosensitive pixel 2124 is acquired as the third brightness value. Do not make any restrictions here.

In the description of the embodiments of the present invention, the terms "first" and "second" are used for description purposes only, and cannot be understood as indicating or implying relative importance or implicitly indicating the quantity of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of said features. In the description of the embodiments of the present invention, "plurality" means two or more, unless otherwise specifically defined.

In the description of the embodiments of the present invention, it should be noted that unless otherwise specified and limited, the terms "installation", "connection", and "connection" should be understood in a broad sense, for example, it can be a fixed connection or a It is a detachable connection, or an integral connection; it can be mechanically connected, it can be electrically connected, or it can communicate with each other; it can be directly connected or indirectly connected through an intermediary, and it can be internal communication between two components or two components interaction relationship. Those of ordinary skill in the art can understand the specific meanings of the above terms in the embodiments of the present invention according to specific situations.

In the description of this specification, reference to the terms "one embodiment", "some embodiments", "exemplary embodiments", "example", "specific examples" or "some examples" etc. Specific features, structures, materials, or features described in an embodiment or example are included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the described specific features, structures, materials or characteristics may be combined in any suitable manner in any one or more embodiments or examples.

Any process or method descriptions in flowcharts or otherwise described herein may be understood to represent modules, segments or portions of code comprising one or more executable instructions for implementing specific logical functions or steps of the process , and the scope of preferred embodiments of the invention includes alternative implementations in which functions may be performed out of the order shown or discussed, including substantially concurrently or in reverse order depending on the functions involved, which shall It is understood by those skilled in the art to which the embodiments of the present invention pertain.

The logic and/or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a sequenced listing of executable instructions for implementing logical functions, can be embodied in any computer-readable medium, For use with instruction execution systems, devices, or devices (such as computer-based systems, systems including processing modules, or other systems that can fetch instructions from instruction execution systems, devices, or devices and execute instructions), or in conjunction with these instruction execution systems, devices or equipment used. For the purposes of this specification, a "computer-readable medium" may be any device that can contain, store, communicate, propagate or transmit a program for use in or in conjunction with an instruction execution system, device or device. More specific examples (non-exhaustive list) of computer-readable media include the following: electrical connection with one or more wires (electronic device), portable computer disk case (magnetic device), random access memory (RAM), Read Only Memory (ROM), Erasable and Editable Read Only Memory (EPROM or Flash Memory), Fiber Optic Devices, and Portable Compact Disc Read Only Memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program can be printed, since the program can be read, for example, by optically scanning the paper or other medium, followed by editing, interpretation or other suitable processing if necessary. The program is processed electronically and stored in computer memory.

It should be understood that each part of the embodiments of the present invention may be implemented by hardware, software, firmware or a combination thereof. In the embodiments described above, various steps or methods may be implemented by software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented by any one or combination of the following techniques known in the art: Discrete logic circuits, ASICs with suitable combinational logic gates, programmable gate arrays (PGAs), field programmable gate arrays (FPGAs), etc.

Those of ordinary skill in the art can understand that all or part of the steps carried by the methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium. During execution, one or a combination of the steps of the method embodiments is included.

In addition, each functional unit in each embodiment of the present invention may be integrated into one processing module, each unit may exist separately physically, or two or more units may be integrated into one module. The above-mentioned integrated modules can be implemented in the form of hardware or in the form of software function modules. If the integrated modules are realized in the form of software function modules and sold or used as independent products, they can also be stored in a computer-readable storage medium.

The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, and the like.

Although the embodiments of the present invention have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limiting the present invention, those skilled in the art can make the above-mentioned The embodiments are subject to changes, modifications, substitutions and variations.

Claims (10)

1. a kind of imaging sensor, it is characterised in that described image sensor includes sensitive pixel elements array, the light-sensitive image Plain unit includes：

First photosensitive pixel, first photosensitive pixel has the first sensitivity；With

Second photosensitive pixel, second photosensitive pixel has the second sensitivity, and second sensitivity is less than the described first spirit Sensitivity.

2. imaging sensor as claimed in claim 1, it is characterised in that first photosensitive pixel and second light-sensitive image The ratio between number of element is located between 7-63.

3. imaging sensor as claimed in claim 1, it is characterised in that the dynamic range of second photosensitive pixel is more than institute State the dynamic range of the first photosensitive pixel.

4. a kind of image processing method for obtaining ambient brightness, includes as described in claim 1-3 any one for handling The scene image of the imaging device collection of imaging sensor, it is characterised in that described image processing method comprises the following steps：

The scene image is handled to obtain corresponding first brightness value of first photosensitive pixel；

Judge whether first brightness value is more than luminance threshold；

Handle the scene image to obtain second photosensitive pixel when first brightness value is more than the luminance threshold Corresponding second brightness value；With

The ambient brightness is used as using second brightness value.

5. image processing method as claimed in claim 4, it is characterised in that described image processing method comprises the following steps：

The ambient brightness is used as using first brightness value when first brightness value is less than or equal to the luminance threshold.

6. image processing method as claimed in claim 4, it is characterised in that described image processing method comprises the following steps：

The average value for obtaining first brightness value of four adjacent first photosensitive pixels of second photosensitive pixel is made For the 3rd brightness value of second photosensitive pixel.

7. a kind of image processing apparatus for obtaining ambient brightness, includes as described in claim 1-3 any one for handling The scene image of the imaging device collection of imaging sensor, it is characterised in that described image processing unit includes：

First processing module, the first processing module is used to handle the scene image to obtain first photosensitive pixel pair The first brightness value answered；

Judge module, the judge module is used to judge whether first brightness value is more than luminance threshold；

Second processing module, the Second processing module is used to handle institute when first brightness value is more than the luminance threshold Scene image is stated to obtain corresponding second brightness value of second photosensitive pixel；With

3rd processing module, the 3rd processing module is used to be used as the ambient brightness using second brightness value.

8. image processing apparatus as claimed in claim 7, it is characterised in that described image processing unit includes：

Fourth processing module, the fourth processing module is used to adopt when first brightness value is less than or equal to the luminance threshold The ambient brightness is used as with first brightness value.

9. image processing apparatus as claimed in claim 7, it is characterised in that described image processing unit includes：

Acquisition module, the acquisition module is used to obtain four adjacent first photosensitive pixels of second photosensitive pixel The average value of first brightness value as second photosensitive pixel the 3rd brightness value.

10. a kind of electronic installation, it is characterised in that including：

Imaging device；With

Image processing apparatus as described in claim 7-9 any one.