CN107341486B - Multi-sensor image captures analysis platform - Google Patents

Abstract

The invention relates to a multi-sensor image capture and analysis platform, comprising: a temperature sensor, which is arranged on a tablet computer and is far away from the heating element of the tablet computer, and includes a PT100 temperature sensitive resistor, which is used to determine whether the nearby temperature is close to the temperature of the human body, and accordingly Send out a signal close to the temperature of the human body or far away from the temperature of the human body; a composite imaging device, set on the tablet computer, including filters, lenses, lens holders, multiple lenses and multiple CMOS sensors, used to monitor the environment of the tablet computer For imaging processing, the number of multiple lenses and multiple CMOS sensors is the same, and each CMOS sensor corresponds to a lens, which is used to photoelectrically convert the transmitted light from the corresponding lens to obtain a high-definition transmitted image. Through the present invention, the recognition accuracy of the environment where the tablet computer is located can be improved.

Description

Multi-sensor image capture analysis platform

technical field

The invention relates to the field of image acquisition, in particular to a multi-sensor image capture and analysis platform.

Background technique

Tablet PC is also called portable computer (English: Tablet Personal Computer, referred to as Tablet PC, Flat Pc, Tablet, Slates), is a kind of small-sized, portable personal computer, uses touch screen as basic input device. It has a touch screen (also known as digital tablet technology) that allows users to work with a stylus or digital pen instead of a traditional keyboard or mouse. Users can input via built-in handwriting recognition, an on-screen soft keyboard, voice recognition, or a real keyboard (if the model is equipped).

Tablet PCs were proposed by Bill Gates and should support chip architectures from X86 (Intel, AMD) and ARM. Tablet PCs are divided into ARM architecture (representative products are ipad and Android tablets) and X86 architecture (representative products are Surface Pro) The latter X86-based tablet computer generally uses an Intel processor and a Windows operating system, has complete computer and tablet functions, and supports exe programs. The development of tablet PCs is accompanied by the rapid development of communication technology. As an emerging technology, CDMA and CDMA2000 are rapidly becoming popular all over the world and have already occupied 18% of the wireless market. As of 2012, the number of CDMA2000 users worldwide has exceeded 256 million, and 156 operators in 70 countries have commercialized 3G CDMA services. Looking at the tablet computer concept product proposed by Microsoft, the tablet computer is a PC that does not need to be flipped, has no keyboard, is small enough to be put into a lady's handbag, but has complete functions.

The miniaturization of the tablet makes it easy to be carried in the user's pocket for convenient use. However, this also brings a disadvantage, which is likely to cause the user to miss some important calls or important information. Therefore, it is necessary to carry out the pocket environment Accurate discrimination, however, is difficult for image processing-based environment recognition in the dim light of the pocket environment.

Contents of the invention

In order to solve the above problems, the present invention provides a multi-sensor image capture and analysis platform, which introduces multiple targeted image acquisition devices and image processing devices to realize accurate recognition of the user's pocket environment in dim light, and also introduces A temperature sensor is used to determine whether the tablet is close to the temperature of the human body, and when receiving a signal close to the temperature of the human body, start the composite imaging device for imaging, and when receiving the pocket-carrying environment signal from the data extraction device, control the tablet computer to increase the message notification sound on the tablet.

According to an aspect of the present invention, a multi-sensor image capture and analysis platform is provided, the platform includes:

The temperature sensor is set on the tablet computer and away from the heating element of the tablet computer, including a PT100 temperature sensitive resistor, which is used to determine whether the nearby temperature is close to the temperature of the human body, so as to send a signal close to the temperature of the human body or away from the temperature signal of the human body; compound imaging The device, set on the tablet computer, includes optical filters, lenses, lens holders, multiple lenses and multiple CMOS sensors, and is used for imaging processing of the environment where the tablet computer is located, and the number of multiple lenses and multiple CMOS sensors is the same , each CMOS sensor corresponds to a lens, which is used to photoelectrically convert the transmitted light from the corresponding lens to obtain a high-definition transmitted image;

The data extraction device is connected with the composite imaging device, and is used to receive multiple high-definition transmission images output by multiple CMOS sensors, determine the center point of each high-definition transmission image and determine the depth of field of each high-definition transmission image, and average each The depth of field of the high-definition transmission image is obtained to obtain the scene depth; the data composite device is connected with the data extraction device, and is used to take the center point of each high-definition transmission image as the center, and take the scene depth as the radius to obtain a square sub-image, and according to The arrangement sequence of multiple lenses connects the corresponding multiple high-definition transmission images to obtain on-site high-definition images;

The gradient extraction device is connected with the data compounding device and is used to receive the live high-definition image. For the pixel value of each pixel in the live high-definition image, subtract the pixel value of its pixel value from the pixel value of the lower row pixel in the same column and take the absolute value to obtain the first An absolute value, subtract the pixel value of its pixel value from the pixel value of the following pixels in the same row and take the absolute value to obtain the second absolute value, and the sum obtained after adding the first absolute value and the second absolute value is used as the gradient, where the field The gradient of each pixel in the last row of the high-definition image directly uses the gradient of the same column of pixels in the previous row, and the gradient of each pixel in the last column of the live high-definition image directly uses the gradient of the same row of pixels in the previous column; the gradient processing equipment, and The gradient extraction device is connected to compare its gradient with a preset gradient threshold for the pixel value of each pixel in the on-site high-definition image, and for each pixel whose gradient is greater than or equal to the preset gradient threshold, its pixel value Sharpening processing to obtain the processed pixel value, for each pixel whose gradient is less than the preset gradient threshold, directly use its pixel value as the processed pixel value, and the processed pixel values of all pixels in the live high-definition image form the live high-definition image The gradient processing image corresponding to the image;

The data extraction device is used to extract the image data from the gradient processed image to determine the environment type of the tablet computer, to send an indoor environment signal, an outdoor environment signal, a pocket environment signal or a dark environment signal, and the location of the tablet computer The environment types include indoor environment, outdoor environment, pocket environment and dark environment;

The ARM11 processing chip is respectively connected with the data extraction device and the temperature sensor, and is used to start the composite imaging device when receiving a signal close to the temperature of the human body, and receive the pocket carrying environment signal from the data extraction device , control your tablet to increase the message notification sound on your tablet.

More specifically, in the multi-sensor image capture and analysis platform: the ARM11 processing chip is also used to shut down the composite imaging device when receiving a temperature signal far away from the human body.

More specifically, in the multi-sensor image capture and analysis platform: in the composite imaging device, the optical filter, the lens, the plurality of lenses and the plurality of CMOS sensors are all arranged on the on the lens holder.

More specifically, in the multi-sensor image capture and analysis platform: the optical filter is arranged in front of the lens.

More specifically, in the multi-sensor image capture and analysis platform: the multiple lenses are arranged in front of the lens.

More specifically, in the multi-sensor image capture and analysis platform: the multiple CMOS sensors are arranged in front of the multiple lenses.

More specifically, in the multi-sensor image capture and analysis platform: each CMOS sensor is arranged in the composite imaging device facing the corresponding lens.

More specifically, in the multi-sensor image capture analysis platform: in the gradient processing device, the preset gradient threshold is determined according to the sharpening level, the higher the sharpening level, the smaller the preset gradient threshold.

More specifically, in the multi-sensor image capture and analysis platform: in the gradient processing device, a built-in storage unit is also included for storing the mapping relationship between the sharpening level and the preset gradient threshold.

Description of drawings

Embodiments of the present invention will be described below in conjunction with the accompanying drawings, wherein:

Fig. 1 is a structural block diagram of a multi-sensor image capture and analysis platform according to an embodiment of the present invention.

Fig. 2 is a structural block diagram of a temperature sensor of a multi-sensor image capture and analysis platform according to an embodiment of the present invention.

Reference signs: 1 temperature sensor; 2 composite imaging device; 3 data extraction device; 4 data composite device; 5 gradient extraction device; 6 gradient processing device; 21 PT100 temperature sensitive resistor; 22 microcontroller

Detailed ways

The implementation of the multi-sensor image capture and analysis platform of the present invention will be described in detail below with reference to the accompanying drawings.

In 2010, the Apple iPad set off a tablet boom all over the world. In 2010, the growth rate of keyword searches for tablet computers reached 1328%. Tablet computers have had a revolutionary impact on the traditional PC industry and even the entire 3C industry. At the same time, as the popularity of tablet PCs heats up, manufacturers from different industries, such as consumer electronics, PC, communications, and software manufacturers, have joined the tablet PC industry one after another. Consulting agencies are also optimistic about the entire tablet PC industry. For a time, from the upstream to the terminal, from the operating system to the software application, an ecological chain of the tablet PC industry has been formed, and the links in the ecological chain of the tablet PC industry have developed rapidly. %, of which tablet sales were 1.74 million units, accounting for about 3.58%. With the rapid development of tablet PCs, the status of tablet PCs in the PC industry will become more and more important, and its proportion in the PC industry will also increase.

From the perspective of the international market, in 2017, the global tablet market is expected to reach 49 billion US dollars. From the perspective of industry development stage, in the three years from 2010 to 2012, tablet PC will go through the stage from birth to maturity, and the whole industry will present a rapid development trend. During this period, the direction of industrial development, market size, industry structure, and consumer demand are not clear, and there are many market opportunities, and new brands will appear in every link of the industrial chain. Among them, the three links of hardware terminal equipment, service content provision and peripheral supporting equipment will be more concentrated and obvious.

The portability of the tablet computer also makes it difficult for the user to answer the sound of incoming calls or messages when the user carries it. In order to overcome the above disadvantages, the present invention builds a multi-sensor image capture and analysis platform, and the specific implementation is as follows.

Fig. 1 is the structural block diagram of the multi-sensor image capture analysis platform shown according to the embodiment of the present invention, and described platform comprises:

The temperature sensor is arranged on the tablet computer and is far away from the heating element of the tablet computer, including a PT100 temperature sensitive resistor, which is used to determine whether the nearby temperature is close to the human body temperature, so as to send a signal close to the human body temperature or far away from the human body temperature signal accordingly;

Alternatively, as shown in Figure 2, the temperature sensor may include a PT100 temperature sensitive resistor and a microcontroller, the PT100 temperature sensitive resistor is used to determine whether the nearby temperature is close to the temperature of the human body, and the microcontroller is connected with the PT100 temperature sensitive resistor for Based on the output of the PT100 temperature sensitive resistor, it sends out a signal close to the human body temperature or far away from the human body temperature signal.

Composite imaging equipment, set on a tablet computer, including optical filters, lenses, lens holders, multiple lenses and multiple CMOS sensors, used for imaging processing of the environment where the tablet computer is located, multiple lenses and multiple CMOS sensors The number is the same, and each CMOS sensor corresponds to a lens, which is used to photoelectrically convert the transmitted light from the corresponding lens to obtain a high-definition transmitted image;

The data extraction device is connected with the composite imaging device, and is used to receive multiple high-definition transmission images output by multiple CMOS sensors, determine the center point of each high-definition transmission image and determine the depth of field of each high-definition transmission image, and average each Depth of field in high-definition transmission images to obtain live field depth;

The data compounding device is connected with the data extraction device, and is used to take the center point of each high-definition transmission image as the center and the field depth of the scene as the radius to obtain a square sub-image, and according to the arrangement order of the multiple lenses, the corresponding multiple A high-definition transmission image is connected to obtain a live high-definition image;

The gradient extraction device is connected with the data compounding device and is used to receive the live high-definition image. For the pixel value of each pixel in the live high-definition image, subtract the pixel value of its pixel value from the pixel value of the lower row pixel in the same column and take the absolute value to obtain the first An absolute value, subtract the pixel value of its pixel value from the pixel value of the following pixels in the same row and take the absolute value to obtain the second absolute value, and the sum obtained after adding the first absolute value and the second absolute value is used as the gradient, where the field The gradient of each pixel in the last row of the high-definition image directly uses the gradient of the same column of pixels in the previous row, and the gradient of each pixel in the last column of the live high-definition image directly uses the gradient of the same row of pixels in the previous column;

The gradient processing device is connected with the gradient extraction device, and is used for comparing the gradient of each pixel value in the on-site high-definition image with the preset gradient threshold, and for each pixel whose gradient is greater than or equal to the preset gradient threshold, Sharpen its pixel value to obtain the processed pixel value. For each pixel whose gradient is less than the preset gradient threshold, directly use its pixel value as the processed pixel value. The processed pixel value of all pixels in the live high-definition image The pixel value forms the gradient processing image corresponding to the live high-definition image;

The data extraction device is used to extract the image data from the gradient processed image to determine the environment type of the tablet computer, to send an indoor environment signal, an outdoor environment signal, a pocket environment signal or a dark environment signal, and the location of the tablet computer The environment types include indoor environment, outdoor environment, pocket environment and dark environment;

The ARM11 processing chip is respectively connected with the data extraction device and the temperature sensor, and is used to start the composite imaging device when receiving a signal close to the temperature of the human body, and receive the pocket carrying environment signal from the data extraction device , control your tablet to increase the message notification sound on your tablet.

Next, continue to further describe the specific structure of the multi-sensor image capture and analysis platform of the present invention.

In said platform:

The ARM11 processing chip is also used for shutting down the composite imaging device when receiving a temperature signal away from the human body.

In said platform:

In the composite imaging device, the optical filter, the lens, the plurality of lenses and the plurality of CMOS sensors are all arranged on the lens holder.

In said platform:

The filter is arranged in front of the lens.

In said platform:

The plurality of lenses are arranged in front of the lens.

In said platform:

The plurality of CMOS sensors are disposed in front of the plurality of lenses.

In said platform:

Each CMOS sensor is disposed in the compound type imaging device facing a corresponding lens.

In said platform:

In the gradient processing device, the preset gradient threshold is determined according to the sharpening level, the higher the sharpening level, the smaller the preset gradient threshold.

and in said platform:

In the gradient processing device, a built-in storage unit is also included for storing the mapping relationship between the sharpening level and the preset gradient threshold.

In addition, the CMOS image sensor is a typical solid-state imaging sensor and has a common historical origin with CCD. A CMOS image sensor is usually composed of several parts such as image sensor array, row driver, column driver, timing control logic, AD converter, data bus output interface, control interface, etc. These parts are usually integrated on the same silicon chip. Its working process can generally be divided into several parts: reset, photoelectric conversion, integration, and readout.

Other digital signal processing circuits can also be integrated on the CMOS image sensor chip, such as AD converter, automatic exposure control, non-uniformity compensation, white balance processing, black level control, gamma correction, etc. DSP devices with programmable functions are integrated with CMOS devices to form a single-chip digital camera and image processing system.

In 1963, Morrison published the computable sensor, which is a structure that can use the photoconductive effect to measure the spot position, which became the beginning of the development of CMOS image sensor. In 1995, the low-noise CMOS active pixel sensor single-chip digital camera was successful.

The CMOS image sensor has the following advantages: 1) Random window reading capability. The random window read operation is an aspect of the CMOS image sensor that is superior to the CCD in function, and it is also called the region of interest selection. In addition, the high integration feature of CMOS image sensor makes it easy to realize the function of opening multiple tracking windows at the same time. 2), anti-radiation ability. In general, the potential radiation resistance performance of CMOS image sensors has an important enhancement relative to the performance of CCDs. 3), system complexity and reliability. Using CMOS image sensor can greatly simplify the system hardware structure. 4), non-destructive data readout method. 5), optimized exposure control. It is worth noting that due to the integration of multiple functional transistors in the pixel structure, CMOS image sensors also have some shortcomings, mainly two indicators of noise and filling rate. In view of the relatively superior performance of the CMOS image sensor, the CMOS image sensor has been widely used in various fields.

Adopting the multi-sensor image capture and analysis platform of the present invention, aiming at the technical problem that tablet computers cannot realize pocket environment detection in the prior art, by introducing temperature sensors, composite imaging equipment, data extraction equipment, data composite equipment, gradient extraction equipment, gradient Targeted image acquisition equipment or image processing equipment such as processing equipment can obtain image data suitable for environment detection, thereby providing more valuable reference data for tablet computer pocket environment detection and tablet computer volume control.

It can be understood that although the present invention has been disclosed above with preferred embodiments, the above embodiments are not intended to limit the present invention. For any person skilled in the art, without departing from the scope of the technical solution of the present invention, the technical content disclosed above can be used to make many possible changes and modifications to the technical solution of the present invention, or be modified into equivalent changes, etc. effective example. Therefore, any simple modifications, equivalent changes and modifications made to the above embodiments according to the technical essence of the present invention, which do not deviate from the technical solution of the present invention, still fall within the protection scope of the technical solution of the present invention.

Claims (5)

1. a kind of multi-sensor image captures analysis platform, including：

Temperature sensor, is arranged on tablet computer and the heater element away from tablet computer, including PT100 temperature-sensitive resistors Device, for determining that nearby whether temperature is close to human body temperature, correspondingly to send close to human body temperature signal or away from human temperature Spend signal；

Compound imaging device, is arranged on tablet computer, including optical filter, camera lens, lens bracket, multiple lens and multiple Cmos sensor, for carrying out the quantity of imaging, multiple lens and multiple cmos sensors to tablet computer local environment Identical, each cmos sensor corresponds to a lens, for being obtained from the transmitted light progress opto-electronic conversion of corresponding lens Obtain high definition transmission image；

Data extraction device, is connected with the compound imaging device, for receiving the multiple high of multiple cmos sensor outputs Clear transmission image, determines the central point of each high definition transmission image and determines the depth of field of each high definition transmission image, and The depth of field of average each high definition transmission image is to obtain the live depth of field；

Data equipment complex, is connected with the data extraction device, for using the central point of each high definition transmission image in The heart, square subgraph is obtained using the live depth of field as the length of side, and will be corresponding multiple according to putting in order for multiple lens High definition transmission image is connected to obtain live high-definition image；

Gradient extraction equipment, is connected with data equipment complex, for receiving live high-definition image, in live high-definition image The pixel value of each pixel, its pixel value is subtracted take absolute value after the pixel value of its same column downlink pixel it is exhausted to obtain first To value, take absolute value after its pixel value is subtracted the pixel value of its following pixel of going together to obtain the second absolute value, it is exhausted by first It is being obtained after being added to value with the second absolute value and as gradient, wherein, each picture of last column of live high-definition image The gradient of element directly takes the gradient of the same column pixel of lastrow, the ladder of each pixel of last row of live high-definition image Degree directly takes the gradient of colleague's pixel of previous column；

Gradient processing equipment, is connected with the gradient extraction equipment, for for each pixel in live high-definition image Pixel value, by its gradient compared with predetermined gradient threshold value, each pixel of predetermined gradient threshold value is more than or equal to for gradient, Processing is sharpened to its pixel value with the pixel value after handle, to each pixel of the gradient less than predetermined gradient threshold value, Directly using its pixel value as the pixel value after processing, the pixel value after the processing of all pixels in live high-definition image is formed The corresponding gradient processing image of live high-definition image；

Data extraction device, for carrying out image data extraction to gradient processing image, to determine the present of tablet computer Environmental form, environmental signal or dark surrounds signal, tablet are carried to send indoor environment signal, outdoor environment signal, pocket The present environmental form of computer includes indoor environment, outdoor environment, pocket and carries environment and dark surrounds；

ARM11 process chips, are connected with the data extraction device and the temperature sensor respectively, for receive it is close During human body temperature signal, start the compound imaging device, and ring is carried receiving the pocket from data extraction device During the signal of border, tablet computer is controlled to improve the information notification voice of tablet computer.

2. multi-sensor image as claimed in claim 1 captures analysis platform, it is characterised in that：

ARM11 process chips are additionally operable to, when receiving away from human body temperature signal, close the compound imaging device.

3. multi-sensor image as claimed in claim 2 captures analysis platform, it is characterised in that：

In the compound imaging device, the optical filter, the camera lens, the multiple lens and the multiple CMOS sensings Device is all disposed within the lens bracket.

4. multi-sensor image as claimed in claim 3 captures analysis platform, it is characterised in that：

The optical filter is arranged on the front of the camera lens.

5. multi-sensor image as claimed in claim 4 captures analysis platform, it is characterised in that：

The multiple lens are arranged on the front of the camera lens.