CN114708232A - Image processing method and device, storage medium and electronic equipment - Google Patents

Abstract

The embodiment of the application discloses an image processing method, an image processing device, a storage medium and electronic equipment, wherein the method comprises the following steps: the method comprises the steps of obtaining a reflection intensity map, determining at least one light spot in the reflection intensity map, obtaining pixel characteristic parameters corresponding to light spot pixel points in the light spot, and determining an interference light spot from the at least one light spot based on the pixel characteristic parameters. By adopting the embodiment of the application, the interference light spots can be accurately identified.

Description

An image processing method, device, storage medium and electronic device

technical field

The present application relates to the field of computer technology, and in particular, to an image processing method, apparatus, storage medium, and electronic device.

Background technique

At present, in the 3D imaging scene, it is mainly based on the binocular camera, the result light method and the time-of-flight method. Taking the indirect time-of-flight method (iTOF method) in the time-of-flight method as an example, the iTOF method is a method for determining the distance to the target to be measured by measuring the phase relationship between the modulated emitted light and the received light, and it is widely used. In 3D depth vision related gesture recognition, face recognition, 3D modeling, somatosensory games, machine vision, assisted focus, security, automatic driving and other related application scenarios.

SUMMARY OF THE INVENTION

Embodiments of the present application provide an image processing method, device, storage medium, and electronic device, and the technical solutions are as follows:

In a first aspect, an embodiment of the present application provides an image processing method, the method comprising:

acquiring a reflection intensity map, and determining at least one light spot in the reflection intensity map;

obtaining the pixel characteristic parameters corresponding to the pixel points of the light spot in the light spot;

Interfering light spots are determined from the at least one light spot based on the pixel characteristic parameters.

In a second aspect, an embodiment of the present application provides an image processing apparatus, and the apparatus includes:

a light spot determination module, configured to acquire a reflection intensity map, and determine at least one light spot in the reflection intensity map;

a parameter acquisition module, configured to acquire pixel characteristic parameters corresponding to the pixel points of the light spot in the light spot;

An interference determination module, configured to determine an interference light spot from the at least one light spot based on the pixel characteristic parameter.

In a third aspect, an embodiment of the present application provides a computer storage medium, where the computer storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor and executing the above method steps.

In a fourth aspect, an embodiment of the present application provides an electronic device, which may include: a processor and a memory; wherein, the memory stores a computer program, and the computer program is adapted to be loaded by the processor and execute the above method steps .

The beneficial effects brought by the technical solutions provided by some embodiments of the present application include at least:

In one or more embodiments of the present application, by acquiring a reflection intensity map, at least one light spot in the reflection intensity map is determined, and then pixel feature parameters corresponding to the pixel points of the light spot in the light spot are obtained, and based on the pixel feature parameters, the difference between the interference light spot and the light spot is obtained. The difference of pixel characteristic parameters on the effective light spot can accurately determine the interference light spot from at least one light spot, which can help to eliminate the light spot interference caused by the superposition of the edge signals of multiple light spots; Relying on , realizes accurate identification of light spot interference, can assist in outputting high-precision depth measurement results, and improves the robustness in depth measurement scenarios.

Description of drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the accompanying drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only These are some embodiments of the present application. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings without any creative effort.

1 is a schematic flowchart of an image processing method provided by an embodiment of the present application;

2 is a schematic diagram of a partial area in a reflection intensity map involved in an image processing method provided by an embodiment of the present application;

3 is a partial schematic diagram of a scene involving light spot interference points involved in the image processing method provided by the embodiment of the present application;

4 is a schematic flowchart of another image processing method provided by an embodiment of the present application;

5 is a schematic flowchart of another image processing method provided by an embodiment of the present application;

6 is a schematic diagram of a scene of a light spot provided by an embodiment of the present application;

7 is a schematic diagram of a scene involving interference light spots provided by an embodiment of the present application;

FIG. 8 is a schematic structural diagram of an image processing apparatus provided by an embodiment of the present application;

9 is a schematic structural diagram of an interference determination module provided by an embodiment of the present application;

10 is a schematic structural diagram of another image processing apparatus provided by an embodiment of the present application;

11 is a schematic structural diagram of an interference verification module provided by an embodiment of the present application;

12 is a schematic structural diagram of an electronic device provided by an embodiment of the present application;

13 is a schematic structural diagram of an operating system and a user space provided by an embodiment of the present application;

Figure 14 is an architecture diagram of the Android operating system in Figure 13;

FIG. 15 is an architectural diagram of the IOS operating system in FIG. 13 .

Detailed ways

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

In the description of the present application, it should be understood that the terms "first", "second" and the like are used for descriptive purposes only, and should not be construed as indicating or implying relative importance. In the description of the present application, it should be noted that, unless otherwise expressly specified and defined, "including" and "having" and any modifications thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but optionally also includes unlisted steps or units, or optionally also includes For other steps or units inherent to these processes, methods, products or devices. For those of ordinary skill in the art, the specific meanings of the above terms in this application can be understood in specific situations. Also, in the description of the present application, unless otherwise specified, "a plurality" means two or more. "And/or", which describes the association relationship of the associated objects, means that there can be three kinds of relationships, for example, A and/or B, which can mean that A exists alone, A and B exist at the same time, and B exists alone. The character "/" generally indicates that the associated objects are an "or" relationship.

The present application will be described in detail below with reference to specific embodiments.

In one embodiment, as shown in FIG. 1 , an image processing method is proposed, which can be implemented by relying on a computer program and can be run on an image processing device based on the von Neumann system. The computer program can be integrated into an application or run as a stand-alone utility application. The image processing apparatus may be a terminal, including but not limited to: a personal computer, a tablet computer, a handheld device, a vehicle-mounted device, a wearable device, a computing device, or other processing devices connected to a wireless modem. Terminal equipment may be called by different names in different networks, for example: user equipment, access terminal, subscriber unit, subscriber station, mobile station, mobile station, remote station, remote terminal, mobile device, user terminal, terminal, wireless communication Equipment, user agent or user equipment, cellular phone, cordless phone, terminal equipment in 5G network or future evolution network, etc.

Specifically, the image processing method includes:

S101: Obtain a reflection intensity map, and determine at least one light spot in the reflection intensity map;

The reflection intensity map is obtained based on the TOF camera mounted on the terminal. The TOF camera is a camera that uses the time-of-flight method for 3D imaging; the terminal emits a modulated light beam through the TOF camera, and when it encounters an object, it is reflected. The element (Sensor) receives the reflected light signal reflected by the object, and generates a reflection intensity map. The reflection intensity map can be used to feed back the pixel intensity corresponding to at least one pixel, and the light energy received corresponding to at least one pixel is proportional to the pixel intensity of the pixel in the reflection intensity map, that is, the stronger the light energy corresponding to the pixel is received, The pixel intensity of the pixel is also higher. In practical application scenarios, the terminal performs scene imaging through the TOF camera, and can obtain the depth map (target scene depth distance map, also called Depth map) and reflection intensity map in the current scene.

It is understandable that the TOF camera is based on the time-of-flight method for 3D imaging. The time-of-flight method can be divided into direct time-of-flight (dTOF) and indirect time-of-flight (iTOF) according to the obtained signal results. Signals obtained by the direct time-of-flight method The result is the time difference, and the result of the signal obtained by the indirect time-of-flight method is the phase difference;

It can be understood that a TOF camera can be understood as a TOF module integrated in the terminal, and the TOF module usually includes a light receiving end (RX) and a light transmitting end (TX);

Schematically, the light-receiving end usually includes a lens, a narrow-band filter (such as a 940nm narrow-band filter), a TOF sensor, etc., and the light-emitting end usually includes a Vertical-Cavity Surface-Emitting Laser (VCSEL) array, Collimating lens, Diffractive Optical Elements (DOE), etc., DOE can be a diffuser Diffuser. The VCSELs in the VCSEL array are evenly arranged on a plane, and are used to emit light beams with two-dimensional regular patterns; the collimating lens is used to receive and condense the light beams to ensure that the light beams have parallel light paths; DOE is used to The light beam is scattered and projected onto the object in the scene according to a certain field of view. The bright spot formed after being reflected on the object and received by the light receiving end is the speckle. For example, a VCSEL array usually includes at least two VCSELs, and the at least two VCSELs are uniformly arranged. When the uniformly arranged VCSELs are used to emit light beams, regular and uniform speckles can be obtained.

In practical applications, the light intensity value received by each pixel (pixel) at the light emitting end (RX) is usually used to represent the confidence of the pixel (point) to generate a depth value, that is, Confidence. The reflection intensity map can be understood as being used to feed back the light intensity value received by each Pixel. In some embodiments, the reflection intensity map can also be called a confidence map (Confidence map). The parameter value of each pixel is characterized by the intensity of the received signal light, which is then used as the confidence of the measured value of the target distance at that point.

Optionally, the TOF camera mounted on the terminal can be a device that supports the speckle iTOF mode, that is, it can be used as a speckle iTOF camera. Light is several beams of light beams. After being reflected by the target object in the scene, several light spots are received on the RX end, that is, the area illuminated by the light spot on the depth map has an effective depth value, and the remaining areas are all Invalid value, if there are valid depth values in the remaining areas, the depth value pixels in this area are interference points and can be removed accordingly. In the present application, speckle interference can be accurately identified based on the reflection intensity map by executing the image processing method.

It can be understood that after being reflected by the target object in the scene, several light spots received on the RX end are actually several speckles, usually at least one speckle will cover several pixels, but each speckle is used in depth images, ranging and other application scenarios. Only one valid depth value is generated in correspondence.

In a feasible implementation manner, after acquiring the reflection intensity map, the terminal may determine at least one light spot based on the intensity distribution information of at least one pixel point in the reflection intensity map.

Schematically, usually, the shape of the spot in the reflection intensity graph is a two-dimensional Gaussian shape, that is, the intensity of the spot is high at the center and gradually weakens around the spot; that is, at least one spot can be identified based on the spot shape in the reflection intensity graph. .

S102: Obtain pixel feature parameters corresponding to the pixel points of the light spot in the light spot;

Illustratively, in the reflection intensity map, the region where at least one light spot is located may cover multiple light spot pixel points. As shown in Figure 2, Figure 2 is a schematic diagram of a part of the reflection intensity map. In Figure 2, the dotted circular frame can be regarded as a light spot, and the light spot corresponding to the dotted circular frame presents a two-dimensional Gaussian shape, which is also That is, the intensity of the center of the light spot is high, and the surrounding area of the light spot gradually becomes weaker. In Fig. 2, each pixel grid can be regarded as a pixel point, and the area covered by the dotted circle frame usually contains multiple pixel points, and these pixel points are also called light spot pixels. The pixel characteristic parameter may be a pixel intensity parameter, a pixel position parameter, and the like of the light spot pixel point. Pixel feature parameters can be directly obtained based on the reflection intensity map.

S103: Determine an interference light spot from the at least one light spot based on the pixel characteristic parameter.

It is understandable that since the reflected light beam will diverge (that is, a light spot covers multiple pixels in the reflection intensity map), the light spot shape of the light spot in the reflection intensity map is usually a two-dimensional Gaussian shape, and the central light energy The strongest, i.e. the spot image shown in Figure 2. The interference light spot is caused by the superposition of the divergent edge signals of at least two light spots. For example, if the distance between multiple light spots is relatively close, the divergent edge signals of these multiple light spots will be superimposed on each other, forming a "pseudo" between the light spots. Light spot", this "pseudo light spot" also conforms to the two-dimensional Gaussian shape, but the received light energy is much smaller than the theoretical design value, and the signal-to-noise ratio is poor in subsequent application scenarios. In practical application scenarios, these "pseudo light spots" can also be generated because A rough depth value will be mistakenly identified as a light spot, but because most of the poor precision will cause interference to the back-end application, that is, the interference point.

Schematically, for example, due to the existence of objective factors such as lens distortion and optical signal divergence, when the distance between at least two light spots is relatively close (for example, the distance between two light spots is about twice the distance between the central areas of the light spots) , a "pseudo-spot", that is, a spot interference point, will appear between the mutually influencing spots. For example, as shown in Figure 3, Figure 3 is a partial schematic diagram of a scene involving spot interference points. There are actually 4 light spots on the surface of an object, and the light spot intensities of the generated 4 light spots are very close (the values in the depth map (Depth map) or reflection intensity map are equivalent), and the back-end applications such as virtual reality and ranging The scene will think that the surface of the area corresponding to these 4 light spots is a plane, but due to signal divergence between these 4 light spots (it can be understood that due to the hardware design at the factory stage, etc., at least one light spot may actually be covered by a light spot in the reflection intensity map. The case of multiple pixels) and at least the edge signals between the two light spots have a signal superposition phenomenon, that is to say, there will be an interference light spot due to the signal superposition phenomenon, as shown in Figure 3, the middle light spot shown in Figure 3 is actually An interference light spot caused by the superposition of edge signals between multiple light spots. Usually, the pixel with high pixel intensity in the area covered by the interference light spot can be used as the interference light spot. It is understandable that the parameters of the interference light spot and the actual light spot are quite different. (For example, the depth value in the depth map, the pixel intensity value in the reflection intensity map), in the back-end application scenario, the surface of these 5 points may be fitted to a surface, causing serious errors. Depth pixel points such as corresponding positions in the depth map can be eliminated based on the interference light spots, so as to reduce the light spot interference.

It can be understood that the interference spot detection is performed on at least one light spot based on the pixel characteristic parameters, so as to detect the pixel feature corresponding to the pixel point between the actual light spot and the interference light spot caused by the superposition of the edge light signals of multiple actual light spots. Differences in parameters (such as pixel intensity parameters, pixel position parameters), based on this, the terminal can determine the interference light spot from at least one light spot; for example, based on the pixel intensity parameter, the interference light spot intensity detection can be performed on at least one light spot, and the terminal can determine the interference light spot from at least one light spot. For example, based on the pixel position parameters, the intensity of at least one light spot can be detected by the interference light spot, and the terminal can determine the interference light spot from at least one light spot; in practical application scenarios, the terminal can usually determine an interference light spot from at least one light spot Light spot, based on the actual application scenario, at least one pixel point covered by the interference light spot is used as the interference light spot.

In one or more embodiments involved in the present application, by acquiring a reflection intensity map, at least one light spot in the reflection intensity map is determined, and then pixel feature parameters corresponding to the pixel points of the light spots in the light spot are obtained, and based on the pixel feature parameters, the interference The difference between the pixel characteristic parameters of the light spot and the effective light spot can accurately determine the interference light spot from at least one light spot, which can help to eliminate the light spot interference caused by the superposition of the edge signals of multiple light spots; The dependence of the algorithm realizes the accurate identification of light spot interference, can assist in the output of high-precision depth measurement results, and improve the robustness in depth measurement scenarios.

Please refer to FIG. 4 , which is a schematic flowchart of another embodiment of an image processing method proposed in the present application. specific:

S201: Obtain a reflection intensity map, and determine at least one light spot in the reflection intensity map;

For details, reference may be made to the method steps involved in one or more embodiments of the present application, which will not be repeated here.

S202: Acquire the number of light spots corresponding to the light spots; if the number of light spots is greater than a number threshold, adjust the exposure time of the camera.

According to some embodiments, after acquiring the reflection intensity map, the terminal may determine at least one light spot based on the intensity distribution information of at least one pixel point in the reflection intensity map.

Schematically, usually, the shape of the spot in the reflection intensity graph is a two-dimensional Gaussian shape, that is, the intensity of the spot is high at the center and gradually weakens around the spot; that is, at least one spot can be identified based on the spot shape in the reflection intensity graph. , and then calculate the number of all light spots to obtain the number of light spots corresponding to the light spots, that is, the number of light spots in the reflection intensity graph. It is understandable that there may be interfering light spots in the counted number of light spots.

In the specification of this application, the number threshold is a threshold value set for measuring the influence caused by the number of interfering light spots. If the number of detected light spots is greater than the number threshold, it can be considered that the number of interference light spots is huge, and the energy of these interference light spots is usually large, and the depth value of the interference light spots corresponding to the interference light spots in the depth map is also high.

It is understandable that, in practical application scenarios, the TOF camera mounted on the terminal may be a device that supports the speckle iTOF mode, such as the speckle iTOF camera; the TOF camera mounted on the terminal is usually used according to the actual scene ( Target detection distance, detection accuracy, target object reflectivity, whether it is used outdoors, etc.) are customized for software and hardware, and the software and hardware are adjusted, including Sensor selection, light source design (including light source optical power, luminous point arrangement), and lens design. and other hardware and software. In the terminal use stage after leaving the factory, the terminal works based on a speckle iTOF mode, and the theoretical number of light spots received by the light receiving end in a speckle iTOF mode and the theoretical number of pixels in the coverage area of the light spot have been determined. For example, after the speckle iTOF camera is in the actual application stage, how many spots will theoretically be in the field of view, and each spot will theoretically cover several pixels. . Optionally, the threshold for the number of light spots is generally greater than the theoretical number of light spots (ie, the actual number of light spots).

In a specific implementation scenario, if the number of detected light spots is greater than the number threshold, it can be considered that the number of interference light spots is huge, and the energy of these interference light spots is usually large, and the depth of the interference light spots corresponding to the interference light spots in the depth map value is also higher. At this time, a possible scenario is that the entire field of view of the TOF camera of the terminal is a close-range target object. Even the original non-spot area in the depth map or reflection intensity map can receive the light signal due to the close-range target object. Scattered light, and the interference light spots or interference light spots generated by each other are more in number, energy and intensity are higher. The terminal can adjust the camera exposure time. There is usually a camera module in the terminal, such as the automatic exposure (Auto Focus, AE) algorithm module. The terminal can send an indication signal to the AE (Auto Focus automatic exposure) algorithm module based on the number of light spots fed back by the reflection intensity map, assisting the AE (Auto Focus) algorithm module. Focus automatic exposure) algorithm module overexposure judgment process, reduce the exposure time when judging overexposure, in order to ensure the measurement of the depth and distance of the close target object. Thereby, a number of light spots in the reflection intensity map can be realized as an overexposure judgment condition, the automatic exposure process is optimized, and the practicability of a terminal such as an iTOF camera is further increased.

Optionally, after adjusting the exposure time of the camera, the terminal may perform the step of acquiring the reflection intensity map, and determining at least one light spot in the reflection intensity map, that is, after completing the exposure time of the camera, re-acquiring the reflection intensity. Intensity map to achieve interference optimization and optimize the optimization effect in practical application scenarios.

S203: Obtain pixel feature parameters corresponding to the pixel points of the light spot in the light spot;

For details, reference may be made to the method steps involved in one or more embodiments of the present application, which will not be repeated here.

S204: Determine at least one spot interference detection method for the reflection intensity map.

In a specific implementation scenario, the current application scenario can be obtained, and different application scenarios correspond to different combinations of light spot interference detection methods (the combination includes at least one light spot interference detection method). , interference adaptability and other requirements, different reference spot interference detection method combinations can be set in advance based on different reference application scenarios, that is, the interference detection combination mapping relationship of at least one reference application scenario and its corresponding reference spot interference detection method combination is established. In the application, after the current application scenario is determined, the combination of light spot interference detection methods in the current application scenario can be determined based on the aforementioned interference detection combination mapping relationship, and at least one light spot for the reflection intensity map can be determined based on the light spot interference detection method combination. Interference detection method.

Illustratively, the interference detection combination mapping relationship may be represented in the form of a mapping combination, a mapping list, a mapping array, etc., which is not specifically limited here.

Illustratively, different reference application scenarios may be application scenarios involving remote depth measurement, virtual reality applications, visual environment reconstruction, virtual reality games, plane detection, etc. involved in the terminal.

Optionally, the light spot interference detection method includes but is not limited to one or more detection methods such as pixel point intensity detection method, energy ratio detection method, pixel point average value detection method, total energy intensity detection method, and spot position detection method. species fit.

Optionally, when it is determined that there are at least two light spot interference detection methods in the current application scenario, the detection timing between at least one light spot interference detection method is not limited, and at least one light spot interference detection method can be executed synchronously or asynchronously. .

Optionally, when it is determined that the light spot interference detection method in the current application scenario is at least one, a corresponding interference light spot can be obtained based on each light spot interference detection method, and whether the interference light spot or the interference light spot corresponding to the interference light spot is removed. It can be removed in combination with the interference spot removal strategy set in the actual scene. For example, multiple interference light spots (such as the first interference light spot, the second interference light spot, the third interference light spot, the fourth interference light spot, the fifth interference light spot, etc.) can be obtained by combining a variety of light spot interference detection methods. Intersection, for example, based on the interference spot removal strategy set in the actual application scenario, a certain spot is simultaneously detected as belonging to "the first interference spot, the second interference spot, the third interference spot, the fourth interference spot, and the fifth interference spot", Then remove the certain spot.

S205: Perform a spot interference detection process on at least one of the light spots by using the light spot interference detection method based on the pixel characteristic parameters, to obtain an interference light spot.

In a feasible implementation manner, the light spot interference detection method may be a pixel point intensity detection method. By detecting the pixel point intensity of the light spot pixel points, the interference light spots are distinguished from the identified multiple light spots. In practical application scenarios, for example, the light receiving end of the speckle iTOF camera is used to obtain the reflection intensity map in the current field of view. The ratio is usually relatively small, such as about a few percent to ten percent. In the area not covered by the actual light spot, the value of the pixel in the reflection intensity map or the depth map or the pixel intensity value (that is, the Confidence value) is not 0. This is due to the existence of noise such as ambient light, circuits, and dark circuits. of. However, the pixel intensity value (that is, the Confidence value) of the area not covered by these actual light spots is usually very low (even if there is interference light spot, the pixel intensity value of the interference light spot is usually much lower than the pixel intensity value of the actual light spot), in some scenes Under these conditions, the interference spot or edge area is one to several orders of magnitude lower than the pixel intensity value (ie, the Confidence value) of the actual spot signal light. Based on this, the spot intensity parameters corresponding to the spot pixels covered by all the spots identified in the reflection intensity map can be detected, so as to determine the interference spot from the identified spots, that is, determine the disturbance spot.

It can be understood that after the terminal determines the light spot interference detection method, if the light spot interference detection method is the pixel intensity detection method, the terminal can perform pixel intensity detection processing on at least one light spot based on the pixel intensity parameter, so as to obtain the first interference. Light spot; the first interference light spot is the interference light spot detected by the pixel intensity detection method.

Optionally, the pixel intensity detection method may specifically be to draw an intensity statistics map, such as an intensity histogram representing the intensity of the pixel point, and the intensity histogram can be used to represent the pixel intensity of the pixel points of the light spot covered by all the light spots, for example, it can be fed back. For spot pixels with the same pixel intensity value, for example, the intensity priority of each spot pixel can be fed back.

It is understandable that in the terminal use stage after leaving the factory, the terminal works based on a speckle iTOF mode, and the theoretical number of light spots received by the light receiving end in a speckle iTOF mode and the theoretical number of pixels in the coverage area of the light spot have been determined. For example, after the speckle iTOF camera is in the actual application stage, how many spots will theoretically be in the field of view, and how many pixels will be covered by the theory of the spot; It can be determined, or, the target light spot ratio (the ratio of the theoretical light spot pixel points to the total image pixel points) of the theoretical light spot pixel points (excluding interference points) can be determined. Based on this, it is only necessary to determine the proportion of target light spots or the reference light spots indicated by the number of target light spots from all the light spot pixels during pixel intensity detection (in some embodiments, it may be referred to as the first light spot and the second light spot) , the light spot intensity parameter corresponding to the reference light spot can be used as the threshold to filter the interference points.

In a feasible implementation manner, the light spot interference detection method may be an energy ratio detection method. By detecting the energy ratio of the pixel points included in the light spot, the interference light spot can be distinguished from the identified multiple light spots. Illustratively, the energy ratio detection method can traverse at least one spot, and calculate the energy ratio of the pixel intensity parameter (that is, the Confidence value) of the pixel at the center of the spot and the Confidence value of other pixels of the spot except the center of the spot. The signal light energy of the spot is concentrated, and its energy ratio will be greater than that of the interference spot.

It can be understood that after the terminal determines the light spot interference detection method, if the light spot interference detection method is the energy proportional detection method, the terminal can perform energy proportional detection processing on at least one light spot based on the pixel intensity parameter, so as to obtain the second interference. Light spot; the second interference light spot is the interference light spot detected by the energy proportional detection method. Illustratively, it is usually possible to calculate the energy ratio of adjacent pairs of light spots, and then compare the energy ratio of each pair of light spots. Usually, if a certain light spot is an interference light spot, the actual energy ratio of the light spot is much larger than that of the adjacent interference light spots.

In a feasible implementation manner, the light spot interference detection method may be the pixel point average detection method, and the pixel point average detection method may be to detect "a certain light spot covers all pixels in the reflection intensity map. The pixel intensity parameter (such as the Confidence value) The ratio of the "sum" to "the sum of all pixel intensity parameters (such as the Confidence value) of all pixels within a certain neighborhood of the spot", to use the ratio as the pixel mean;

Optionally, the pixel point mean detection method can be "the sum of the pixel intensity parameters of the spot (such as the Confidence value) divided by the average value of the number of pixels covered" and "the spot pixel intensity parameter of all pixels within a certain neighborhood range of the spot. (such as the Confidence value) divided by the ratio of "the average value of the number of pixels covered" to use the ratio as the average value of the pixel points; generally, the smaller the ratio, the greater the possibility that the light spot is an interference light spot

It can be understood that, after the terminal determines the light spot interference detection method, if the light spot interference detection method is the pixel point average detection method, the terminal performs pixel point average detection processing on at least one light spot based on the pixel intensity parameter, so as to obtain information from multiple light spots. The third interference light spot is obtained from the light spot; for example, the pixel average value of the light spot is obtained after the pixel point average detection processing, and then the pixel average value is compared with the average threshold value, so as to determine the interference light spot from multiple light spots, that is, determine the interference light spot .

In a feasible implementation manner, the light spot interference detection method may be a total energy intensity detection method. Usually, in at least one small area corresponding to the reflection intensity map, the number of theoretical light spots in the small area can be preset in advance, and the The total energy of multiple actual light spots (excluding interference light spots) in a small area should be equal to each other. For example, the small area corresponds to a two-dimensional plane of the target object in the visual scene, and the total energy of the interference light spots is much less than The actual spot in this small area. Schematically, a small area may not be determined. Usually, the number of light spots in the reflected energy map is large, and a light spot usually only corresponds to the parameter value of one pixel in the depth map after processing, that is to say, you can set any value of n( n is usually a small value) adjacent spots are compared.

It can be understood that after the terminal determines the light spot interference detection method, if the light spot interference detection method is the total energy intensity detection method, the terminal performs total energy intensity detection processing on at least one light spot based on the pixel intensity parameter to obtain a fourth interference light spot. ; For example, after the total energy intensity detection processing, the total energy intensity of multiple light spots is obtained, and then the total energy intensity of at least one light spot is compared. When the total energy intensity of a certain light spot is small, the light spot indicated by the smaller total energy intensity can be used as For the interference light spot, at least one of all the pixel points covered by the interference light spot can be determined as the interference light spot, for example, the pixel point indicated by the maximum pixel intensity parameter covered by the interference light spot is used as the interference light spot.

In a feasible implementation manner, the speckle interference detection method may be a light spot position detection method, which can also be understood as a fixed position detection. The position on the intensity map is usually relatively fixed, but it may be offset by several pixels due to objective reasons such as distance and distance parallax, signal divergence, etc. Based on this, the position range of the light spot can be set at the factory, and the position range of at least one light spot can be set. Record, if the spot appears outside these positions, there is a high probability of disturbing the spot.

It can be understood that, after the terminal determines the light spot interference detection method, if the light spot interference detection method is the light spot position detection method, the terminal can perform light spot position detection processing on at least one light spot based on the pixel position parameter to obtain a fifth interference light spot. For example, in the light spot position detection process, the light spot positions of multiple light spots are obtained, and the light spot positions of the light spots are matched with their corresponding position ranges, so that the interference light spots can be determined from the plurality of light spots.

In some embodiments, after determining the interference light spot, the terminal performs interference cancellation processing on the interference light spot. Illustratively, after determining the interference light spot based on the Confidence information in the reflection intensity map, the terminal may remove at least one pixel point included in the interference light spot in the reflection intensity map, such as removing the entire interference light spot. Illustratively, after determining the interference light spot, the terminal may determine the interference position of the interference light spot in the depth map based on the position of the interference light spot, so as to eliminate the interference light spot on the depth map.

In a feasible implementation scenario, after the interference light spot is determined, the interference light spot can also be verified by combining a color image such as RGB and a depth image to determine whether the interference light spot is a correct interference spot.

It can be understood that the terminal can obtain a color image and a depth image, and the reflection intensity map, the depth image, and the color image are different types of images for the same target object; that is, the terminal is targeting the target in the current scene. When imaging an object, the terminal may simultaneously capture the reflection intensity map, the depth map and the color image for the same target object, and the color image may usually be an RGB image. After identifying the interfering light spot, the captured color image is then acquired.

It can be understood that the terminal can determine the first position of the interference light spot in the depth image, and can determine the second position of at least one adjacent light spot corresponding to the interference light spot in the depth image; based on the first position and the The second position is used to obtain a depth pixel verification result for the interference light spot.

It is understandable that the terminal then determines the third position of the interference light spot in the color image, and determines the fourth position of at least one adjacent light spot corresponding to the interference light spot in the color image; position and the fourth position, obtain the color pixel verification result for the interference light spot;

Illustratively, in the reflection intensity map, at least one light spot usually covers multiple light spot pixels, while in the depth image and color image, it usually corresponds to only one pixel;

Illustratively, a light spot may cover multiple pixels, but the light spot is only calculated at a certain pixel position of the light spot (such as the pixel point of the center of gravity of the light spot) to generate a depth value, which represents the proximity of the light spot in the depth image. The depth distance value of a small area; for example, it can be generated by combining the original information of all pixels covered by the light spot in the original image (RAW image, the original image output by the TOF camera), and taking the pixel point corresponding to the center of gravity of the light spot. An effective depth value. The effective depth value calculates a depth value based on the original information that the spot covers all pixels.

Schematically, usually after the interference spot is determined, its position in the depth image, that is, the first position, that is, the position of the interference spot corresponding to the effective depth value is taken as the first position (usually the largest pixel of the interference spot). The pixels indicated by the intensity correspond to the positions in the depth image).

Further, depth images are usually captured by the TOF camera of the device, and color images such as RGB images are usually captured by the RGB camera of the device. By calibrating the RGB camera and the TOF camera, the coordinate transformation between the depth image and the color image can be determined. relationship (which can exist in the form of image coordinate transformation), after the terminal determines the interference light spot, obtains the interference light spot corresponding to the interference light spot in the depth image, and takes the position of the interference light spot in the depth image as the first position; then; Based on the first position in the aforementioned depth image, determine the third position of the interference light spot in the color image according to the coordinate transformation relationship; in the same way, obtain at least one adjacent light spot corresponding to the interference light spot, and then obtain the depth of the at least one adjacent light spot by referring to the foregoing method. For the corresponding adjacent light spot in the image, the position of the adjacent light spot in the depth image is taken as the second position; then based on the second position, the fourth position of the adjacent light spot in the color image is determined according to the coordinate transformation relationship; When the number of adjacent light spots corresponding to the interference light spot is multiple, the second position of at least one adjacent light spot in the depth image and the fourth position in the color image are acquired in the aforementioned manner.

Optionally, the positional relationship between at least one adjacent light spot and the interference light spot is an adjacent relationship, and the number of adjacent light spots may be a preset number. The second position of the adjacent pixel point corresponding to the adjacent light spot can be obtained in the depth image, and based on the second position of the adjacent pixel point, the fourth position of the adjacent light spot in the color image is determined according to the transformation relationship of the image coordinate system;

In practical applications: color images such as RGB images have higher resolution and can provide color information; depth images can provide depth information. If it is determined as an interference light spot in the preceding steps, the interference verification can be performed on the interference light spot. In the specific implementation, the first position of the interference light spot corresponding to the interference light spot can be found in the depth image, and the second position of the adjacent light spot corresponding to the adjacent light spot can be found in the depth image, and then the interference light spot can be found on the RGB image. The third position corresponding to the interference light spot and the fourth position of the adjacent light spot corresponding to the interference light spot in the adjacent light spot of the RGB image;

Illustratively, the depth value indicated by the interference light spot at the first position is quite different from the depth value indicated by other adjacent light spots at the second position, such as "depth value of the first position" and "depth of the second position". The depth difference between the "value" is greater than a certain depth difference threshold, and in the color information given by the RGB image, the color information corresponding to the "interference light spots and adjacent light spots" of the interference spots in the RGB image is similar. , such as the color pixel difference (such as RGB difference) between "color pixel value (such as RGB value) at the third position" and "color pixel value (such as RGB value) at the third position" is less than a certain color difference value Threshold value (such as the threshold value or critical value set for RGB value); indicating that these points may be the same surface of the object, the probability of correct judgment of the interference light spot is high; otherwise, the interference light spot corresponding to the interference light spot may come from For long-distance or other low-reflectivity targets or targets with smooth specular reflection, it is possible that the interference spot is not an interference spot type but an effective spot type.

In a specific implementation, based on the first position and the second position, obtaining the depth pixel verification result for the interference light spot may be: obtaining a first depth value corresponding to the first position and obtaining a second depth value corresponding to the second position Depth value, calculate the depth value difference between the first depth value and the second depth value, if the depth value difference is greater than the depth threshold, the depth pixel verification result is the first result, such as True, otherwise it is the second result, such as False;

Further, based on the third position and the fourth position, obtaining the color pixel verification result for the interference light spot may be: obtaining the first color value (such as RGB value) corresponding to the third position and obtaining the fourth color value. The second color value corresponding to the position, calculate the color value difference between the first color value and the second color value, if the color value difference is less than the color threshold, the color information of the interference light spot and the adjacent light spot is usually similar, and the color pixel verification The result is the first result, such as True; otherwise, the color pixel verification result is the second result, such as False;

Then combined with the depth pixel verification results and the color pixel verification results, the interference verification results for the interference light spots are determined. Illustratively, if both the depth pixel verification result and the color pixel verification result are the first results, such as True, then the interference verification result for the interference light spot is: the interference verification is passed, otherwise, the interference verification for the interference light spot is The result is: the interference verification fails.

In the embodiment of the present application, by acquiring the reflection intensity map, at least one light spot in the reflection intensity map is determined, and then the pixel feature parameters corresponding to the pixel points of the light spot in the light spot are obtained, and the pixels on the interference light spot and the effective light spot are based on the pixel feature parameters. The difference in characteristic parameters can accurately determine the interference spot from at least one spot, and can assist in eliminating the spot interference caused by the superposition of the edge signals of multiple spots; Accurately identify light spot interference, which can assist in outputting high-precision depth measurement results and improve the robustness in depth measurement scenarios; and can combine depth map and color map for interference light spot verification, further improving the accuracy of interference light spot identification ; Combined with different detection methods or different timing detection methods determined based on actual application scenarios, the detection process of interference light spots is optimized.

Please refer to FIG. 5 , which is a schematic flowchart of another embodiment of an image processing method proposed in the present application. specific:

S301: Obtain a reflection intensity map, determine at least one light spot in the reflection intensity map, and obtain pixel feature parameters corresponding to the pixel points of the light spot in the light spot;

S302: Determine at least one light spot interference detection method for the reflection intensity map.

For details, reference may be made to the method steps involved in one or more embodiments of the present application, which will not be repeated here.

S303: If the light spot interference detection method is a pixel point intensity detection method, determine the first intensity priority corresponding to the light spot pixel point based on the pixel intensity parameter, and obtain the target light spot ratio for the reflection intensity map;

According to some embodiments, the light spot interference detection method may be a pixel point intensity detection method, by detecting the pixel point intensity of the light spot pixel points, so as to distinguish the interference light spots from the identified multiple light spots. In practical application scenarios, for example, the light receiving end of the speckle iTOF camera is used to obtain the reflection intensity map in the current field of view. The ratio is usually relatively small, such as about a few percent to ten percent. In the area not covered by the actual light spot, the value of the pixel in the reflection intensity map or the depth map or the pixel intensity value (that is, the Confidence value) is not 0. This is due to the existence of noise such as ambient light, circuits, and dark circuits. of. However, the pixel intensity values of the areas not covered by these actual light spots (which can also be understood as effective light spots other than the interference light spots) (that is, the Confidence value of the pixels in the uncovered areas) are usually very low (even if there are interference light spots). In some cases, the pixel intensity value in the interference spot is usually much lower than the pixel intensity value in the actual spot). In some scenarios, the pixel intensity value of the interference spot or edge area is one lower than the pixel intensity value of the actual spot signal light (that is, the Confidence value). to several orders of magnitude. Based on this, the spot intensity parameters (such as the Confidence value corresponding to the spot pixels) corresponding to the spot pixels covered by all the spots identified in the reflection intensity map can be detected, so as to determine the interference from the identified multiple spots. Light spot, that is, to determine the interference light spot.

It can be understood that after the terminal determines the light spot interference detection method, if the light spot interference detection method is the pixel intensity detection method, the terminal can perform pixel intensity detection processing on at least one light spot based on the pixel intensity parameter. In the reflection intensity map (also called the Confidence map), the pixel intensity parameters of the pixel points of the light spot covered by the light spot (such as the Confidence value of the pixel points of the light spot covered by the light spot), according to the pixel points covered by all the light spots (the number of pixels of the light spot covered by the light spot) The number of pixel intensity parameters can be more than one) to prioritize the value of the pixel intensity parameters, and determine the first intensity priority corresponding to at least one spot pixel point (that is, the spot pixel point covered by the spot). An intensity priority is positively correlated, and the ratio of target light spots obtained for the reflection intensity map.

Optionally, determining the first intensity priority corresponding to at least one light spot pixel point (that is, the pixel point covered by the light spot) may be determined by drawing an intensity statistic graph, for example, the intensity statistic graph may represent the pixel intensity of the light spot pixel point. The intensity histogram of the parameter (confidence value corresponding to the spot pixel), the intensity histogram can be used to represent the pixel intensity of the spot pixel covered by all the spots, for example, the spot with the same pixel intensity parameter (such as the same Confidence value) can be fed back For pixel points, for example, the intensity priority of at least one light spot pixel point can be fed back.

It can be understood that in the terminal use stage after leaving the factory, the terminal works based on a speckle iTOF mode. The theoretical number of light spots received by the light receiving end in a speckle iTOF mode, the theoretical number of light spot pixels covered by the light spot, etc. have been determined. . For example, after the actual application stage of a speckle iTOF camera, how many spots will theoretically be in the field of view, and at least one spot will theoretically cover several spot pixels; further, due to the theoretical number of spot pixels covered by the spot (in some implementations It can also be referred to as the number of reference pixels in the method) can be determined, or, the target light spot ratio (the ratio of theoretical light spot pixels to total image pixels) of theoretical light spot pixels (excluding interference points) can be determined. Based on this, the pixel intensity detection only needs to determine the reference light spot (which may be referred to as the first light spot or the second light spot in some embodiments) from all the light spot pixels, and use the pixel intensity parameter corresponding to the reference light spot as the threshold Perform interference screening.

Schematically, the histogram statistics is to count the histogram of the entire Confidence image. According to prior knowledge, after the speckle iTOF camera is designed, how many light spots will be in the field of view, and at least one light spot will cover several pixels. can be derived from theory.

S304: Determine an intensity threshold based on the target light spot ratio, the first intensity priority of the light spot pixel points, and the pixel intensity parameter of the light spot pixel point; based on the intensity threshold, select from at least one of the light spots Determine the first interference spot.

In a feasible implementation manner, the first light spot may be determined based on the proportion of the target light spot and the first intensity priority of the light spot pixel point, and the first intensity threshold corresponding to the first light spot may be obtained. the first intensity threshold determines a first interference spot from at least one of the spots;

Illustratively, it is assumed that the proportion of target light spots (that is, the proportion of the number of light spot pixels) is x%, and the first intensity priority of the light spot pixels can be represented in the form of a histogram corresponding to the Confidence graph, then the terminal can Determine the number of points from the top to the bottom of the histogram to determine the first light spot (the number of points in the x% can be used as the first light spot), and the pixel intensity parameter of the first light spot is used as the first intensity threshold, For example, the Confidence value corresponding to the first light spot is Cx, that is, theoretically, the Confidence value of the light spot pixel points covered by all the light spots should be greater than Cx. That is, Cx can be used as the threshold to filter the interference light spots, that is, the Confidence value of all light spot pixels can be compared with Cx. If the Confidence value of the light spot pixel point is greater than Cx, the light spot pixel point is a non-interfering light spot. If If the Confidence value of the light spot pixel is less than or equal to Cx, then the light spot pixel can be used as an interference light spot, or the light spot pixel can be used as a suspected interference light spot, and then further determined by combining other light spot interference detection methods, such as using other light spot interference methods It is determined that the light spot pixel point meets the interference point judgment condition, and the light spot pixel point is regarded as the interference light spot. After the interference light spot is screened out, the first interference light spot is determined. For example, the light spot corresponding to the interference light spot can be used as the first interference light spot. In the actual light spot interference detection, usually most of the light spot pixels covered by the first interference light spot are used. All may be interfering light spots.

In a feasible implementation manner, the number of reference pixels corresponding to the light spot is obtained, and the number of reference pixels is the theoretical number of light spot pixels covered by the light spot, and the proportion of the target light spot and the reference pixel are determined. The target ratio corresponding to the number, the second light spot is determined based on the target ratio and the first intensity priority of the light spot pixel point, and the second intensity threshold corresponding to the second light spot is obtained, based on the second intensity threshold from at least determining a first interference light spot in one of the light spots;

Schematically, in practical application scenarios, the Confidence value (which can be understood as the pixel intensity of the pixel) of the pixel in the area where the edge of the actual light spot meets is usually greater than the Confidence value of the pixel such as ambient light noise or circuit noise; and in general scenarios , there may be long-distance or low-reflectivity objects that exceed the design range. At this time, the signal light reflected by these long-distance objects is displayed on the light-receiving end Sensor with very weak spot energy, that is, the pixel intensity parameter The value is low , may be lower than the Confidence value of the pixels in the interference spot at the intersection of the edge of the near spot The set threshold, such as the number of pixels below the target light spot ratio. In order to improve the recognition accuracy of light spot interference. Interference spot detection can be performed by combining the number of reference pixel points corresponding to the spot.

The number of reference pixel points corresponding to the light spot can be understood as the number of light spot pixel points theoretically covered by at least one light spot. It can be understood that, for example, in the terminal use stage after delivery, the terminal works based on a speckle iTOF mode. The theoretical number of pixels in the coverage area (that is, the number of reference pixels) has been determined or can be calculated.

It can be understood that the second intensity threshold can be determined based on the combination of the number of reference pixels corresponding to at least one light spot and the proportion of the target light spot, so as to improve the recognition accuracy of light spot interference.

Illustratively, if a spot covers n spot pixels (that is, the number of reference pixels is n), taking the aforementioned histogram as an example, the first intensity priority of the spot pixels may be the histogram corresponding to the Confidence graph. The terminal can determine the number of points from the top to the bottom (x/n)% of the histogram, so as to determine the second light spot (the (x/n)% number of points can be used as the second light spot ), the pixel intensity parameter of the second light spot is used as the second intensity threshold, for example, the Confidence value corresponding to the second light spot is used as the second intensity threshold Cxna. Further, theoretically, the Confidence value of all the pixels covered by the light spot should be greater than Cxna. That is, Cxna can be used as the threshold to filter the interference light spots. If the Confidence value of the light spot pixel point is greater than Cxna, the light spot pixel point is not an interfering light spot. If the Confidence value of the light spot pixel point is less than or equal to Cxna, then the light spot pixel point can be used as interference Light spots. The first interference light spot may be determined according to the light spot corresponding to the interference light spot.

In a feasible implementation manner, the first intensity threshold and the second intensity threshold can also be used for balance, so as to avoid errors in light spot interference judgment, achieve accuracy in interference judgment, and avoid missing interference light spots. Illustratively, from the first intensity threshold and the second intensity threshold, that is, Cx to Cxna, the Confidence threshold increases from small to large, corresponding to fewer and fewer interference points, and can be calculated from the first intensity threshold and the second intensity. The threshold value is combined with the actual application scenario to determine an appropriate threshold value to judge the spot interference.

It can be understood that the terminal can perform determining the first light spot based on the proportion of the target light spot and the first intensity priority of the light spot pixel point and obtain the first intensity threshold corresponding to the first light spot; based on the target light The spot ratio and the first intensity priority of the light spot pixels determine the first light spot and obtain the first intensity threshold corresponding to the first light spot, obtain the number of reference pixels corresponding to the light spot, and determine the target light spot accounting for a target ratio corresponding to the number of reference pixels, determining a second light spot based on the target ratio and the first intensity priority of the light spot pixels, and obtaining a second intensity threshold corresponding to the second light spot, based on The first intensity threshold and the second intensity threshold determine a first interfering light spot from at least one of the light spots.

It can be understood that from the first intensity threshold and the second intensity threshold, that is, from Cx to Cxna, the Confidence threshold increases from small to large, corresponding to fewer and fewer interference points. From the first intensity threshold and the second intensity threshold A threshold is combined to determine a threshold for interference filtering; in a specific implementation, the terminal determines a threshold reference range based on the first intensity threshold and the second intensity threshold. For example, the threshold reference range is: [Cx, Cxna], and then from the A target intensity threshold is obtained from the threshold reference range, and a first interference light spot is determined from at least one of the light spots based on the target intensity threshold.

Optionally, the target intensity threshold may be any value randomly selected from the threshold reference range as the target intensity threshold.

Optionally, a threshold may also be determined from the first intensity threshold and the second intensity threshold in combination with the current practical application scenario to perform interference filtering.

Illustratively, in different application scenarios for different requirements such as measurement accuracy and interference adaptability, an adjustment factor can be set, and a reference adjustment factor is set separately for multiple reference application scenarios, and the reference factor is used to determine the target intensity from the threshold reference range. threshold. In the actual application stage, the adjustment factor mapping relationship between multiple reference application scenarios and their corresponding reference adjustment factors is established. After the current application scenario is determined, the adjustment factor corresponding to the application scenario can be determined based on the adjustment factor mapping relationship. Based on the adjustment factor, a preset calculation formula is used to determine the target intensity threshold from the threshold reference range. For example, the threshold reference range is: [Cx, Cxna], assuming that the adjustment factor is a, then the target intensity threshold b=(Cxna-Cx)*a +Cx;

S305: If the light spot interference detection method is an energy proportional detection method, obtain the intensity value of the center point and the edge point intensity value corresponding to at least one of the light spots;

According to some embodiments, the energy ratio detection method detects the energy ratio of the pixel points included in the light spot, so as to distinguish the interference light spot from the identified multiple light spots. Illustratively, the energy ratio detection method can traverse at least one spot, and calculate the energy ratio of the pixel intensity parameter (that is, the Confidence value) of the pixel at the center of the spot and the Confidence value of other pixels of the spot except the center of the spot. The signal light energy of the spot is concentrated, and its energy ratio will be greater than that of the interference spot.

The center point intensity value can be understood as the pixel intensity value (equivalent to the Confidence value) of the center pixel point of the light spot. The pixel intensity value is taken as the center point intensity value. The edge point intensity value can be understood as the pixel intensity value (that is, the Confidence value) of the edge pixel points in the light spot except the center pixel point. Both the center point intensity value and the edge point intensity value can be directly obtained based on the reflection intensity map.

Schematically, it is assumed that the light spot covers m light spot pixels. In theory, from the center of the light spot to the edge, the intensity of the light spot pixel point gradually decays, that is, the corresponding Confidence value gradually decays, and the attenuation law basically conforms to the Gaussian surface. The area covered by the light spot The Confidence value of a certain row or column of spot pixels conforms to a Gaussian curve. As shown in Figure 6, Figure 6 is a schematic diagram of a scene involving light spots. In Figure 6, the relationship between the Confidence value of a certain column of light spot pixel points and the position of the light spot pixel points is shown in Figure 6. In Figure 6, the light spots on both sides are visible As an actual light spot or an effective signal light spot, the middle light spot is generated by the superposition of the edge signal light of the two light spots. The effective signal light spot is shown in (a) in Figure 6. The energy intensity of the effective signal light spot is concentrated, the central pixel Confidence value is high, and the effective signal light spot is The surrounding spot energy rapidly decays. That is to say, the ratio of the Confidence value of the central pixel point of the effective light spot to the Confidence value of the surrounding area of the light spot is larger. The interference signal spot is shown in (b) in Figure 6. The distribution of the interference signal spot also presents a Gaussian surface, but the energy of the interference signal spot is dispersed, and the ratio of the central pixel Confidence value of the interference signal spot to the edge Confidence value is much smaller than the adjacent two effective The ratio corresponding to the spot.

S306: Perform energy proportional detection processing on at least one light spot based on the central point intensity value and the edge point intensity value, and determine a second interference light spot from at least one of the light spots.

According to some embodiments, the terminal may perform energy proportional detection processing on at least one light spot based on the pixel intensity parameter, so as to obtain a second interference light spot; the second interference light spot is the interference light spot detected by the energy proportional detection method . Illustratively, it is usually possible to calculate the energy ratio of adjacent pairs of light spots, and then compare the energy ratio of each pair of light spots. Usually, if a certain light spot is an interference light spot, the actual energy ratio of the light spot is much larger than that of the adjacent interference light spots.

It can be understood that 1. The terminal can determine the first intensity ratio of the light spot based on the intensity value of the center point and the intensity value of the edge point;

The first intensity ratio is the ratio of the intensity value of the center point of the light spot to the intensity value of the edge point.

Schematically, the edge distance can be set, and the center point of the light spot is taken as a reference, and the edge point of the same light spot indicated by the edge distance is taken. Pixel points are used as edge points.

2. The terminal obtains the first light spot corresponding to the light spot respectively, and determines the second intensity ratio of the first light spot;

Illustratively, adjacent light spots adjacent to the light spot may be acquired, and the adjacent light spot may be used as the first light spot corresponding to the light spot; considering that the ratio of the adjacent light spots is comparable. Proximity can be understood as the fact that the light spot is very close to its corresponding first light spot. In a real scene, the two light spots may come from adjacent positions on the surface of the same object, that is, the energy of the two light spots and the calculated distance are similar. That is, the two spots are comparable. If the distance between the light spot and its corresponding first light spot is far, it is possible that one of the light spots comes from the surface of the object at a close distance, and the other comes from the background at a distance. The reflected light from the background at a distance itself, due to its long optical path, hits the TOF camera and receives light. The light spot on the end Sensor may have weak or inconcentrated energy, so the comparability between the light spots with a longer distance is poor. Based on this, adjacent light spots adjacent to the light spots may be acquired, and the adjacent light spots may be used as the first light spots corresponding to the light spots.

Illustratively, a fixed distance may also be set, and a light spot within a fixed distance from at least one of the light spots is acquired as the first light spot.

Illustratively, a distance threshold may be set, and the terminal may obtain the target distance between the light spot and at least one second light spot (the second light spot is the light spot other than the current light spot), based on the target distance and the distance threshold. The first light spot corresponding to the light spot is determined from the at least one second light spot, that is, the first light spot corresponding to when the target distance between the second light spot and the light spot is less than or equal to the distance threshold is taken.

It can be understood that the determination method of the second intensity ratio is similar to the determination method of the first intensity ratio, that is, the ratio of the intensity value of the center point to the intensity value of the edge point of the first light spot. Schematically, the "center point and edge point selection" of the first light spot should be consistent with the "center point and edge point selection" of the light spot, for example, the edge point corresponding to the center point of the light spot and the center of the first light spot are selected based on the same edge distance. point corresponding to the edge point.

3. The terminal determines a second interference light spot from at least one of the light spots based on the first intensity ratio and the second intensity ratio of the light spot.

In a possible implementation manner, the target difference value may be determined based on the first intensity ratio and the second intensity ratio; 2. Interference spot;

Illustratively, the target difference is the difference between the first intensity ratio and the second intensity ratio. The difference threshold is a threshold value determined for the target difference based on the hardware requirements and application scenarios in advance. If the target difference is greater than the difference threshold, it can be determined that the first light spot is an interference light spot. If the target difference value is less than or equal to the difference value threshold, it is usually possible that the first light spot is not an interference light spot.

In a feasible implementation manner, the first intensity ratio may be used as a reference, and an intensity ratio fluctuation range for the first light spot may be determined based on at least one of the second intensity ratios; based on the intensity ratio fluctuation range and the reference fluctuation range, a second disturbing light spot is determined from at least one of the light spots.

For example, if there are adjacent k×k first light spots in a certain small area including the current light spot, the fluctuation range of the ratio of the central pixel Confidence value to the edge Confidence value of these k×k first light spots (that is, a plurality of second intensities The intensity ratio fluctuation range corresponding to the ratio and the current first intensity ratio) should not exceed the reference fluctuation range. The reference fluctuation range can be preset. If it deviates from the reference fluctuation range, the light spot is identified as the second interference light spot.

In one or more embodiments, the first intensity ratio of the light spot may be determined based on the intensity value of the center point and the intensity value of the edge point; a threshold value of the intensity ratio, that is, the intensity ratio threshold; then, based on the first intensity ratio and the intensity ratio threshold, a second interference spot is determined from at least one of the spots, for example, if the first intensity ratio is greater than the intensity If the first intensity ratio is less than or equal to the intensity ratio threshold, the light spot is an interference light spot.

S307: If the light spot interference detection method is a pixel point mean detection method, determine at least one target pixel area corresponding to the light spot, and the light spot is located in the target pixel area;

According to some embodiments, the light spot interference detection method may be a pixel point average detection method, and the pixel point average detection method may be to detect "a certain light spot covers the pixel intensity parameters (such as the Confidence value) of all the light spot pixels in the reflection intensity map. The ratio of "sum" to "the sum of the pixel intensity parameters (such as the Confidence value) of all pixels in a certain neighborhood of the spot", to use the ratio as the pixel mean;

Wherein, a certain neighborhood range of the light spot is the aforementioned target pixel area, and the target pixel area may be a rectangular area, and the rectangular area only needs to include the pixel points covered by one or more light spots. The target pixel area can be a circular area. The target pixel area can be all pixels that are larger than a certain ratio of the maximum intensity in a rectangular area. For example, the maximum value of Confidence in a rectangular area is 100, and a certain ratio is set to a (for example, a is 40%), then It is determined that all pixels with a Confidence value greater than (a*100) in a certain rectangular area meet the requirements.

S308: Determine, based on the pixel intensity parameter, a pixel mean value of a light spot for at least one of the light spots, and determine a regional pixel mean value corresponding to at least one of the target pixel regions;

Schematically, the first total intensity corresponding to all the light spot pixels of at least one of the light spots and the total number of light spot pixels can be obtained, and the quotient of the first total intensity and the total number of light spot pixels can be used as the light spot of the light spot. pixel mean;

The first total intensity is the sum of the pixel intensity parameters (Confidenc values) of all the pixel points of the light spot covered by one light spot. The total number of light spot pixel points is the total number of all light spot pixel points of the light spot. Assuming that the light spot a covers 6 light spot pixels, the first total intensity is the sum of the Confidenc values of the 6 light spot pixels, and the total number of light spot pixels is 6.

Illustratively, the second total intensity and the total number of regional pixels corresponding to all regional pixels of at least one of the target pixel regions can be obtained, and the quotient of the second total intensity and the total number of regional pixels is used as the target. The mean value of the area pixels corresponding to the pixel area.

The second total intensity is also the sum of the pixel intensity parameters of all area pixels of the target pixel area. The total number of area pixels is the number of all light spot pixels in the area. Assuming that the target pixel area b covers 8 area pixels, the second total intensity is the sum of the Confidenc values of the 8 area pixels, and the total number of area pixels is 8.

S309: Perform pixel point average detection processing on at least one light spot based on the light spot pixel average value and the regional pixel average value, and determine a third interference light spot from at least one of the light spots.

Optionally, the pixel point mean detection method may be: for a certain light spot, calculate "the sum of the pixel intensity parameters of the light spot (such as the Confidence value) (that is, the first total intensity) divided by the number of pixels covered (that is, the light spot). The average value C ₁ ” of the total number of pixel points), the average value C ₁ can be used as the average value of the light spot pixel points; and the calculation of “the pixel intensity parameter of all the pixel points in the target pixel area (that is, the second total intensity) divided by the coverage to the pixel The average value C ₂ ” of the number of points (that is, the total number of pixel points in the area), the average value C ₂ can be used as the average value of the area pixel points; the ratio of the average value C ₁ to the average value C ₂ (that is, C ₁ /C ₂ ), usually The smaller the ratio, the greater the possibility that a certain light spot is an interference light spot. Among them, the ratio (ie C ₁ /C ₂ ) can be referred to as a reference ratio.

It can be understood that, if the light spot interference detection method is a pixel point mean detection method, the terminal performs pixel point average detection processing on at least one light spot based on the pixel intensity parameter, so as to obtain a third interference light spot from multiple light spots; For example, after the pixel point mean detection process, the ratio for at least one light spot (ie C ₁ /C ₂ ) is obtained, and then the ratio (ie C ₁ /C ₂ ) is compared with a certain ratio threshold, if the ratio (ie C 1 /C 2 ) is compared with a certain ratio threshold ₁ /C ₂ ) is less than a certain ratio threshold, then the light spot is regarded as an interference light spot. On the contrary, the light spot is usually an effective light spot; in the case of multiple light spots, you can refer to the above method to compare the plurality of light spots in turn to get The interfering light spot is determined from the plurality of light spots, and the interfering light spot can be determined based on the interfering light spot.

The ratio threshold is a threshold or critical value set for the reference ratio (ie, C ₁ /C ₂ ).

Illustratively, the reference ratio corresponding to the pixel point mean value of the light spot and the regional pixel point average value is determined; the terminal may perform pixel point average value detection processing on at least one light spot based on the reference ratio value and the ratio threshold value, and obtain the pixel value from at least one of the light spots. Determine the third interference spot.

It can be understood that the basis of the pixel point mean detection method is that the light energy of the effective spot is strong and concentrated, and the energy and confidence decay rapidly from the center of the spot to the edge. For the effective light spot, the ratio of C ₁ /C ₂ , that is, the reference ratio is much larger than the ratio corresponding to the interference point. This is because the center of the light spot of the interference point itself is generated by the superposition of at least two effective light spot edges, and the energy is very weak. Illustratively, by setting a threshold value for the reference ratio, the judgment of the interference light spot can be realized.

S310: If the light spot interference detection method is the total energy intensity detection method, then, based on the pixel intensity parameter, determine the light spot intensity of the third light spot in the reference area and the average value of the light spot intensity corresponding to all the fourth light spots;

According to some embodiments, the light spot interference detection method may be a total energy intensity detection method, usually within at least one reference area corresponding to the reflection intensity map (the reference area may usually be a small area range of a preset size specification), in the reference area The theoretical number of regional light spots can be pre-set or calculated and derived, and the total energy (also understood as the light spot energy) of multiple actual light spots (excluding interference light spots) in the reference area should be equivalent to each other. If the reference area corresponds to A certain two-dimensional plane of the target object in the visual scene, and the spot energy of the interference spot generated at this time is usually much smaller than the spot energy of the actual spot in the reference area.

The light spot energy can be understood as the sum of the pixel intensity parameters of the light spot covering all the light spot pixel points, that is, the sum of the Confidence values of the light spot covering all the light spot pixel points. In some embodiments, the light spot energy can also be called the light spot intensity or Spot energy intensity.

The reference area can generally be understood as a small area range with a preset size specification; illustratively: the reference area may be an area with r×c pixels, which can be understood as r×c pixels as the preset size specification .

In a specific implementation scenario, it is assumed that the reference area should theoretically contain 2×2 light spots.

A feasible implementation manner may be: the third light spot may be any light spot among all the light spots in the reference area; the third light spot may also be the light spot with the smallest light spot intensity among all the light spots in the reference area;

Optionally, the relationship between the third light spot and the fourth light spot may be: all the light spots included in the reference area are the fourth light spot and the third light spot is one of all the fourth light spots;

For example, suppose there are 4 spots in the reference area: spot A1, spot A2, spot A3, spot A4; the fourth spot is spot A1, spot A2, spot A3, spot A4: the third spot is also called spot A1, spot A2, spot A3, spot A4. A2, one of the light spot A3, the light spot A4", for example, the third light spot can be the light spot with the smallest light spot intensity among the "light spot A1, light spot A2, light spot A3, light spot A4", assuming: the light spot intensity of light spot A1 is: 45, The light spot intensity of light spot A2 is: 99, the light spot intensity of light spot A3 is: 100, and the light spot intensity of light spot A4 is: 101, then light spot A1 in the reference area can be used as the third light spot.

Optionally, the relationship between the third light spot and the fourth light spot may be: the reference area includes the third light spot and all fourth light spots

For example, suppose there are 4 light spots in the reference area: light spot B1, light spot B2, light spot B3, light spot B4; the third light spot can be one of "light spot B1, light spot B2, light spot B3, light spot B4"; the fourth light spot is also That is, the light spots except the third light spot in "light spot B1, light spot B2, light spot B3, light spot B4": for example, the third light spot can be the light spot B3 in "light spot B1, light spot B2, light spot B3, light spot B4", then the third light spot The four light spots are: "light spot B1, light spot B2 light spot B4" except for light spot B3; schematically, it is assumed that the light spot intensity of light spot B1 is: 45, the light spot intensity of light spot B2 is: 99, and the light spot intensity of light spot B3 is: : 100, the light spot intensity of the light spot B4 is: 101, then the light spot B1 with the smallest light spot intensity in the reference area can be used as the third light spot.

The light spot intensity is the sum of the pixel intensity parameters (Confidence values) of all the light spot pixel points covered by a light spot; for example: the light spot has 3 light spot pixel points, and the Confidence value of at least one light spot pixel point is the light spot pixel point 1:x _1. Light spot pixel point 2: x ₂ , light spot pixel point 1: x ₃ ; then the light spot intensity is: (x ₁₊ x ₂₊ x ₃ )/3.

The "average light spot intensity" in the average light spot intensity corresponding to all the fourth light spots can be understood as: the sum of the light spot intensities of all the fourth light spots (that is, the sum of the Confidence values of all the fourth light spots) and the number of the fourth light spots. ratio.

The mean value of the light spot intensity corresponding to all the fourth light spots is the ratio of the sum of the light spot intensities of all the fourth light spots to the total number of the fourth light spots. For example: the fourth spot has 4 spots, namely spot A1, spot A2, spot A3, spot A4. Suppose: spot intensity of spot A1 is: 45, spot intensity of spot A2 is: 99, spot intensity of spot A3 is: 100 , the light spot intensity of light spot A4 is: 101; then the total number of fourth light spots is 4, and the average light spot intensity corresponding to all fourth light spots is: (45+99+100+101)/4.

In a feasible implementation manner, the total energy intensity detection method can also be a non-maximum value suppression detection method. According to some embodiments, such as the speckle iTOF camera in the actual application stage, how many theoretically there will be a certain reference area in the field of view At least one effective spot will theoretically cover several pixels, and the theoretical number threshold of the corresponding spot is usually determined; non-maximum suppression detection can be performed based on the actual number of spots in the reference area: all spots in the reference area correspond to When the total number of light spots is less than or equal to the threshold of the number of light spots corresponding to the reference area (in some embodiments, the threshold of the number of light spots can be determined based on the number of theoretical light spots), there is no need to perform total energy intensity detection. The light spots are usually effective light spots; when the actual number of light spots is greater than the theoretical light spot number corresponding to the reference area, it can usually be considered that there are interference light spots.

Illustratively, the reference area can be an area of r×c pixels, which can be understood as the preset size specification of r×c pixels. It is assumed that the reference area should theoretically contain 2×2 light spots, that is, the reference area. The threshold of the number of light spots is: 4, and in practice, the number of light spots detected in the reference area in the reflection intensity graph is: 5, that is, the total number of light spots corresponding to the reference area is 5; at this time: the total number of light spots is greater than the number of light spots If the threshold is set, the total energy intensity detection is performed, that is, “determine the spot intensity of the third spot in the reference area and the average spot intensity corresponding to all the fourth spots based on the pixel intensity parameter in S310” and S311.

Optionally, the light spot with the smallest light spot intensity of all light spots in the reference area may be used as the interference light spot.

Optionally, the threshold of the number of light spots can be determined based on the number of theoretical light spots. For example, the threshold of the number of light spots can be equal to the number of theoretical light spots; the threshold of the number of light spots can also be slightly larger than the number of theoretical light spots, so as to consider the light spot error in the actual scene and improve the robustness.

In a feasible implementation manner, the number of effective light spots that should theoretically appear in the above-mentioned reference area has been predetermined. Therefore, the distance between at least one theoretically occurring light spot or the range of the distance change can also be determined, that is to say, it can be The non-maximum value suppression detection is performed based on the distance between the two light spots in the reference area to further determine the non-maximum value suppression. After the light spot separation distance between the two, if the light spot separation distance is less than the separation distance threshold, it is possible that an interfering light spot appears in the reference area of the field of view, and the "Based on the pixel intensity parameter, determine the third light spot in the reference area" in S310 can be performed. The spot intensity of , and the mean value of the spot intensity corresponding to all the fourth spots” and S311.

Optionally, if the light spot separation distance is greater than or equal to the separation distance threshold, ignore processing may be performed.

Optionally, the positional relationship between the aforementioned two light spots may be an adjacent positional relationship.

Illustratively, if the spot spacing distance between the two adjacent spots is less than the spot spacing distance threshold, and the spot intensity of the middle spot is less than the average spot intensity of the two sides, or the spot intensity of the middle spot is less than The intensity of the light spots on both sides, the energy intensity of the middle light spot is weaker, and the middle light spot can be used as the interference light spot, as shown in Figure 7, which is a schematic diagram of a scene involving interference light spots. The distance between the two spots between the spots is less than the separation distance threshold, and the spot intensity of the middle spot is smaller than the spot intensity of the two sides, that is, the energy intensity of the middle spot is weak, and the middle spot can be used as an interference spot, and all the light spots covered by the interference spot can be used. At least one of the pixel points can be used as the interference light spot, for example, the point indicated by the maximum value of the pixel intensity parameter (Confidence value) among all the pixel points covered by the interference light spot is used as the interference point.

S311: Perform a total energy intensity detection process on the third light spot based on the light spot intensity of the third light spot and the average value of the light spot intensity, and determine a fourth interference light spot.

In a feasible implementation manner, the intensity difference between the spot intensity of the third spot and the average intensity of the spots is determined, that is, the difference between the intensity of the spot and the average intensity of the spots; the first difference threshold of the value to realize the decision;

If the intensity difference is less than or equal to the first difference threshold, the third light spot with the intensity difference less than or equal to the first difference threshold is used as the fourth interference light spot.

If the intensity difference is greater than the first difference threshold, ignore processing is performed.

In a feasible implementation manner, determining the intensity ratio of the spot intensity of the third spot to the mean value of the spot intensity, that is, the ratio of the spot intensity to the mean value of the spot intensity;

If the intensity ratio is less than or equal to the first ratio threshold, the third light spot is used as the fourth interference light spot. Wherein, the first ratio threshold is a threshold value set for the intensity ratio.

If the intensity ratio is greater than the first ratio threshold, ignore processing is performed.

In a specific implementation scenario, it can be understood that if the reference area includes a plurality of light spots, the corresponding light spot intensities of the plurality of light spots can be obtained, and the third light spot indicated by the minimum light spot intensity is selected based on the light spot intensities of the plurality of light spots , and then determine all fourth spots within the reference area. One way is: take all the light spots in the reference area as the fourth light spot; one way is: take all the light spots in the reference area except the third light spot as the fourth light spot; then calculate the light spots corresponding to all the fourth light spots The average intensity is calculated, and then the quotient of the intensity of the third spot and the average intensity of the spot is calculated as the intensity ratio. If the intensity ratio is less than the first ratio threshold, the third spot is used as the fourth interference spot.

S312: If the light spot interference detection method is a light spot position detection method, determine a light spot position for the at least one light spot based on a pixel position parameter; obtain a reference position range corresponding to at least one of the light spots;

It can be understood that the spot interference detection method can be a spot position detection method, and can also be understood as a fixed position detection. According to some embodiments, the speckle iTOF camera is in the factory stage, and the spot position on the reflection intensity map obtained by the light receiving end. Usually it is relatively fixed, but a few pixels may shift due to objective reasons such as distance and distance parallax, signal divergence, etc. Based on this, the position range of the light spot can be set at the factory stage, and the position range of at least one light spot can be recorded. Appearing outside these positions is likely to interfere with light spots.

S313: If the light spot position of the light spot does not match the reference position range, determine the light spot as a fifth interference light spot.

It can be understood that, after the terminal determines the light spot interference detection method, if the light spot interference detection method is the light spot position detection method, the terminal can perform light spot position detection processing on at least one light spot based on the pixel position parameter to obtain a fifth interference light spot. For example, during the spot position detection process, the spot positions of multiple spots are obtained, and the spot position of the spot is matched with the position range corresponding to the spot to detect whether the spot position falls within the reference position range, and the spot is determined. is the fifth interference light spot; and so on, the fifth interference light spot can be determined from the plurality of light spots.

In the embodiment of the present application, by acquiring the reflection intensity map, at least one light spot in the reflection intensity map is determined, and then the pixel feature parameters corresponding to the pixel points of the light spot in the light spot are obtained, and the pixels on the interference light spot and the effective light spot are based on the pixel feature parameters. The difference in characteristic parameters can accurately determine the interference light spot from at least one light spot, and can assist in eliminating the light spot interference caused by the superposition of the edge signals of multiple light spots; Accurately identify light spot interference, which can assist in outputting high-precision depth measurement results and improve the robustness in depth measurement scenarios; and, it can combine different light spot interference detection methods or different time sequence light spot interference detection methods determined based on actual application scenarios. , which optimizes the detection process of interference light spots.

The image processing apparatus provided by the embodiment of the present application will be described in detail below with reference to FIG. 8 . It should be noted that the image processing apparatus shown in FIG. 8 is used to execute the methods of the embodiments shown in FIG. 1 to FIG. 6 of the present application. For convenience of description, only the parts related to the embodiments of the present application are shown, and the specific technology For details not disclosed, please refer to the embodiments shown in FIGS. 1 to 7 of the present application.

Please refer to FIG. 8 , which shows a schematic structural diagram of an image processing apparatus according to an embodiment of the present application. The image processing apparatus 1 can be implemented as all or a part of the user terminal through software, hardware or a combination of the two. According to some embodiments, the image processing apparatus 1 includes a light spot determination module 11, a parameter acquisition module 12 and an interference determination module 13, which are specifically used for:

a light spot determination module 11, configured to acquire a reflection intensity map, and determine at least one light spot in the reflection intensity map;

A parameter acquisition module 12, configured to acquire pixel characteristic parameters corresponding to the pixel points of the light spot in the light spot;

The interference determination module 13 is configured to determine the interference light spot from the at least one light spot based on the pixel characteristic parameter.

Optionally, as shown in FIG. 9 , the interference determination module 13 includes:

A detection and determination unit 131, configured to determine at least one speckle interference detection method for the reflection intensity map;

The interference detection unit 132 is configured to perform a light spot interference detection process on at least one of the light spots by using the light spot interference detection method based on the pixel characteristic parameter to obtain an interference light spot.

Optionally, the interference detection unit 132 is specifically configured to:

If the light spot interference detection method is a pixel intensity detection method, performing pixel intensity detection processing on at least one light spot based on a pixel intensity parameter to obtain a first interference light spot;

If the light spot interference detection method is an energy proportional detection method, performing energy proportional detection processing on at least one light spot based on the pixel intensity parameter to obtain a second interference light spot;

If the light spot interference detection method is a pixel point average detection method, performing pixel point average detection processing on at least one light spot based on the pixel intensity parameter to obtain a third interference light spot;

If the light spot interference detection method is a total energy intensity detection method, performing total energy intensity detection processing on at least one light spot based on the pixel intensity parameter to obtain a fourth interference light spot;

If the light spot interference detection method is a light spot position detection method, the light spot position detection process is performed on at least one light spot based on the pixel position parameter to obtain a fifth interference light spot.

Optionally, the interference detection unit 132 is specifically configured to:

Determine the first intensity priority corresponding to the light spot pixel point based on the pixel intensity parameter, and obtain the target light spot ratio for the reflection intensity map;

An intensity threshold is determined based on the target light spot ratio, the first intensity priority of the light spot pixels, and the pixel intensity parameters of the light spot pixels.

Based on the intensity threshold, a first interfering light spot is determined from at least one of the light spots.

Optionally, the interference detection unit 132 is specifically configured to:

Determine a first light spot based on the proportion of the target light spot and the first intensity priority of the light spot pixel point and obtain a first intensity threshold corresponding to the first light spot, based on the first intensity threshold from at least one of the Identify the first interfering spot in the spot; or,

Obtain the number of reference pixels corresponding to the light spot, determine the target ratio corresponding to the proportion of the target light spots and the number of reference pixels, and determine the first intensity priority based on the target ratio and the first intensity priority of the light spot pixels. two light spots and acquiring a second intensity threshold corresponding to the second light spot, and determining a first interference light spot from at least one of the light spots based on the second intensity threshold; or,

Determine the first light spot based on the proportion of the target light spot and the first intensity priority of the light spot pixel points, obtain the first intensity threshold corresponding to the first light spot, obtain the number of reference pixels corresponding to the light spot, and determine The target ratio corresponding to the proportion of the target light spots and the number of reference pixels, the second light spot is determined based on the target ratio and the first intensity priority of the light spot pixels, and the first light spot corresponding to the second light spot is obtained. Two intensity thresholds, a first interfering light spot is determined from at least one of the light spots based on the first intensity threshold and the second intensity threshold.

Optionally, the interference detection unit 132 is specifically configured to:

determining a threshold reference range based on the first intensity threshold and the second intensity threshold;

A target intensity threshold is obtained from the threshold reference range, and a first interfering light spot is determined from at least one of the light spots based on the target intensity threshold.

Optionally, the interference detection unit 132 is specifically configured to:

acquiring at least one intensity value of the center point corresponding to the light spot and the intensity value of the edge point;

An energy ratio detection process is performed on at least one light spot based on the central point intensity value and the edge point intensity value, and a second interference light spot is determined from at least one of the light spots.

Optionally, the interference detection unit 132 is specifically configured to:

determining a first intensity ratio of the light spot based on the center point intensity value and the edge point intensity value; determining a second interfering light spot from at least one of the light spots based on the first intensity ratio and an intensity ratio threshold; and /or,

Determine the first intensity ratio of the light spot based on the intensity value of the center point and the edge point intensity value; obtain the first light spot corresponding to the light spot respectively, and determine the second intensity ratio of the first light spot; based on the The first intensity ratio and the second intensity ratio of the light spots determine a second interfering light spot from at least one of the light spots.

Optionally, the interference detection unit 132 is specifically configured to:

determining a target difference based on the first intensity ratio and the second intensity ratio; determining a second interfering light spot from at least one of the light spots based on the target difference and a difference threshold; or,

Using the first intensity ratio as a reference, determine an intensity ratio fluctuation range for the first light spot based on at least one of the second intensity ratios; based on the intensity ratio fluctuation range and the reference fluctuation range, from at least one of the light spots Determine the second interference spot.

Optionally, the interference detection unit 132 is specifically configured to:

Acquire the adjacent light spot adjacent to the light spot, and use the adjacent light spot as the first light spot corresponding to the light spot; or,

A target distance between the light spot and at least one second light spot is acquired, and a first light spot corresponding to the light spot is determined from the at least one second light spot based on the target distance and a distance threshold.

Optionally, the interference detection unit 132 is specifically configured to:

determining at least one target pixel area corresponding to the light spot, and the light spot is located in the target pixel area;

Determine, based on the pixel intensity parameter, a pixel mean value of a light spot for at least one of the light spots, and determine a regional pixel mean value corresponding to at least one of the target pixel regions;

Perform pixel point average detection processing on at least one light spot based on the light spot pixel average value and the regional pixel average value of the light spot, and determine a third interference light spot from at least one of the light spots.

Optionally, the interference detection unit 132 is specifically configured to:

Obtain the first total intensity and the total number of light spot pixels corresponding to all the light spot pixels of at least one of the light spots, and use the quotient of the first total intensity and the total number of light spot pixels as the light spot pixel mean of the light spot;

Obtain the second total intensity and the total number of regional pixels corresponding to all area pixels of at least one of the target pixel areas, and take the quotient of the second total intensity and the total number of regional pixels as the area corresponding to the target pixel area pixel mean.

Optionally, the interference detection unit 132 is specifically configured to:

determining the reference ratio corresponding to the pixel mean value of the light spot and the pixel mean value of the area;

Perform pixel point mean detection processing on at least one light spot based on the reference ratio and the ratio threshold, and determine a third interference light spot from at least one of the light spots.

Optionally, the interference detection unit 132 is specifically configured to:

Based on the pixel intensity parameter, determine the spot intensity of the third spot in the reference area and the mean value of the spot intensity corresponding to all the fourth spots;

The total energy intensity detection process is performed on the third light spot based on the light spot intensity of the third light spot and the average value of the light spot intensity, and a fourth interference light spot is determined.

Optionally, all light spots included in the reference area are fourth light spots and the third light spot is one of all the fourth light spots; or, the reference area includes the third light spot and all fourth light spots spot.

Optionally, the interference detection unit 132 is specifically configured to:

Acquire the total number of light spots corresponding to the reference area; if the total number of light spots is greater than the threshold of the number of light spots, execute the step of determining the light spot intensity of the third light spot in the reference area and the light spot intensities corresponding to all the fourth light spots based on the pixel intensity parameter mean steps; or,

Obtain the spot interval distance between two spots in the reference area, and if the spot interval distance is less than the interval distance threshold, execute the determining the spot intensity of the third spot in the reference area and all fourth spots based on the pixel intensity parameter. Steps for the mean value of the spot intensity corresponding to the spot.

Optionally, the interference detection unit 132 is specifically configured to:

determining the intensity ratio of the light spot intensity of the third light spot to the mean value of the light spot intensity;

If the intensity ratio is less than or equal to the first ratio threshold, the third light spot is used as the fourth interference light spot.

Optionally, the interference detection unit 132 is specifically configured to:

Determine a spot position for the at least one spot based on the pixel position parameter; obtain a reference position range corresponding to at least one of the spots;

If the light spot position of the light spot does not match the reference position range, the light spot is determined as the fifth interference light spot.

Optionally, as shown in FIG. 10 , the device 1 further includes:

an image acquisition module 14, configured to acquire a color image and a depth image, wherein the reflection intensity map, the depth image and the color image are images of different types for the same target object;

The interference verification module 15 is configured to perform interference verification on the interference light spot based on the color image and the depth image.

Optionally, as shown in FIG. 11 , the interference verification module 15 includes:

The color verification unit 151 is configured to determine the third position of the interference light spot in the color image, and determine the fourth position of at least one adjacent light spot corresponding to the interference light spot in the color image; Three positions and the fourth position, obtain the color pixel verification result for the interference light spot;

The depth verification unit 152 is configured to determine the first position of the interference light spot in the depth image, and determine the second position of at least one adjacent light spot corresponding to the interference light spot in the depth image; A position and the second position to obtain a depth pixel verification result for the interference light spot;

The result generating unit 153 is configured to obtain an interference verification result for the interference light spot based on the color pixel verification result and the depth pixel verification result.

Optionally, as shown in FIG. 10 , the device 1 further includes:

The exposure adjustment module 15 is configured to obtain the number of light spots corresponding to the light spots, and if the number of light spots is greater than the number threshold, adjust the exposure time of the camera.

It should be noted that, when the image processing apparatus provided in the above embodiments executes the image processing method, only the division of the above at least one functional module is used as an example for illustration. In practical applications, the above functions may be allocated to different functional modules as required. To complete, that is, to divide the internal structure of the device into different functional modules to complete all or part of the functions described above. In addition, the image processing apparatus and the image processing method embodiments provided by the above embodiments belong to the same concept, and the embodiment and implementation process thereof are detailed in the method embodiments, which will not be repeated here.

The above-mentioned serial numbers of the embodiments of the present application are only for description, and do not represent the advantages or disadvantages of the embodiments.

In the embodiment of the present application, by acquiring the reflection intensity map, at least one light spot in the reflection intensity map is determined, and then the pixel feature parameters corresponding to the pixel points of the light spot in the light spot are obtained, and the pixels on the interference light spot and the effective light spot are based on the pixel feature parameters. The difference in characteristic parameters can accurately determine the interference light spot from at least one light spot, and can assist in eliminating the light spot interference caused by the superposition of the edge signals of multiple light spots; Accurately identify light spot interference, which can assist in outputting high-precision depth measurement results and improve the robustness in depth measurement scenarios; and, it can combine different light spot interference detection methods or different time sequence light spot interference detection methods determined based on actual application scenarios. , which optimizes the detection process of interference light spots.

Embodiments of the present application further provide a computer storage medium, where the computer storage medium can store multiple instructions, and the instructions are suitable for being loaded by a processor and executing the above-described embodiments shown in FIG. 1 to FIG. 7 . For the specific execution process of the image processing method, reference may be made to the specific descriptions of the embodiments shown in FIG. 1 to FIG. 7 , which will not be repeated here.

The present application also provides a computer program product, where the computer program product stores at least one instruction, and the at least one instruction is loaded by the processor and executes the image processing in the embodiments shown in FIG. 1 to FIG. 7 . For the specific execution process, reference may be made to the specific descriptions of the embodiments shown in FIG. 1 to FIG. 7 , which will not be repeated here.

Please refer to FIG. 12 , which shows a structural block diagram of an electronic device provided by an exemplary embodiment of the present application. An electronic device in this application may include one or more of the following components: a processor 110 , a memory 120 , an input device 130 , an output device 140 and a bus 150 . The processor 110 , the memory 120 , the input device 130 and the output device 140 may be connected through a bus 150 .

The processor 110 may include one or more processing cores. The processor 110 uses at least one kind of interface and line to connect at least one part in the whole electronic device, by running or executing the instructions, programs, code sets or instruction sets stored in the memory 120, and calling the data stored in the memory 120. , perform at least one function of the electronic device 100 and process data. Optionally, the processor 110 may employ at least one of digital signal processing (digital signal processing, DSP), field-programmable gate array (field-programmable gate array, FPGA), and programmable logic array (programmable logic array, PLA). implemented in hardware. The processor 110 may integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), a modem, and the like. Among them, the CPU mainly handles the operating system, user interface and application programs, etc.; the GPU is used for rendering and drawing of the display content; the modem is used to handle wireless communication. It can be understood that, the above-mentioned modem may also not be integrated into the processor 110, and is implemented by a communication chip alone.

The memory 120 may include random access memory (RAM), or may include read-only memory (ROM). Optionally, the memory 120 includes a non-transitory computer-readable storage medium. Memory 120 may be used to store instructions, programs, codes, sets of codes, or sets of instructions. The memory 120 may include a stored program area and a stored data area, wherein the stored program area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.) , instructions for implementing the following at least one method embodiment, etc., the operating system can be an Android (Android) system, including a system based on the deep development of the Android system, an IOS system developed by Apple, including a system based on the deep development of the IOS system. system or other systems. The storage data area can also store data created by the electronic device in use, such as a phone book, audio and video data, chat record data, and the like.

Referring to FIG. 13 , the memory 120 can be divided into an operating system space and a user space. The operating system runs in the operating system space, and native and third-party applications run in the user space. In order to ensure that different third-party applications can achieve better running effects, the operating system allocates corresponding system resources to different third-party applications. However, different application scenarios in the same third-party application also have different requirements for system resources. For example, in the local resource loading scenario, the third-party application has higher requirements on the disk read speed; in the animation rendering scenario, the first Third-party applications have higher requirements on GPU performance. The operating system and the third-party application are independent of each other, and the operating system often cannot perceive the current application scenario of the third-party application in time, so that the operating system cannot perform targeted system resource adaptation according to the specific application scenario of the third-party application.

In order to enable the operating system to distinguish the specific application scenarios of third-party applications, it is necessary to open up the data communication between the third-party application and the operating system, so that the operating system can obtain the current scene information of the third-party application at any time, and then perform the operation based on the current scene. Targeted system resource adaptation.

Taking the Android system as the operating system as an example, the programs and data stored in the memory 120 are shown in FIG. 14 . The memory 120 can store the Linux kernel layer 320, the system runtime library layer 340, the application framework layer 360 and the application layer 380, Among them, the Linux kernel layer 320, the system runtime layer 340 and the application framework layer 360 belong to the operating system space, and the application layer 380 belongs to the user space. The Linux kernel layer 320 provides underlying drivers for at least one type of hardware of the electronic device, such as display drivers, audio drivers, camera drivers, Bluetooth drivers, Wi-Fi drivers, power management, and the like. The system runtime layer 340 provides main feature support for the Android system through some C/C++ libraries. For example, the SQLite library provides database support, the OpenGL/ES library provides 3D drawing support, and the Webkit library provides browser kernel support. An Android runtime library (Android runtime) is also provided in the system runtime library layer 340, which mainly provides some core libraries, which can allow developers to use Java language to write Android applications. The application framework layer 360 provides at least one API that may be used when building applications. Developers can also build their own applications by using these APIs, such as activity management, window management, view management, notification management, and content providers. , package management, call management, resource management, location management. There is at least one application running in the application layer 380, and these applications may be native applications that come with the operating system, such as contact programs, SMS programs, clock programs, camera applications, etc.; they may also be developed by third-party developers Third-party applications, such as game applications, instant messaging programs, photo enhancement programs, etc.

Taking the operating system as the IOS system as an example, the programs and data stored in the memory 120 are shown in FIG. 15 . The IOS system includes: a core operating system layer 420 (Core OS layer), a core service layer 440 (Core Services layer), a media layer 460 (Media layer), touchable layer 480 (Cocoa Touch Layer). The core operating system layer 420 includes the operating system kernel, drivers, and low-level program frameworks, which provide functions closer to hardware for use by the program frameworks located in the core service layer 440 . The core service layer 440 provides system services and/or program frameworks required by application programs, such as a foundation framework, an account framework, an advertisement framework, a data storage framework, a network connection framework, a geographic location framework, a motion framework, and the like. The media layer 460 provides audiovisual interfaces for applications, such as graphics and image related interfaces, audio technology related interfaces, video technology related interfaces, and audio and video transmission technology wireless playback (AirPlay) interfaces. The touchable layer 480 provides at least one common interface-related framework for application development, and the touchable layer 480 is responsible for the user's touch interaction operation on the electronic device. Such as local notification service, remote push service, advertising framework, game tool framework, message user interface interface (User Interface, UI) framework, user interface UIKit framework, map framework and so on.

Among the frameworks shown in FIG. 15 , frameworks related to most applications include but are not limited to: the basic framework in the core service layer 440 and the UIKit framework in the touchable layer 480 . The basic framework provides many basic object classes and data types, and provides the most basic system services for all applications, regardless of UI. The classes provided by the UIKit framework are the basic UI class libraries for creating touch-based user interfaces. iOS applications can provide UI based on the UIKit framework, so it provides the application's infrastructure for building user interfaces, drawing , handling and user interaction events, responding to gestures, and more.

The method and principle of implementing data communication between a third-party application and an operating system in the IOS system may refer to the Android system, which will not be repeated in this application.

The input device 130 is used for receiving input instructions or data, and the input device 130 includes but is not limited to a keyboard, a mouse, a camera, a microphone or a touch device. The output device 140 is used for outputting instructions or data, and the output device 140 includes, but is not limited to, a display device, a speaker, and the like. In one example, the input device 130 and the output device 140 may be co-located, and the input device 130 and the output device 140 are a touch display screen, the touch display screen is used to receive any suitable objects such as a user's finger, a touch pen, etc. Nearby touch operations, and displaying a user interface of at least one application. The touch display is usually provided on the front panel of the electronic device. The touch screen can be designed as a full screen, a curved screen or a special-shaped screen. The touch display screen can also be designed to be a combination of a full screen and a curved screen, or a combination of a special-shaped screen and a curved screen, which is not limited in the embodiments of the present application.

In addition, those skilled in the art can understand that the structure of the electronic device shown in the above drawings does not constitute a limitation to the electronic device, and the electronic device may include more or less components than those shown in the drawings, or a combination of certain components may be included. some components, or a different arrangement of components. For example, the electronic device further includes components such as a radio frequency circuit, an input unit, a sensor, an audio circuit, a wireless fidelity (WiFi) module, a power supply, and a Bluetooth module, which will not be repeated here.

In this embodiment of the present application, the execution body of at least one step may be the electronic device described above. Optionally, the execution body of at least one step is an operating system of the electronic device. The operating system may be an Android system, an IOS system, or other operating systems, which are not limited in this embodiment of the present application.

The electronic device according to the embodiment of the present application may also be mounted with a display device, and the display device may be at least one device capable of realizing a display function, such as a cathode ray tube display (CR for short), a light emitting diode display ( light-emitting diode display, referred to as LED), electronic ink screen, liquid crystal display (liquid crystal display, referred to as LCD), plasma display panel (plasma display panel, referred to as PDP) and so on. The user can use the display device on the electronic device 101 to view the displayed text, image, video and other information. The electronic device may be a smart phone, a tablet computer, a game device, an AR (Augmented Reality, augmented reality) device, a car, a data storage device, an audio playback device, a video playback device, a notebook, a desktop computing device,

In the electronic device shown in FIG. 12, the electronic device may be a terminal, and the processor 110 may be used to invoke the network optimization application program stored in the memory 120, and specifically perform the following operations:

acquiring a reflection intensity map, and determining at least one light spot in the reflection intensity map;

obtaining the pixel characteristic parameters corresponding to the pixel points of the light spot in the light spot;

Interfering light spots are determined from the at least one light spot based on the pixel characteristic parameters.

In one embodiment, the processor 110 specifically performs the following operations when performing the determining of the interference light spot from the at least one light spot based on the pixel characteristic parameter:

At least one light spot interference detection method for the reflection intensity map is determined, and based on the pixel characteristic parameter, the light spot interference detection method is used to perform a light spot interference detection process on at least one of the light spots to obtain an interference light spot.

In one embodiment, the processor 110 specifically performs the following operations when performing the speckle interference detection processing on at least one of the light spots based on the pixel characteristic parameters using the light spot interference detection method to obtain the interference light spot:

If the light spot interference detection method is a pixel intensity detection method, performing pixel intensity detection processing on at least one light spot based on a pixel intensity parameter to obtain a first interference light spot;

If the light spot interference detection method is an energy proportional detection method, performing energy proportional detection processing on at least one light spot based on the pixel intensity parameter to obtain a second interference light spot;

If the light spot interference detection method is a pixel point average detection method, performing pixel point average detection processing on at least one light spot based on the pixel intensity parameter to obtain a third interference light spot;

If the light spot interference detection method is a total energy intensity detection method, performing total energy intensity detection processing on at least one light spot based on the pixel intensity parameter to obtain a fourth interference light spot;

If the light spot interference detection method is a light spot position detection method, the light spot position detection process is performed on at least one light spot based on the pixel position parameter to obtain a fifth interference light spot.

In one embodiment, when the processor 110 performs the pixel intensity detection process on at least one light spot based on the pixel intensity parameter to obtain the first interference light spot, the processor 110 specifically performs the following steps:

Determine the first intensity priority corresponding to the light spot pixel point based on the pixel intensity parameter, and obtain the target light spot ratio for the reflection intensity map;

An intensity threshold is determined based on the target light spot ratio, the first intensity priority of the light spot pixels, and the pixel intensity parameters of the light spot pixels.

Based on the intensity threshold, a first interfering light spot is determined from at least one of the light spots.

In one embodiment, the processor 110 determines the intensity threshold value based on the target light spot ratio, the first intensity priority of the light spot pixel point, and the pixel intensity parameter of the light spot pixel point, Based on the intensity threshold, when the first interference light spot is determined from at least one of the light spots, the following steps are specifically performed:

Determine a first light spot based on the proportion of the target light spot and the first intensity priority of the light spot pixel point and obtain a first intensity threshold corresponding to the first light spot, based on the first intensity threshold from at least one of the Identify the first interfering spot in the spot; or,

Obtain the number of reference pixels corresponding to the light spot, determine the target ratio corresponding to the proportion of the target light spots and the number of reference pixels, and determine the first intensity priority based on the target ratio and the first intensity priority of the light spot pixels. two light spots and acquiring a second intensity threshold corresponding to the second light spot, and determining a first interference light spot from at least one of the light spots based on the second intensity threshold; or,

Determine the first light spot based on the proportion of the target light spot and the first intensity priority of the light spot pixel points, obtain the first intensity threshold corresponding to the first light spot, obtain the number of reference pixels corresponding to the light spot, and determine The target ratio corresponding to the proportion of the target light spots and the number of reference pixels, the second light spot is determined based on the target ratio and the first intensity priority of the light spot pixels, and the first light spot corresponding to the second light spot is obtained. Two intensity thresholds, a first interfering light spot is determined from at least one of the light spots based on the first intensity threshold and the second intensity threshold.

In one embodiment, when the processor 110 performs the determining of the first interference light spot from at least one of the light spots based on the first intensity threshold and the second intensity threshold, the processor 110 specifically performs the following steps:

determining a threshold reference range based on the first intensity threshold and the second intensity threshold;

A target intensity threshold is obtained from the threshold reference range, and a first interfering light spot is determined from at least one of the light spots based on the target intensity threshold.

In one embodiment, when the processor 110 performs the energy ratio detection process on at least one light spot based on the pixel intensity parameter to obtain the second interference light spot, the processor 110 specifically performs the following steps:

acquiring at least one intensity value of the center point corresponding to the light spot and the intensity value of the edge point;

An energy ratio detection process is performed on at least one light spot based on the central point intensity value and the edge point intensity value, and a second interference light spot is determined from at least one of the light spots.

In one embodiment, the processor 110 is performing the energy ratio detection process on at least one light spot based on the central point intensity value and the edge point intensity value, and determines the second interference from at least one of the light spots When spotting, perform the following steps:

determining a first intensity ratio of the light spot based on the center point intensity value and the edge point intensity value; determining a second interfering light spot from at least one of the light spots based on the first intensity ratio and an intensity ratio threshold; and /or,

Determine the first intensity ratio of the light spot based on the intensity value of the center point and the edge point intensity value; obtain the first light spot corresponding to the light spot respectively, and determine the second intensity ratio of the first light spot; based on the The first intensity ratio and the second intensity ratio of the light spots determine a second interfering light spot from at least one of the light spots.

In one embodiment, when the processor 110 executes the determining of a second interference light spot from at least one of the light spots based on the first intensity ratio and the second intensity ratio of the light spot, the processor 110 specifically executes The following steps:

determining a target difference based on the first intensity ratio and the second intensity ratio; determining a second interfering light spot from at least one of the light spots based on the target difference and a difference threshold; or,

Using the first intensity ratio as a reference, determine an intensity ratio fluctuation range for the first light spot based on at least one of the second intensity ratios; based on the intensity ratio fluctuation range and the reference fluctuation range, from at least one of the light spots Determine the second interference spot.

In one embodiment, when the processor 110 performs the acquiring of the first light spot corresponding to the light spot, the processor 110 specifically performs the following steps: acquiring adjacent light spots adjacent to the light spot, and using the adjacent light spots as the the first light spot corresponding to the light spot; or,

A target distance between the light spot and at least one second light spot is acquired, and a first light spot corresponding to the light spot is determined from the at least one second light spot based on the target distance and a distance threshold.

In one embodiment, the processor 110 specifically performs the following steps when performing the pixel point mean detection process on at least one light spot based on the pixel intensity parameter to obtain a third interference light spot: determining at least one of the light spots a corresponding target pixel area, the light spot is located in the target pixel area;

Determine, based on the pixel intensity parameter, a pixel mean value of a light spot for at least one of the light spots, and determine a regional pixel mean value corresponding to at least one of the target pixel regions;

Perform pixel point average detection processing on at least one light spot based on the light spot pixel average value and the regional pixel average value, and determine a third interference light spot from at least one of the light spots.

In one embodiment, when the processor 110 executes the determining, based on the pixel intensity parameter, the average pixel point value of the light spot for at least one of the light spots, and determining the average pixel point value of the region corresponding to the at least one target pixel area, Perform the following steps specifically:

Obtain the first total intensity and the total number of light spot pixels corresponding to all the light spot pixels of at least one of the light spots, and use the quotient of the first total intensity and the total number of light spot pixels as the light spot pixel mean of the light spot;

Obtain the second total intensity and the total number of regional pixels corresponding to all area pixels of at least one of the target pixel areas, and take the quotient of the second total intensity and the total number of regional pixels as the area corresponding to the target pixel area pixel mean.

In one embodiment, the processor 110 is performing the pixel point average detection process on at least one light spot based on the light spot pixel average value and the regional pixel point average value, and determines the third light spot from the at least one light spot. When the light spot is disturbed, the following steps are specifically performed: determining the reference ratio corresponding to the average value of the pixel points of the light spot and the average value of the pixel points of the region;

Perform pixel point mean detection processing on at least one light spot based on the reference ratio and the ratio threshold, and determine a third interference light spot from at least one of the light spots.

In one embodiment, the processor 110 specifically performs the following steps when performing the total energy intensity detection process on at least one light spot based on the pixel intensity parameter to obtain a fourth interference light spot: based on the pixel intensity parameter , determine the light spot intensity of the third light spot in the reference area and the average value of the light spot intensity corresponding to all the fourth light spots; the fourth light spot is the light spot other than the third light spot in the reference area;

The total energy intensity detection process is performed on the third light spot based on the light spot intensity of the third light spot and the average value of the light spot intensity, and a fourth interference light spot is determined.

In one embodiment, all the light spots included in the reference area are fourth light spots and the third light spot is one of all the fourth light spots; or, the reference area includes the third light spot and all the fourth light spots Fourth spot.

In one embodiment, before the processor 110 determines the spot intensity of the third spot in the reference area and the mean value of spot intensities corresponding to all the fourth spots based on the pixel intensity parameter, the processor 110 further performs the following step: acquiring the reference area The corresponding total number of light spots; if the total number of light spots is greater than the threshold of the number of light spots, the step of determining the light spot intensity of the third light spot in the reference area and the average value of the light spot intensity corresponding to all the fourth light spots based on the pixel intensity parameter is performed ;or,

Obtain the spot interval distance between two spots in the reference area, and if the spot interval distance is less than the interval distance threshold, execute the determining the spot intensity of the third spot in the reference area and all fourth spots based on the pixel intensity parameter. Steps for the mean value of the spot intensity corresponding to the spot.

In one embodiment, when the processor 110 performs the total energy intensity detection process on the third light spot based on the light spot intensity of the third light spot and the average value of the light spot intensity, and determines the fourth interference light spot, Perform the following steps specifically:

determining the intensity ratio of the light spot intensity of the third light spot to the mean value of the light spot intensity;

If the intensity ratio is less than or equal to the first ratio threshold, the third light spot is used as the fourth interference light spot.

In one embodiment, when the processor 110 performs the spot position detection process on at least one spot based on the pixel position parameter to obtain a fifth interference spot, the processor 110 specifically performs the following steps:

Determine a spot position for the at least one spot based on the pixel position parameter; obtain a reference position range corresponding to at least one of the spots;

If the light spot position of the light spot does not match the reference position range, the light spot is determined as the fifth interference light spot.

In one embodiment, after the processor 110 performs the determining of the interference light spot from the at least one light spot, the processor 110 further performs the following steps:

acquiring a color image and a depth image, wherein the reflection intensity map, the depth image and the color image are images of different types for the same target object;

Based on the color image and the depth image, interference verification is performed on the interference light spot.

In one embodiment, when the processor 110 performs the interference verification on the interference light spot based on the color image and the depth image, the processor 110 specifically performs the following steps:

determining a first position of the interference light spot in the depth image, and determining a second position of at least one adjacent light spot corresponding to the interference light spot in the depth image; based on the first position and the second position position, obtain the depth pixel verification result for the interference light spot;

determining a third position of the interference light spot in the color image, and determining a fourth position of at least one adjacent light spot corresponding to the interference light spot in the color image; based on the third position and the fourth position position, obtain the color pixel verification result for the interference light spot;

Based on the color pixel verification result and the depth pixel verification result, an interference verification result for the interference light spot is obtained.

In one embodiment, after the processor 110 performs the acquiring of the reflection intensity map and determines at least one light spot in the reflection intensity map, the processor 110 further performs the following steps:

Obtain the number of light spots corresponding to the light spots;

If the number of light spots is greater than the number threshold, adjust the camera exposure time

In the embodiment of the present application, by acquiring the reflection intensity map, at least one light spot in the reflection intensity map is determined, and then the pixel feature parameters corresponding to the pixel points of the light spot in the light spot are obtained, and the pixels on the interference light spot and the effective light spot are based on the pixel feature parameters. The difference in characteristic parameters can accurately determine the interference spot from at least one spot, and can assist in eliminating the spot interference caused by the superposition of the edge signals of multiple spots; Accurately identify light spot interference, which can assist in outputting high-precision depth measurement results, and improve the robustness in depth measurement scenarios; and, it can combine different light spot interference detection methods or different time sequence light spot interference detection methods determined based on actual application scenarios. , which optimizes the detection process of interference light spots.

Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium. When executed, the flow of at least one method embodiment described above may be included. Wherein, the storage medium may be a magnetic disk, an optical disk, a read-only storage memory or a random storage memory, and the like.

The above disclosures are only the preferred embodiments of the present application, and of course, the scope of the rights of the present application cannot be limited by this. Therefore, equivalent changes made according to the claims of the present application are still within the scope of the present application.

Claims (24)

1. an image processing method, is characterized in that, described method comprises: acquiring a reflection intensity map, and determining at least one light spot in the reflection intensity map; obtaining the pixel characteristic parameters corresponding to the pixel points of the light spot in the light spot; Interfering light spots are determined from the at least one light spot based on the pixel characteristic parameters. 2 . The method according to claim 1 , wherein the determining an interference light spot from the at least one light spot based on the pixel characteristic parameter comprises: 2 . At least one light spot interference detection method for the reflection intensity map is determined, and based on the pixel characteristic parameter, the light spot interference detection method is used to perform a light spot interference detection process on at least one light spot to obtain an interference light spot. 3. The method according to claim 2, characterized in that, performing a spot interference detection process on at least one light spot by using the light spot interference detection method based on the pixel characteristic parameters to obtain an interference light spot, comprising: If the light spot interference detection method is a pixel intensity detection method, performing pixel intensity detection processing on at least one light spot based on a pixel intensity parameter to obtain a first interference light spot; If the light spot interference detection method is an energy proportional detection method, performing energy proportional detection processing on at least one light spot based on the pixel intensity parameter to obtain a second interference light spot; If the light spot interference detection method is a pixel point average detection method, performing pixel point average detection processing on at least one light spot based on the pixel intensity parameter to obtain a third interference light spot; If the light spot interference detection method is a total energy intensity detection method, performing total energy intensity detection processing on at least one light spot based on the pixel intensity parameter to obtain a fourth interference light spot; If the light spot interference detection method is a light spot position detection method, the light spot position detection process is performed on at least one light spot based on the pixel position parameter to obtain a fifth interference light spot. 4. The method according to claim 3, wherein the pixel intensity detection processing is performed on at least one light spot based on the pixel intensity parameter to obtain the first interference light spot, comprising: Determine the first intensity priority corresponding to the light spot pixel point based on the pixel intensity parameter, and obtain the target light spot ratio for the reflection intensity map; An intensity threshold is determined based on the target light spot ratio, the first intensity priority of the light spot pixels, and the pixel intensity parameters of the light spot pixels. Based on the intensity threshold, a first interfering light spot is determined from at least one of the light spots. 5 . The method according to claim 4 , wherein the intensity is determined based on the target light spot ratio, the first intensity priority of the light spot pixel points, and the pixel intensity parameter of the light spot pixel points. 6 . A threshold, based on the intensity threshold, determining a first interference light spot from at least one of the light spots, including: Determine a first light spot based on the proportion of the target light spot and the first intensity priority of the light spot pixel point and obtain a first intensity threshold corresponding to the first light spot, based on the first intensity threshold from at least one of the Identify the first interfering spot in the spot; or, Obtain the number of reference pixels corresponding to the light spot, determine the target ratio corresponding to the proportion of the target light spot and the number of reference pixels, and determine the first intensity priority based on the target ratio and the first intensity priority of the light spot pixels. two light spots and acquiring a second intensity threshold corresponding to the second light spot, and determining a first interference light spot from at least one of the light spots based on the second intensity threshold; or, Determine the first light spot based on the proportion of the target light spot and the first intensity priority of the pixel points of the light spot, obtain the first intensity threshold corresponding to the first light spot, obtain the number of reference pixels corresponding to the light spot, and determine The target ratio corresponding to the proportion of the target light spots and the number of reference pixels, the second light spot is determined based on the target ratio and the first intensity priority of the light spot pixels, and the first light spot corresponding to the second light spot is obtained. Two intensity thresholds, a first interfering light spot is determined from at least one of the light spots based on the first intensity threshold and the second intensity threshold. 6. The method according to claim 5, wherein the determining a first interference light spot from at least one of the light spots based on the first intensity threshold and the second intensity threshold comprises: determining a threshold reference range based on the first intensity threshold and the second intensity threshold; A target intensity threshold is obtained from the threshold reference range, and a first interfering light spot is determined from at least one of the light spots based on the target intensity threshold. 7 . The method according to claim 3 , wherein, performing energy proportional detection processing on at least one light spot based on the pixel intensity parameter to obtain a second interference light spot, comprising: 8 . acquiring at least one intensity value of the center point corresponding to the light spot and the intensity value of the edge point; An energy ratio detection process is performed on at least one light spot based on the central point intensity value and the edge point intensity value, and a second interference light spot is determined from at least one of the light spots. 8 . The method according to claim 7 , wherein the energy ratio detection process is performed on at least one light spot based on the intensity value of the center point and the intensity value of the edge point, and the first light spot is determined from the at least one light spot. 9 . Two interference spots, including: determining a first intensity ratio of the light spot based on the center point intensity value and the edge point intensity value; determining a second interfering light spot from at least one of the light spots based on the first intensity ratio and an intensity ratio threshold; and /or, Determine the first intensity ratio of the light spot based on the intensity value of the center point and the edge point intensity value; obtain the first light spot corresponding to the light spot respectively, and determine the second intensity ratio of the first light spot; based on the The first intensity ratio and the second intensity ratio of the light spots determine a second interfering light spot from at least one of the light spots. 9 . The method of claim 8 , wherein the determining a second interfering light spot from at least one of the light spots based on the first intensity ratio and the second intensity ratio of the light spot, comprising: 10 . : determining a target difference based on the first intensity ratio and the second intensity ratio; determining a second interfering light spot from at least one of the light spots based on the target difference and a difference threshold; or, Taking the first intensity ratio as a reference, determining an intensity ratio fluctuation range for the first light spot based on at least one of the second intensity ratios; based on the intensity ratio fluctuation range and the reference fluctuation range, from at least one of the light spots Determine the second interference spot. 10. The method according to claim 8, wherein the acquiring the first light spot corresponding to the light spot comprises: Acquire the adjacent light spot adjacent to the light spot, and use the adjacent light spot as the first light spot corresponding to the light spot; or, A target distance between the light spot and at least one second light spot is acquired, and a first light spot corresponding to the light spot is determined from the at least one second light spot based on the target distance and a distance threshold. 11. The method according to claim 3, wherein the pixel point mean detection processing is performed on at least one light spot based on the pixel intensity parameter to obtain a third interference light spot, comprising: determining at least one target pixel area corresponding to the light spot, and the light spot is located in the target pixel area; Determine, based on the pixel intensity parameter, a mean value of light spot pixel points for at least one of the light spots, and determine an average value of regional pixel points corresponding to at least one of the target pixel areas; Based on the average pixel point value of the light spot and the average pixel point value of the area, a pixel point average value detection process is performed on at least one light spot, and a third interference light spot is determined from at least one of the light spots. 12 . The method according to claim 11 , wherein the pixel point mean value of the light spot for at least one of the light spots is determined based on the pixel intensity parameter, and the regional pixel point average value corresponding to at least one target pixel area is determined. 12 . ,include: Acquiring the first total intensity and the total number of light spot pixels corresponding to all the light spot pixels of at least one of the light spots, and taking the quotient of the first total intensity and the total number of light spot pixels as the average value of the light spot pixels of the light spot; Obtain the second total intensity and the total number of regional pixels corresponding to all regional pixels of at least one of the target pixel areas, and use the quotient of the second total intensity and the total number of regional pixels as the area corresponding to the target pixel area pixel mean. 13 . The method according to claim 11 , wherein the pixel point average value detection process is performed on at least one light spot based on the light spot pixel point average value and the regional pixel point average value, and is determined from the at least one light spot. 14 . The third interference spot, including: determining the reference ratio corresponding to the pixel mean value of the light spot and the pixel mean value of the area; Perform pixel point mean detection processing on at least one light spot based on the reference ratio and the ratio threshold, and determine a third interference light spot from at least one of the light spots. 14 . The method according to claim 3 , wherein, performing total energy intensity detection processing on at least one light spot based on the pixel intensity parameter to obtain a fourth interference light spot, comprising: 14 . Based on the pixel intensity parameter, determine the spot intensity of the third spot in the reference area and the mean value of the spot intensity corresponding to all the fourth spots; The total energy intensity detection process is performed on the third light spot based on the light spot intensity of the third light spot and the average value of the light spot intensity, and a fourth interference light spot is determined. 15. The method according to claim 14, wherein all the light spots included in the reference area are fourth light spots and the third light spot is one of all the fourth light spots; or, the reference The area includes the third spot and all fourth spots. 16 . The method according to claim 14 , wherein, before determining the spot intensity of the third spot in the reference area and the mean value of the spot intensity corresponding to all the fourth spots based on the pixel intensity parameter, the method further comprises: 16 . Acquire the total number of light spots corresponding to the reference area; if the total number of light spots is greater than the threshold of the number of light spots, execute the step of determining the light spot intensity of the third light spot in the reference area and the light spot intensities corresponding to all the fourth light spots based on the pixel intensity parameter mean steps; or, Obtain the spot interval distance between two spots in the reference area, and if the spot interval distance is less than the interval distance threshold, execute the determining the spot intensity of the third spot in the reference area and all fourth spots based on the pixel intensity parameter. Steps for the mean value of the spot intensity corresponding to the spot. 17 . The method according to claim 14 , wherein the total energy intensity detection process is performed on the third light spot based on the light spot intensity of the third light spot and the average value of the light spot intensity, and a fourth interference light spot is determined. 18 . ,include: determining the intensity ratio of the light spot intensity of the third light spot to the mean value of the light spot intensity; If the intensity ratio is less than or equal to the first ratio threshold, the third light spot is used as the fourth interference light spot. 18. The method according to claim 3, wherein, performing spot position detection processing on at least one spot based on pixel position parameters to obtain a fifth interference spot, comprising: Determine a spot position for the at least one spot based on the pixel position parameter; obtain a reference position range corresponding to at least one of the spots; If the light spot position of the light spot does not match the reference position range, the light spot is determined as the fifth interference light spot. 19. The method according to claim 1, wherein after determining the interference light spot from the at least one light spot, the method further comprises: acquiring a color image and a depth image, wherein the reflection intensity map, the depth image and the color image are images of different types for the same target object; Based on the color image and the depth image, interference verification is performed on the interference light spot. 20. The method according to claim 19, wherein the performing interference verification on the interference light spot based on the color image and the depth image comprises: determining a first position of the interference light spot in the depth image, and determining a second position of at least one adjacent light spot corresponding to the interference light spot in the depth image; based on the first position and the second position, obtain the depth pixel verification result for the interference light spot; determining a third position of the interference light spot in the color image, and determining a fourth position of at least one adjacent light spot corresponding to the interference light spot in the color image; based on the third position and the fourth position position, obtain the color pixel verification result for the interference light spot; Based on the color pixel verification result and the depth pixel verification result, an interference verification result for the interference light spot is obtained. 21. The method according to claim 1, wherein, after the acquiring a reflection intensity map and determining at least one light spot in the reflection intensity map, the method further comprises: Obtain the number of light spots corresponding to the light spots; If the number of light spots is greater than the number threshold, the camera exposure time is adjusted. 22. An image processing device, characterized in that the device comprises: a light spot determination module, configured to acquire a reflection intensity map, and determine at least one light spot in the reflection intensity map; a parameter acquisition module, configured to acquire pixel characteristic parameters corresponding to the pixel points of the light spot in the light spot; An interference determination module, configured to determine an interference light spot from the at least one light spot based on the pixel characteristic parameter. 23. A computer storage medium, characterized in that the computer storage medium stores a plurality of instructions, the instructions are suitable for being loaded by a processor and performing the method steps of any one of claims 1-20. 24. An electronic device, comprising: a processor and a memory; wherein the memory stores a computer program, and the computer program is adapted to be loaded by the processor and execute any one of claims 1 to 21 method steps for the item.

Images