CN115469333A - Depth measuring device and system - Google Patents

Abstract

The application relates to a degree of depth measuring device and system, the device include central processing unit, RGB sensor, microcontroller and TOF unit, wherein: the central processing unit is used for respectively generating RGB control signals and TOF control signals to the RGB sensor and the TOF unit and simultaneously sending correction signals to the microcontroller; the RGB sensor is used for acquiring RGB images of a target scene according to the RGB control signals, sending the RGB images to the central processor and generating feedback signals to the microcontroller; the microcontroller is used for correcting the feedback signal according to the correction signal to generate a TOF trigger signal; the TOF unit is used for emitting different modulation light beams according to the TOF trigger signal, collecting the reflected modulation light beams to generate at least two frames of catch images and transmitting the images to the central processing unit according to the TOF control signal; and the central processing unit is also used for calculating at least two frames of the rake hue images to obtain a TOF depth image so as to synchronously output the RGB image and the TOF depth image. The embodiment of the application can improve the synchronization effect of the images.

Description

A depth measuring device and system

technical field

The present application relates to the technical field of image acquisition and processing, in particular to a depth measurement device and system.

Background technique

Depth cameras can be divided into time of flight (Time of Flight, TOF) and structured light according to the working principle. Depth cameras have been gradually applied in various fields of human production and life, such as: robots, interactive games, augmented reality (Augmented Reality, AR), virtual reality (Virtual Reality, VR), unmanned driving and 3D modeling, etc.

Among them, the TOF depth measurement technology has the advantages of good stability and strong practicability, and it is a relatively outstanding technology among many optical three-dimensional measurement technologies. The TOF depth measurement technology includes two types, one is often called the pulse ranging method, the basic principle is to calculate the measured object (or the measured object detection distance) through the time interval of the laser pulse emitted by the TOF depth measurement device from emission to reception. area) to the distance between the TOF depth measurement equipment; the other is often called the phase difference distance measurement method, the basic principle of which is to calculate the phase change generated by the laser emitted by the TOF depth measurement equipment going back and forth to the object once. The distance between the measured object (or the detection area of the measured object) and the TOF depth measurement device.

On the one hand, in order to improve the detection accuracy and detection range, TOF depth measurement equipment generally uses multi-frequency fusion for depth detection, but when using the multi-frequency fusion algorithm, it is necessary to obtain multiple frames of images for phase unwrapping. When using images for processing, the acquired multi-frame images need spatial alignment and temporal alignment. Spatial alignment can be achieved through image calibration, but temporal alignment is difficult to achieve.

On the other hand, in order to improve the texture of TOF depth measurement equipment, RGB cameras are usually used to obtain RGB images for texture mapping. However, in the existing technology, the RGB camera and the TOF depth measurement device use their own time stamps to achieve synchronization. When this implementation method is adopted, on the one hand, due to the difference in exposure time between the two, the time stamps are offset, and eventually there will be Causes synchronization deviation; on the other hand, because there is task priority scheduling in the SOC chip, and the synchronization task between the RGB camera and the TOF depth measurement device is behind the task priority, it is difficult to realize the synchronous output of the RGB camera and the TOF depth measurement device .

Contents of the invention

In view of this, the embodiments of the present application provide a depth measurement device and system, which can solve at least one technical problem in the related art.

In the first aspect, an embodiment of the present application provides a depth measurement device, including: an RGB sensor, a TOF unit including a projection module and a TOF sensor, a central processing unit, and a microcontroller; Generate RGB control signal and TOF control signal to RGB sensor and TOF unit, and send correction signal to microcontroller at the same time; RGB sensor is used to collect RGB image according to RGB control signal and send RGB image to central processing unit, and generate selection communication at the same time No. or vertical synchronization signal to the microcontroller; the microcontroller is used to modify the strobe signal or vertical synchronization signal according to the correction signal to generate a TOF trigger signal to trigger the TOF unit to start working; the TOF unit is used to synchronize the TOF trigger signal When triggered, the projection module emits at least two different modulated beams to the target scene, and the TOF sensor collects different modulated beams reflected by the target scene to generate at least two frames of rawphase images, and transmits the rawphase images to the central processing unit according to the TOF control signal; The central processing unit is also used to perform phase unwrapping and depth fusion calculation on at least two frames of rawphase images to obtain TOF depth images, so as to synchronously output RGB images and TOF depth images.

In a second aspect, an embodiment of the present application provides a depth measurement system, including: a plurality of depth measurement devices as described in the embodiment of the first aspect.

The beneficial effect of the embodiment of the present application is that, while the depth measurement device realizes the measurement distance expansion, the exposure synchronization is achieved through software and hardware synchronization, which improves the synchronization effect of the image.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the accompanying drawings that need to be used in the descriptions of the embodiments or the prior art will be briefly introduced below. Obviously, the accompanying drawings in the following description are only for the present application For some embodiments, those of ordinary skill in the art can also obtain other drawings based on these drawings without paying creative efforts.

Fig. 1 is a schematic structural diagram of a depth measuring device provided by an embodiment of the present application;

Fig. 2 is a schematic diagram of image shapes obtained by two exposure methods provided by an embodiment of the present application;

Fig. 3 is a working sequence diagram of each module of a depth measurement device provided by an embodiment of the present application;

FIG. 4 is a working sequence diagram of each module of a master device provided by an embodiment of the present application;

Fig. 5 is a working sequence diagram of each module when a slave device is synchronously triggered according to an embodiment of the present application;

FIG. 6 is a working sequence diagram of each module when another slave device is synchronously triggered according to an embodiment of the present application;

Fig. 7 is a schematic diagram of a connection mode between multiple depth measuring devices provided by another embodiment of the present application;

Fig. 8 is a schematic diagram of another connection mode between multiple depth measuring devices provided by an embodiment of the present application;

Fig. 9 is a schematic structural diagram of a depth measurement system provided by an embodiment of the present application;

Fig. 10 is a schematic structural diagram of another depth measurement system provided by an embodiment of the present application.

detailed description

In the following description, specific details such as specific system structures and technologies are presented for the purpose of illustration rather than limitation, so as to thoroughly understand the embodiments of the present application. It will be apparent, however, to one skilled in the art that the present application may be practiced in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.

The term "and/or" used in the description of the present application and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

"One embodiment" or "some embodiments" or the like described in the specification of the present application means that a specific feature, structure or characteristic described in connection with the embodiment is included in one or more embodiments of the present application. Thus, appearances of the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in other embodiments," etc. in various places in this specification are not necessarily All refer to the same embodiment, but mean "one or more but not all embodiments" unless specifically stated otherwise. The terms "including", "comprising", "having" and variations thereof mean "including but not limited to", unless specifically stated otherwise. In addition, in the description of the present application, "plurality" means two or more.

In order to illustrate the technical solutions described in this application, specific examples are used below to illustrate.

FIG. 1 is a schematic structural diagram of a depth measuring device provided by an embodiment of the present application. As shown in Figure 1, the depth measurement device includes an RGB sensor 11, a TOF unit 12, a central processing unit 13 and a microcontroller (MicrocontrollerUnit, MCU) 14, wherein:

The central processing unit 13 is used to send the RGB control signal to the RGB sensor 11 to control the RGB sensor 11 to start working and synchronously send the TOF control signal to the TOF unit 12, and at the same time send the correction signal to the microcontroller 14;

The RGB sensor 11 is used to receive the RGB control signal, is also used to perform exposure according to the RGB control signal to collect the RGB image of the target scene, and is also used to simultaneously generate a feedback signal to the microcontroller 14; wherein the feedback signal includes a gate signal or vertical sync signal;

The microcontroller 14 is used to receive the correction signal and the feedback signal, and is also used to correct the feedback signal according to the correction signal to obtain a TOF trigger signal and send it to the TOF unit 12;

TOF unit 12, including a projection module and a TOF sensor, is used to receive a TOF trigger signal; under the synchronous trigger of the TOF trigger signal, the projection module is used to emit at least two different modulated light beams to the target scene, and the TOF sensor is exposed at least twice To collect different modulated light beams reflected back by the target scene to generate at least two frames of rawphase images, that is, one modulated light beam corresponds to at least one frame of rawphase images, and transmit the rawphase images to the central processing unit 13 according to the TOF control signal; wherein the rawphase images are The TOF sensor converts the collected optical signal into the original data of the digital signal, and the total time of the TOF sensor being exposed at least twice is aligned with the exposure time of the RGB sensor 11;

The central processing unit 13 is also used to synchronously receive RGB images and at least two frames of rawphase images according to the RGB control signal and the TOF control signal, and perform phase unwrapping and depth fusion calculation on the received at least two frames of rawphase images to obtain TOF depth images for synchronization Output RGB image and TOF depth image.

It should be noted that the time intervals for at least two exposures of the TOF sensor may be the same or different, but the time for one exposure of the RGB sensor 11 needs to be aligned with the total time for at least two exposures of the TOF sensor.

In one embodiment, the microcontroller 14 can be equipped with a real-time system for receiving various signals in real time to generate trigger signals to the sensors in real time. That is to say, the microcontroller 14 is a single-task processor, which is only used for processing synchronous trigger tasks in the application, so that the microcontroller 14 receives various signals and feeds back trigger signals in real time. Specifically, the microcontroller 14 receives the correction signal and the feedback signal in real time, and corrects the feedback signal according to the correction signal to generate a TOF trigger signal, which is used for the TOF sensor to start working.

Under the triggering of the TOF trigger signal, the synchronization method of the projection module and the TOF sensor in the TOF unit 12 includes: the TOF sensor in the TOF unit 12 receives the TOF trigger signal to generate a TOF exposure signal to control the exposure of the body, and synchronously outputs a projection trigger signal to The projection module is used to drive the projection module to emit modulated beams of different frequencies to the target scene. Preferably, the modulated light beam comprises continuous waves of different frequencies or pulsed light beams of different pulse periods.

In one embodiment, in order to ensure that the exposure time of the RGB sensor 11 is aligned with the time of at least two exposures of the TOF sensor, the images obtained by each exposure of the RGB sensor 11 or the TOF sensor can be given corresponding time stamps to synchronize according to the time stamps Send the RGB image and the rawphase image to the central processing unit 13; if a frame of RGB image and two frames of rawphase images are sent synchronously according to the time stamp, because the exposure time of one frame of RGB image is aligned with the total exposure time of two frames of rawphase images, by Therefore, when the exposure is completed, the corresponding time stamps are assigned to one frame of RGB image and two frames of rawphase images, so that two frames of rawphase images can be transmitted synchronously while one frame of RGB image is being transmitted.

In one embodiment, the exposure mode of the RGB sensor 11 or the TOF sensor is rolling exposure or global exposure. When the exposure method is rolling exposure, it is also necessary to ensure that the exposure center of the RGB sensor 11 is aligned with the exposure center of the TOF sensor for at least two exposures, so as to avoid poor alignment between the RGB image and the rawphase image. It should be noted that the exposure center can be understood as the center of the image obtained after the sensor is exposed.

Specifically, when the RGB sensor 11 is subjected to rolling shutter exposure, the obtained image is parallelogram; while when the TOF sensor is globally exposed, the obtained image is rectangular, as shown in FIG. 2 . It can be seen that the images obtained based on the two exposure methods are not aligned, so the central processing unit 13 needs to adjust the exposure centers of the images to align the exposure centers, so as to obtain aligned images. Specifically, the exposure centers of the exposure images obtained by different exposure methods are compared in advance, and if the exposure centers are not aligned, the offset is calculated, and then the image is adjusted according to the offset for image alignment and synchronously sent to the central processing unit 13 . It should be noted that the above calculated offset may be obtained in advance and saved for use in subsequent image synchronization.

In one embodiment, the TOF depth image obtained by phase unwrapping and depth fusion calculation based on at least two frames of rawphase images, the corresponding exposure time of the TOF depth image is a certain time between the exposure time periods of the two frames of rawphase images, so , in order to ensure the synchronous output effect of the RGB image and the TOF depth image, the central processing unit 13 also needs to correct the exposure center of the RGB image and the TOF depth image to align the exposure center.

In one embodiment, the TOF sensor includes at least one pixel, and each pixel includes at least 3 and more than 3 taps (charge accumulation elements, used to collect electrical signals generated by light beams reflected back from the target scene), and each tap is continuously The exposure moment within the frame period is fixed and constant. In order to unwrap the phases of different modulated light beams reflected back from the target scene to obtain the number of winding cycles, the TOF sensor needs to obtain at least one frame of rawphase images corresponding to different modulated light beams in consecutive different frame periods and transmit them to the central processing unit 13. The processor 13 performs unwrapping calculations according to the rawphase images corresponding to different modulated beams to obtain the number of wrapping cycles of different modulated beams; calculates the depth values based on different modulated beams by using the number of wrapping cycles, and finally assigns each depth corresponding to the different modulated beams The weight corresponding to the value is fused to obtain the TOF depth image.

In one embodiment, in order to improve the precision of the depth value, the taps included in each pixel preferably collect the amount of charge generated by the reflected light beam or the background light in a circular mode. Take the three taps a, b, and c as an example to realize the rotation mode. In the first frame period, the first tap, the second tap, and the third tap turn on the accumulated charge signal in turn; in the second frame period, the second tap, The third tap and the first tap turn on the accumulated charge signal sequentially. The TOF sensor obtains at least two frames of rawphase images according to the above method under different modulated light beams and transmits them to the central processing unit 13; taking two different modulated light beams as an example, the TOF sensor needs to obtain 4 frames of rawphase images at this time, that is, the TOF sensor Four exposures are required. The central processing unit 13 correspondingly superimposes the rawphase images obtained based on different frame periods according to different modulated beams to eliminate noise to obtain a high-precision rawphase image, thereby improving depth accuracy; for example, the same modulated beam is obtained based on different frame periods. Do superposition to remove noise. That is, when the projection module projects n different modulated beams, the TOF sensor can acquire at least n frames of rawphase images; in order to improve the depth accuracy, the TOF sensor can acquire at least 2n frames of rawphase images under n different modulated beams. Perform noise cancellation.

It should be noted that the rotation mode of the taps is not limited to the above-mentioned method, and it can also be the third tap, the first tap, and the second tap that are turned on in turn; when each pixel includes more than 3 taps, and so on, it will not be described here. .

For ease of understanding, the embodiments in FIG. 3 to FIG. 6 are illustrated by taking the TOF sensor acquiring at least two frames of rawphase images under at least two different modulated light beams and transmitting them to the central processing unit 13 to generate a frame of TOF depth map as an example.

Fig. 3 is according to the working timing chart of each module of the depth measurement device that the present application provides, in conjunction with Fig. 1 and Fig. 3 shown, when RGB sensor 11 receives RGB control signal, can produce a high-level RGB strobe signal (marked as RGB strobeout), the RGB strobe signal will be monitored by the real-time system carried by the microcontroller 14 . When the real-time system carried by the microcontroller 14 monitors the RGB strobe signal, the RGB strobe signal can be corrected according to the synchronously received correction signal from the central processing unit 13 to generate a high-level TOF trigger signal (denoted as TOFtriggle in) and send to TOF unit. Triggered by the TOF trigger signal, the TOF sensor in the TOF unit generates a TOF exposure signal (denoted as TOF exposure) and performs at least four exposures according to the TOF exposure signal to collect at least four frames of rawphase images, and synchronously sends the projection trigger signal to the projection module to emit at least two different modulated light beams towards the target scene. It should be noted that this embodiment only uses the RGB strobe signal as an example for illustration. In some other embodiments, when the RGB sensor 11 receives the RGB control signal, it can also generate a vertical synchronization (VSYNC) signal or generate other synchronization signals as feedback. Signal to achieve synchronous exposure, no limitation here.

The embodiment of the present application also provides a depth measurement system. The depth measurement system includes a plurality of the above-mentioned depth measurement devices (referred to as measurement devices), and any measurement device is selected as the master device (Master device), and the measurement device is in the master mode (Master mode) , the rest of the measurement devices are slave devices (Slave devices), that is to say, the rest of the measurement devices are in slave mode (Slave mode). Wherein, the master device can generate a synchronous trigger signal (denoted as Ext Triggle out) to the slave device to trigger the slave device to work synchronously, and the synchronous trigger signal can be generated by any one of the RGB sensor 11, the TOF sensor or the microcontroller 14 in the master device . Preferably, in the case where the microcontroller 14 is equipped with a real-time system, the microcontroller 14 in the master device can generate a synchronous trigger signal (marked as Ext Triggle out) to trigger the slave device to work synchronously, so as to ensure accurate and stable synchronization between multiple devices .

Specifically, FIG. 4 shows the working timing diagram of each module of the master device. In combination with FIG. 1 and FIG. , the microcontroller 14 can send a TOF trigger signal (TOF trigger in) to the TOF unit and at the same time send a synchronous trigger signal (ExtTriggle out) to the slave device to realize the synchronous operation of the master and slave devices. In one embodiment, in order to avoid interference between the master and slave devices, the rising edge of the synchronous trigger signal sent to the slave device corresponds to the TOF trigger signal received by the TOF unit of the master device or the TOF exposure signal corresponding to the first exposure of the TOF sensor There is a delay time ΔT between the rising edge of , and the configuration of ΔT needs to be greater than the exposure time of the single frame phase of the TOF sensor and less than the two-frame time interval, where the two-frame time interval depends on the number of master-slave devices.

Further, because this application needs to collect at least two frames of rawphase images for TOF depth image synthesis, therefore, based on the gap between two frames of rawphase images, multiple measurement devices are used to work together, which can maximize the use of the camera's own characteristics to achieve multi-machine synchronization and avoid Multi-device interference. Specifically, assuming that the maximum exposure time of the TOF sensor is 600us, and the idle time between each frame of IR images is 8ms, theoretically, within this idle time, about 13 measurement devices can be inserted to work together, or more or more It can be designed according to the exposure time of the actual TOF sensor, and there is no limitation here.

Fig. 5 is a working timing diagram of each module when the slave device is triggered synchronously. Specifically, as shown in FIG. 1 and FIG. 5 , the microcontroller 14 of the slave device receives the synchronous trigger signal (Ext Triggle in) sent by the master device to generate an RGB trigger signal (denoted as RGB trigger in, which can be selected for the above-mentioned RGB pass signal or vertical distribution signal) to make the RGB sensor 11 expose to obtain an RGB image, and generate a TOF trigger signal to the TOF unit 12 at the same time so that the TOF sensor in the TOF unit 12 generates a TOF exposure signal and performs at least four exposures to obtain at least four frames rawphase image.

On the basis of the timing diagram shown in Figure 5, Figure 6 shows a complete timing diagram for receiving an external trigger signal from the measuring device to control the operation of the machine. As shown in Figure 1 and Figure 6, the microcontroller 14 receives the external trigger signal (marked as Ext Triggle in) sent by the synchronization device to control the exposure of the RGB sensor 11 and the TOF sensor of the machine, and outputs an output signal (ExtTriggle out) to trigger The synchronous exposure of the next measurement device makes the exposure of other equipment and this machine not interfere.

It should be noted that the master and slave devices can also periodically generate corresponding trigger signals through their respective microcontrollers 14 and send them to the TOF unit and the RGB sensor 11 to achieve data synchronization between the master and slave devices.

In addition, in the case of multiple measuring devices, in addition to sending the TOF trigger signal and the RGB trigger signal to the TOF unit and the RGB sensor 11 at the same time, the microcontroller 14 of the slave device can also send the RGB trigger signal to the The RGB sensor 11 generates a TOF trigger signal according to the RGB trigger signal to realize synchronous output. It should be understood that, either way, the exposure center should be corrected to ensure synchronous output.

In one embodiment, the connection mode among multiple measurement devices may include a star mode or a chain mode. Specifically, any measuring device among the multiple measuring devices is selected as the master device, and the remaining measuring devices are slave measuring devices. Among them, as shown in Figure 7, the signal output terminal of the master device is connected to the signal input terminal of a slave device in the chain mode, and the signal output terminal of a slave device is connected to the signal input terminal of the next slave device, and so on until the end A signal output terminal of a slave device is connected to a signal input terminal of a master device, thereby forming a chain structure. As shown in FIG. 8 , in the star mode, the signal input terminals of each slave device are connected to the signal output terminals of the master device.

In some embodiments, the depth measurement system may further include a server in addition to the above-mentioned measurement devices, and each measurement device is connected to the server. Among them, the server is used as the host, and each measuring device is used as the slave device. This arrangement facilitates the transmission of images acquired by multiple measuring devices to the server for synchronous processing. The connection between the measurement device and the server can be a wired or wireless connection. As a possible implementation manner, as shown in FIG. 9 , the connection among multiple measurement devices is in a chain mode, and each measurement device is connected to the server. As another possible implementation manner, as shown in FIG. 10 , the connection among multiple measurement devices is in a star mode, and each measurement device is connected to the server.

In one embodiment, when the measurement device acquires each frame of image, it needs to stamp each frame of image with a time stamp. The depth measurement system implements the following methods to synchronize image time stamps or system time of multiple measurement devices, so as to realize image synchronization output. It should be noted that, when performing the following method, the server in the depth measurement system is a master (Master device), and each measuring device is a slave device (Slave device) of the master.

S1: The server sends its own time and a request signal to multiple measuring devices to request to record the time of each measuring device.

S2: Each measuring device returns its own time, and sets its own time as the time T sent by the server.

S3: The server records the RTT time (i.e. the round-trip time) of multiple measuring devices; for the measuring device with abnormal RTT time deviation, repeat steps S1, S2 and S3; wherein, the RRT time is the server sending its own time to the measuring device, and the measuring device is set The own time is the time sent by the server and the time used to send the setting success signal to the server.

S4: The server sends RTT to the measuring device, and then the measuring device sets its own time as T+RTT/2, where only the time stamp of the written image is modified or the system time of the measuring device is directly changed.

In some embodiments, the system time of each measuring device can be time calibrated periodically.

In some embodiments, further, the system time of each measuring device may be corrected according to the trigger signal received by each measuring device to achieve synchronization. More specifically include:

S10: The microcontroller in each measuring device receives the rising edge of the synchronous trigger signal (Ext trigger in), normally triggers the corresponding frame synchronization function, and records the system time at this time as Ts;

S20: Polling to read the level of the current synchronous trigger signal of each measuring device; calculate the duration of the high level t ms (unit: millisecond); if t is greater than the preset threshold, it is judged as a time synchronous signal, and if |T0+( n-1)×NT-Ts|>NT, then set n=floor[(Ts-T0+NT/2)/NT]; then configure the system time of each measuring device as T0+(n-1)×NT+ t; where, floor[] means rounding down, the preset threshold is preferably 10, and T0 is the initial start time of the system (which can be uniformly defined as 0 minutes 0 seconds 0 milliseconds 0 microseconds on January 1, 1970), n represents the number of received time synchronization signals.

It should be noted that this embodiment multiplexes Ext trigger in and Ext trigger out signals. When the system receives the time synchronization signal of Ext trigger in, the device configures the system time as T0+NT; N is the time synchronization of multiple measurement devices Cycle length, the unit can be set to 1/30 second or frame time, N should not be too large or too small, the recommended interval can be [1000, 10000].

In the embodiments of the present application, exposure synchronization is achieved through hardware and software synchronization, which improves the image synchronization effect. In addition, in some embodiments, the microcontroller is equipped with a real-time system, which can perform real-time feedback on various data and signals in real time, and the control accuracy can reach the level of microseconds, ensuring stable and accurate synchronization effects.

The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: it can still apply to the foregoing embodiments Modifications to the technical solutions recorded, or equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of each embodiment of the application, and should be included in this application. within the scope of protection.

Claims (13)

1. A depth measurement device, comprising: an RGB sensor, a TOF unit including a projection module and a TOF sensor, a central processing unit and a microcontroller,

the central processing unit is used for respectively generating an RGB control signal and a TOF control signal to the RGB sensor and the TOF unit and simultaneously sending a correction signal to the microcontroller;

the RGB sensor is used for collecting RGB images according to the RGB control signals, sending the RGB images to the central processing unit, and simultaneously generating gating signals or vertical synchronizing signals to be fed back to the microcontroller;

the microcontroller is used for correcting the gating signal or the vertical synchronization signal according to the correction signal to generate a TOF trigger signal so as to trigger the TOF unit to start working;

the TOF unit is used for transmitting at least two different modulation light beams to a target scene by the projection module under the synchronous triggering of the TOF trigger signal, and the TOF sensor is used for collecting the light beams reflected back by the target scene to generate at least two frames of rapphase images corresponding to the different modulation light beams and transmitting the rapphase images to the central processing unit according to the TOF control signal;

the central processing unit is further configured to perform phase unwrapping and depth fusion calculation on the at least two frames of rapphase images to obtain a TOF depth image, so as to synchronously output the RGB image and the TOF depth image.

2. The depth measurement apparatus of claim 1, wherein an exposure time for the RGB sensor to acquire an RGB image is aligned with a total exposure time for the TOF sensor to acquire the at least two frames of the rapphase image.

3. The depth measurement device of claim 1 or 2, wherein the microcontroller carries a real-time system for receiving signals in real-time to generate trigger signals to the TOF sensor or the RGB sensor in real-time.

4. The depth measurement apparatus of claim 1 or 2, wherein the central processor is further configured to align the exposure centers of the RGB image and the at least two swathhase images; and the central processor is also used for aligning the exposure centers of the TOF depth image and the RGB image and then synchronously outputting the aligned images.

5. A depth measurement system, comprising: a plurality of depth measuring devices as claimed in any one of claims 1 to 4.

6. The depth measurement system of claim 5, wherein any of the depth measurement devices is a master device, and the remaining depth measurement devices are slave devices, the master device being configured to generate a synchronization trigger signal to a microcontroller of the slave device to trigger the slave device to operate synchronously with the microcontroller of the slave device; wherein the synchronization trigger signal is generated by any one of the RGB sensor, TOF sensor, or the microcontroller in the master device.

7. The depth measurement system of claim 5, wherein any one of the depth measurement devices is a master device, and the remaining depth measurement devices are slave devices, and the master device and the slave devices are timed by the respective microcontrollers to generate synchronization trigger signals to be sent to the respective RGB sensors and TOF sensors to achieve data synchronization.

8. The depth measurement system of any one of claims 5 to 7, wherein the plurality of depth measurement devices are connected in a manner comprising a star mode or a chain mode.

9. The depth measurement system of any of claims 6 to 7, wherein a rising edge of the synchronized trigger signal sent to the slave device differs by a delay time Δ T from a rising edge of the TOF trigger signal received by the TOF unit of the master device or from a TOF exposure signal of a first exposure of the TOF sensor.

10. The depth measurement system of any of claim 9, further comprising a server, wherein a plurality of the depth measurement devices are each connected to the server, wherein the server is a master, wherein a plurality of the depth measurement devices are slaves to the master, and wherein the master receives the RGB image and the depth image output by each of the depth measurement devices.

11. The depth measurement system of claim 10, wherein the system time or image time stamp of each of the depth measurement devices is calibrated to achieve image synchronization.

12. The depth measurement system of claim 11, wherein calibrating the system time for each of the depth measurement devices comprises:

the server is used for sending self time T and a request signal to the depth measuring devices to request to record the time of each depth measuring device;

each depth measuring device is used for returning self time according to the request signal and setting the self time as self time T issued by the server;

the server is further used for recording round trip times RTT of a plurality of depth measuring devices and sending the round trip times RTT to the depth measuring devices so that the depth measuring devices set the self time to be T + RTT/2, wherein the round trip time is the time when the server sends the self time to the depth measuring devices;

the depth measuring device sets self time as the time transmitted by the server and sends a setting success signal to the time used by the server.

13. The depth measurement system of claim 11, wherein calibrating the system time for each of the depth measurement devices comprises:

recording the system time Ts when the microcontroller in each measuring device normally triggers the corresponding frame synchronization function after receiving the rising edge of the synchronization trigger signal;

polling and reading the current synchronous trigger signal level of each measuring device, and calculating the duration time t of high level; if T is larger than a preset threshold, judging the signal to be a time synchronization signal, and if | T0+ (N-1) × NT-Ts | > NT, setting N = floor [ (Ts-T0 + NT/2)/NT ], wherein T0 is the initial starting time of the system, and N is the period length of time synchronization of each measuring device;

configuring the system time of each measuring device to be T0+ (n-1) multiplied by NT + T; where n represents the number of received time synchronization signals.

Images