[go: up one dir, main page]

US20180306905A1 - Method of Providing a Dynamic Region of interest in a LIDAR System - Google Patents

Method of Providing a Dynamic Region of interest in a LIDAR System Download PDF

Info

Publication number
US20180306905A1
US20180306905A1 US15/492,771 US201715492771A US2018306905A1 US 20180306905 A1 US20180306905 A1 US 20180306905A1 US 201715492771 A US201715492771 A US 201715492771A US 2018306905 A1 US2018306905 A1 US 2018306905A1
Authority
US
United States
Prior art keywords
region
interest
sampling resolution
light beam
spatial sampling
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US15/492,771
Inventor
Ronald A. Kapusta
Benjamin Luey
Harvy Weinberg
Scott R. Davis
Michael H. Anderson
Scott D. Rommel
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Analog Devices Inc
Original Assignee
Analog Devices Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Analog Devices Inc filed Critical Analog Devices Inc
Priority to US15/492,771 priority Critical patent/US20180306905A1/en
Priority to US16/606,721 priority patent/US20200150228A1/en
Priority to DE112017007467.3T priority patent/DE112017007467T5/en
Priority to PCT/US2017/065392 priority patent/WO2018194721A1/en
Priority to CN201780089765.9A priority patent/CN110537108B/en
Assigned to ANALOG DEVICES, INC. reassignment ANALOG DEVICES, INC. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: DAVIS, SCOTT ROBERT, LUEY, Benjamin, ANDERSON, MICHAEL HOWARD, ROMMEL, SCOTT D., WEINBERG, HARVEY, KAPUSTA, RONALD A.
Publication of US20180306905A1 publication Critical patent/US20180306905A1/en
Priority to US18/217,366 priority patent/US20240069172A1/en
Abandoned legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G01MEASURING; TESTING
    • G01SRADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S7/00Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
    • G01S7/48Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00
    • G01S7/481Constructional features, e.g. arrangements of optical elements
    • G01S7/4817Constructional features, e.g. arrangements of optical elements relating to scanning
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01SRADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/02Systems using the reflection of electromagnetic waves other than radio waves
    • G01S17/50Systems of measurement based on relative movement of target
    • G01S17/58Velocity or trajectory determination systems; Sense-of-movement determination systems
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01SRADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/88Lidar systems specially adapted for specific applications
    • G01S17/89Lidar systems specially adapted for specific applications for mapping or imaging
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01SRADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/88Lidar systems specially adapted for specific applications
    • G01S17/93Lidar systems specially adapted for specific applications for anti-collision purposes
    • G01S17/931Lidar systems specially adapted for specific applications for anti-collision purposes of land vehicles
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01SRADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S7/00Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
    • G01S7/48Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00
    • G01S7/4808Evaluating distance, position or velocity data
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01SRADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S7/00Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
    • G01S7/48Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00
    • G01S7/483Details of pulse systems

Definitions

  • the present disclosure relates to systems and methods for providing a dynamic region of interest in a LIDAR system.
  • Certain lidar systems include a laser that can be discretely scanned over a series of points in a target region and a detector that can detect a reflected portion of the discretely scanned laser, such as to provide an image of the target region.
  • An angular resolution of the lidar system can depend on the number of points that can be scanned by the laser within a field of view of the lidar system.
  • a large field of view can include a ⁇ 30° horizontal field of view and a ⁇ 0.6° vertical field of view
  • a very fine angular resolution can include a 0.1° horizontal angular resolution and a 0.2° vertical angular resolution
  • the lidar system can include a 20 Hz frame update rate.
  • the lidar system can scan 540,000 points per second, and can correspond to an average laser power of 720 mW for a laser outputting 1 ⁇ J per pulse.
  • An average laser power of 720 mW can be high enough to cause considerable thermal design challenges.
  • the average laser power can be much higher. For example, if five pulses per point in the 2D field of view are used, the average laser power can be 3.6 W.
  • the inventors have recognized, among other things, that it is possible to reduce a number of points scanned per lidar image, such as by providing varying spatial resolution in the lidar images, such as to overcome difficulties with thermal management and receive-side electronics. Further features of the disclosure are provided in the appended claims, which features may optionally be combined with each other in any permutation or combination, unless expressly indicated otherwise elsewhere in this document.
  • the disclosure can feature a method for providing a dynamic region of interest, such as in a lidar system.
  • the method can include scanning a light beam over a field of view, such as to capture a first lidar image.
  • the method can also include identifying a first object, such as within the captured first lidar image.
  • the method can also include selecting a first region of interest, such as within a field of view that contains at least a portion of the identified first object.
  • the method can also include capturing a second lidar image, where capturing the second lidar image can include scanning the light beam over the first region of interest, such as at a first spatial sampling resolution and scanning the light beam over the field of view outside of the first region of interest, such as at a second spatial sampling resolution, wherein the second sampling resolution can be different than the first spatial sampling resolution.
  • the second sampling resolution can be less than the first spatial sampling resolution.
  • the method can also include identifying a second object, such as can be outside of the first region of interest, selecting a second region of interest that can contain at least a portion of the identified second object, and capturing a third lidar image, where capturing the third lidar image can include scanning the light beam over the first region of interest and the second region of interest at the first spatial sampling resolution and scanning the light beam over the field of view outside of both the first region of interest and the second region of interest at a third spatial sampling resolution, where the third sampling resolution can be different than the second spatial sampling resolution.
  • the third sampling resolution can be less than the second spatial sampling resolution.
  • the second object can be identified outside of the first region of interest using the captured second lidar image.
  • the method can also include detecting a movement of the identified first object, and adjusting a characteristic of the first region of interest, such as in response to the detected movement of the identified first object.
  • the method can also include adjusting a size of the first region of interest, such as in response to the detected movement of the identified first object.
  • the method can also include adjusting a size and position of the first region of interest in response to the detected movement of the identified first object.
  • the method can also include detecting a change in the size of the identified first object and adjusting a size of the first region of interest, such as to accommodate the detected change in size of the identified first object.
  • the method can also include reducing a second spatial sampling resolution, such as in response to an increase in the size of the first region of interest.
  • the method can also include increasing a second spatial sampling resolution, such as in response to a decrease in the size of the first region of interest.
  • the method can also include scanning the light beam over the field of view to capture successive lidar images, wherein the region of interest is capable of being adjusted after the capture of each successive lidar image.
  • the method can also include identifying a second object outside of the first region of interest, selecting a second region of interest that can contain at least a portion of the identified second object, and capturing a third lidar image, where capturing the third lidar image can include scanning the light beam over the first region of interest at the first spatial sampling resolution, scanning the light beam over the second region of interest at a third spatial sampling resolution, and scanning the light beam over the field of view outside of the first region of interest and the second region of interest, at the second spatial sampling resolution, wherein the third sampling resolution can be different than the first spatial sampling resolution.
  • the third sampling resolution can be less than the second spatial sampling resolution.
  • the second object can be identified outside of the first region of interest by using the captured second lidar image. Identifying a first object within the captured first lidar image can include detecting at least one edge of the first object. Identifying a first object within the captured first lidar image can include detecting at least one lane marker.
  • the disclosure can feature a system for providing a dynamic region of interest in a lidar system.
  • the system can include a laser configured to emit a light beam, such as towards a target region.
  • the system can also include control circuitry configured to instruct an optical system to scan the light beam over the target region.
  • the system can also include an optical system having a field of view and can configured to direct a portion of the light beam received from the target region.
  • the system can also include a photodetector configured to receive the portion of the light beam directed from the optical system, such as to form a first lidar image.
  • the system can also include detection circuitry that can be configured to identify a first object within the first lidar image.
  • the control circuitry can be further configured to select a first region of interest within the field of view that can contain at least a portion of the identified first object, instruct the optical system to scan the light beam over the first region of interest at a first spatial sampling resolution, and instruct the optical system to scan the light beam over the field of view outside of the first region of interest at a second spatial sampling resolution that can be different than the first spatial sampling resolution.
  • the first spatial sampling resolution can be less than the second spatial sampling resolution.
  • the photodetector can be further configured to receive a corresponding portion of the light beam to form a second lidar image.
  • the detection circuitry can be further configured to identify a second object outside of the first region of interest in the second lidar image.
  • the control circuitry can be further configured to select a second region of interest that can contain a portion of the identified second object, instruct the optical system to scan the light beam over the first region of interest and the second region of interest at the first spatial sampling resolution, and instruct the optical system to scan the light beam over the field of view outside of both the first region of interest and the second region of interest at a third spatial sampling resolution that can be different than the first spatial sampling resolution.
  • the third spatial sampling resolution can be less than the first spatial sampling resolution.
  • the detection circuitry can be further configured to detect a movement of the identified first object and the control circuitry can be configured to adjust a characteristic of the first region of interest, such as in response to the detected movement of the identified first object.
  • the control circuitry can be further configured to adjust a size of the first region of interest, such as in response to the detected movement of the identified first object.
  • the control circuitry can be further configured to adjust a size and position of the first region of interest, such as in response to the detected movement of the identified first object.
  • the detection circuitry can be further configured to detect a change in the size of the identified first object and the control circuitry can be further configured to adjust a size of the first region of interest, such as to accommodate the detected change in size of the identified first object.
  • the control circuitry can be further configured to reduce the second spatial sampling resolution, such as in response to an increase in the size of the first region of interest.
  • the control circuitry can be further configured to increase the second spatial sampling resolution, such as in response to a decrease in the size of the first region of interest.
  • the disclosure can feature a system for providing a dynamic region of interest in a lidar system.
  • the system can include a means for scanning a light beam over a field of view, such as to capture a first lidar image.
  • the means for scanning can include control circuitry and a scanning laser, such as control circuitry 104 and scanning laser 106 as shown in FIG. 1A .
  • the system can also include a means for identifying a first object within the captured first lidar image.
  • the means for identifying can include detection circuitry, such as detection circuitry 124 as shown in FIG. 1A .
  • the system can also include a means for selecting a first region of interest within the field of view that contains at least a portion of the identified first object.
  • the means for selecting can include control circuitry, such as control circuitry 104 as shown in FIG. 1A .
  • the system can also include a means for capturing a second lidar image, where capturing the second lidar image can includes scanning the light beam over the first region of interest at a first spatial sampling resolution and scanning the light beam over the field of view outside of the first region of interest at a second spatial sampling resolution, wherein the second sampling resolution can be different than the first spatial sampling resolution. In an example, the second sampling resolution can be less than the first spatial sampling resolution.
  • the means for capturing a second lidar image can include control circuitry and a scanning laser, such as control circuitry 104 and scanning laser 106 as shown in FIG. 1A .
  • FIG. 1A illustrates a diagram of a lidar system.
  • FIGS. 1B-1D illustrate examples of a frame in a lidar system.
  • FIGS. 2A-2C illustrate an example of a sequence of frames in a lidar system.
  • FIGS. 3A-3B illustrate an example of a sequence of frames in a lidar system.
  • FIG. 4 illustrates a method of operation of a lidar system.
  • FIG. 1A shows an example of a lidar system 100 .
  • the lidar system 100 can include control circuitry 104 , a scanning laser 108 , an optical system 116 , a photosensitive detector 120 , and detection circuitry 124 .
  • the control circuitry 104 can be connected to the scanning laser 108 and the detection circuitry 124 .
  • the photosensitive detector 120 can be connected to the detection circuitry 124 .
  • the control circuitry 104 can provide instructions to the scanning laser 108 , such as to cause the scanning laser to scan a light beam over a target region 112 .
  • the scanning laser 108 can include laser that can emit a light beam and an optical system, such as an electro-optic waveguide.
  • the electro-optic waveguide can adjust an angle of the light beam based on the received instructions from the control circuitry 104 .
  • the target region 112 can correspond to a field of view of the optical system 116 .
  • the scanning laser 108 can scan a light beam over the target region 112 in a series of scanned points 114 .
  • the optical system 116 can receive at least a portion of the light beam from the target region 112 and can image the scanned points 114 onto the photosensitive detector 120 (e.g., a CCD).
  • the detection circuitry 124 can receive and process the image of the scanned points from the photosensitive detector 120 , such as to form a frame.
  • control circuitry 104 can select a region of interest that is a subset of the field of view of the optical system and instruct the scanning laser to scan over the region of interest.
  • detection circuitry 124 can include circuitry for digitizing the received image.
  • the lidar system 100 can be installed in an automobile, such as to facilitate an autonomous self-driving automobile.
  • FIG. 1B illustrates an example of a frame 130 corresponding to a 2D image, such as that captured with lidar system 100 .
  • the frame 130 can include a collection of scanned points 114 .
  • the scanned points 114 can be regularly spaced by a distance d, along a grid.
  • the spacing d of the scanned points 114 can determine the angular resolution of a lidar system, such as the lidar system 100 .
  • a larger spacing can correspond to a coarser angular resolution and a smaller spacing can correspond to a finer angular resolution.
  • the frame 130 can include a region of interest 135 that corresponds to a field of view of the optical system 116 (e.g., all points within the field of view can be scanned).
  • FIG. 1C illustrates an example of a frame 130 , such as that captured with lidar system 100 .
  • the frame 130 can include a collection of scanned points 114 .
  • the scanned points 114 can be regularly spaced along a grid.
  • the spacing of the scanned points 114 can determine the angular resolution of a lidar system, such as the lidar system 100 .
  • a larger spacing can correspond to a coarser angular resolution and a smaller spacing can correspond to a finer angular resolution.
  • the frame 130 can include a region of interest 135 that corresponds to a subset of a field of view of the optical system 116 .
  • the scanning laser 108 scan a light beam over the region of interest 135 , but not other points within the field of view of the lidar system 100 (e.g., only a fraction of points within the field of view can be scanned).
  • FIG. 1D illustrates an example of a frame 130 , such as that captured with lidar system 100 .
  • the frame 130 can include a collection of scanned points 114 .
  • the scanned points 114 can be regularly spaced along a grid.
  • the spacing of the scanned points 114 can determine the angular resolution of a lidar system, such as the lidar system 100 .
  • a larger spacing can correspond to a coarser angular resolution and a smaller spacing can correspond to a finer angular resolution.
  • the frame 130 can include a region of interest 135 that corresponds to a subset of a field of view of the optical system 116 .
  • the scanning laser 108 scan a light beam over the region of interest 135 , but not other points within the field of view of the lidar system 100 (e.g., only a fraction of points within the field of view can be scanned).
  • FIGS. 2A-2C illustrate an example of a sequence of frames 230 - 232 where the scanned points can be irregularly spaced across a field of view of the optical system 116 .
  • the first frame 230 as illustrated in FIG. 2A can include a first region of interest 235 .
  • the first region of interest 235 can include a collection of regularly spaced scanned points.
  • the scanned points in the first region of interest 235 can correspond to a first angular resolution. Outside of the first region of interest 235 , the scanned points can be regularly spaced with a larger spacing than the first region of interest 235 , corresponding to a coarser angular resolution than in the first region of interest 235 .
  • Every third column in every other row can be scanned as illustrated in FIG. 2A .
  • other patterns of scanning can be utilized outside of the first region of interest 235 .
  • a scanning pattern outside of the first region of interest can include every second column, in every third row.
  • the scanning pattern outside of the first region of interest 235 can include every n th column in every m th row.
  • the first region of interest 235 can be dynamically adjusted on a frame-to-frame basis, such as based on an analysis of the frame by the detection circuitry 124 . In the example shown in FIG.
  • the first frame can accommodate up to 144 scanned points
  • the first region of interest 235 can include 36 scanned points
  • the portion of the frame outside of the region of interest can include 17 scanned points, for a total of 53 scanned points out of a total of 144 possible scanned points.
  • the second frame 231 as illustrated in FIG. 2B can include a second region of interest 236 .
  • the second region of interest 236 can be determined based on an object detected in the first frame 230 .
  • the second region of interest 236 can be smaller than the first region of interest 235 and can include a collection of regularly spaced scanned points.
  • the scanned points in the second region of interest 236 can correspond to a first angular resolution.
  • the scanned points can be regularly spaced with a larger spacing than the second region of interest 236 , corresponding to a coarser angular resolution than in the second region of interest 236 .
  • the second region of interest 236 can be dynamically adjusted on a frame-to-frame basis, such as based on an analysis of the first frame 230 by the detection circuitry 124 .
  • a total number of scanned points in the frame 231 can be smaller than the total number of scanned points in the frame 230 . In the example shown in FIG.
  • the second frame can accommodate up to 144 scanned points
  • the second region of interest 236 can include 12 scanned points
  • the portion of the frame outside of the region of interest can include 23 scanned points, for a total of 45 scanned points out of a total of 144 possible scanned points.
  • the third frame 232 as illustrated in FIG. 2C can include a third region of interest 237 and a region of disinterest 240 .
  • the third region of interest 237 can be determined based on an object detected in the second frame 231 .
  • the third region of interest 237 can be the same size as the second region of interest 236 and can include a collection of regularly spaced scanned points.
  • the scanned points in the third region of interest 237 can correspond to a first angular resolution.
  • the scanned points can be regularly spaced with a larger spacing than the third region of interest 237 , corresponding to a coarser angular resolution than in the third region of interest 237 .
  • the third region of interest 237 can be dynamically adjusted on a frame-to-frame basis, such as based on an analysis of the second frame 231 by the detection circuitry 124 .
  • the scanned points can be regularly spaced with a larger spacing than outside of the third region of interest 237 .
  • no points are scanned in the region of disinterest 240 .
  • the region of disinterest can correspond to an area in the frame that includes a quasi-stationary object.
  • the size and location of the region of disinterest 240 can be determined based on the identification of one or more objects within the second frame 231 . Similar to the regions of interest 235 - 237 , the region of disinterest 240 can be dynamically adjusted on a frame-to-frame basis. In an example where the third region of interest 236 can be the same size as the second region of interest 235 , a total number of scanned points in the third frame 232 can be smaller than the number of scanned points in the second frame 231 . In the example shown in FIG.
  • the third frame can accommodate up to 144 scanned points
  • the third region of interest 237 can include 12 scanned points
  • the region of disinterest 240 can exclude up to 20 scanned points
  • the portion of the frame outside of the region of interest can include 18 scanned points, for a total of 30 scanned points out of a total of 144 possible scanned points.
  • FIGS. 3A-3B illustrate a sequence of frames 330 - 331 , such as can be collected by a lidar system in an automobile where the scanned points can be irregularly spaced across a field of view that can include a road and associated landscape.
  • the first frame 330 as illustrated in FIG. 3A can include a first region of interest 335 , a second region of interest 345 , and a region of disinterest 340 .
  • the first region of interest 335 can include a collection of regularly spaced scanned points.
  • the scanned points in the first region of interest 335 can correspond to a first angular resolution.
  • the first region of interest 335 can correspond to a portion of a road having at least one lane, where each lane can be approximately 4 meters wide.
  • a width of the first region of interest 335 can be selected, such as to accommodate the width of three lanes (e.g., a lane that an automobile is driving in and additionally, one lane on either side of the lane that the automobile is driving in).
  • the width of the first region of interest 335 can be sized to accommodate a radius of curvature of the road. For example, at a relatively high speed of 150 km/hr, a radius of curvature of the road can be approximately 1 km, corresponding to a road that can be 4° off of a longitudinal axis at a distance of 150 m.
  • a radius of curvature of the road can be approximately 200 m, corresponding to a road that can be 10° off of a longitudinal axis at a distance of 60 m.
  • the first region of interest 335 can extend 20° in a horizontal direction, and to account for a vertical extent of other automobiles (e.g. an automobile can extend 4 m and the region of interest can be sized to accommodate twice the vehicle height at a distance of 60 m), the first region of interest can extend 4° in a vertical direction.
  • the second region of interest 345 can be smaller than the first region of interest 335 and can include a collection of regularly spaced scanned points.
  • the scanned points in the second region of interest 345 can correspond to the first angular resolution.
  • the second region of interest 345 can correspond to a portion of a lane marker on a road.
  • the scanned points can be regularly spaced with a larger spacing than the first region of interest 335 and the second region of interest 345 , corresponding to a coarser angular resolution than in the first region of interest 335 or the second region of interest 345 .
  • every m th column in every n th row can be scanned with the exception of the region of disinterest 340 .
  • the region of disinterest 340 can designate an area within the frame 330 where the scanned points can be regularly spaced with a larger spacing than in the first region of interest 335 , the second region of interest 345 , or the region outside of the first region of interest 335 and the second region of interest 345 . In an example, no points are scanned within the region of disinterest 340 .
  • the region of disinterest 340 can include fixed road infrastructure, such as guard rails and the road shoulder.
  • the region of disinterest can include a road surface near an automobile.
  • the region of disinterest 340 can correspond to objects such as trees, rocks, or mountains within a field of view of a lidar system, such as lidar system 100 .
  • the first region of interest 335 , the second region of interest 345 , and the region of disinterest 340 can be adjusted dynamically, such as based on the motion of objects within the field of view of the lidar system 100 .
  • FIG. 3B illustrates a second frame 331 where the regions of interest and disinterest have been dynamically updated, such as based on a change in the relative position of the road and lane markers within the field of view of the lidar system 100 .
  • the second frame 331 as illustrated in FIG. 3 b can include a first region of interest 336 , a second region of interest 346 , and a region of disinterest 341 .
  • the first region of interest 336 can include a collection of regularly spaced scanned points.
  • the scanned points in the first region of interest 335 can correspond to a first angular resolution.
  • the first region of interest 335 can correspond to a portion of a road having at least one lane, where each lane can be approximately 4 meters wide.
  • a width of the first region of interest 335 can be selected, such as to accommodate the width of three lanes (e.g., a lane that an automobile is driving in and additionally, one lane on either side of the lane that the automobile is driving in).
  • the width of the first region of interest 335 can be sized to accommodate a radius of curvature of the road.
  • a radius of curvature of the road can be approximately 1 km, corresponding to a road that can be 40 off of a longitudinal axis at a distance of 150 m.
  • a radius of curvature of the road can be approximately 200 m, corresponding to a road that can be 10° off of a longitudinal axis at a distance of 60 m.
  • the first region of interest 335 can extend 20° in a horizontal direction, and to account for a vertical extent of other automobiles (e.g.
  • an automobile can extend 4 m and the region of interest can be sized to accommodate twice the vehicle height at a distance of 60 m), the first region of interest can extend 4° in a vertical direction.
  • the second region of interest 346 can be smaller than the first region of interest 336 and can include a collection of regularly spaced scanned points. The scanned points in the second region of interest 346 can correspond to the first angular resolution.
  • the second region of interest 346 can correspond to a portion of a lane marker on a road.
  • the scanned points can be regularly spaced with a larger spacing than the first region of interest 336 and the second region of interest 346 , corresponding to a coarser angular resolution than in the first region of interest 336 or the second region of interest 346 .
  • every m th column in every n th row can be scanned with the exception of the region of disinterest 341 .
  • the region of disinterest 341 can designate an area within the frame 331 where the scanned points can be regularly spaced with a larger spacing than in the first region of interest 336 , the second region of interest 346 , or the region outside of the first region of interest 336 and the second region of interest 346 . In an example, no points are scanned within the region of disinterest 341 .
  • the region of disinterest 341 can correspond to objects such as trees, rocks, or mountains within a field of view of a lidar system, such as lidar system 100 .
  • FIG. 4 illustrates a method of adjusting a field of view in a lidar system, such as lidar system 100 .
  • a light beam such as can be emitted by the scanning laser 108 can be scanned over a target region within a field of view of an optical system, such as optical system 116 and a first image can be captured by a photosensitive detector, such as the photosensitive detector 120 (step 410 ).
  • a first object can be identified within the first image by detection circuitry, such as the detection circuitry 124 (step 420 ).
  • Control circuitry, such as control circuitry 104 can select a first region of interest that includes at least a portion of the identified first object (step 430 ).
  • a second lidar image can then be captured (step 440 ).
  • the capturing of the second lidar image can include steps 450 and 460 described below.
  • a light beam, such as can be emitted by the scanning laser 108 can be scanned over the first region of interest at a first spatial sampling resolution (step 450 ).
  • a light beam, such as can be emitted by the scanning laser 108 can be scanned over the field of view outside of the first region of interest at a second spatial sampling resolution, wherein the second sampling resolution can be less than the first spatial sampling resolution (step 460 ).
  • detection circuitry such as the detection circuitry 124 can identify a second object outside of the first region of interest in the captured second lidar image.
  • Control circuitry, such as control circuitry 104 can select a second region of interest that can contain at least a portion of the identified second object.
  • a third lidar image can then be captured, where capturing the third lidar image can include scanning a light beam, such as that emitted by the scanning laser 108 , over both the first and second regions of interest at the first spatial sampling resolution and over a field of view outside of both the first and second regions of interest at a third spatial sampling resolution that can be less than the second spatial sampling resolution.
  • the control circuitry 104 can receive external data, such as from an inertial sensor, GPS, radar, camera, or wheel speed sensor data, and in response to the received external data, the control circuitry 104 can adjust a size or position of the first region of interest.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • General Physics & Mathematics (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Electromagnetism (AREA)
  • Optical Radar Systems And Details Thereof (AREA)
  • Traffic Control Systems (AREA)

Abstract

A system and method for providing a dynamic region of interest in a lidar system can include scanning a light beam over a field of view to capture a first lidar image, identifying a first object within the captured first lidar image, selecting a first region of interest within the field of view that contains at least a portion of the identified first object, and capturing a second lidar image, where capturing the second lidar image includes scanning the light beam over the first region of interest at a first spatial sampling resolution, and scanning the light beam over the field of view outside of the first region of interest at a second spatial sampling resolution, wherein the second sampling resolution is less than the first spatial sampling resolution.

Description

    FIELD OF THE DISCLOSURE
  • The present disclosure relates to systems and methods for providing a dynamic region of interest in a LIDAR system.
  • BACKGROUND
  • Certain lidar systems include a laser that can be discretely scanned over a series of points in a target region and a detector that can detect a reflected portion of the discretely scanned laser, such as to provide an image of the target region. An angular resolution of the lidar system can depend on the number of points that can be scanned by the laser within a field of view of the lidar system.
  • SUMMARY OF THE DISCLOSURE
  • In certain lidar systems with a large field of view and a very fine angular resolution, thermal management of the lidar system and receive-side analog-to-digital conversion circuitry can present design challenges. In an example, a large field of view can include a ±30° horizontal field of view and a ±0.6° vertical field of view, a very fine angular resolution can include a 0.1° horizontal angular resolution and a 0.2° vertical angular resolution, and the lidar system can include a 20 Hz frame update rate. In such an example, the lidar system can scan 540,000 points per second, and can correspond to an average laser power of 720 mW for a laser outputting 1 μJ per pulse. An average laser power of 720 mW can be high enough to cause considerable thermal design challenges. In an example where multiple laser pulses can be used for each point in a 2D field of view, the average laser power can be much higher. For example, if five pulses per point in the 2D field of view are used, the average laser power can be 3.6 W. The inventors have recognized, among other things, that it is possible to reduce a number of points scanned per lidar image, such as by providing varying spatial resolution in the lidar images, such as to overcome difficulties with thermal management and receive-side electronics. Further features of the disclosure are provided in the appended claims, which features may optionally be combined with each other in any permutation or combination, unless expressly indicated otherwise elsewhere in this document.
  • In an aspect, the disclosure can feature a method for providing a dynamic region of interest, such as in a lidar system. The method can include scanning a light beam over a field of view, such as to capture a first lidar image. The method can also include identifying a first object, such as within the captured first lidar image. The method can also include selecting a first region of interest, such as within a field of view that contains at least a portion of the identified first object. The method can also include capturing a second lidar image, where capturing the second lidar image can include scanning the light beam over the first region of interest, such as at a first spatial sampling resolution and scanning the light beam over the field of view outside of the first region of interest, such as at a second spatial sampling resolution, wherein the second sampling resolution can be different than the first spatial sampling resolution. In an example, the second sampling resolution can be less than the first spatial sampling resolution. The method can also include identifying a second object, such as can be outside of the first region of interest, selecting a second region of interest that can contain at least a portion of the identified second object, and capturing a third lidar image, where capturing the third lidar image can include scanning the light beam over the first region of interest and the second region of interest at the first spatial sampling resolution and scanning the light beam over the field of view outside of both the first region of interest and the second region of interest at a third spatial sampling resolution, where the third sampling resolution can be different than the second spatial sampling resolution. In an example, the third sampling resolution can be less than the second spatial sampling resolution. In an example, the second object can be identified outside of the first region of interest using the captured second lidar image. The method can also include detecting a movement of the identified first object, and adjusting a characteristic of the first region of interest, such as in response to the detected movement of the identified first object. The method can also include adjusting a size of the first region of interest, such as in response to the detected movement of the identified first object. The method can also include adjusting a size and position of the first region of interest in response to the detected movement of the identified first object. The method can also include detecting a change in the size of the identified first object and adjusting a size of the first region of interest, such as to accommodate the detected change in size of the identified first object. The method can also include reducing a second spatial sampling resolution, such as in response to an increase in the size of the first region of interest. The method can also include increasing a second spatial sampling resolution, such as in response to a decrease in the size of the first region of interest. The method can also include scanning the light beam over the field of view to capture successive lidar images, wherein the region of interest is capable of being adjusted after the capture of each successive lidar image. The method can also include identifying a second object outside of the first region of interest, selecting a second region of interest that can contain at least a portion of the identified second object, and capturing a third lidar image, where capturing the third lidar image can include scanning the light beam over the first region of interest at the first spatial sampling resolution, scanning the light beam over the second region of interest at a third spatial sampling resolution, and scanning the light beam over the field of view outside of the first region of interest and the second region of interest, at the second spatial sampling resolution, wherein the third sampling resolution can be different than the first spatial sampling resolution. In an example, the third sampling resolution can be less than the second spatial sampling resolution. In an example, the second object can be identified outside of the first region of interest by using the captured second lidar image. Identifying a first object within the captured first lidar image can include detecting at least one edge of the first object. Identifying a first object within the captured first lidar image can include detecting at least one lane marker.
  • In an aspect, the disclosure can feature a system for providing a dynamic region of interest in a lidar system. The system can include a laser configured to emit a light beam, such as towards a target region. The system can also include control circuitry configured to instruct an optical system to scan the light beam over the target region. The system can also include an optical system having a field of view and can configured to direct a portion of the light beam received from the target region. The system can also include a photodetector configured to receive the portion of the light beam directed from the optical system, such as to form a first lidar image. The system can also include detection circuitry that can be configured to identify a first object within the first lidar image. The control circuitry can be further configured to select a first region of interest within the field of view that can contain at least a portion of the identified first object, instruct the optical system to scan the light beam over the first region of interest at a first spatial sampling resolution, and instruct the optical system to scan the light beam over the field of view outside of the first region of interest at a second spatial sampling resolution that can be different than the first spatial sampling resolution. In an example, the first spatial sampling resolution can be less than the second spatial sampling resolution. The photodetector can be further configured to receive a corresponding portion of the light beam to form a second lidar image. The detection circuitry can be further configured to identify a second object outside of the first region of interest in the second lidar image. The control circuitry can be further configured to select a second region of interest that can contain a portion of the identified second object, instruct the optical system to scan the light beam over the first region of interest and the second region of interest at the first spatial sampling resolution, and instruct the optical system to scan the light beam over the field of view outside of both the first region of interest and the second region of interest at a third spatial sampling resolution that can be different than the first spatial sampling resolution. In an example, the third spatial sampling resolution can be less than the first spatial sampling resolution. The detection circuitry can be further configured to detect a movement of the identified first object and the control circuitry can be configured to adjust a characteristic of the first region of interest, such as in response to the detected movement of the identified first object. The control circuitry can be further configured to adjust a size of the first region of interest, such as in response to the detected movement of the identified first object. The control circuitry can be further configured to adjust a size and position of the first region of interest, such as in response to the detected movement of the identified first object. The detection circuitry can be further configured to detect a change in the size of the identified first object and the control circuitry can be further configured to adjust a size of the first region of interest, such as to accommodate the detected change in size of the identified first object. The control circuitry can be further configured to reduce the second spatial sampling resolution, such as in response to an increase in the size of the first region of interest. The control circuitry can be further configured to increase the second spatial sampling resolution, such as in response to a decrease in the size of the first region of interest.
  • In an aspect, the disclosure can feature a system for providing a dynamic region of interest in a lidar system. The system can include a means for scanning a light beam over a field of view, such as to capture a first lidar image. The means for scanning can include control circuitry and a scanning laser, such as control circuitry 104 and scanning laser 106 as shown in FIG. 1A. The system can also include a means for identifying a first object within the captured first lidar image. The means for identifying can include detection circuitry, such as detection circuitry 124 as shown in FIG. 1A. The system can also include a means for selecting a first region of interest within the field of view that contains at least a portion of the identified first object. The means for selecting can include control circuitry, such as control circuitry 104 as shown in FIG. 1A. The system can also include a means for capturing a second lidar image, where capturing the second lidar image can includes scanning the light beam over the first region of interest at a first spatial sampling resolution and scanning the light beam over the field of view outside of the first region of interest at a second spatial sampling resolution, wherein the second sampling resolution can be different than the first spatial sampling resolution. In an example, the second sampling resolution can be less than the first spatial sampling resolution. The means for capturing a second lidar image can include control circuitry and a scanning laser, such as control circuitry 104 and scanning laser 106 as shown in FIG. 1A.
  • BRIEF DESCRIPTION OF THE DRAWINGS
  • The present disclosure will now be described, by way of example, with reference to the accompanying drawings, in which:
  • FIG. 1A illustrates a diagram of a lidar system.
  • FIGS. 1B-1D illustrate examples of a frame in a lidar system.
  • FIGS. 2A-2C illustrate an example of a sequence of frames in a lidar system.
  • FIGS. 3A-3B illustrate an example of a sequence of frames in a lidar system.
  • FIG. 4 illustrates a method of operation of a lidar system.
  • DETAILED DESCRIPTION OF EMBODIMENTS OF THE DISCLOSURE
  • FIG. 1A shows an example of a lidar system 100. The lidar system 100 can include control circuitry 104, a scanning laser 108, an optical system 116, a photosensitive detector 120, and detection circuitry 124. The control circuitry 104 can be connected to the scanning laser 108 and the detection circuitry 124. The photosensitive detector 120 can be connected to the detection circuitry 124. During operation, the control circuitry 104 can provide instructions to the scanning laser 108, such as to cause the scanning laser to scan a light beam over a target region 112. In an example, the scanning laser 108 can include laser that can emit a light beam and an optical system, such as an electro-optic waveguide. The electro-optic waveguide can adjust an angle of the light beam based on the received instructions from the control circuitry 104. The target region 112 can correspond to a field of view of the optical system 116. The scanning laser 108 can scan a light beam over the target region 112 in a series of scanned points 114. The optical system 116 can receive at least a portion of the light beam from the target region 112 and can image the scanned points 114 onto the photosensitive detector 120 (e.g., a CCD). The detection circuitry 124 can receive and process the image of the scanned points from the photosensitive detector 120, such as to form a frame. In an example, the control circuitry 104 can select a region of interest that is a subset of the field of view of the optical system and instruct the scanning laser to scan over the region of interest. In an example, the detection circuitry 124 can include circuitry for digitizing the received image. In an example, the lidar system 100 can be installed in an automobile, such as to facilitate an autonomous self-driving automobile.
  • FIG. 1B illustrates an example of a frame 130 corresponding to a 2D image, such as that captured with lidar system 100. The frame 130 can include a collection of scanned points 114. The scanned points 114 can be regularly spaced by a distance d, along a grid. The spacing d of the scanned points 114 can determine the angular resolution of a lidar system, such as the lidar system 100. For example, a larger spacing can correspond to a coarser angular resolution and a smaller spacing can correspond to a finer angular resolution. In an example, the frame 130 can include a region of interest 135 that corresponds to a field of view of the optical system 116 (e.g., all points within the field of view can be scanned).
  • FIG. 1C illustrates an example of a frame 130, such as that captured with lidar system 100. The frame 130 can include a collection of scanned points 114. The scanned points 114 can be regularly spaced along a grid. The spacing of the scanned points 114 can determine the angular resolution of a lidar system, such as the lidar system 100. For example, a larger spacing can correspond to a coarser angular resolution and a smaller spacing can correspond to a finer angular resolution. In an example, the frame 130 can include a region of interest 135 that corresponds to a subset of a field of view of the optical system 116. In an example, the scanning laser 108 scan a light beam over the region of interest 135, but not other points within the field of view of the lidar system 100 (e.g., only a fraction of points within the field of view can be scanned).
  • FIG. 1D illustrates an example of a frame 130, such as that captured with lidar system 100. The frame 130 can include a collection of scanned points 114. The scanned points 114 can be regularly spaced along a grid. The spacing of the scanned points 114 can determine the angular resolution of a lidar system, such as the lidar system 100. For example, a larger spacing can correspond to a coarser angular resolution and a smaller spacing can correspond to a finer angular resolution. In an example, the frame 130 can include a region of interest 135 that corresponds to a subset of a field of view of the optical system 116. In an example, the scanning laser 108 scan a light beam over the region of interest 135, but not other points within the field of view of the lidar system 100 (e.g., only a fraction of points within the field of view can be scanned).
  • FIGS. 2A-2C illustrate an example of a sequence of frames 230-232 where the scanned points can be irregularly spaced across a field of view of the optical system 116. The first frame 230 as illustrated in FIG. 2A can include a first region of interest 235. The first region of interest 235 can include a collection of regularly spaced scanned points. The scanned points in the first region of interest 235 can correspond to a first angular resolution. Outside of the first region of interest 235, the scanned points can be regularly spaced with a larger spacing than the first region of interest 235, corresponding to a coarser angular resolution than in the first region of interest 235. Outside of the first region of interest 235, every third column in every other row can be scanned as illustrated in FIG. 2A. However, other patterns of scanning can be utilized outside of the first region of interest 235. For example, a scanning pattern outside of the first region of interest can include every second column, in every third row. More generally, the scanning pattern outside of the first region of interest 235 can include every nth column in every mth row. The first region of interest 235 can be dynamically adjusted on a frame-to-frame basis, such as based on an analysis of the frame by the detection circuitry 124. In the example shown in FIG. 2A, the first frame can accommodate up to 144 scanned points, the first region of interest 235 can include 36 scanned points, and the portion of the frame outside of the region of interest can include 17 scanned points, for a total of 53 scanned points out of a total of 144 possible scanned points. The second frame 231 as illustrated in FIG. 2B can include a second region of interest 236. The second region of interest 236 can be determined based on an object detected in the first frame 230. The second region of interest 236 can be smaller than the first region of interest 235 and can include a collection of regularly spaced scanned points. The scanned points in the second region of interest 236 can correspond to a first angular resolution. Outside of the second region of interest 236, the scanned points can be regularly spaced with a larger spacing than the second region of interest 236, corresponding to a coarser angular resolution than in the second region of interest 236. The second region of interest 236 can be dynamically adjusted on a frame-to-frame basis, such as based on an analysis of the first frame 230 by the detection circuitry 124. In an example where the second region of interest 236 can be smaller than a first region of interest 235, a total number of scanned points in the frame 231 can be smaller than the total number of scanned points in the frame 230. In the example shown in FIG. 2B, the second frame can accommodate up to 144 scanned points, the second region of interest 236 can include 12 scanned points, and the portion of the frame outside of the region of interest can include 23 scanned points, for a total of 45 scanned points out of a total of 144 possible scanned points. The third frame 232 as illustrated in FIG. 2C can include a third region of interest 237 and a region of disinterest 240. The third region of interest 237 can be determined based on an object detected in the second frame 231. The third region of interest 237 can be the same size as the second region of interest 236 and can include a collection of regularly spaced scanned points. The scanned points in the third region of interest 237 can correspond to a first angular resolution. Outside of the third region of interest 237, the scanned points can be regularly spaced with a larger spacing than the third region of interest 237, corresponding to a coarser angular resolution than in the third region of interest 237. The third region of interest 237 can be dynamically adjusted on a frame-to-frame basis, such as based on an analysis of the second frame 231 by the detection circuitry 124. In the region of disinterest 240, the scanned points can be regularly spaced with a larger spacing than outside of the third region of interest 237. In an example, no points are scanned in the region of disinterest 240. In an example, the region of disinterest can correspond to an area in the frame that includes a quasi-stationary object. The size and location of the region of disinterest 240 can be determined based on the identification of one or more objects within the second frame 231. Similar to the regions of interest 235-237, the region of disinterest 240 can be dynamically adjusted on a frame-to-frame basis. In an example where the third region of interest 236 can be the same size as the second region of interest 235, a total number of scanned points in the third frame 232 can be smaller than the number of scanned points in the second frame 231. In the example shown in FIG. 2C, the third frame can accommodate up to 144 scanned points, the third region of interest 237 can include 12 scanned points, the region of disinterest 240 can exclude up to 20 scanned points, and the portion of the frame outside of the region of interest can include 18 scanned points, for a total of 30 scanned points out of a total of 144 possible scanned points.
  • FIGS. 3A-3B illustrate a sequence of frames 330-331, such as can be collected by a lidar system in an automobile where the scanned points can be irregularly spaced across a field of view that can include a road and associated landscape. The first frame 330 as illustrated in FIG. 3A can include a first region of interest 335, a second region of interest 345, and a region of disinterest 340. The first region of interest 335 can include a collection of regularly spaced scanned points. The scanned points in the first region of interest 335 can correspond to a first angular resolution. The first region of interest 335 can correspond to a portion of a road having at least one lane, where each lane can be approximately 4 meters wide. A width of the first region of interest 335 can be selected, such as to accommodate the width of three lanes (e.g., a lane that an automobile is driving in and additionally, one lane on either side of the lane that the automobile is driving in). The width of the first region of interest 335 can be sized to accommodate a radius of curvature of the road. For example, at a relatively high speed of 150 km/hr, a radius of curvature of the road can be approximately 1 km, corresponding to a road that can be 4° off of a longitudinal axis at a distance of 150 m. At a medium speed of 80 km/hr, a radius of curvature of the road can be approximately 200 m, corresponding to a road that can be 10° off of a longitudinal axis at a distance of 60 m. To account for the radius of curvature of the road, the first region of interest 335 can extend 20° in a horizontal direction, and to account for a vertical extent of other automobiles (e.g. an automobile can extend 4 m and the region of interest can be sized to accommodate twice the vehicle height at a distance of 60 m), the first region of interest can extend 4° in a vertical direction. The second region of interest 345 can be smaller than the first region of interest 335 and can include a collection of regularly spaced scanned points. The scanned points in the second region of interest 345 can correspond to the first angular resolution. The second region of interest 345 can correspond to a portion of a lane marker on a road. Outside of the first region of interest 335 and the second region of interest 345, the scanned points can be regularly spaced with a larger spacing than the first region of interest 335 and the second region of interest 345, corresponding to a coarser angular resolution than in the first region of interest 335 or the second region of interest 345. Outside of the first region of interest 335 and the second region of interest 345, every mth column in every nth row can be scanned with the exception of the region of disinterest 340. The region of disinterest 340 can designate an area within the frame 330 where the scanned points can be regularly spaced with a larger spacing than in the first region of interest 335, the second region of interest 345, or the region outside of the first region of interest 335 and the second region of interest 345. In an example, no points are scanned within the region of disinterest 340. The region of disinterest 340 can include fixed road infrastructure, such as guard rails and the road shoulder. The region of disinterest can include a road surface near an automobile. The region of disinterest 340 can correspond to objects such as trees, rocks, or mountains within a field of view of a lidar system, such as lidar system 100. The first region of interest 335, the second region of interest 345, and the region of disinterest 340 can be adjusted dynamically, such as based on the motion of objects within the field of view of the lidar system 100. FIG. 3B illustrates a second frame 331 where the regions of interest and disinterest have been dynamically updated, such as based on a change in the relative position of the road and lane markers within the field of view of the lidar system 100. The second frame 331 as illustrated in FIG. 3b can include a first region of interest 336, a second region of interest 346, and a region of disinterest 341. The first region of interest 336 can include a collection of regularly spaced scanned points. The scanned points in the first region of interest 335 can correspond to a first angular resolution. The first region of interest 335 can correspond to a portion of a road having at least one lane, where each lane can be approximately 4 meters wide. A width of the first region of interest 335 can be selected, such as to accommodate the width of three lanes (e.g., a lane that an automobile is driving in and additionally, one lane on either side of the lane that the automobile is driving in). The width of the first region of interest 335 can be sized to accommodate a radius of curvature of the road. For example, at a relatively high speed of 150 km/hr, a radius of curvature of the road can be approximately 1 km, corresponding to a road that can be 40 off of a longitudinal axis at a distance of 150 m. At a medium speed of 80 km/hr, a radius of curvature of the road can be approximately 200 m, corresponding to a road that can be 10° off of a longitudinal axis at a distance of 60 m. To account for the radius of curvature of the road, the first region of interest 335 can extend 20° in a horizontal direction, and to account for a vertical extent of other automobiles (e.g. an automobile can extend 4 m and the region of interest can be sized to accommodate twice the vehicle height at a distance of 60 m), the first region of interest can extend 4° in a vertical direction. The second region of interest 346 can be smaller than the first region of interest 336 and can include a collection of regularly spaced scanned points. The scanned points in the second region of interest 346 can correspond to the first angular resolution. The second region of interest 346 can correspond to a portion of a lane marker on a road. Outside of the first region of interest 336 and the second region of interest 346, the scanned points can be regularly spaced with a larger spacing than the first region of interest 336 and the second region of interest 346, corresponding to a coarser angular resolution than in the first region of interest 336 or the second region of interest 346. Outside of the first region of interest 336 and the second region of interest 346, every mth column in every nth row can be scanned with the exception of the region of disinterest 341. The region of disinterest 341 can designate an area within the frame 331 where the scanned points can be regularly spaced with a larger spacing than in the first region of interest 336, the second region of interest 346, or the region outside of the first region of interest 336 and the second region of interest 346. In an example, no points are scanned within the region of disinterest 341. The region of disinterest 341 can correspond to objects such as trees, rocks, or mountains within a field of view of a lidar system, such as lidar system 100.
  • FIG. 4 illustrates a method of adjusting a field of view in a lidar system, such as lidar system 100. A light beam, such as can be emitted by the scanning laser 108 can be scanned over a target region within a field of view of an optical system, such as optical system 116 and a first image can be captured by a photosensitive detector, such as the photosensitive detector 120 (step 410). A first object can be identified within the first image by detection circuitry, such as the detection circuitry 124 (step 420). Control circuitry, such as control circuitry 104 can select a first region of interest that includes at least a portion of the identified first object (step 430). A second lidar image can then be captured (step 440). The capturing of the second lidar image can include steps 450 and 460 described below. A light beam, such as can be emitted by the scanning laser 108 can be scanned over the first region of interest at a first spatial sampling resolution (step 450). A light beam, such as can be emitted by the scanning laser 108 can be scanned over the field of view outside of the first region of interest at a second spatial sampling resolution, wherein the second sampling resolution can be less than the first spatial sampling resolution (step 460). In an example, detection circuitry, such as the detection circuitry 124 can identify a second object outside of the first region of interest in the captured second lidar image. Control circuitry, such as control circuitry 104 can select a second region of interest that can contain at least a portion of the identified second object. A third lidar image can then be captured, where capturing the third lidar image can include scanning a light beam, such as that emitted by the scanning laser 108, over both the first and second regions of interest at the first spatial sampling resolution and over a field of view outside of both the first and second regions of interest at a third spatial sampling resolution that can be less than the second spatial sampling resolution. In an example, the control circuitry 104 can receive external data, such as from an inertial sensor, GPS, radar, camera, or wheel speed sensor data, and in response to the received external data, the control circuitry 104 can adjust a size or position of the first region of interest.

Claims (20)

1. A method for providing a dynamic region of interest in a lidar system, the method comprising:
scanning a light beam over a field of view to capture a first lidar image;
identifying a first object within the captured first lidar image;
selecting a first region of interest within the field of view that contains at least a portion of the identified first object; and
capturing a second lidar image, where capturing the second lidar image includes:
scanning the light beam over the first region of interest at a first spatial sampling resolution; and
scanning the light beam over the field of view outside of the first region of interest at a second spatial sampling resolution, wherein the second sampling resolution is different than the first spatial sampling resolution.
2. The method of claim 1 further comprising:
identifying a second object outside of the first region of interest;
selecting a second region of interest that contains at least a portion of the identified second object; and
capturing a third lidar image, where capturing the third lidar image includes:
scanning the light beam over the first region of interest and the second region of interest at the first spatial sampling resolution; and
scanning the light beam over the field of view outside of both the first region of interest and the second region of interest at a third spatial sampling resolution, wherein the third sampling resolution is different than the first spatial sampling resolution.
3. The method of claim 1 further comprising:
detecting a movement of the identified first object; and
adjusting a characteristic of the first region of interest in response to the detected movement of the identified first object.
4. The method of claim 3 comprising adjusting a size of the first region of interest in response to the detected movement of the identified first object.
5. The method of claim 3 comprising adjusting a size and position of the first region of interest in response to the detected movement of the identified first object.
6. The method of claim 1 further comprising:
detecting a change in the size of the identified first object; and
adjusting a size of the first region of interest to accommodate the detected change in size of the identified first object.
7. The method of claim 6 comprising reducing a second spatial sampling resolution in response to an increase in the size of the first region of interest.
8. The method of claim 6 comprising increasing a second spatial sampling resolution in response to a decrease in the size of the first region of interest.
9. The method of claim 1 further comprising scanning the light beam over the field of view to capture successive lidar images, wherein the region of interest is capable of being adjusted after the capture of each successive lidar image.
10. The method of claim 1 further comprising:
identifying a second object outside of the first region of interest;
selecting a second region of interest that contains at least a portion of the identified second object; and
capturing a third lidar image, where capturing the third lidar image includes:
scanning the light beam over the first region of interest at the first spatial sampling resolution;
scanning the light beam over the second region of interest at a third spatial sampling resolution; and
scanning the light beam over the field of view outside of the first region of interest and the second region of interest at the second spatial sampling resolution, wherein the third sampling resolution is different than the second spatial sampling resolution.
11. The method of claim 1 wherein identifying a first object within the captured first lidar image includes detecting at least one edge of the first object.
12. The method of claim 1 wherein identifying a first object within the captured first lidar image includes detecting at least one lane marker.
13. A system for providing a dynamic region of interest in a lidar system, the system comprising:
a laser configured to emit a light beam towards a target region;
control circuitry configured to instruct an optical system to scan the light beam over the target region;
an optical system having a field of view and configured to direct a portion of the light beam received from the target region;
a photodetector configured to receive the portion of the light beam directed from the optical system to form a first lidar image; and
detection circuitry configured to identify a first object within the first lidar image; wherein
the control circuitry is further configured to select a first region of interest within the field of view that contains at least a portion of the identified first object, instruct the optical system to scan the light beam over the first region of interest at a first spatial sampling resolution, and instruct the optical system to scan the light beam over the field of view outside of the first region of interest at a second spatial sampling resolution different than the first spatial sampling resolution, and wherein the photodetector is further configured to receive a corresponding portion of the light beam to form a second lidar image.
14. The system of claim 13 wherein the detection circuitry is further configured to identify a second object outside of the first region of interest, the control circuitry is further configured to select a second region of interest that contains a portion of the identified second object, instruct the optical system to scan the light beam over the first region of interest and the second region of interest at the first spatial sampling resolution, and instruct the optical system to scan the light beam over the field of view outside of both the first region of interest and the second region of interest at a third spatial sampling resolution different than the first spatial sampling resolution.
15. The system of claim 13 wherein the detection circuitry is further configured to detect a movement of the identified first object and the control circuitry is configured to adjust a characteristic of the first region of interest in response to the detected movement of the identified first object.
16. The system of claim 15 wherein the control circuitry is further configured to adjust a size of the first region of interest in response to the detected movement of the identified first object.
17. The system of claim 15 wherein the control circuitry is further configured to adjust a size and position of the first region of interest in response to the detected movement of the identified first object.
18. The system of claim 13 wherein the detection circuitry is further configured to detect a change in the size of the identified first object and the control circuitry is further configured to adjust a size of the first region of interest to accommodate the detected change in size of the identified first object.
19. The system of claim 18 wherein the control circuitry is further configured to reduce the second spatial sampling resolution in response to an increase in the size of the first region of interest.
20. A system for providing a dynamic region of interest in a lidar system, the system comprising:
means for scanning a light beam over a field of view to capture a first lidar image;
means for identifying a first object within the captured first lidar image;
means for selecting a first region of interest within the field of view that contains at least a portion of the identified first object; and
means for capturing a second lidar image, where capturing the second lidar image includes:
scanning the light beam over the first region of interest at a first spatial sampling resolution; and
scanning the light beam over the field of view outside of the first region of interest at a second spatial sampling resolution, wherein the second sampling resolution is different than the first spatial sampling resolution.
US15/492,771 2017-04-20 2017-04-20 Method of Providing a Dynamic Region of interest in a LIDAR System Abandoned US20180306905A1 (en)

Priority Applications (6)

Application Number Priority Date Filing Date Title
US15/492,771 US20180306905A1 (en) 2017-04-20 2017-04-20 Method of Providing a Dynamic Region of interest in a LIDAR System
US16/606,721 US20200150228A1 (en) 2017-04-20 2017-12-08 Method of Providing Interference Reduction and a Dynamic Region of Interest in a LIDAR System
DE112017007467.3T DE112017007467T5 (en) 2017-04-20 2017-12-08 Method for providing interference reduction and a dynamic range of interest in a LIDAR system
PCT/US2017/065392 WO2018194721A1 (en) 2017-04-20 2017-12-08 Method of providing interference reduction and a dynamic region of interest in a lidar system
CN201780089765.9A CN110537108B (en) 2017-04-20 2017-12-08 Method for providing reduced interference and target dynamic area in lidar systems
US18/217,366 US20240069172A1 (en) 2017-04-20 2023-06-30 Method of Providing Interference Reduction and a Dynamic Region of Interest in a LIDAR System

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
US15/492,771 US20180306905A1 (en) 2017-04-20 2017-04-20 Method of Providing a Dynamic Region of interest in a LIDAR System

Related Child Applications (3)

Application Number Title Priority Date Filing Date
PCT/US2017/065392 Continuation WO2018194721A1 (en) 2017-04-20 2017-12-08 Method of providing interference reduction and a dynamic region of interest in a lidar system
US16/606,721 Continuation US20200150228A1 (en) 2017-04-20 2017-12-08 Method of Providing Interference Reduction and a Dynamic Region of Interest in a LIDAR System
US16/606,721 Continuation-In-Part US20200150228A1 (en) 2017-04-20 2017-12-08 Method of Providing Interference Reduction and a Dynamic Region of Interest in a LIDAR System

Publications (1)

Publication Number Publication Date
US20180306905A1 true US20180306905A1 (en) 2018-10-25

Family

ID=60888666

Family Applications (2)

Application Number Title Priority Date Filing Date
US15/492,771 Abandoned US20180306905A1 (en) 2017-04-20 2017-04-20 Method of Providing a Dynamic Region of interest in a LIDAR System
US18/217,366 Pending US20240069172A1 (en) 2017-04-20 2023-06-30 Method of Providing Interference Reduction and a Dynamic Region of Interest in a LIDAR System

Family Applications After (1)

Application Number Title Priority Date Filing Date
US18/217,366 Pending US20240069172A1 (en) 2017-04-20 2023-06-30 Method of Providing Interference Reduction and a Dynamic Region of Interest in a LIDAR System

Country Status (4)

Country Link
US (2) US20180306905A1 (en)
CN (1) CN110537108B (en)
DE (1) DE112017007467T5 (en)
WO (1) WO2018194721A1 (en)

Cited By (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190064322A1 (en) * 2017-08-25 2019-02-28 Institute For Information Industry Vehicle detecting method and vehicle detecting system
US20190179317A1 (en) * 2017-12-13 2019-06-13 Luminar Technologies, Inc. Controlling vehicle sensors using an attention model
US20200090501A1 (en) * 2018-09-19 2020-03-19 International Business Machines Corporation Accident avoidance system for pedestrians
US10598788B1 (en) * 2018-10-25 2020-03-24 Aeye, Inc. Adaptive control of Ladar shot selection using spatial index of prior Ladar return data
US10641897B1 (en) 2019-04-24 2020-05-05 Aeye, Inc. Ladar system and method with adaptive pulse duration
WO2021255110A1 (en) * 2020-06-17 2021-12-23 Jaguar Land Rover Limited Vehicle control system using a scanning system
GB2597928A (en) * 2020-08-05 2022-02-16 Envisics Ltd Light detection and ranging
CN114296102A (en) * 2021-12-31 2022-04-08 深圳市镭神智能系统有限公司 Lidar scanning method and scanning device
US11402462B2 (en) * 2017-11-06 2022-08-02 Echodyne Corp. Intelligent sensor and intelligent feedback-based dynamic control of a parameter of a field of regard to which the sensor is directed
US20220299433A1 (en) * 2019-07-03 2022-09-22 Sony Group Corporation Information processing apparatus, information processing method, program, and sensing system
US11573299B2 (en) 2020-06-03 2023-02-07 Seagate Technology Llc LIDAR scan profile parameterization
US20230096608A1 (en) * 2021-07-02 2023-03-30 Seagate Technology Llc Field of view laser power rebalancing
US11656341B2 (en) 2019-06-24 2023-05-23 Infineon Technologies Ag Lidar system selectively changing a size of the field of view
US20240029389A1 (en) * 2020-09-01 2024-01-25 Volkswagen Aktiengesellschaft Method and device for sensing the environment of a vehicle driving with at least semi-automation

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20230341526A1 (en) * 2020-01-07 2023-10-26 Baraja Pty Ltd Adaptive spatial estimation system
DE102020107377A1 (en) 2020-03-18 2021-09-23 Valeo Schalter Und Sensoren Gmbh Method for processing a three-dimensional point cloud
CN114325726B (en) * 2020-09-27 2022-12-13 北京一径科技有限公司 Anti-crosstalk processing method of laser radar, storage medium and laser radar
CN115685219A (en) * 2021-07-30 2023-02-03 北京万集科技股份有限公司 Target detection method and device based on laser scanning and target detection terminal
US12360247B1 (en) * 2022-04-29 2025-07-15 Zoox, Inc. Partial sensor data processing for latency reduction
US20230403471A1 (en) * 2022-06-13 2023-12-14 Motional Ad Llc Multiple position rolling shutter imaging device
CN116593997B (en) * 2023-07-17 2023-09-19 北京亮道智能汽车技术有限公司 Anti-crosstalk and anti-interference method and device for laser radar

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5638164A (en) * 1994-08-09 1997-06-10 Development Authority, Rafael State of Israel--Ministry of Defense Armament Apparatus and method for laser imaging
US9383753B1 (en) * 2012-09-26 2016-07-05 Google Inc. Wide-view LIDAR with areas of special attention

Family Cites Families (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5621514A (en) * 1995-01-05 1997-04-15 Hughes Electronics Random pulse burst range-resolved doppler laser radar
IL200332A0 (en) * 2008-08-19 2010-04-29 Rosemount Aerospace Inc Lidar system using a pseudo-random pulse sequence
US9091754B2 (en) * 2009-09-02 2015-07-28 Trimble A.B. Distance measurement methods and apparatus
CN102338872A (en) * 2010-07-16 2012-02-01 李少勤 Method for preventing and processing interference signal of automotive anti-collision radar
KR102038533B1 (en) * 2012-06-14 2019-10-31 한국전자통신연구원 Laser Radar System and Method for Acquiring Target Image
WO2015199736A1 (en) * 2014-06-27 2015-12-30 Hrl Laboratories, Llc Compressive scanning lidar
US9625582B2 (en) * 2015-03-25 2017-04-18 Google Inc. Vehicle with multiple light detection and ranging devices (LIDARs)
US10061020B2 (en) * 2015-09-20 2018-08-28 Qualcomm Incorporated Light detection and ranging (LIDAR) system with dual beam steering
US10690756B2 (en) * 2016-05-10 2020-06-23 Texas Instruments Incorporated Methods and apparatus for LIDAR operation with pulse position modulation
US10305247B2 (en) * 2016-08-30 2019-05-28 Apple Inc. Radiation source with a small-angle scanning array
US10295660B1 (en) * 2016-12-30 2019-05-21 Panosense Inc. Aligning optical components in LIDAR systems

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5638164A (en) * 1994-08-09 1997-06-10 Development Authority, Rafael State of Israel--Ministry of Defense Armament Apparatus and method for laser imaging
US9383753B1 (en) * 2012-09-26 2016-07-05 Google Inc. Wide-view LIDAR with areas of special attention

Cited By (30)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10684357B2 (en) * 2017-08-25 2020-06-16 Institute For Information Industry Vehicle detection assisting method and vehicle detection assisting system
US20190064322A1 (en) * 2017-08-25 2019-02-28 Institute For Information Industry Vehicle detecting method and vehicle detecting system
US11402462B2 (en) * 2017-11-06 2022-08-02 Echodyne Corp. Intelligent sensor and intelligent feedback-based dynamic control of a parameter of a field of regard to which the sensor is directed
US20190179317A1 (en) * 2017-12-13 2019-06-13 Luminar Technologies, Inc. Controlling vehicle sensors using an attention model
US10984257B2 (en) 2017-12-13 2021-04-20 Luminar Holdco, Llc Training multiple neural networks of a vehicle perception component based on sensor settings
US10768304B2 (en) 2017-12-13 2020-09-08 Luminar Technologies, Inc. Processing point clouds of vehicle sensors having variable scan line distributions using interpolation functions
US10754037B2 (en) 2017-12-13 2020-08-25 Luminar Technologies, Inc. Processing point clouds of vehicle sensors having variable scan line distributions using voxel grids
US20200090501A1 (en) * 2018-09-19 2020-03-19 International Business Machines Corporation Accident avoidance system for pedestrians
US10598788B1 (en) * 2018-10-25 2020-03-24 Aeye, Inc. Adaptive control of Ladar shot selection using spatial index of prior Ladar return data
US10670718B1 (en) 2018-10-25 2020-06-02 Aeye, Inc. System and method for synthetically filling ladar frames based on prior ladar return data
US10656277B1 (en) * 2018-10-25 2020-05-19 Aeye, Inc. Adaptive control of ladar system camera using spatial index of prior ladar return data
US10656252B1 (en) 2018-10-25 2020-05-19 Aeye, Inc. Adaptive control of Ladar systems using spatial index of prior Ladar return data
US11327177B2 (en) 2018-10-25 2022-05-10 Aeye, Inc. Adaptive control of ladar shot energy using spatial index of prior ladar return data
US11733387B2 (en) 2018-10-25 2023-08-22 Aeye, Inc. Adaptive ladar receiver control using spatial index of prior ladar return data
US12372654B2 (en) 2018-10-25 2025-07-29 Aeye, Inc. Adaptive control of ladar systems using spatial index of prior ladar return data
US10656272B1 (en) 2019-04-24 2020-05-19 Aeye, Inc. Ladar system and method with polarized receivers
US10921450B2 (en) 2019-04-24 2021-02-16 Aeye, Inc. Ladar system and method with frequency domain shuttering
US11513223B2 (en) 2019-04-24 2022-11-29 Aeye, Inc. Ladar system and method with cross-receiver
US10641897B1 (en) 2019-04-24 2020-05-05 Aeye, Inc. Ladar system and method with adaptive pulse duration
US11656341B2 (en) 2019-06-24 2023-05-23 Infineon Technologies Ag Lidar system selectively changing a size of the field of view
US12111251B2 (en) * 2019-07-03 2024-10-08 Sony Group Corporation Information processing apparatus, information processing method, program, and sensing system
US20220299433A1 (en) * 2019-07-03 2022-09-22 Sony Group Corporation Information processing apparatus, information processing method, program, and sensing system
US11573299B2 (en) 2020-06-03 2023-02-07 Seagate Technology Llc LIDAR scan profile parameterization
WO2021255110A1 (en) * 2020-06-17 2021-12-23 Jaguar Land Rover Limited Vehicle control system using a scanning system
GB2597928B (en) * 2020-08-05 2024-09-25 Envisics Ltd Light detection and ranging
GB2597928A (en) * 2020-08-05 2022-02-16 Envisics Ltd Light detection and ranging
US20240029389A1 (en) * 2020-09-01 2024-01-25 Volkswagen Aktiengesellschaft Method and device for sensing the environment of a vehicle driving with at least semi-automation
US12412363B2 (en) * 2020-09-01 2025-09-09 Volkswagen Aktiengesellschaft Method and device for sensing the environment of a vehicle driving with at least semi-automation
US20230096608A1 (en) * 2021-07-02 2023-03-30 Seagate Technology Llc Field of view laser power rebalancing
CN114296102A (en) * 2021-12-31 2022-04-08 深圳市镭神智能系统有限公司 Lidar scanning method and scanning device

Also Published As

Publication number Publication date
DE112017007467T5 (en) 2020-01-02
US20240069172A1 (en) 2024-02-29
CN110537108A (en) 2019-12-03
WO2018194721A1 (en) 2018-10-25
CN110537108B (en) 2023-11-14

Similar Documents

Publication Publication Date Title
US20180306905A1 (en) Method of Providing a Dynamic Region of interest in a LIDAR System
US20230137192A1 (en) Detection method of lidar, lidar, and system for vehicle including the same
US20200150228A1 (en) Method of Providing Interference Reduction and a Dynamic Region of Interest in a LIDAR System
US11287523B2 (en) Method and apparatus for enhanced camera and radar sensor fusion
EP2843615B1 (en) Device for detecting three-dimensional object and method for detecting three-dimensional object
US11061122B2 (en) High-definition map acquisition system
KR20230004425A (en) Autonomous Vehicle Environment Cognitive Software Architecture
US12140671B2 (en) Velocity determination with a scanned lidar system
US11780436B2 (en) On-board sensor system
USRE50261E1 (en) System and method for multipurpose traffic detection and characterization
KR101395089B1 (en) System and method for detecting obstacle applying to vehicle
EP3848725B1 (en) Prioritized sensor data processing using map information for automated vehicles
KR101428239B1 (en) Road marker recognition device and recognition method of the same
KR101880593B1 (en) Lidar sensor device for automatic driving of unmanned vehicles
US20060111841A1 (en) Method and apparatus for obstacle avoidance with camera vision
KR102008263B1 (en) Convergence detector and traffic enforcement system therewith
WO2017009848A1 (en) Gated structured imaging
JP2008026997A (en) Pedestrian recognition device and pedestrian recognition method
JP2014006588A (en) Road surface boundary estimation device and program
CN108732573B (en) Parking space detection method and parking space detection system
KR20190093301A (en) Method and system for error correction of radar for vehicle
JP4691701B2 (en) Number detection device and method
KR101734867B1 (en) Apparatus for detecting object
JP2014130429A (en) Photographing device and three-dimensional object area detection program
JP2001188988A (en) Vehicle detection device

Legal Events

Date Code Title Description
STPP Information on status: patent application and granting procedure in general

Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION

AS Assignment

Owner name: ANALOG DEVICES, INC., MASSACHUSETTS

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:KAPUSTA, RONALD A.;LUEY, BENJAMIN;WEINBERG, HARVEY;AND OTHERS;SIGNING DATES FROM 20170531 TO 20180104;REEL/FRAME:044913/0950

STPP Information on status: patent application and granting procedure in general

Free format text: NON FINAL ACTION MAILED

STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION