[go: up one dir, main page]

KR100902176B1 - 3D scanner using rotating face mirror - Google Patents

3D scanner using rotating face mirror Download PDF

Info

Publication number
KR100902176B1
KR100902176B1 KR1020080128763A KR20080128763A KR100902176B1 KR 100902176 B1 KR100902176 B1 KR 100902176B1 KR 1020080128763 A KR1020080128763 A KR 1020080128763A KR 20080128763 A KR20080128763 A KR 20080128763A KR 100902176 B1 KR100902176 B1 KR 100902176B1
Authority
KR
South Korea
Prior art keywords
light
optical path
rotating
scanner
mirror
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related
Application number
KR1020080128763A
Other languages
Korean (ko)
Inventor
박윤창
Original Assignee
선문대학교 산학협력단
지스캔(주)
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 선문대학교 산학협력단, 지스캔(주) filed Critical 선문대학교 산학협력단
Priority to KR1020080128763A priority Critical patent/KR100902176B1/en
Application granted granted Critical
Publication of KR100902176B1 publication Critical patent/KR100902176B1/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B26/00Optical devices or arrangements for the control of light using movable or deformable optical elements
    • G02B26/08Optical devices or arrangements for the control of light using movable or deformable optical elements for controlling the direction of light
    • G02B26/10Scanning systems
    • G02B26/105Scanning systems with one or more pivoting mirrors or galvano-mirrors
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B26/00Optical devices or arrangements for the control of light using movable or deformable optical elements
    • G02B26/08Optical devices or arrangements for the control of light using movable or deformable optical elements for controlling the direction of light
    • G02B26/10Scanning systems
    • G02B26/12Scanning systems using multifaceted mirrors
    • G02B26/121Mechanical drive devices for polygonal mirrors
    • G02B26/122Control of the scanning speed of the polygonal mirror
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B26/00Optical devices or arrangements for the control of light using movable or deformable optical elements
    • G02B26/08Optical devices or arrangements for the control of light using movable or deformable optical elements for controlling the direction of light
    • G02B26/10Scanning systems
    • G02B26/12Scanning systems using multifaceted mirrors
    • G02B26/123Multibeam scanners, e.g. using multiple light sources or beam splitters
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B26/00Optical devices or arrangements for the control of light using movable or deformable optical elements
    • G02B26/08Optical devices or arrangements for the control of light using movable or deformable optical elements for controlling the direction of light
    • G02B26/10Scanning systems
    • G02B26/12Scanning systems using multifaceted mirrors
    • G02B26/124Details of the optical system between the light source and the polygonal mirror
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N1/00Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
    • H04N1/04Scanning arrangements, i.e. arrangements for the displacement of active reading or reproducing elements relative to the original or reproducing medium, or vice versa
    • H04N1/113Scanning arrangements, i.e. arrangements for the displacement of active reading or reproducing elements relative to the original or reproducing medium, or vice versa using oscillating or rotating mirrors

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Optics & Photonics (AREA)
  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Length Measuring Devices By Optical Means (AREA)

Abstract

본 발명은 단일 회전 다면경과 서로 다른 색상의 광원 및 필터를 이용하여 다양한 색상의 1차원 패턴광을 영사하고 각 색상별 영상을 한 번에 획득함으로써 측정 속도를 향상시킬 수 있는 회전다면경을 이용한 3D 스캐너에 관한 것이다. The present invention is a 3D using a rotating multi-faceted mirror that can improve the measurement speed by projecting a one-dimensional pattern light of a variety of colors using a single rotating multi-faceted mirror and light sources and filters of different colors and acquires the image for each color at once It is about a scanner.

이를 위한, 본 발명은 3D 스캐너에 있어서, 상기 회전다면경의 일측에 구비되며 서로 다른 색정보를 갖는 점광을 발생하는 다수의 광원과, 상기 각 광원으로부터 발생한 다양한 방향에서의 광을 동일한 광축으로 출사시키기 위한 광필터링 수단과, 상기 회전다면경의 반사면에서 반사되는 광을 수광하여 광로를 인식하여 동기신호를 발생하는 광경로인식수단과, 상기 광경로인식수단으로부터 발생하는 동기신호에 동기시켜 상기 광원의 점멸주기를 조정하여 직선 줄무늬의 주기와 위상을 제어하는 제어수단과, 상기 측정대상물 표면으로부터 반사되는 광을 촬상하는 컬러카메라를 포함한다.To this end, the present invention, in the 3D scanner, a plurality of light sources provided on one side of the rotating polygon mirror and generating point light having different color information, and to emit light in various directions generated from the respective light sources to the same optical axis An optical filtering means for receiving the light reflected from the reflecting surface of the rotating multi-facet mirror, the optical path recognition means for recognizing an optical path, and generating a synchronization signal, and synchronizing with the synchronization signal generated from the optical path recognition means. Control means for controlling the period and phase of the straight stripes by adjusting the flashing period, and a color camera for imaging the light reflected from the surface of the measurement object.

Description

회전다면경을 이용한 3D 스캐너{3D SCANNER USING THE POLYGON MIRROR}3D scanner using rotating face mirror {3D SCANNER USING THE POLYGON MIRROR}

본 발명은 3D 스캐너에 관한 것으로서, 단일 회전 다면경과 서로 다른 색상의 광원 및 필터를 이용하여 다양한 색상의 1차원 패턴광을 영사하고 각 색상별 영상을 한 번에 획득함으로써 측정 속도를 향상시킬 수 있는 회전다면경을 이용한 3D 스캐너에 관한 것이다. The present invention relates to a 3D scanner, which uses a single rotating multi-faceted mirror and light sources and filters of different colors to project one-dimensional patterned light of various colors and to obtain an image for each color at a time, thereby improving measurement speed. A 3D scanner using a rotating mirror.

일반적으로, 자유 곡면으로 이루어진 물체의 3차원 형상측정은 다양한 가공물의 검사나, 캐드/캠(CAD/CAM), 의료, 솔리드 모델링 등 여러 가지 분야에서 폭넓게 적용되고 있다. In general, three-dimensional shape measurement of free-formed objects is widely applied in various fields such as inspection of various workpieces, CAD / CAM, medical care, and solid modeling.

이러한 3차원 형상의 측정기술의 하나 중의 하나가 접촉식 3차원 측정기를 사용하여 곡면상의 한 점씩 측정하여 전체 곡면형상을 측정하는 방식이 사용되어 오고 있었는데, 이러한 접촉식 3차원 형상 측정법은 측정시간이 과다하게 소요되는 문제점을 안고 있다.One of the three-dimensional shape measurement technology has been used to measure the entire curved shape by measuring a point on the curved surface by using a contact type 3D measuring device, this contact type 3D shape measurement method has been used It has an excessive problem.

이에 최근에는 접촉식 3차원 측정기의 단점을 해소하여 측정 효율을 높이기 위하여 광학적인 방식을 이용하고 있다.Recently, the optical method has been used to solve the shortcomings of the contact type 3D measuring device and increase the measuring efficiency.

이와 같이 광학적인 방법을 이용하여 물체의 입체형상을 측정하는 장비를 통 상적으로 "3차원 스캐너(3D Scanner)"라고 부른다. As such, a device for measuring a three-dimensional shape of an object using an optical method is commonly referred to as a "3D scanner".

이러한 3차원 스캐너는, 렌즈와 카메라로 이루어진 영상획득부와 패턴광을 영사하는 패턴광 영사부 및 패턴광 영사부를 통해 획득된 영상을 분석 처리하는 컴퓨터로 구성 되어 있다. The three-dimensional scanner comprises an image acquisition unit consisting of a lens and a camera, a pattern light projection unit for projecting pattern light, and a computer for analyzing and processing an image obtained through the pattern light projection unit.

도 1은 종래의 3D 스캐너의 일 실시예를 나타낸 개념도로서, 렌즈(110)와 카메라(120)로 구성되는 영상획득부(100), 측정물(P) 표면에 패턴광을 영사하는 패턴광 영사부(200)로 구성된다.1 is a conceptual diagram illustrating an embodiment of a conventional 3D scanner, and includes an image acquisition unit 100 including a lens 110 and a camera 120 and a pattern light projection unit projecting pattern light onto a surface of a measurement object P. Referring to FIG. It consists of 200.

여기서, 패턴광 영사부(200)는 디지털 패턴광 영사기로서 LCD 또는 DLP(Digital Light Processing) 방식을 이용한 것이다.The pattern light projector 200 uses an LCD or a digital light processing (DLP) method as a digital pattern light projector.

이러한 디지털 패턴광 영사기를 이용하는 경우에는 R(Red), G(Green), B(Blue) 색상 조합을 통해 다양한 색상 및 다양한 형태의 패턴광 영사가 가능하다.When the digital pattern light projector is used, various colors and various types of pattern light projections are possible through a combination of R (Red), G (Green), and B (Blue) colors.

이러한 디지털 패턴광 영사기는 도 2에 도시된 바와 같이 2차원 영상 데이터를 입력해 주어야 한다.The digital patterned light projector needs to input two-dimensional image data as shown in FIG. 2.

따라서, 1차원 패턴만을 영사하는 3차원 형상 측정에서는, 2차원 영상 데이터를 처리해야 하기 때문에 데이터의 양이 많아지면서 결국 형상 측정에 많은 시간이 소요된다.Therefore, in three-dimensional shape measurement, which projects only one-dimensional pattern, two-dimensional image data must be processed, so that the amount of data increases, and thus, the shape measurement takes a long time.

또한, 고가이며 장비의 부피가 클 뿐만 아니라, 높은 전력 소모와 발열의 문제가 있다.In addition, expensive and bulky equipment, there is a problem of high power consumption and heat generation.

도 3은 종래의 3D 스캐너의 다른 실시예를 나타낸 개념도로서, 렌즈(110)와 카메라(120)로 구성되는 영상획득부(100), 측정물(P) 표면에 패턴광을 영사하는 패 턴광 영사부(300)를 포함한다. 3 is a conceptual diagram illustrating another embodiment of a conventional 3D scanner, and includes an image acquisition unit 100 including a lens 110 and a camera 120 and a pattern light projection unit projecting pattern light onto a surface of a measurement object P. Referring to FIG. 300.

여기서, 패턴광 영사부(300)는 일정 간격의 줄무늬 패턴광을 생성하는 것으로서, 다수의 반사면을 갖는 회전다면경(310)과, 점광을 발생하는 광원(320)과, 광원(320)에서 발생하는 점광을 직선광으로 변환하여 회전다면경(310)의 반사면에 입사시키는 집광렌즈(330)와 원통형 렌즈(340), 그리고 광원(310)으로부터 발생한 광의 경로를 인식하는 광로변환미러(350)와 수광센서(360)를 포함한다. Here, the pattern light projection unit 300 generates the striped pattern light at a predetermined interval, and is generated by the rotating polyhedron 310 having a plurality of reflective surfaces, the light source 320 generating the point light, and the light source 320. Optical path conversion mirror 350 for detecting the path of the light generated from the condenser lens 330, the cylindrical lens 340, and the light source 310 to convert the point light to a linear light to be incident on the reflecting surface of the rotary polygon mirror 310 And a light receiving sensor 360.

이렇게 회전다면경(310)을 갖는 패턴광 영사부(300)는 구조가 단순하고 부피가 작으며, 전력 소모가 낮을 뿐만 아니라 도 4에 도시된 바와 같이 1차원의 패턴을 영사하여 3차원 입체 형상을 측정할 수 있으므로, 처리해야할 데이터가 적어 반응 속도가 빠르며 가격이 저렴한 장점이 있다.The patterned light projection unit 300 having the rotating multi-faceted mirror 310 has a simple three-dimensional structure by projecting a one-dimensional pattern as shown in FIG. Because it can be measured, there is less data to be processed, so the reaction rate is faster and the price is lower.

반면에, 단색광의 광원을 이용하기 때문에 다양한 색상의 패턴을 영사할 수 없는 단점이 있다.On the other hand, since a light source of monochromatic light is used, there is a disadvantage in that it is not possible to project patterns of various colors.

아울러, 종래의 패턴광 영사부를 이용한 입체 형상 측정에서는, 도 5에 도시된 바와 같이 위상 0°의 패턴광과, 위상 120°의 패턴광 및 위상 240° 패턴광 각각을 순차로 측정물에 조사하여, 각 위상별 영상을 순차로 하나씩 획득한 후 획득한 영상 3개를 조합하여 모아레 무늬를 획득한다. In addition, in the three-dimensional shape measurement using a conventional pattern light projection unit, as shown in Fig. 5, each of the pattern light of phase 0 °, the pattern light of phase 120 ° and the phase 240 ° pattern light is sequentially irradiated onto the measurement object. After obtaining one image for each phase one by one, a moire pattern is obtained by combining three acquired images.

따라서, 모아레 무늬를 얻기 위해 패턴광 영사 및 영상 획득 과정을 각각 3번씩 진행해야 하므로, 측정에 오랜 시간이 소요되는 단점이 있다.Therefore, in order to obtain a moire pattern, the pattern light projection and the image acquisition process have to be performed three times, respectively, and thus, a long time is required for measurement.

상기 배경 기술의 문제점을 해결하기 위한 본 발명의 목적은, 서로 다른 색상을 갖는 다수의 광원을 이용하여 서로 다른 위상을 갖도록 각 색상별 패턴광을 측정물에 동시에 영사하고, 컬러카메라로 한 번에 촬상한 후 각 색정보를 분리한 후 이를 조합하여 모아레 무늬를 획득함으로써, 적은 양의 입력 데이터로 다양한 색상을 구현할 수 있고 한번의 촬상으로 위상 천이된 영상을 한번에 획득하여 측정 속도 및 신뢰성을 확보할 수 있도록 하는 회전다면경을 이용한 3D 스캐너를 제공함에 있다.An object of the present invention for solving the problems of the background art, by simultaneously projecting the pattern light for each color to the measurement object to have a different phase by using a plurality of light sources having different colors, and at a time with a color camera By capturing each color information and then combining them to obtain a moire pattern, it is possible to realize various colors with a small amount of input data, and to obtain a measurement speed and reliability by acquiring phase shifted images at one time by one imaging. The present invention provides a 3D scanner using a rotating mirror.

상기 과제를 해결하기 위한 본 발명은 3D 스캐너에 있어서, 상기 회전다면경의 일측에 구비되며 서로 다른 색정보를 갖는 점광을 발생하는 다수의 광원과, 상기 각 광원으로부터 발생한 다양한 방향에서의 광을 동일한 광축으로 출사시키기 위한 광필터링 수단과, 상기 회전다면경의 반사면에서 반사되는 광을 수광하여 광로를 인식하여 동기신호를 발생하는 광경로인식수단과, 상기 광경로인식수단으로부터 발생하는 동기신호에 동기시켜 상기 광원의 점멸주기를 조정하여 직선 줄무늬의 주기와 위상을 제어하는 제어수단과, 상기 측정대상물 표면으로부터 반사되는 광을 촬상하는 컬러카메라를 포함한다.The present invention for solving the above problems, in the 3D scanner, a plurality of light sources provided on one side of the rotating polygon mirror and generating point light having different color information, and the same optical axis to the light in various directions generated from the respective light sources An optical filtering means for outputting the light source, an optical path recognition means for receiving the light reflected from the reflecting surface of the rotating multi-facet mirror, recognizing an optical path and generating a synchronization signal, and a synchronization signal generated from the optical path recognition means. Control means for controlling the period and phase of the straight stripes by adjusting the flashing period of the light source, and a color camera for imaging the light reflected from the surface of the measurement object.

상기 광필터링 수단은 상기 각 광원으로부터의 광을 동일한 광축으로 출사시키는 필터 또는 빔분할기로 구성될 수 있다.The light filtering means may be constituted by a filter or a beam splitter that emits light from each light source to the same optical axis.

상기 컬러카메라의 노출시간의 시작시점과 종료시점은 광경로인식수단에서 발생하는 동기신호와 일치됨이 바람직하다.The start point and the end point of the exposure time of the color camera are preferably matched with the synchronization signal generated by the optical path recognition means.

상기 회전다면경은 고속으로 등속 회전함이 바람직하다.The rotating mirror is preferably rotated at a constant speed at high speed.

본 발명은 다수의 광원으로 다양한 색상의 패턴광을 동시에 영사하고 컬러카메라 한번으로 다양한 색상의 패턴광을 갖는 측정물 표면 영상을 획득함으로써, 적은 양의 입력 데이터로 다양한 색상을 구현할 수 있고 한번의 촬상으로 위상 천이된 영상을 한 번에 획득함으로써, 측정 속도를 향상시킬 수 있다.The present invention can simultaneously project pattern light of various colors with a plurality of light sources and acquire a surface image of a workpiece having pattern light of various colors with a single color camera, thereby realizing various colors with a small amount of input data. By acquiring the phase shifted image at once, the measurement speed can be improved.

또한, 본 발명은 기존의 회전다면경을 갖는 3D 스캐너에 다수의 광원만을 추가함으로써 구조가 단순하여 전력 소모가 적고 부피가 작아 장비 가격을 저렴하게 할 수 있는 이점이 있다.In addition, the present invention has the advantage that the structure is simple by adding a plurality of light sources to the existing 3D scanner having a rotating multi-faceted mirror, so that the power consumption is small and the volume is small, so that the equipment price can be reduced.

도 6은 본 발명에 따른 회전다면경을 이용한 3D 스캐너 구성도이다.Figure 6 is a block diagram of a 3D scanner using a rotating mirror according to the present invention.

도 6을 참조하면, 회전다면경(10)과 다수의 광원(20 : 20a,20b,20c)과 광필터링 수단(30)과, 광경로인식수단(40)과, 제어수단(50)과, 컬러카메라(60)를 포함한다.Referring to FIG. 6, the rotating polyhedron 10, the plurality of light sources 20: 20a, 20b, 20c, the light filtering means 30, the light path recognition means 40, the control means 50, It includes a color camera 60.

여기서, 회전다면경(10)은 회전을 통해 반사면으로 입사된 광을 일정 주기의 패턴광 형태로 측정물(P) 표면에 입사시키기 위한 것으로서, 다수의 반사면을 갖는다. Here, the rotary mirror 10 is for incidence of light incident on the reflective surface through rotation to the surface of the measurement object P in the form of pattern light of a predetermined period, and has a plurality of reflective surfaces.

이러한 회전다면경(10)은 고속 회전을 통해 각 광원(20 : 20a,20b,20c)의 점멸 주기에 따라 일정 간격의 줄무늬를 갖는 패턴광을 형성하여 측정물(P)에 영사할 수 있도록 정속도로 회전 구동되는 스핀들 모터(미도시함)에 결합되어 있고, 스핀 들 모터(미도시함)는 전력을 안정적으로 공급하는 파워 서플라이가 구비된 모터 드라이버(미도시함)에 연결되어 있으며, 이에 대한 구성은 대한민국등록특허 0371078호에 공지된 바 있다.The rotating multi-face mirror 10 is a constant speed so as to form a pattern light having a stripe of a predetermined interval according to the flashing cycle of each light source (20: 20a, 20b, 20c) through a high-speed rotation to project to the measurement object (P) It is coupled to a spindle motor (not shown) that is driven to rotate on the road, and the spindle motor (not shown) is connected to a motor driver (not shown) having a power supply that reliably supplies power. The configuration is known from Republic of Korea Patent No. 0371078.

여기서, 회전다면경(10)은 고속으로 회전하면 회전관성모멘트가 증가하여 회전속도에 변동량이 극히 적어지기 때문에, 고속 회전시키는 것이 바람직하며, 아울러 입사되는 패턴광의 일정한 주기를 생성하기 위하여 등속 회전시키는 것이 바람직하다.  Here, since the rotational inertia 10 is rotated at high speed, the rotational moment of inertia increases and the variation in the rotational speed is extremely small. Therefore, the rotational mirror 10 is preferably rotated at high speed. It is preferable.

그리고, 다수의 광원(20 : 20a,20b,20c)은 회전다면경(10)의 일측에서 서로 다른 방향에서 서로 다른 색정보를 갖는 점광을 발생하는 것이다. In addition, the plurality of light sources 20: 20a, 20b, and 20c generate point light having different color information in different directions at one side of the rotating mirror 10.

즉, 서로 다른 색정보를 갖는다는 것은 파장대가 서로 다른 광을 발생시키는 것으로서, 각 광원광원(20 : 20a,20b,20c)은 예를 들어 R(Red, 20a), G(Green, 20b), B(Blue, 20c) 컬러의 광을 각각 발생시키도록 구성되는 고휘도 LED 또는 레이저 다이오드(Laser Diode, LD) 로 이루어질 수 있다.That is, having different color information generates light having different wavelength bands, and each light source light source 20: 20a, 20b, 20c is, for example, R (Red, 20a), G (Green, 20b), It may be made of a high-brightness LED or a laser diode (LD) configured to generate light of B (Blue, 20c) color, respectively.

아울러, 각 광원(20 : 20a,20b,20c)과 회전다면경(10) 사이에는 각 광원(20 : 20a,20b,20c)으로부터의 광을 집광하기 위한 집광 렌즈(21)가 구비된다. In addition, a condensing lens 21 for condensing light from each light source 20: 20a, 20b, 20c is provided between each light source 20: 20a, 20b, 20c and the rotary mirror 10.

그리고, 집광렌즈(21)를 통해 집광된 광은 광필터링 수단(30)에 의해 동일한 광축으로 출사된다. The light collected through the condenser lens 21 is emitted to the same optical axis by the light filtering means 30.

즉, 광필터링 수단(30)은 서로 다른 방향에서 입사되는 각 광원(20 : 20a,20b,20c)으로부터 발생한 점광의 광 경로를 변환하여 동일한 광축으로 출사시켜 후술하는 원통형 렌즈(22)에 동일한 광축으로 입사시킨다.That is, the optical filtering means 30 converts the optical path of the point light generated from each of the light sources 20: 20a, 20b, and 20c incident in different directions and emits the same optical axis to the cylindrical lens 22 described later. Incident.

이를 위해, 광필터링 수단(30)은 필터 또는 빔분할기가 이용될 수 있다.To this end, the light filtering means 30 may be a filter or a beam splitter.

이때, 제 1 광필터링 수단(31)과 제 2 광필터링 수단(32)의 투과율과 반사율을 도 7에 도시된 바와 같이 구성되는 것이 가장 효율적이다.At this time, it is most efficient to configure the transmittance and reflectance of the first light filtering means 31 and the second light filtering means 32 as shown in FIG.

이렇게 광필터링 수단(30)에 의해 동일한 광축으로 출사된 점광은 원통형 렌즈(22)를 통해 직선광으로 변환된다.The point light emitted by the light filtering means 30 on the same optical axis is converted into linear light through the cylindrical lens 22.

한편, 광경로인식수단(40)은 회전다면경의 반사면에서 반사되는 광을 수광하여 광로를 인식하여 동기신호를 발생한다. On the other hand, the optical path recognition means 40 receives the light reflected from the reflecting surface of the rotating multi-face mirror to recognize the optical path to generate a synchronization signal.

이를 위하여, 광경로인식수단(40)은 광로변환미러(41)와, 수광센서(42)와, 수광 드라이버(43)를 포함한다. To this end, the optical path recognition means 40 includes an optical path conversion mirror 41, a light receiving sensor 42, and a light receiving driver 43.

광로변환미러(41)는 회전다면경(10)의 반사면으로부터 반사되는 광의 경로를 변환한다.The optical path changing mirror 41 converts the path of the light reflected from the reflecting surface of the rotating polygon mirror 10.

수광센서(42)는 광로변환미러(41)를 통해 반사된 광을 수광하여 회전다면경(10) 반사면에서 반사되는 광의 광경로를 인식하는 것으로서 포토다이오드가 이용된다.The light receiving sensor 42 receives the light reflected through the optical path changing mirror 41 and recognizes the optical path of the light reflected from the reflecting surface of the rotating multifaceted mirror 10.

수광 드라이버(43)는 수광센서(42)로부터의 검출 신호에 근거하여 하나의 반사면에서의 반사광이 종료되었다는 신호 즉 동기신호를 발생한다. The light receiving driver 43 generates a signal, i.e., a synchronizing signal, indicating that the reflected light on one reflective surface is terminated based on the detection signal from the light receiving sensor 42.

제어수단(50)은 광경로인식수단(40)으로부터 발생하는 동기신호에 동기시켜 각 광원(20 : 20a,20b,20c)의 점멸주기를 조정하여 직선 줄무늬의 주기와 위상을 제어한다. The control means 50 controls the period and phase of the straight stripes by adjusting the blinking period of each light source 20: 20a, 20b, 20c in synchronization with the synchronization signal generated from the optical path recognition means 40.

즉, 제어수단(50)은 시간제어드라이버(51)를 통해 각 광원(20 : 20a,20b,20c)의 광출력 온/오프 시간을 제어하여 측정물(P)에 조사되는 직선 줄무늬 패턴광의 주기와 위상을 조절한다.That is, the control means 50 controls the light output on / off time of each light source 20: 20a, 20b, 20c through the time control driver 51, so that the period of the straight stripe pattern light irradiated to the measurement object P. Adjust phase with.

아울러, 제어수단(50)은 광원(20 : 20a,20b,20c)의 점멸주기 뿐만 아니라 회전다면경(10)의 회전 속도를 제어하도록 구성된다.In addition, the control means 50 is configured to control the rotation speed of the rotating mirror 10 as well as the flashing period of the light sources 20: 20a, 20b, 20c.

시간제어드라이버(51)는 수광 드라이버(43)에서 발생되는 동기신호의 1주기 동안에 제어수단(50)에서 발생하는 메인 클럭(Main Clock) 신호를 기준으로 하여, 각 광원(20 : 20a,20b,20c)을 점멸시키기 위한 점멸구동신호(VIDEO)를 발생한다.The time control driver 51 is based on the main clock signal generated by the control means 50 during one period of the synchronization signal generated by the light receiving driver 43, and the respective light sources 20: 20a, 20b, Generates a flashing drive signal (VIDEO) to flash 20c).

예를 들어, 도 8에 도시된 바와 같이 위상 0°는 R(Red) 색상 패턴으로, 위상 120° 는 G(Green) 색상 패턴으로, 위상 240°는 B(Blue) 색상 패턴으로 동시에 조사한다.For example, as shown in FIG. 8, the phase 0 ° is irradiated with the R (Red) color pattern, the phase 120 ° with the G (Green) color pattern, and the phase 240 ° with the B (Blue) color pattern.

이렇게 각 위상에 따라 서로 다른 색정보를 갖는 패턴광을 형성한 후에, 측정물(P) 표면 영상을 컬러카메라(60)로 촬상한다.After forming pattern light having different color information according to each phase in this way, the surface image of the measurement object P is captured by the color camera 60.

컬러카메라(60)는 측정물 표면에서 반사된 광으로부터 색상 정보를 분리한 후 이를 조합하여 하나의 영상으로 인식하는 것이다.The color camera 60 separates the color information from the light reflected from the surface of the workpiece and combines the color information into a single image.

따라서, 본 발명은 측정물(P) 표면에 각 위상별로 서로 다른 색상을 갖는 패턴광을 동시에 영사하고, 이를 컬러카메라(60)로 촬상한다.Therefore, the present invention simultaneously project pattern light having a different color for each phase on the surface of the workpiece P, and image it with the color camera 60.

여기서, 회전다면경(30) 하나의 반사면이 회전할 때마다 컬러카메라(60)가 촬상하고 있는 공간에 1번씩 레이저가 지나가므로, 광량이 충분할 경우 레이저가 한번 지나가는 시간 동안만 컬러카메라(60)를 노출시켜도 되지만, 광량이 충분하지 않을 경우에는 레이저가 여러 번 지나가는 동안 컬러카메라(60)를 노출시킨다. Here, since the laser passes once in the space where the color camera 60 is imaging every time one of the rotating mirrors 30 reflects, the color camera 60 only passes for a time when the laser beam passes once. ), But if the amount of light is not sufficient, the color camera 60 is exposed while the laser passes several times.

따라서, 컬러카메라(60)의 노출시간은 광경로인식수단(40)으로부터의 동기신호에 맞추어져야 한다. Therefore, the exposure time of the color camera 60 should be matched with the synchronization signal from the optical path recognition means 40.

즉, 컬러카메라(60) 노출시간의 시작시점과 종료시점은 광경로인식수단(40)으로부터의 동기신호와 일치되어야 한다.That is, the start point and the end point of the exposure time of the color camera 60 should coincide with the synchronization signal from the optical path recognition means 40.

그러면, 컬러카메라(60)에서는 촬상된 영상으로부터 각 위상별로 서로 다른 색상의 영상을 각각 분리한 후 이를 하나의 영상으로 조합하여 모아레 무늬를 획득한다. Then, the color camera 60 separates images of different colors for each phase from the captured image and combines them into one image to obtain a moire fringe.

즉, 종래에는 모아레 무늬 획득을 위하여 위상 천이된 패턴광을 여러번 영사하고, 패턴광을 영사할 때 마다 영상을 획득한 다음 획득된 다수의 영상을 조합해야 하므로, 측정에 많은 시간이 소요되었지만, 본 발명은 서로 다른 위상과 서로 다른 색 정보를 갖는 패턴광을 동시에 조사하고, 이를 컬러카메라로 한번에 촬상하여 모아레 무늬를 획득함으로써, 측정 속도를 향상시킬 수 있는 것이다.That is, in the related art, it is necessary to project the phase shifted pattern light several times for acquiring the moire pattern, and to acquire the images each time the pattern light is projected, and then combine a plurality of the acquired images. The present invention can improve the measurement speed by simultaneously irradiating patterned light having different phases and different color information, and imaging them at once with a color camera to obtain a moire fringe.

이하, 본 발명의 3D 스캐너를 이용한 작용을 간략하게 설명하면 다음과 같다.Hereinafter, a brief description of the operation using the 3D scanner of the present invention.

우선, 회전다면경(10) 일측에 구비된 각 광원(20 : 20a,20b,20c)으로부터 광을 발생시키고 회전다면경(10)을 고속 회전시켜 반사면으로 입사된 광을 반사시킨다.First, light is generated from each of the light sources 20: 20a, 20b, and 20c provided on one side of the rotating mirror mirror 10, and the rotating mirror mirror 10 is rotated at high speed to reflect light incident on the reflecting surface.

즉, 각 광원(20 : 20a,20b,20c)을 온시키면 광원에서 발생된 빛이 집광렌즈(21)를 통해 집광된 후에 광필터링 수단(30)를 통과하면서 동일한 광축으로 원통형렌즈(22)에 입사된다.That is, when each light source 20: 20a, 20b, 20c is turned on, light generated from the light source is collected through the condenser lens 21, and then passes through the light filtering means 30 to the cylindrical lens 22 with the same optical axis. Incident.

그리고, 동일한 광축으로 입사된 점광은 원통형렌즈(22)에서 직선광으로 변환되어 회전다면경(10)의 반사면으로 입사된 후 반사면을 통해 반사된다. Then, the point light incident on the same optical axis is converted into linear light in the cylindrical lens 22 and is incident on the reflecting surface of the rotating polygon mirror 10 and then reflected through the reflecting surface.

그리고, 회전다면경(10)의 반사면에서 반사된 광은 수광센서(42)에서 수광된다. Then, the light reflected from the reflecting surface of the rotating mirror mirror 10 is received by the light receiving sensor 42.

그러면 수광센서(42)는 회전다면경의 반사면에서 연속적으로 반사되는 광을 수광하여 수광된 광경로를 인식한 후 동기신호를 발생한다. Then, the light receiving sensor 42 receives light that is continuously reflected from the reflecting surface of the rotating polyhedron, recognizes the received light path, and generates a synchronization signal.

그리고, 시간제어드라이버(51)는 동기신호에 동기되어 일정 주기와 위상을 갖도록 각 광원(20 : 20a,20b,20c)의 점멸주기를 조절한다.The time control driver 51 adjusts the flashing period of each light source 20: 20a, 20b, 20c to have a predetermined period and phase in synchronization with the synchronous signal.

이때, 각 광원(20 : 20a,20b,20c)의 점멸주기는 상술한 도 8을 참조하여, 위상 0ㅀ는 R(Red) 색상 패턴으로, 위상 120ㅀ 는 G(Green) 색상 패턴으로, 위상 240ㅀ는 B(Blue) 색상 패턴을 생성하도록 한다. 여기서, 각 위상별 색상 예시는 단순한 예시일 뿐 이를 다른 방식으로 변형 실시할 수 있다.At this time, the blinking period of each light source 20: 20a, 20b, 20c is described with reference to FIG. 8, where phase 0 'is a R (Red) color pattern, phase 120 is a G (Green) color pattern, and a phase 240 kHz allows to generate a B (Blue) color pattern. Here, the color examples for each phase are merely examples and may be modified in other ways.

그러면, 각 색상별로 일정 주기와 위상을 갖는 다양한 색상의 패턴광이 측정물(P) 표면에 영사된다.Then, pattern light of various colors having a predetermined period and phase for each color is projected onto the measurement object P surface.

이후, 측정물(P) 표면으로부터 반사되는 패턴광을 컬러카메라(60) 촬상한다.Then, the pattern light reflected from the surface of the measurement object P is picked up by the color camera 60.

이때, 컬러카메라(60) 노출시간의 시작시점과 종료시점은 광경로인식수단(40)으로부터의 동기신호와 일치되는 것이 바람직하다.At this time, it is preferable that the start point and end point of the exposure time of the color camera 60 coincide with the synchronization signal from the optical path recognition means 40.

이어서, 컬러카메라(60)로 촬상된 영상에서 각각의 컬러를 분리하여 위상 0°에서의 영상, 위상 120°에서의 영상, 위상 240°에서의 영상을 얻을 수 있다.Subsequently, each color may be separated from an image captured by the color camera 60 to obtain an image at phase 0 °, an image at phase 120 °, and an image at phase 240 °.

이렇게 분리된 3장의 영상을 이용하여 모아레 무늬를 구하게 된다.Using these three separate images, moire patterns are obtained.

예를 들어, 각 광원(20 : 20a,20b,20c)이 각각 R, G, B 색상이면서, 도7에서의 타이밍 차트에 의해 점멸되었다면, 컬러카메라(60)로 촬상된 영상에서 R성분, G성분, B성분은 각각 위상 0°에서의 영상, 위상 120°에서의 영상, 위상 240°에서의 영상에 해당하게 된다.For example, if each of the light sources 20: 20a, 20b, and 20c are R, G, and B colors, respectively, and flashed by the timing chart in Fig. 7, the R component, G in the image photographed by the color camera 60 is shown. The component and B component correspond to the image at phase 0 degrees, the image at phase 120 degrees, and the image at phase 240 degrees, respectively.

도9는 본 발명에서 R, G, B성분이 합성된 패턴광이 측정물에 영사된 것을 칼라카메라로 획득된 영상이다.9 is an image obtained with a color camera that the pattern light synthesized with the R, G, and B components is projected onto the measurement object in the present invention.

이와 같이 본 발명은 서로 다른 색정보를 갖는 광원으로 측정물에 1차원의 패턴광을 다양한 색상의 조합으로 조사하고, 이를 컬러카메라로 촬상하여 각 색상별 영상을 분리한 다음 이를 기존의 모아레 무늬 계산방법에 따라 모아레 무늬를 한번에 획득할 수 있기 때문에 3차원 형상을 측정하는데 소요되는 시간을 줄일 수 있다.As described above, the present invention is a light source having different color information to irradiate the measurement object with a combination of one-dimensional pattern light in a variety of colors, and then to take a color camera to separate the image for each color and calculate the existing moire pattern According to the method, the moire pattern can be obtained at a time, thereby reducing the time required for measuring the three-dimensional shape.

도 1은 종래의 3D 스캐너의 일 실시예를 나타낸 개념도.1 is a conceptual diagram showing an embodiment of a conventional 3D scanner.

도 2는 도 1에 의해 영사되는 2차원 패턴광.FIG. 2 is a two-dimensional patterned light projected by FIG. 1. FIG.

도 3은 종래의 3D 스캐너의 다른 실시예를 나타낸 개념도.3 is a conceptual diagram showing another embodiment of the conventional 3D scanner.

도 4는 도 3에 의해 영사되는 1차원 패턴광.4 is a one-dimensional patterned light projected by FIG.

도 5는 종래 기술에 따른 3D 스캐너에서의 광원 점멸 타이밍 차트.5 is a timing chart of light source blinking in a 3D scanner according to the prior art.

도 6은 본 발명에 따른 회전다면경을 이용한 3D 스캐너 구성도.6 is a 3D scanner configuration using a rotating mirror according to the present invention.

도 7은 본 발명에 따른 회전다면경을 이용한 3D 스캐너에서의 광원 점멸 타이밍 차트.7 is a light source flashing timing chart in a 3D scanner using a rotating polyscopy according to the present invention.

도 8은 도 6의 각 광원에 대한 광필터링 수단의 가장 효과적인 투과율-반사율 구성 그래프.8 is a graph of the most effective transmittance-reflectance scheme of the light filtering means for each light source of FIG. 6;

도 9는 본 발명에 따라 획득된 모아레 무늬를 나타낸 도면.9 is a view showing a moire fringe obtained according to the present invention.

도9는 본 발명에 따라 R, G, B성분이 합성된 패턴광이 측정물에 영사된 것을 칼라카메라로 획득된 영상이다.FIG. 9 is an image obtained by a color camera that pattern light in which R, G, and B components are synthesized is projected onto a measurement object according to the present invention.

<도면의 주요부분에 대한 부호의 설명> <Description of the symbols for the main parts of the drawings>

10 : 회전다면경10: rotating face mirror

20 : 광원20: light source

20a : R20a: R

20b : G20b: G

20c : B20c: B

21 : 집광렌즈21 condensing lens

22 : 원통형 렌즈22: cylindrical lens

30 : 광필터링 수단30: light filtering means

31 : 제 1 광필터링 수단31: first optical filtering means

32 : 제 2 광필터링 수단32: second optical filtering means

40 : 광경로인식수단40: optical path recognition means

41 : 광로변환미러41: optical path mirror

42 : 수광센서42: light receiving sensor

43 : 수광 드라이버43: light receiving driver

50 : 제어수단50: control means

51 : 시간제어드라이버51: time control driver

60 : 컬러카메라60 color camera

Claims (3)

3D 스캐너에 있어서, In the 3D scanner, 다수의 반사면을 갖고, 일정속도로 회전하여 광을 일정 주기의 패턴광 형태로 측정물(P) 표면에 입사시키기 위한 회전다면경(10)과, A rotating multi-face mirror 10 having a plurality of reflective surfaces and rotating at a constant speed to allow light to be incident on the surface of the workpiece P in the form of pattern light of a predetermined period; 상기 회전다면경의 일측에 구비되어 It is provided on one side of the rotating multi-face mirror 서로 다른 색정보를 갖는 점광Point light with different color information 을 발생하는 Causing 다수의 광원(20)과 A plurality of light sources 20 ,, 상기 각 광원으로부터 발생한 다양한 방향에서의 광을 동일한 광축으로 회전다면경으로 출사시키 위한 광필터링 수단(30)과,An optical filtering means 30 for emitting light in various directions generated from the respective light sources with a rotating polyhedron at the same optical axis; 상기 회전다면경의 반사면에서 반사되는 패턴광을 수광하여 광로를 인식하여 동기신호를 발생하는 광경로인식수단(40)과,Optical path recognition means (40) for receiving pattern light reflected from the reflecting surface of the rotating multi-face mirror and recognizing the optical path to generate a synchronization signal; 상기 광경로인식수단으로부터 발생하는 동기신호에 동기시켜 상기 광원의 점멸주기를 조정하여 직선 줄무늬의 주기와 위상을 제어하는 제어수단(50)과,Control means (50) for controlling the period and phase of the straight stripes by adjusting the blinking period of the light source in synchronization with the synchronization signal generated from the optical path recognition means; 노출시간의 시작시점과 종료시점은 상기 광경로인식수단에서 발생하는 동기신호와 일치되고, 상기 측정대상물 표면으로부터 반사되는 광을 촬상하는 컬러카메라(60) 를 포함함을 특징으로 하는 회전다면경을 이용한 3D 스캐너.The start point and the end point of the exposure time coincide with the synchronization signal generated by the optical path recognition means, and include a color camera 60 for capturing light reflected from the surface of the measurement object. 3D scanner used. 삭제delete 삭제delete
KR1020080128763A 2008-12-17 2008-12-17 3D scanner using rotating face mirror Expired - Fee Related KR100902176B1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
KR1020080128763A KR100902176B1 (en) 2008-12-17 2008-12-17 3D scanner using rotating face mirror

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
KR1020080128763A KR100902176B1 (en) 2008-12-17 2008-12-17 3D scanner using rotating face mirror

Publications (1)

Publication Number Publication Date
KR100902176B1 true KR100902176B1 (en) 2009-06-10

Family

ID=40982602

Family Applications (1)

Application Number Title Priority Date Filing Date
KR1020080128763A Expired - Fee Related KR100902176B1 (en) 2008-12-17 2008-12-17 3D scanner using rotating face mirror

Country Status (1)

Country Link
KR (1) KR100902176B1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101477185B1 (en) * 2013-03-21 2014-12-29 부산대학교 산학협력단 3-Dimension Scanner Platform and Scanning Device Having the Same

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH11264722A (en) 1998-03-17 1999-09-28 Yuusuke Nonomura Three-dimensional measuring device
KR20000077129A (en) * 1999-05-29 2000-12-26 박윤창 striped pattern formatting apparatus and method using polygon mirror
KR20020039583A (en) * 2000-11-22 2002-05-27 임쌍근 Phase Shift Projection Moire Method and Apparatus Applying Moire Pattern Generator
JP2003075134A (en) * 2001-09-04 2003-03-12 Univ Kanazawa Shape measuring method and shape measuring device using optical interference

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH11264722A (en) 1998-03-17 1999-09-28 Yuusuke Nonomura Three-dimensional measuring device
KR20000077129A (en) * 1999-05-29 2000-12-26 박윤창 striped pattern formatting apparatus and method using polygon mirror
KR20020039583A (en) * 2000-11-22 2002-05-27 임쌍근 Phase Shift Projection Moire Method and Apparatus Applying Moire Pattern Generator
JP2003075134A (en) * 2001-09-04 2003-03-12 Univ Kanazawa Shape measuring method and shape measuring device using optical interference

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101477185B1 (en) * 2013-03-21 2014-12-29 부산대학교 산학협력단 3-Dimension Scanner Platform and Scanning Device Having the Same

Similar Documents

Publication Publication Date Title
US12042124B2 (en) Reflection suppression using fluorescence
US7349104B2 (en) System and a method for three-dimensional imaging systems
US20200186777A1 (en) Method and apparatus for colour imaging a three-dimensional structure
US20090322859A1 (en) Method and System for 3D Imaging Using a Spacetime Coded Laser Projection System
US9170097B2 (en) Hybrid system
EP1941843B1 (en) Method and apparatus for colour imaging a three-dimensional structure
JP4379056B2 (en) Three-dimensional imaging apparatus and method
JP3417222B2 (en) Real-time range finder
US20040125205A1 (en) System and a method for high speed three-dimensional imaging
JPH11211443A (en) Three-dimensional shape measuring device
CN112710253A (en) Three-dimensional scanner and three-dimensional scanning method
JP7664658B2 (en) Depth data measuring head, computing device and measuring method
KR101175780B1 (en) 3-Dimension depth camera using the Infrared laser projection display
KR100895856B1 (en) Three-dimensional shape measuring device using a rotating mirror and dual mirror and three-dimensional shape measuring method using the same
CN112712583B (en) Three-dimensional scanner, three-dimensional scanning system, and three-dimensional scanning method
JP3414624B2 (en) Real-time range finder
JP3818028B2 (en) 3D image capturing apparatus and 3D image capturing method
KR100902176B1 (en) 3D scanner using rotating face mirror
JP7298025B2 (en) Three-dimensional scanner and three-dimensional scanning method
WO2011142462A1 (en) Band-shaped member shape measuring method, and device therefor
JP2003014430A (en) Three-dimensional measuring method and three- dimensional measuring apparatus
JP2017037189A (en) Pattern irradiation device, system, operation method, and program
JP3668466B2 (en) Real-time range finder
JP2004053532A (en) Optical shape measuring device
JP2021043296A (en) Image detection device

Legal Events

Date Code Title Description
A201 Request for examination
PA0109 Patent application

St.27 status event code: A-0-1-A10-A12-nap-PA0109

PA0201 Request for examination

St.27 status event code: A-1-2-D10-D11-exm-PA0201

A302 Request for accelerated examination
PA0302 Request for accelerated examination

St.27 status event code: A-1-2-D10-D16-exm-PA0302

St.27 status event code: A-1-2-D10-D17-exm-PA0302

E902 Notification of reason for refusal
PE0902 Notice of grounds for rejection

St.27 status event code: A-1-2-D10-D21-exm-PE0902

E13-X000 Pre-grant limitation requested

St.27 status event code: A-2-3-E10-E13-lim-X000

P11-X000 Amendment of application requested

St.27 status event code: A-2-2-P10-P11-nap-X000

P13-X000 Application amended

St.27 status event code: A-2-2-P10-P13-nap-X000

E701 Decision to grant or registration of patent right
PE0701 Decision of registration

St.27 status event code: A-1-2-D10-D22-exm-PE0701

GRNT Written decision to grant
PR0701 Registration of establishment

St.27 status event code: A-2-4-F10-F11-exm-PR0701

PR1002 Payment of registration fee

Fee payment year number: 1

St.27 status event code: A-2-2-U10-U11-oth-PR1002

PG1601 Publication of registration

St.27 status event code: A-4-4-Q10-Q13-nap-PG1601

PR1001 Payment of annual fee

Fee payment year number: 4

St.27 status event code: A-4-4-U10-U11-oth-PR1001

R18-X000 Changes to party contact information recorded

St.27 status event code: A-5-5-R10-R18-oth-X000

FPAY Annual fee payment

Payment date: 20130405

Year of fee payment: 5

PR1001 Payment of annual fee

Fee payment year number: 5

St.27 status event code: A-4-4-U10-U11-oth-PR1001

FPAY Annual fee payment

Payment date: 20140530

Year of fee payment: 6

PR1001 Payment of annual fee

Fee payment year number: 6

St.27 status event code: A-4-4-U10-U11-oth-PR1001

FPAY Annual fee payment

Payment date: 20150528

Year of fee payment: 7

PR1001 Payment of annual fee

Fee payment year number: 7

St.27 status event code: A-4-4-U10-U11-oth-PR1001

P14-X000 Amendment of ip right document requested

St.27 status event code: A-5-5-P10-P14-nap-X000

R18-X000 Changes to party contact information recorded

St.27 status event code: A-5-5-R10-R18-oth-X000

FPAY Annual fee payment

Payment date: 20160527

Year of fee payment: 8

PR1001 Payment of annual fee

Fee payment year number: 8

St.27 status event code: A-4-4-U10-U11-oth-PR1001

P22-X000 Classification modified

St.27 status event code: A-4-4-P10-P22-nap-X000

FPAY Annual fee payment

Payment date: 20170602

Year of fee payment: 9

PR1001 Payment of annual fee

Fee payment year number: 9

St.27 status event code: A-4-4-U10-U11-oth-PR1001

P22-X000 Classification modified

St.27 status event code: A-4-4-P10-P22-nap-X000

FPAY Annual fee payment

Payment date: 20180531

Year of fee payment: 10

PR1001 Payment of annual fee

Fee payment year number: 10

St.27 status event code: A-4-4-U10-U11-oth-PR1001

FPAY Annual fee payment

Payment date: 20190529

Year of fee payment: 11

PR1001 Payment of annual fee

Fee payment year number: 11

St.27 status event code: A-4-4-U10-U11-oth-PR1001

R18-X000 Changes to party contact information recorded

St.27 status event code: A-5-5-R10-R18-oth-X000

PR1001 Payment of annual fee

Fee payment year number: 12

St.27 status event code: A-4-4-U10-U11-oth-PR1001

PN2301 Change of applicant

St.27 status event code: A-5-5-R10-R11-asn-PN2301

PN2301 Change of applicant

St.27 status event code: A-5-5-R10-R14-asn-PN2301

R18-X000 Changes to party contact information recorded

St.27 status event code: A-5-5-R10-R18-oth-X000

PC1903 Unpaid annual fee

Not in force date: 20210604

Payment event data comment text: Termination Category : DEFAULT_OF_REGISTRATION_FEE

St.27 status event code: A-4-4-U10-U13-oth-PC1903

K11-X000 Ip right revival requested

St.27 status event code: A-6-4-K10-K11-oth-X000

PC1903 Unpaid annual fee

Ip right cessation event data comment text: Termination Category : DEFAULT_OF_REGISTRATION_FEE

Not in force date: 20210604

St.27 status event code: N-4-6-H10-H13-oth-PC1903

PR0401 Registration of restoration

St.27 status event code: A-6-4-K10-K13-oth-PR0401

PR1001 Payment of annual fee

Fee payment year number: 13

St.27 status event code: A-4-4-U10-U11-oth-PR1001

R401 Registration of restoration
PN2301 Change of applicant

St.27 status event code: A-5-5-R10-R11-asn-PN2301

St.27 status event code: A-5-5-R10-R13-asn-PN2301

R18-X000 Changes to party contact information recorded

St.27 status event code: A-5-5-R10-R18-oth-X000

PR1001 Payment of annual fee

Fee payment year number: 14

St.27 status event code: A-4-4-U10-U11-oth-PR1001

PR1001 Payment of annual fee

Fee payment year number: 15

St.27 status event code: A-4-4-U10-U11-oth-PR1001

PC1903 Unpaid annual fee

Not in force date: 20240604

Payment event data comment text: Termination Category : DEFAULT_OF_REGISTRATION_FEE

St.27 status event code: A-4-4-U10-U13-oth-PC1903

PC1903 Unpaid annual fee

Ip right cessation event data comment text: Termination Category : DEFAULT_OF_REGISTRATION_FEE

Not in force date: 20240604

St.27 status event code: N-4-6-H10-H13-oth-PC1903

R18-X000 Changes to party contact information recorded

St.27 status event code: A-5-5-R10-R18-oth-X000