[go: up one dir, main page]

WO2018179421A1 - Computer system, artificial object diagnosis method, and program - Google Patents

Computer system, artificial object diagnosis method, and program Download PDF

Info

Publication number
WO2018179421A1
WO2018179421A1 PCT/JP2017/013821 JP2017013821W WO2018179421A1 WO 2018179421 A1 WO2018179421 A1 WO 2018179421A1 JP 2017013821 W JP2017013821 W JP 2017013821W WO 2018179421 A1 WO2018179421 A1 WO 2018179421A1
Authority
WO
WIPO (PCT)
Prior art keywords
artifact
image
image analysis
acquired
computer system
Prior art date
Application number
PCT/JP2017/013821
Other languages
French (fr)
Japanese (ja)
Inventor
俊二 菅谷
佳雄 奥村
Original Assignee
株式会社オプティム
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 株式会社オプティム filed Critical 株式会社オプティム
Priority to PCT/JP2017/013821 priority Critical patent/WO2018179421A1/en
Publication of WO2018179421A1 publication Critical patent/WO2018179421A1/en

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion

Definitions

  • the present invention relates to a computer system, an artifact diagnosis method, and a program for diagnosing an artifact.
  • thermo image diagnosis it is possible for an operator to know the temperature distribution without contacting facilities or equipment (see Non-Patent Document 1). Moreover, it is also possible for an operator to grasp the damage or discoloration of facilities or devices by monitoring the visible light image.
  • Non-Patent Document 1 or 2 an abnormality such as a facility or device is determined based only on acquired image data or non-image data, and diagnosis is performed by analyzing a plurality of image data. It was not a thing.
  • the present invention provides the following solutions.
  • the present invention is a computer system for diagnosing an artifact, First image acquisition means for acquiring a plurality of first artifact images accompanied by a time-series change of the artifact; First image analysis means for image analysis of the acquired first artifact image; Second image acquisition means for acquiring a plurality of second artifact images accompanied by a time-series change of another artifact in the past; Second image analysis means for image analysis of the acquired second artifact image; Collating means for collating the result of image analysis of the first artifact image with the result of image analysis of the second artifact image; Diagnostic means for diagnosing the artifact based on the collated result; A computer system is provided.
  • a computer system for diagnosing an artifact obtains a plurality of first artifact images accompanied by a time-series change of the artifact, and performs image analysis on the obtained first artifact image, Acquiring a plurality of second artifact images accompanied by a time-series change of another artifact in the past, performing image analysis on the acquired second artifact image, and a result of image analysis of the first artifact image; The result of image analysis of the second artifact image is collated, and the artifact is diagnosed based on the collation result.
  • the present invention is a computer system category, but also in other categories such as an artifact diagnosis method and program, the same actions and effects according to the category are exhibited.
  • the present invention it is possible to provide a computer system, an artifact diagnosis method, and a program in which a plurality of time-series image data are combined and diagnosis accuracy is further improved as compared with diagnosis based on conventional single image analysis. It becomes.
  • FIG. 1 is a diagram showing an outline of the artifact diagnosis system 1.
  • FIG. 2 is an overall configuration diagram of the artifact diagnosis system 1.
  • FIG. 3 is a functional block diagram of the computer 10.
  • FIG. 4 is a flowchart showing a learning process executed by the computer 10.
  • FIG. 5 is a flowchart showing an artifact diagnosis process executed by the computer 10.
  • FIG. 6 is a diagram illustrating a first artifact image and a second artifact image that the computer 10 collates.
  • FIG. 1 is a diagram for explaining an outline of an artifact diagnosis system 1 which is a preferred embodiment of the present invention.
  • the artifact diagnosis system 1 includes a computer 10 and is a computer system that diagnoses artifacts.
  • the artifacts diagnosed by the artifact diagnosis system 1 are, for example, pipes, roads and bridges, buildings, and arbitrary artifacts (vehicles, air conditioners, home appliances, information processing devices, etc.).
  • the artifact diagnosis system 1 acquires an image of a pipe, performs image analysis on a marked (enclosed) image with respect to the pipe image, and diagnoses a pipe crack. To do.
  • the computer 10 is connected to various imaging devices such as an infrared camera, visible light camera, X-ray camera, and ultrasonic camera (not shown), and various devices that store or measure environmental data such as internal flow rate, temperature, and humidity. Is a computing device.
  • imaging devices such as an infrared camera, visible light camera, X-ray camera, and ultrasonic camera (not shown), and various devices that store or measure environmental data such as internal flow rate, temperature, and humidity.
  • the computer 10 acquires a plurality of first artifact images accompanied by a time-series change of the artifact (step S01).
  • the computer 10 acquires any one or a combination of X-ray images, infrared images, ultrasonic images, or visible light images as the first artifact image.
  • the computer 10 obtains the above-described first artifact image captured by the above-described various imaging devices.
  • the first artifact image is not limited to the image described above, and may be other images.
  • the computer 10 performs image analysis on the acquired first artifact image (step S02).
  • the computer 10 performs image analysis by analyzing either or both of the feature point and the feature amount of the first artifact image.
  • a feature point is something reflected in an image, specifically, shape, color, brightness, outline, and the like.
  • the feature amount is a statistical numerical value such as various numerical values (average of pixel values, variance, histogram) calculated from image data.
  • the computer 10 performs machine learning in advance on one or both of feature points and feature amounts of a second artifact image, which will be described later, as teacher data, and performs image analysis on the first artifact image based on the learning result. Good. Further, the computer 10 may perform image analysis on an image marked (enclosed) with respect to the first artifact image by a terminal device (not shown) or the like. The mark means to enclose each specific part of the image.
  • the computer 10 acquires a plurality of second artifact images accompanied by a time-series change of another artifact in the past (step S03).
  • the computer 10 acquires the second artifact image from another computer or database not shown. At this time, the computer 10 acquires one or a plurality of second artifact images.
  • the computer 10 performs image analysis on the acquired second artifact image (step S04).
  • the computer 10 performs image analysis by analyzing either or both of the feature point and the feature amount of the second artifact image.
  • the computer 10 executes image analysis similar to the image analysis of the first artifact image described above. That is, when the computer 10 analyzes the feature point with respect to the first artifact image, the computer 10 analyzes the feature point of the second artifact image, and analyzes the feature amount with respect to the first artifact image.
  • the feature amount of the second artifact image is analyzed and the feature point and feature amount are analyzed for the first artifact image, the feature point and feature amount of the second artifact image are analyzed.
  • the computer 10 may perform image analysis on the marked image with respect to the second artifact image by a terminal device (not shown) or the like.
  • the computer 10 collates the image analysis result of the first artifact image with the image analysis result of the second artifact image (step S05).
  • the computer 10 collates either or both of the feature point or feature amount analyzed from the first artifact image with either or both of the feature point or feature amount analyzed from the second artifact image.
  • the computer 10 diagnoses the artifact based on the collation result (step S06). For example, the computer 10 calculates the similarity between the first artifact image and the second artifact image based on the collation result, and diagnoses the artifact.
  • the computer 10 may diagnose a risk related to a defect such as a crack of an artifact based on the collation result.
  • the risk related to defects indicates, for example, a percentage of the occurrence rate of defects in an artificial object that is diagnosed.
  • FIG. 2 is a diagram showing a system configuration of the artifact diagnosis system 1 which is a preferred embodiment of the present invention.
  • the artifact diagnosis system 1 includes a computer 10 and a public line network (Internet network, third generation, fourth generation communication network, etc.) 5 and is a computer system that diagnoses artifacts.
  • the computer 10 is the above-described computing device having the functions described later.
  • FIG. 3 is a functional block diagram of the computer 10.
  • the computer 10 includes a CPU (Central Processing Unit), a RAM (Random Access Memory), a ROM (Read Only Memory), etc. as the control unit 11, and a device for enabling communication with other devices as the communication unit 12. For example, a WiFi (Wireless Fidelity) compatible device compliant with IEEE 802.11 is provided.
  • the computer 10 also includes a data storage unit such as a hard disk, a semiconductor memory, a recording medium, or a memory card as the storage unit 13. Further, the computer 10 includes, as the processing unit 14, a device for executing various processes such as image processing, state diagnosis, and learning process.
  • control unit 11 reads a predetermined program, thereby realizing the artifact image acquisition module 20 and the diagnosis result acquisition module 21 in cooperation with the communication unit 12. Further, in the computer 10, the control unit 11 reads a predetermined program, thereby realizing the storage module 30 in cooperation with the storage unit 13. Further, in the computer 10, the control unit 11 reads a predetermined program, thereby realizing an analysis module 40, a learning module 41, a matching module 42, and a diagnosis module 43 in cooperation with the processing unit 14.
  • FIG. 4 is a diagram illustrating a flowchart of the learning process executed by the computer 10. Processing executed by each module described above will be described together with this processing.
  • the artifact image acquisition module 20 acquires a known second artifact image (step S10).
  • the second artifact image is obtained by making a plurality of images accompanied by a time-series change of the artifact into one image.
  • the second artifact image acquired by the artifact image acquisition module 20 is, for example, at least one of an X-ray image, an infrared image, an ultrasonic image, or a visible light image of the artifact.
  • the artifact image acquisition module 20 may acquire the second artifact image from the corresponding imaging device, may acquire it via a computer (not shown), or is stored in the computer or the like.
  • the second artifact image may be acquired from a database or the like. In the following description, the artifact image acquisition module 20 will be described as having acquired a pipe image as the second artifact image.
  • the analysis module 40 performs image analysis on either or both of the feature point and the feature amount from the acquired second artifact image (step S11).
  • the feature point is something reflected in the second artifact image, and specifically, the shape, brightness, color, contour, etc. of the artifact reflected in the image.
  • the feature amount is a statistical numerical value such as various numerical values (average pixel value, variance, histogram) calculated from the second artifact image.
  • the analysis module 40 extracts feature points and feature amounts by performing image analysis on the second artifact image. Specifically, the analysis module 40 performs image analysis on the second artifact image by executing image matching technology, blob analysis, and the like. Moreover, the analysis module 40 extracts a feature amount by executing a predetermined calculation on the second artifact image.
  • the diagnosis result acquisition module 21 acquires the diagnosis result of the artifact corresponding to the second artifact image acquired this time (step S12). In step S12, the diagnosis result acquisition module 21 acquires a diagnosis result for the artifact associated with the acquired second artifact image.
  • the diagnosis result acquisition module 21 may acquire this diagnosis result via a computer or the like (not shown), or acquire it from a database or the like stored in this computer or the like.
  • the diagnosis result in the present embodiment is, for example, identification of symptoms, identification of necessary treatment, and the like.
  • the learning module 41 learns by associating the second artifact image and the diagnosis result (step S13).
  • step S ⁇ b> 13 the learning module 41 learns by associating the second artifact image acquired by the artifact image acquisition module 20 with the diagnosis result acquired by the diagnosis result acquisition module 21.
  • the learning module 41 learns at least one of the above-described X-ray image, infrared image, ultrasonic image, or visible light image of the artifact in association with the diagnosis result.
  • the learning performed by the learning module 41 is machine learning that repeatedly learns from data and finds a pattern hidden in the learning.
  • or S13 mentioned above instead of the analysis module 40 extracting either a feature point or a feature-value, or both, a target location is detected with the terminal device etc. which the worker who is not shown in figure holds.
  • the learning module 41 may learn by associating the second artifact image with the diagnosis result. In this case, the analysis module 40 learns the marked second artifact image and the diagnosis result in association with each other.
  • the analysis module 40 performs image analysis on the marked image. That is, one or both of the feature point and the feature amount of the marked part are extracted.
  • the analysis module 40 may extract the area, shape, etc. of the marked part as a feature point or feature amount.
  • the storage module 30 stores the learning result as a learning result (step S14).
  • the artifact diagnosis system 1 executes the above-described learning process a sufficient number of times and stores the learning result.
  • FIG. 5 is a diagram illustrating a flowchart of the artifact diagnosis process executed by the computer 10. Processing executed by each module described above will be described together with this processing. In the following description, the artifact diagnosis system 1 will be described as diagnosing a pipe crack based on a pipe image.
  • the artifact image acquisition module 20 acquires a first artifact image (step S20).
  • the first artifact image is a single image formed from a plurality of images accompanying a time-series change of the artifact.
  • the first artifact image acquired by the artifact image acquisition module 20 is, for example, at least one of an X-ray image, an infrared image, an ultrasonic image, or a visible light image of the artifact.
  • the artifact image acquisition module 20 may acquire the first artifact image from a corresponding imaging device, may acquire via a computer not shown, or the like, and is stored in this computer or the like
  • the first artifact image may be acquired from a database or the like.
  • an image of an artificial object from before (for example, an image stored in a computer or the like not shown) and an image captured by an imaging device or the like this time are collected along a time-series change.
  • the first image is acquired as the first artifact image.
  • the artifact image acquisition module 20 will be described as having acquired a pipe image as the first artifact image.
  • the analysis module 40 performs image analysis on either or both of the feature point and the feature amount from the acquired first artifact image (step S21).
  • the feature points and feature amounts are as described above.
  • the analysis module 40 extracts feature points and feature amounts of the first artifact image, as in step S11 described above.
  • the analysis module 40 performs image analysis on the marked first artifact image.
  • the marked first artifact image is an image surrounded by a specific part of the image (a part where the hue is different, such as a part where a crack occurs or a part where the temperature is high). This mark is given by a terminal device (not shown) or automatically.
  • the analysis module 40 extracts feature points and feature amounts based on differences in image pigments in the marked area.
  • the artifact image acquisition module 20 acquires a second artifact image (step S22).
  • step S22 the artifact image acquisition module 20 acquires the learning result stored in the storage module 30 as the second artifact image by the process in step S14 described above. At this time, the second artifact image acquired by the artifact image acquisition module 20 is marked.
  • the artifact image acquisition module 20 may acquire not a learning result but a plurality of images accompanied by a time-series change of another artifact in the past as the second artifact image. Further, the artifact image acquisition module 20 may acquire a second artifact image that is not marked.
  • the analysis module 40 performs image analysis on either or both of the feature point and the feature amount from the acquired second artifact image (step S23).
  • the feature points and feature amounts of the second artifact image are extracted in the same manner as the processes of steps S11 and S21 described above.
  • the collation module 42 collates the result of image analysis of the first artifact image and the result of image analysis of the second artifact image (step S24).
  • step S24 the collation module 42 collates either or both of the feature points or feature amounts extracted from the first artifact image with either or both of the feature points analyzed from the second artifact image.
  • the matching module 42 checks the feature point of the first artifact image and the feature point of the second artifact image.
  • the feature amount is extracted from the first artifact image
  • the feature point of the first artifact image and the feature amount of the second artifact image are collated, and the feature point and feature amount are extracted from the first artifact image. Is extracted, the feature point and feature amount of the first artifact image are collated with the feature point and feature amount of the second artifact image.
  • a specific collation method will be described later.
  • FIG. 6 is a diagram illustrating a first artifact image and a second artifact image that are collated by the collation module 42.
  • the matching module 42 arranges the first artifact images (first piping image 200, second piping image 210, and third piping image 220) in the first artifact image display area 100, and The second artifact image (the fourth piping image 300, the fifth piping image 310) is aligned and collated in the two artifact image display area 110.
  • the first piping image 200, the second piping image 210, and the third piping image 220 show time-series changes of the artifacts in this order.
  • the 4th piping image 300 and the 5th piping image 310 show the change of the time series of another past artifact different from the 1st artifact image.
  • the fifth piping image 310 is a piping image in a state where a crack has occurred.
  • the collation module 42 collates the change from the fourth piping image 300 to the fifth piping image 310 with the first artifact image as a feature point or feature amount.
  • the collation module 42 determines, as a score, the degree of similarity of the degree of similarity between the change in the first artifact image and the change in the second artifact image. For example, this score is determined as a high score as there are many similarities in change, and as a low score as there are few similarities in change.
  • the collation module 42 collates the change in the first artifact image as a feature point or feature amount, and whether or not the similarity with the pipe image in the state where the second artifact image is cracked has a high score. It is determined whether or not the artifact is cracked.
  • the number of the first artifact images and the second artifact images that are collated by the collation module 42 is not limited to the above-described number, and can be changed as appropriate. Further, the collation module 42 may collate more artifact images in addition to the first artifact image and the second artifact image.
  • the diagnosis module 43 diagnoses the artifact based on the collated result (step S25).
  • the diagnosis module 43 diagnoses the artifact based on the similarity calculated by the matching module 42. For example, when the similarity is a score equal to or higher than a predetermined value as a result of the collation, the diagnosis module 43 gives a diagnosis result similar to the diagnosis result performed on the second artifact image to the artifact. Diagnose.
  • the diagnosis module 43 may diagnose a symptom or treatment itself as a diagnosis result, or may diagnose a risk related to a defect of an artifact.
  • the risk related to the defect is, for example, what percentage the probability of occurrence of the diagnosed defect is, what percentage is the probability of occurrence of the corresponding defect in the future.
  • the diagnostic module 43 further collates the other second artifact image to obtain another diagnosis result for this artifact. Diagnose it.
  • the means and functions described above are realized by a computer (including a CPU, an information processing apparatus, and various terminals) reading and executing a predetermined program.
  • the program is provided, for example, in a form (SaaS: Software as a Service) provided from a computer via a network.
  • the program is provided in a form recorded on a computer-readable recording medium such as a flexible disk, CD (CD-ROM, etc.), DVD (DVD-ROM, DVD-RAM, etc.).
  • the computer reads the program from the recording medium, transfers it to the internal storage device or the external storage device, stores it, and executes it.
  • the program may be recorded in advance in a storage device (recording medium) such as a magnetic disk, an optical disk, or a magneto-optical disk, and provided from the storage device to a computer via a communication line.

Landscapes

  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Pathology (AREA)
  • General Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Surgery (AREA)
  • Biochemistry (AREA)
  • Biophysics (AREA)
  • Analytical Chemistry (AREA)
  • Chemical & Material Sciences (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Medical Informatics (AREA)
  • Molecular Biology (AREA)
  • Immunology (AREA)
  • Animal Behavior & Ethology (AREA)
  • Public Health (AREA)
  • Veterinary Medicine (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Theoretical Computer Science (AREA)
  • Image Analysis (AREA)

Abstract

[Problem] To provide a computer system, an artificial object diagnosis method, and a program, which are capable of, by combining a plurality of time-sequential image data sets, improving the accuracy of diagnosis more than a conventional diagnosis based on a single image analysis. [Solution] This computer system that diagnoses an artificial object acquires a plurality of first artificial object images including temporal change of the artificial object, performs image analysis on the acquired first artificial object images, acquires a plurality of second artificial object images obtained in the past and including temporal change of another artificial object, performs image analysis on the acquired second artificial object images, compares the image analysis result of the first artificial object images with the image analysis result of the second artificial object images, and diagnoses the artificial object on the basis of the comparison result.

Description

コンピュータシステム、人工物診断方法及びプログラムComputer system, artifact diagnosis method and program
　本発明は、人工物を診断するコンピュータシステム、人工物診断方法及びプログラムに関する。 The present invention relates to a computer system, an artifact diagnosis method, and a program for diagnosing an artifact.
　近年、画像診断により、インフラとなる道路や工場の施設や配管や装置等を診断する方法が知られている。例えば、サーモ画像診断では、作業者が施設や装備等に接触せずに温度分布を知ることが可能である（非特許文献１参照）。また、作業者が可視光画像を監視することにより、施設や装置等の破損や変色等を把握することも可能である。 In recent years, a method of diagnosing infrastructure roads, factory facilities, piping, equipment, etc. by image diagnosis is known. For example, in thermo image diagnosis, it is possible for an operator to know the temperature distribution without contacting facilities or equipment (see Non-Patent Document 1). Moreover, it is also possible for an operator to grasp the damage or discoloration of facilities or devices by monitoring the visible light image.
　一方、工場や施設では、画像外データ（配管の温度データ、配管を流れる流体の流量変化データ）等の画像とは異なる外部リソースとなるデータで、施設や装備等の診断を行うことも可能となっている（非特許文献２参照）。 On the other hand, in factories and facilities, it is possible to diagnose facilities and equipment with data that is an external resource different from images such as data outside the image (pipe temperature data, flow rate change data of fluid flowing in the pipe). (See Non-Patent Document 2).
　しかしながら、しかしながら、非特許文献１又は２の構成では、取得できた画像データ又は画像外データのみで施設や装置等の異常を判定するものであり、複数の画像データを解析して、診断を行うものではなかった。 However, in the configuration of Non-Patent Document 1 or 2, an abnormality such as a facility or device is determined based only on acquired image data or non-image data, and diagnosis is performed by analyzing a plurality of image data. It was not a thing.
　本発明は、時系列の複数の画像データを組み合わせ、従来の単体の画像解析による診断よりも、さらに診断の精度を向上させたコンピュータシステム、人工物診断方法及びプログラムを提供することを目的とする。 It is an object of the present invention to provide a computer system, an artifact diagnosis method, and a program in which a plurality of time-series image data are combined and the diagnosis accuracy is further improved as compared with a diagnosis based on a conventional single image analysis. .
　本発明では、以下のような解決手段を提供する。 The present invention provides the following solutions.
　本発明は、人工物を診断するコンピュータシステムであって、
　前記人工物の時系列の変化を伴う複数枚の第１人工物画像を取得する第１画像取得手段と、
　取得した前記第１人工物画像を画像解析する第１画像解析手段と、
　過去の別の人工物の時系列の変化を伴う複数枚の第２人工物画像を取得する第２画像取得手段と、
　取得した前記第２人工物画像を画像解析する第２画像解析手段と、
　前記第１人工物画像の画像解析の結果と、前記第２人工物画像の画像解析の結果とを照合する照合手段と、
　照合した結果に基づいて、前記人工物を診断する診断手段と、
　を備えることを特徴とするコンピュータシステムを提供する。 The present invention is a computer system for diagnosing an artifact,
First image acquisition means for acquiring a plurality of first artifact images accompanied by a time-series change of the artifact;
First image analysis means for image analysis of the acquired first artifact image;
Second image acquisition means for acquiring a plurality of second artifact images accompanied by a time-series change of another artifact in the past;
Second image analysis means for image analysis of the acquired second artifact image;
Collating means for collating the result of image analysis of the first artifact image with the result of image analysis of the second artifact image;
Diagnostic means for diagnosing the artifact based on the collated result;
A computer system is provided.
　本発明によれば、人工物を診断するコンピュータシステムは、前記人工物の時系列の変化を伴う複数枚の第１人工物画像を取得し、取得した前記第１人工物画像を画像解析し、過去の別の人工物の時系列の変化を伴う複数枚の第２人工物画像を取得し、取得した前記第２人工物画像を画像解析し、前記第１人工物画像の画像解析の結果と、前記第２人工物画像の画像解析の結果とを照合し、照合した結果に基づいて、前記人工物を診断する。 According to the present invention, a computer system for diagnosing an artifact obtains a plurality of first artifact images accompanied by a time-series change of the artifact, and performs image analysis on the obtained first artifact image, Acquiring a plurality of second artifact images accompanied by a time-series change of another artifact in the past, performing image analysis on the acquired second artifact image, and a result of image analysis of the first artifact image; The result of image analysis of the second artifact image is collated, and the artifact is diagnosed based on the collation result.
　本発明は、コンピュータシステムのカテゴリであるが、人工物診断方法及びプログラム等の他のカテゴリにおいても、そのカテゴリに応じた同様の作用・効果を発揮する。 The present invention is a computer system category, but also in other categories such as an artifact diagnosis method and program, the same actions and effects according to the category are exhibited.
　本発明によれば、時系列の複数の画像データを組み合わせ、従来の単体の画像解析による診断よりも、さらに診断の精度を向上させたコンピュータシステム、人工物診断方法及びプログラムを提供することが可能となる。 According to the present invention, it is possible to provide a computer system, an artifact diagnosis method, and a program in which a plurality of time-series image data are combined and diagnosis accuracy is further improved as compared with diagnosis based on conventional single image analysis. It becomes.
図１は、人工物診断システム１の概要を示す図である。FIG. 1 is a diagram showing an outline of the artifact diagnosis system 1. 図２は、人工物診断システム１の全体構成図である。FIG. 2 is an overall configuration diagram of the artifact diagnosis system 1. 図３は、コンピュータ１０の機能ブロック図である。FIG. 3 is a functional block diagram of the computer 10. 図４は、コンピュータ１０が実行する学習処理を示すフローチャートである。FIG. 4 is a flowchart showing a learning process executed by the computer 10. 図５は、コンピュータ１０が実行する人工物診断処理を示すフローチャートである。FIG. 5 is a flowchart showing an artifact diagnosis process executed by the computer 10. 図６は、コンピュータ１０が照合する第１人工物画像と、第２人工物画像とを示す図である。FIG. 6 is a diagram illustrating a first artifact image and a second artifact image that the computer 10 collates.
　以下、本発明を実施するための最良の形態について図を参照しながら説明する。なお、これはあくまでも一例であって、本発明の技術的範囲はこれに限られるものではない。 Hereinafter, the best mode for carrying out the present invention will be described with reference to the drawings. This is merely an example, and the technical scope of the present invention is not limited to this.
　［人工物診断システム１の概要］
　本発明の好適な実施形態の概要について、図１に基づいて説明する。図１は、本発明の好適な実施形態である人工物診断システム１の概要を説明するための図である。人工物診断システム１は、コンピュータ１０から構成され、人工物を診断するコンピュータシステムである。人工物診断システム１が診断する人工物とは、例えば、配管、道路や橋、建築物、任意の人工物（車両、冷暖房装置、家庭電化製品、情報処理機器等）である。 [Outline of Artifact Diagnosis System 1]
An outline of a preferred embodiment of the present invention will be described with reference to FIG. FIG. 1 is a diagram for explaining an outline of an artifact diagnosis system 1 which is a preferred embodiment of the present invention. The artifact diagnosis system 1 includes a computer 10 and is a computer system that diagnoses artifacts. The artifacts diagnosed by the artifact diagnosis system 1 are, for example, pipes, roads and bridges, buildings, and arbitrary artifacts (vehicles, air conditioners, home appliances, information processing devices, etc.).
　なお、以下において、人工物診断システム１は、配管の画像を取得し、この配管の画像に対してマークされた（囲まれた）画像について画像解析し、配管のクラックの診断をするものとして説明する。 In the following description, it is assumed that the artifact diagnosis system 1 acquires an image of a pipe, performs image analysis on a marked (enclosed) image with respect to the pipe image, and diagnoses a pipe crack. To do.
　コンピュータ１０は、図示していない赤外線カメラ、可視光カメラ、Ｘ線カメラ、超音波カメラ等の各種撮像装置等や、内部流量、温度、湿度等の環境データ等を記憶又は計測する各種装置に接続された計算装置である。 The computer 10 is connected to various imaging devices such as an infrared camera, visible light camera, X-ray camera, and ultrasonic camera (not shown), and various devices that store or measure environmental data such as internal flow rate, temperature, and humidity. Is a computing device.
　はじめに、コンピュータ１０は、人工物の時系列の変化を伴う複数枚の第１人工物画像を取得する（ステップＳ０１）。コンピュータ１０は、第１人工物画像として、レントゲン画像、赤外線画像、超音波画像又は可視光画像のいずれか又は複数の組み合わせを取得する。コンピュータ１０は、上述した各種撮像装置が撮像した上述した第１人工物画像を取得する。なお、第１人工物画像は、上述した画像に限らず、その他の画像であってもよい。 First, the computer 10 acquires a plurality of first artifact images accompanied by a time-series change of the artifact (step S01). The computer 10 acquires any one or a combination of X-ray images, infrared images, ultrasonic images, or visible light images as the first artifact image. The computer 10 obtains the above-described first artifact image captured by the above-described various imaging devices. The first artifact image is not limited to the image described above, and may be other images.
　コンピュータ１０は、取得した第１人工物画像を画像解析する（ステップＳ０２）。コンピュータ１０は、第１人工物画像の特徴点又は特徴量のいずれか又は双方を解析することにより、画像解析を実行する。特徴点とは、画像に映っている何かであり具体的には、形状、色、輝度、輪郭等である。特徴量とは、画像データから算出した各種数値（画素値の平均、分散、ヒストグラム）等の統計的な数値である。 The computer 10 performs image analysis on the acquired first artifact image (step S02). The computer 10 performs image analysis by analyzing either or both of the feature point and the feature amount of the first artifact image. A feature point is something reflected in an image, specifically, shape, color, brightness, outline, and the like. The feature amount is a statistical numerical value such as various numerical values (average of pixel values, variance, histogram) calculated from image data.
　なお、コンピュータ１０は、後述する第２人工物画像の特徴点又は特徴量のいずれか又は双方を教師データとして予め機械学習し、この学習結果に基づいて、第１人工物画像を画像解析しもよい。また、コンピュータ１０は、図示していない端末装置等により、第１人工物画像に対してマークされた（囲まれた）画像について画像解析を行ってもよい。マークとは、画像の特定部位毎等に囲みを付けることを意味する。 Note that the computer 10 performs machine learning in advance on one or both of feature points and feature amounts of a second artifact image, which will be described later, as teacher data, and performs image analysis on the first artifact image based on the learning result. Good. Further, the computer 10 may perform image analysis on an image marked (enclosed) with respect to the first artifact image by a terminal device (not shown) or the like. The mark means to enclose each specific part of the image.
　コンピュータ１０は、過去の別の人工物の時系列の変化を伴う複数枚の第２人工物画像を取得する（ステップＳ０３）。コンピュータ１０は、図示していない他のコンピュータやデータベース等から、第２人工物画像を取得する。このとき、コンピュータ１０は、一又は複数の第２人工物画像を取得する。 The computer 10 acquires a plurality of second artifact images accompanied by a time-series change of another artifact in the past (step S03). The computer 10 acquires the second artifact image from another computer or database not shown. At this time, the computer 10 acquires one or a plurality of second artifact images.
　コンピュータ１０は、取得した第２人工物画像を画像解析する（ステップＳ０４）。コンピュータ１０は、第２人工物画像の特徴点又は特徴量のいずれか又は双方を解析することにより、画像解析を実行する。このとき、コンピュータ１０は、上述した第１人工物画像の画像解析と同様の画像解析を実行する。すなわち、コンピュータ１０は、第１人工物画像に対して、特徴点を解析した場合、第２人工物画像の特徴点を解析し、第１人工物画像に対して、特徴量を解析した場合、第２人工物画像の特徴量を解析し、第１人工物画像に対して、特徴点及び特徴量を解析した場合、第２人工物画像の特徴点及び特徴量を解析する。 The computer 10 performs image analysis on the acquired second artifact image (step S04). The computer 10 performs image analysis by analyzing either or both of the feature point and the feature amount of the second artifact image. At this time, the computer 10 executes image analysis similar to the image analysis of the first artifact image described above. That is, when the computer 10 analyzes the feature point with respect to the first artifact image, the computer 10 analyzes the feature point of the second artifact image, and analyzes the feature amount with respect to the first artifact image. When the feature amount of the second artifact image is analyzed and the feature point and feature amount are analyzed for the first artifact image, the feature point and feature amount of the second artifact image are analyzed.
　なお、コンピュータ１０は、図示していない端末装置等により、第２人工物画像に対してマークされた画像について画像解析を行ってもよい。 Note that the computer 10 may perform image analysis on the marked image with respect to the second artifact image by a terminal device (not shown) or the like.
　コンピュータ１０は、第１人工物画像の画像解析の結果と、第２人工物画像の画像解析の結果とを照合する（ステップＳ０５）。コンピュータ１０は、第１人工物画像から解析した特徴点又は特徴量のいずれか又は双方と、第２人工物画像から解析した特徴点又は特徴量のいずれか又は双方とを照合する。 The computer 10 collates the image analysis result of the first artifact image with the image analysis result of the second artifact image (step S05). The computer 10 collates either or both of the feature point or feature amount analyzed from the first artifact image with either or both of the feature point or feature amount analyzed from the second artifact image.
　コンピュータ１０は、照合した結果に基づいて、人工物を診断する（ステップＳ０６）。コンピュータ１０は、例えば、照合した結果に基づいて、第１人工物画像と、第２人工物画像との間の類似度を算出し、人工物を診断する。 The computer 10 diagnoses the artifact based on the collation result (step S06). For example, the computer 10 calculates the similarity between the first artifact image and the second artifact image based on the collation result, and diagnoses the artifact.
　なお、コンピュータ１０は、照合した結果に基づいて、人工物の割れ等の不具合に関するリスクを診断してもよい。不具合に関するリスクとは、例えば診断した人工物に何パーセント程度、その人工物における不具合の発生率の数字を示す。 Note that the computer 10 may diagnose a risk related to a defect such as a crack of an artifact based on the collation result. The risk related to defects indicates, for example, a percentage of the occurrence rate of defects in an artificial object that is diagnosed.
　以上が、人工物診断システム１の概要である。 The above is the outline of the artifact diagnosis system 1.
　［人工物診断システム１のシステム構成］
　図２に基づいて、本発明の好適な実施形態である人工物診断システム１のシステム構成について説明する。図２は、本発明の好適な実施形態である人工物診断システム１のシステム構成を示す図である。人工物診断システム１は、コンピュータ１０、公衆回線網（インターネット網や、第３、第４世代通信網等）５から構成され、人工物を診断するコンピュータシステムである。 [System configuration of the artifact diagnosis system 1]
Based on FIG. 2, the system configuration | structure of the artifact diagnostic system 1 which is suitable embodiment of this invention is demonstrated. FIG. 2 is a diagram showing a system configuration of the artifact diagnosis system 1 which is a preferred embodiment of the present invention. The artifact diagnosis system 1 includes a computer 10 and a public line network (Internet network, third generation, fourth generation communication network, etc.) 5 and is a computer system that diagnoses artifacts.
　コンピュータ１０は、後述の機能を備えた上述した計算装置である。 The computer 10 is the above-described computing device having the functions described later.
　［各機能の説明］
　図３に基づいて、本発明の好適な実施形態である人工物診断システム１の機能について説明する。図３は、コンピュータ１０の機能ブロック図を示す図である。 [Description of each function]
Based on FIG. 3, the function of the artifact diagnosis system 1 which is a preferred embodiment of the present invention will be described. FIG. 3 is a functional block diagram of the computer 10.
　コンピュータ１０は、制御部１１として、ＣＰＵ（Ｃｅｎｔｒａｌ　Ｐｒｏｃｅｓｓｉｎｇ　Ｕｎｉｔ）、ＲＡＭ（Ｒａｎｄｏｍ　Ａｃｃｅｓｓ　Ｍｅｍｏｒｙ）、ＲＯＭ（Ｒｅａｄ　Ｏｎｌｙ　Ｍｅｍｏｒｙ）等を備え、通信部１２として、他の機器と通信可能にするためのデバイス、例えば、ＩＥＥＥ８０２．１１に準拠したＷｉＦｉ（Ｗｉｒｅｌｅｓｓ　Ｆｉｄｅｌｉｔｙ）対応デバイスを備える。また、コンピュータ１０は、記憶部１３として、ハードディスクや半導体メモリ、記録媒体、メモリカード等によるデータのストレージ部を備える。また、コンピュータ１０は、処理部１４として、画像処理、状態診断、学習処理等の各種処理を実行するためのデバイス等を備える。 The computer 10 includes a CPU (Central Processing Unit), a RAM (Random Access Memory), a ROM (Read Only Memory), etc. as the control unit 11, and a device for enabling communication with other devices as the communication unit 12. For example, a WiFi (Wireless Fidelity) compatible device compliant with IEEE 802.11 is provided. The computer 10 also includes a data storage unit such as a hard disk, a semiconductor memory, a recording medium, or a memory card as the storage unit 13. Further, the computer 10 includes, as the processing unit 14, a device for executing various processes such as image processing, state diagnosis, and learning process.
　コンピュータ１０において、制御部１１が所定のプログラムを読み込むことにより、通信部１２と協働して、人工物画像取得モジュール２０、診断結果取得モジュール２１を実現する。また、コンピュータ１０において、制御部１１が所定のプログラムを読み込むことにより、記憶部１３と協働して、記憶モジュール３０を実現する。また、コンピュータ１０において、制御部１１が所定のプログラムを読み込むことにより、処理部１４と協働して解析モジュール４０、学習モジュール４１、照合モジュール４２、診断モジュール４３を実現する。 In the computer 10, the control unit 11 reads a predetermined program, thereby realizing the artifact image acquisition module 20 and the diagnosis result acquisition module 21 in cooperation with the communication unit 12. Further, in the computer 10, the control unit 11 reads a predetermined program, thereby realizing the storage module 30 in cooperation with the storage unit 13. Further, in the computer 10, the control unit 11 reads a predetermined program, thereby realizing an analysis module 40, a learning module 41, a matching module 42, and a diagnosis module 43 in cooperation with the processing unit 14.
　［学習処理］
　図４に基づいて、人工物診断システム１が実行する学習処理について説明する。図４は、コンピュータ１０が実行する学習処理のフローチャートを示す図である。上述した各モジュールが実行する処理について、本処理に併せて説明する。 [Learning process]
A learning process executed by the artifact diagnosis system 1 will be described with reference to FIG. FIG. 4 is a diagram illustrating a flowchart of the learning process executed by the computer 10. Processing executed by each module described above will be described together with this processing.
　人工物画像取得モジュール２０は、既知の第２人工物画像を取得する（ステップＳ１０）。第２人工物画像は、人工物の時系列の変化を伴う複数枚の画像を一の画像としたものである。ステップＳ１０において、人工物画像取得モジュール２０が取得する第２人工物画像とは、例えば、人工物のレントゲン画像、赤外線画像、超音波画像又は可視光画像の少なくとも一つである。人工物画像取得モジュール２０は、第２人工物画像を、対応する撮像装置から取得してもよいし、図示してないコンピュータ等を介して取得してもよいし、このコンピュータ等に記憶されたデータベース等から、第２人工物画像を取得してもよい。以下の説明において、人工物画像取得モジュール２０は、第２人工物画像として、配管の画像を取得したものとして説明する。 The artifact image acquisition module 20 acquires a known second artifact image (step S10). The second artifact image is obtained by making a plurality of images accompanied by a time-series change of the artifact into one image. In step S10, the second artifact image acquired by the artifact image acquisition module 20 is, for example, at least one of an X-ray image, an infrared image, an ultrasonic image, or a visible light image of the artifact. The artifact image acquisition module 20 may acquire the second artifact image from the corresponding imaging device, may acquire it via a computer (not shown), or is stored in the computer or the like. The second artifact image may be acquired from a database or the like. In the following description, the artifact image acquisition module 20 will be described as having acquired a pipe image as the second artifact image.
　解析モジュール４０は、取得した第２人工物画像から特徴点又は特徴量のいずれか又は双方を画像解析する（ステップＳ１１）。特徴点とは、第２人工物画像に映っている何かであり、具体的には、画像に映っている人工物等の形状、輝度、色、輪郭等である。また、特徴量とは、第２人工物画像から算出した各種数値（画素値の平均、分散、ヒストグラム）等の統計的な数値である。ステップＳ１１において、解析モジュール４０は、第２人工物画像を画像解析することにより、特徴点や特徴量を抽出する。具体的には、解析モジュール４０は、第２人工物画像に対して、画像マッチング技術、ブロブ解析等を実行することにより、画像解析を行う。また、解析モジュール４０は、第２人工物画像に対して、所定の計算を実行することにより、特徴量を抽出する。 The analysis module 40 performs image analysis on either or both of the feature point and the feature amount from the acquired second artifact image (step S11). The feature point is something reflected in the second artifact image, and specifically, the shape, brightness, color, contour, etc. of the artifact reflected in the image. The feature amount is a statistical numerical value such as various numerical values (average pixel value, variance, histogram) calculated from the second artifact image. In step S <b> 11, the analysis module 40 extracts feature points and feature amounts by performing image analysis on the second artifact image. Specifically, the analysis module 40 performs image analysis on the second artifact image by executing image matching technology, blob analysis, and the like. Moreover, the analysis module 40 extracts a feature amount by executing a predetermined calculation on the second artifact image.
　診断結果取得モジュール２１は、今回取得した第２人工物画像に該当する人工物の診断結果を取得する（ステップＳ１２）。ステップＳ１２において、診断結果取得モジュール２１は、取得した第２人工物画像に紐付けられたこの人工物に対する診断結果を取得する。診断結果取得モジュール２１は、この診断結果を、図示してないコンピュータ等を介して取得してもよいし、このコンピュータ等に記憶されたデータベース等から取得する。本実施形態における診断結果とは、例えば、症状の特定、必要な処置の特定等である。 The diagnosis result acquisition module 21 acquires the diagnosis result of the artifact corresponding to the second artifact image acquired this time (step S12). In step S12, the diagnosis result acquisition module 21 acquires a diagnosis result for the artifact associated with the acquired second artifact image. The diagnosis result acquisition module 21 may acquire this diagnosis result via a computer or the like (not shown), or acquire it from a database or the like stored in this computer or the like. The diagnosis result in the present embodiment is, for example, identification of symptoms, identification of necessary treatment, and the like.
　学習モジュール４１は、第２人工物画像と、診断結果とを対応付けて学習する（ステップＳ１３）。ステップＳ１３において、学習モジュール４１は、人工物画像取得モジュール２０が取得した第２人工物画像と、診断結果取得モジュール２１が取得した診断結果とを対応付けて学習する。学習モジュール４１は、上述した人工物のレントゲン画像、赤外線画像、超音波画像又は可視光画像の少なくとも一つを診断結果と対応付けて学習する。学習モジュール４１が実行する学習とは、データから反復的に学習し、そこに潜むパターンを見つけ出す機械学習である。 The learning module 41 learns by associating the second artifact image and the diagnosis result (step S13). In step S <b> 13, the learning module 41 learns by associating the second artifact image acquired by the artifact image acquisition module 20 with the diagnosis result acquired by the diagnosis result acquisition module 21. The learning module 41 learns at least one of the above-described X-ray image, infrared image, ultrasonic image, or visible light image of the artifact in association with the diagnosis result. The learning performed by the learning module 41 is machine learning that repeatedly learns from data and finds a pattern hidden in the learning.
　なお、上述したステップＳ１１乃至Ｓ１３の処理において、解析モジュール４０が特徴点又は特徴量のいずれか又は双方を抽出する代わりに、図示していない作業従事者が保有する端末装置等により、対象箇所をマークした（囲った）第２人工物画像に基づいて、学習モジュール４１は、第２人工物画像と診断結果とを対応付けて学習してもよい。この場合、解析モジュール４０は、マークされた第２人工物画像と、診断結果とを対応付けて学習する。 In addition, in the process of step S11 thru | or S13 mentioned above, instead of the analysis module 40 extracting either a feature point or a feature-value, or both, a target location is detected with the terminal device etc. which the worker who is not shown in figure holds. Based on the marked (enclosed) second artifact image, the learning module 41 may learn by associating the second artifact image with the diagnosis result. In this case, the analysis module 40 learns the marked second artifact image and the diagnosis result in association with each other.
　第２人工物画像における各画像の其々は、上述した端末装置等により、マークを付与される。解析モジュール４０は、このマークされた画像について画像解析する。すなわち、マークされた部位の特徴点又は特徴量のいずれか又は双方を抽出する。なお、解析モジュール４０は、このマークされた部位の面積や、形状等を特徴点や特徴量として抽出してもよい。 Each mark in the second artifact image is given a mark by the terminal device described above. The analysis module 40 performs image analysis on the marked image. That is, one or both of the feature point and the feature amount of the marked part are extracted. The analysis module 40 may extract the area, shape, etc. of the marked part as a feature point or feature amount.
　記憶モジュール３０は、学習した結果を、学習結果として記憶する（ステップＳ１４）。 The storage module 30 stores the learning result as a learning result (step S14).
　人工物診断システム１は、上述した学習処理を、十分な回数実行し、学習した結果を記憶する。 The artifact diagnosis system 1 executes the above-described learning process a sufficient number of times and stores the learning result.
　以上が、学習処理である。 The above is the learning process.
　［人工物診断処理］
　図５に基づいて、人工物診断システム１が実行する人工物診断処理について説明する。図５は、コンピュータ１０が実行する人工物診断処理のフローチャートを示す図である。上述した各モジュールが実行する処理について、本処理に併せて説明する。なお、以下の説明において、人工物診断システム１は、配管の画像に基づいて配管の割れを診断するものとして説明する。 [Artifact diagnosis processing]
Based on FIG. 5, the artifact diagnosis processing executed by the artifact diagnosis system 1 will be described. FIG. 5 is a diagram illustrating a flowchart of the artifact diagnosis process executed by the computer 10. Processing executed by each module described above will be described together with this processing. In the following description, the artifact diagnosis system 1 will be described as diagnosing a pipe crack based on a pipe image.
　人工物画像取得モジュール２０は、第１人工物画像を取得する（ステップＳ２０）。第１人工物画像とは、人工物の時系列の変化に伴う複数枚の画像を一の画像としたものである。ステップＳ２０において、人工物画像取得モジュール２０が取得する第１人工物画像とは、例えば、人工物のレントゲン画像、赤外線画像、超音波画像又は可視光画像の少なくとも一つである。人工物画像取得モジュール２０は、第１人工物画像を、対応する撮像装置から取得してもよいし、図示してないコンピュータ等を介して取得してもよいし、このコンピュータ等に記憶されたデータベース等から、第１人工物画像を取得してもよい。具体的には、人工物の以前からの画像（例えば、図示していないコンピュータ等に記憶しておいた画像）と、今回撮像装置等により撮像した画像とを、時系列の変化に沿ってまとめた一の画像を第１人工物画像として取得する。以下の説明において、人工物画像取得モジュール２０は、第１人工物画像として、配管の画像を取得したものとして説明する。 The artifact image acquisition module 20 acquires a first artifact image (step S20). The first artifact image is a single image formed from a plurality of images accompanying a time-series change of the artifact. In step S20, the first artifact image acquired by the artifact image acquisition module 20 is, for example, at least one of an X-ray image, an infrared image, an ultrasonic image, or a visible light image of the artifact. The artifact image acquisition module 20 may acquire the first artifact image from a corresponding imaging device, may acquire via a computer not shown, or the like, and is stored in this computer or the like The first artifact image may be acquired from a database or the like. Specifically, an image of an artificial object from before (for example, an image stored in a computer or the like not shown) and an image captured by an imaging device or the like this time are collected along a time-series change. The first image is acquired as the first artifact image. In the following description, the artifact image acquisition module 20 will be described as having acquired a pipe image as the first artifact image.
　解析モジュール４０は、取得した第１人工物画像から特徴点又は特徴量のいずれか又は双方を画像解析する（ステップＳ２１）。特徴点及び特徴量は、上述した通りである。解析モジュール４０は、上述したステップＳ１１と同様に、第１人工物画像の特徴点や特徴量を抽出する。このとき、解析モジュール４０は、マーク（囲み）された第１人工物画像について画像解析する。マークされた第１人工物画像とは、画像の特定部位毎（割れが発生している箇所、高温になっている箇所等の色相が異なる部位等）等に囲った画像である。このマークは、図示していない端末装置等により付与されたり、自動的に付与される。解析モジュール４０は、マークされた領域内の画像色素の違い等に基づいて、特徴点や特徴量の抽出を行う。 The analysis module 40 performs image analysis on either or both of the feature point and the feature amount from the acquired first artifact image (step S21). The feature points and feature amounts are as described above. The analysis module 40 extracts feature points and feature amounts of the first artifact image, as in step S11 described above. At this time, the analysis module 40 performs image analysis on the marked first artifact image. The marked first artifact image is an image surrounded by a specific part of the image (a part where the hue is different, such as a part where a crack occurs or a part where the temperature is high). This mark is given by a terminal device (not shown) or automatically. The analysis module 40 extracts feature points and feature amounts based on differences in image pigments in the marked area.
　人工物画像取得モジュール２０は、第２人工物画像を取得する（ステップＳ２２）。ステップＳ２２において、人工物画像取得モジュール２０は、上述したステップＳ１４の処理により、記憶モジュール３０が記憶した学習結果を第２人工物画像として取得する。このとき、人工物画像取得モジュール２０が取得する第２人工物画像には、マークが付与されている。 The artifact image acquisition module 20 acquires a second artifact image (step S22). In step S22, the artifact image acquisition module 20 acquires the learning result stored in the storage module 30 as the second artifact image by the process in step S14 described above. At this time, the second artifact image acquired by the artifact image acquisition module 20 is marked.
　なお、人工物画像取得モジュール２０は、学習結果ではなく、過去の別の人工物の時系列の変化を伴う複数枚の画像を第２人工物画像として取得してもよい。また、人工物画像取得モジュール２０は、マークが付与されていない状態の第２人工物画像を取得してもよい。 Note that the artifact image acquisition module 20 may acquire not a learning result but a plurality of images accompanied by a time-series change of another artifact in the past as the second artifact image. Further, the artifact image acquisition module 20 may acquire a second artifact image that is not marked.
　解析モジュール４０は、取得した第２人工物画像から特徴点又は特徴量のいずれか又は双方を画像解析する（ステップＳ２３）。ステップＳ２３の処理は、上述したステップＳ１１及びステップＳ２１の処理と同様に、第２人工物画像の特徴点や特徴量を抽出する。 The analysis module 40 performs image analysis on either or both of the feature point and the feature amount from the acquired second artifact image (step S23). In the process of step S23, the feature points and feature amounts of the second artifact image are extracted in the same manner as the processes of steps S11 and S21 described above.
　照合モジュール４２は、第１人工物画像の画像解析の結果と、第２人工物画像の画像解析の結果とを照合する（ステップＳ２４）。ステップＳ２４において、照合モジュール４２は、第１人工物画像から抽出した特徴点又は特徴量のいずれか又は双方と、第２人工物画像から解析した特徴点のいずれか又は双方とを照合する。このとき、照合モジュール４２は、第１人工物画像から特徴点を抽出した場合、第１人工物画像の特徴点と、第２人工物画像の特徴点とを照合する。同様に、第１人工物画像から特徴量を抽出した場合、第１人工物画像の特徴点と、第２人工物画像の特徴量とを照合し、第１人工物画像から特徴点及び特徴量を抽出した場合、第１人工物画像の特徴点及び特徴量と、第２人工物画像の特徴点及び特徴量とを照合する。具体的な照合の方法については、後述する。 The collation module 42 collates the result of image analysis of the first artifact image and the result of image analysis of the second artifact image (step S24). In step S24, the collation module 42 collates either or both of the feature points or feature amounts extracted from the first artifact image with either or both of the feature points analyzed from the second artifact image. At this time, when the feature point is extracted from the first artifact image, the matching module 42 checks the feature point of the first artifact image and the feature point of the second artifact image. Similarly, when the feature amount is extracted from the first artifact image, the feature point of the first artifact image and the feature amount of the second artifact image are collated, and the feature point and feature amount are extracted from the first artifact image. Is extracted, the feature point and feature amount of the first artifact image are collated with the feature point and feature amount of the second artifact image. A specific collation method will be described later.
　図６に基づいて、照合モジュール４２が照合する第１人工物画像と、第２人工物画像とについて説明する。図６は、照合モジュール４２が照合する第１人工物画像と、第２人工物画像とを示す図である。図６において、照合モジュール４２は、第１人工物画像表示領域１００に、第１人工物画像（第１の配管画像２００、第２の配管画像２１０、第３の配管画像２２０）を並べ、第２人工物画像表示領域１１０に、第２人工物画像（第４の配管画像３００、第５の配管画像３１０）を並べて照合する。第１の配管画像２００、第２の配管画像２１０、第３の配管画像２２０は、この順番にある人工物の時系列の変化を示す。また、第４の配管画像３００、第５の配管画像３１０は、第１人工物画像とは異なる過去の別の人工物の時系列の変化を示す。ここで、第５の配管画像３１０は、割れが発生した状態の配管画像である。照合モジュール４２は、第４の配管画像３００から第５の配管画像３１０への変化を、特徴点や特徴量として、第１人工物画像と照合する。このとき、照合モジュール４２は、第１人工物画像における変化と、第２人工物画像における変化との間に類似点がどの程度存在するかの類似度をスコアとして判断する。このスコアは、例えば、変化の類似点が多い程、高スコアとして判断し、変化の類似点が少ない程、低スコアとして判断する。照合モジュール４２は、第１人工物画像の変化を、特徴点や特徴量として照合し、第２の人工物画像における割れが発生した状態の配管画像との類似点が高スコアであるか否かを判断することにより、この人工物に割れが発生しているか否かを判断する。 The first artifact image and the second artifact image that are collated by the collation module 42 will be described with reference to FIG. FIG. 6 is a diagram illustrating a first artifact image and a second artifact image that are collated by the collation module 42. In FIG. 6, the matching module 42 arranges the first artifact images (first piping image 200, second piping image 210, and third piping image 220) in the first artifact image display area 100, and The second artifact image (the fourth piping image 300, the fifth piping image 310) is aligned and collated in the two artifact image display area 110. The first piping image 200, the second piping image 210, and the third piping image 220 show time-series changes of the artifacts in this order. Moreover, the 4th piping image 300 and the 5th piping image 310 show the change of the time series of another past artifact different from the 1st artifact image. Here, the fifth piping image 310 is a piping image in a state where a crack has occurred. The collation module 42 collates the change from the fourth piping image 300 to the fifth piping image 310 with the first artifact image as a feature point or feature amount. At this time, the collation module 42 determines, as a score, the degree of similarity of the degree of similarity between the change in the first artifact image and the change in the second artifact image. For example, this score is determined as a high score as there are many similarities in change, and as a low score as there are few similarities in change. The collation module 42 collates the change in the first artifact image as a feature point or feature amount, and whether or not the similarity with the pipe image in the state where the second artifact image is cracked has a high score. It is determined whether or not the artifact is cracked.
　なお、照合モジュール４２が照合する第１人工物画像及び第２人工物画像の数は、上述した数に限らず、適宜変更可能である。また、照合モジュール４２は、第１人工物画像及び第２人工物画像に加え、さらに多くの人工物画像を照合してもよい。 In addition, the number of the first artifact images and the second artifact images that are collated by the collation module 42 is not limited to the above-described number, and can be changed as appropriate. Further, the collation module 42 may collate more artifact images in addition to the first artifact image and the second artifact image.
　診断モジュール４３は、照合した結果に基づいて、人工物を診断する（ステップＳ２５）。ステップＳ２５において、診断モジュール４３は、照合モジュール４２が算出した類似度に基づいて、人工物を診断する。例えば、診断モジュール４３は、照合した結果、類似度が所定の値以上のスコアであった場合、第２人工物画像に対して行われた診断結果と同様の診断結果を、この人工物に対して診断する。このとき、診断モジュール４３は、症状や処置そのものを診断結果として診断してもよいし、人工物の不具合に関するリスクを診断してもよい。不具合に関するリスクとは、例えば、診断した不具合が発生している確率が何パーセント程度であるか、将来の該当する不具合が発生する確率が何パーセント程度であるか等である。 The diagnosis module 43 diagnoses the artifact based on the collated result (step S25). In step S25, the diagnosis module 43 diagnoses the artifact based on the similarity calculated by the matching module 42. For example, when the similarity is a score equal to or higher than a predetermined value as a result of the collation, the diagnosis module 43 gives a diagnosis result similar to the diagnosis result performed on the second artifact image to the artifact. Diagnose. At this time, the diagnosis module 43 may diagnose a symptom or treatment itself as a diagnosis result, or may diagnose a risk related to a defect of an artifact. The risk related to the defect is, for example, what percentage the probability of occurrence of the diagnosed defect is, what percentage is the probability of occurrence of the corresponding defect in the future.
　なお、診断モジュール４３は、照合した結果、類似度が所定の値よりも小さいスコアであった場合、他の第２人工物画像をさらに照合することにより、その他の診断結果を、この人工物に対して診断する。 If the similarity is a score smaller than a predetermined value as a result of the collation, the diagnostic module 43 further collates the other second artifact image to obtain another diagnosis result for this artifact. Diagnose it.
　上述した手段、機能は、コンピュータ（ＣＰＵ、情報処理装置、各種端末を含む）が、所定のプログラムを読み込んで、実行することによって実現される。プログラムは、例えば、コンピュータからネットワーク経由で提供される（ＳａａＳ：ソフトウェア・アズ・ア・サービス）形態で提供される。また、プログラムは、例えば、フレキシブルディスク、ＣＤ（ＣＤ－ＲＯＭなど）、ＤＶＤ（ＤＶＤ－ＲＯＭ、ＤＶＤ－ＲＡＭなど）等のコンピュータ読取可能な記録媒体に記録された形態で提供される。この場合、コンピュータはその記録媒体からプログラムを読み取って内部記憶装置又は外部記憶装置に転送し記憶して実行する。また、そのプログラムを、例えば、磁気ディスク、光ディスク、光磁気ディスク等の記憶装置（記録媒体）に予め記録しておき、その記憶装置から通信回線を介してコンピュータに提供するようにしてもよい。 The means and functions described above are realized by a computer (including a CPU, an information processing apparatus, and various terminals) reading and executing a predetermined program. The program is provided, for example, in a form (SaaS: Software as a Service) provided from a computer via a network. The program is provided in a form recorded on a computer-readable recording medium such as a flexible disk, CD (CD-ROM, etc.), DVD (DVD-ROM, DVD-RAM, etc.). In this case, the computer reads the program from the recording medium, transfers it to the internal storage device or the external storage device, stores it, and executes it. The program may be recorded in advance in a storage device (recording medium) such as a magnetic disk, an optical disk, or a magneto-optical disk, and provided from the storage device to a computer via a communication line.
　以上、本発明の実施形態について説明したが、本発明は上述したこれらの実施形態に限るものではない。また、本発明の実施形態に記載された効果は、本発明から生じる最も好適な効果を列挙したに過ぎず、本発明による効果は、本発明の実施形態に記載されたものに限定されるものではない。 As mentioned above, although embodiment of this invention was described, this invention is not limited to these embodiment mentioned above. The effects described in the embodiments of the present invention are only the most preferable effects resulting from the present invention, and the effects of the present invention are limited to those described in the embodiments of the present invention. is not.
　１　人工物診断システム１０　コンピュータ 1 Artifact diagnosis system 10 Computer

Claims (11)

  1. 　人工物を診断するコンピュータシステムであって、
    　前記人工物の時系列の変化を伴う複数枚の第１人工物画像を取得する第１画像取得手段と、
    　取得した前記第１人工物画像を画像解析する第１画像解析手段と、
    　過去の別の人工物の時系列の変化を伴う複数枚の第２人工物画像を取得する第２画像取得手段と、
    　取得した前記第２人工物画像を画像解析する第２画像解析手段と、
    　前記第１人工物画像の画像解析の結果と、前記第２人工物画像の画像解析の結果とを照合する照合手段と、
    　照合した結果に基づいて、前記人工物を診断する診断手段と、
    　を備えることを特徴とするコンピュータシステム。     A computer system for diagnosing artifacts,
    First image acquisition means for acquiring a plurality of first artifact images accompanied by a time-series change of the artifact;
    First image analysis means for image analysis of the acquired first artifact image;
    Second image acquisition means for acquiring a plurality of second artifact images accompanied by a time-series change of another artifact in the past;
    Second image analysis means for image analysis of the acquired second artifact image;
    Collating means for collating the result of image analysis of the first artifact image with the result of image analysis of the second artifact image;
    Diagnostic means for diagnosing the artifact based on the collated result;
    A computer system comprising:
  2. 　前記照合手段は、前記第１人工物画像の特徴点と、前記第２人工物画像の特徴点とを照合する、
    　ことを特徴とする請求項１に記載のコンピュータシステム。     The collation means collates the feature points of the first artifact image with the feature points of the second artifact image;
    The computer system according to claim 1.
  3. 　前記照合手段は、前記第１人工物画像の特徴量と、前記第２人工物画像の特徴量とを照合する、
    　ことを特徴とする請求項１に記載のコンピュータシステム。     The collation means collates the feature quantity of the first artifact image with the feature quantity of the second artifact image;
    The computer system according to claim 1.
  4. 　前記診断手段は、照合した結果に基づいて、前記第１人工物画像と、前記第２人工物画像との間の類似度を算出し、診断する、
    　ことを特徴とする請求項１に記載のコンピュータシステム。     The diagnostic means calculates a degree of similarity between the first artifact image and the second artifact image based on the collated result, and diagnoses it.
    The computer system according to claim 1.
  5. 　前記診断手段は、照合した結果に基づいて、前記人工物の劣化に関するリスクを診断する、
    　ことを特徴とする請求項１に記載のコンピュータシステム。     The diagnostic means diagnoses a risk related to deterioration of the artifact based on the collated result,
    The computer system according to claim 1.
  6. 　前記第１画像解析手段は、既知の第２人工物画像の特徴点を教師データとして機械学習し、取得した前記第１人工物画像を画像解析する、
    　ことを特徴とする請求項１に記載のコンピュータシステム。     The first image analysis means performs machine learning using feature points of a known second artifact image as teacher data, and performs image analysis on the acquired first artifact image.
    The computer system according to claim 1.
  7. 　前記第１画像解析手段は、既知の第２人工物画像の特徴量を教師データとして機械学習し、取得した前記第１人工物画像を画像解析する、
    　ことを特徴とする請求項１に記載のコンピュータシステム。     The first image analysis means performs machine learning using a feature amount of a known second artifact image as teacher data, and performs image analysis on the acquired first artifact image.
    The computer system according to claim 1.
  8. 　前記第１画像解析手段は、取得した前記第１人工物画像に対してマークされた画像について画像解析する、
    　ことを特徴とする請求項１に記載のコンピュータシステム。     The first image analysis means performs image analysis on the marked image with respect to the acquired first artifact image.
    The computer system according to claim 1.
  9. 　前記第１人工物画像は、配管の画像であって、
    　前記第１画像解析手段は、取得した前記配管の画像に対してマークされた画像について画像解析し、
    　前記診断手段は、前記配管の割れの診断をする、
    　ことを特徴とする請求項１に記載のコンピュータシステム。     The first artifact image is an image of piping,
    The first image analysis means performs image analysis on an image marked with respect to the acquired image of the pipe,
    The diagnosis means diagnoses cracks in the pipe;
    The computer system according to claim 1.
  10. 　人工物を診断するコンピュータシステムが実行する人工物診断方法であって、
    　前記人工物の時系列の変化を伴う複数枚の第１人工物画像を取得するステップと、
    　取得した前記第１人工物画像を画像解析するステップと、
    　過去の別の人工物の時系列の変化を伴う複数枚の第２人工物画像を取得するステップと、
    　取得した前記第２人工物画像を画像解析するステップと、
    　前記第１人工物画像の画像解析の結果と、前記第２人工物画像の画像解析の結果とを照合するステップと、
    　照合した結果に基づいて、前記人工物を診断するステップと、
    　を備えることを特徴とする人工物診断方法。     An artifact diagnosis method executed by a computer system for diagnosing an artifact,
    Obtaining a plurality of first artifact images with a time-series change of the artifact;
    Image analysis of the acquired first artifact image;
    Acquiring a plurality of second artifact images accompanied by a time-series change of another artifact in the past;
    Image analysis of the acquired second artifact image;
    Collating the result of image analysis of the first artifact image with the result of image analysis of the second artifact image;
    Diagnosing the artifact based on the matching result;
    An artifact diagnosis method comprising:
  11. 　人工物を診断するコンピュータシステムに、
    　前記人工物の時系列の変化を伴う複数枚の第１人工物画像を取得するステップ、
    　取得した前記第１人工物画像を画像解析するステップ、
    　過去の別の人工物の時系列の変化を伴う複数枚の第２人工物画像を取得するステップ、
    　取得した前記第２人工物画像を画像解析するステップ、
    　前記第１人工物画像の画像解析の結果と、前記第２人工物画像の画像解析の結果とを照合するステップ、
    　照合した結果に基づいて、前記人工物を診断するステップ、
    　を実行させるためのプログラム。     For computer systems that diagnose artifacts,
    Obtaining a plurality of first artifact images with a time-series change of the artifact,
    Image analysis of the acquired first artifact image;
    Obtaining a plurality of second artifact images with a time-series change of another artifact in the past;
    Image analysis of the acquired second artifact image;
    Collating the result of image analysis of the first artifact image with the result of image analysis of the second artifact image;
    Diagnosing the artifact based on the collated result;
    A program for running
PCT/JP2017/013821 2017-03-31 2017-03-31 Computer system, artificial object diagnosis method, and program WO2018179421A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/JP2017/013821 WO2018179421A1 (en) 2017-03-31 2017-03-31 Computer system, artificial object diagnosis method, and program

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2017/013821 WO2018179421A1 (en) 2017-03-31 2017-03-31 Computer system, artificial object diagnosis method, and program

Publications (1)

Publication Number Publication Date
WO2018179421A1 true WO2018179421A1 (en) 2018-10-04

Family

ID=63677510

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2017/013821 WO2018179421A1 (en) 2017-03-31 2017-03-31 Computer system, artificial object diagnosis method, and program

Country Status (1)

Country Link
WO (1) WO2018179421A1 (en)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPS63132381A (en) * 1986-11-25 1988-06-04 Hitachi Ltd Image data search and display system
JP2010165127A (en) * 2009-01-14 2010-07-29 Canon Inc Apparatus and method for processing information
JP2016012346A (en) * 2014-06-04 2016-01-21 パナソニック株式会社 Control method and program
JP2016045662A (en) * 2014-08-21 2016-04-04 富士フイルム株式会社 Similar image retrieval device, operation method of similar image retrieval device and similar image retrieval program

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPS63132381A (en) * 1986-11-25 1988-06-04 Hitachi Ltd Image data search and display system
JP2010165127A (en) * 2009-01-14 2010-07-29 Canon Inc Apparatus and method for processing information
JP2016012346A (en) * 2014-06-04 2016-01-21 パナソニック株式会社 Control method and program
JP2016045662A (en) * 2014-08-21 2016-04-04 富士フイルム株式会社 Similar image retrieval device, operation method of similar image retrieval device and similar image retrieval program

Similar Documents

Publication Publication Date Title
JP7340265B2 (en) Abnormality detection device, abnormality detection method, and program
JP6924413B2 (en) Data generator, data generation method and data generation program
US11715190B2 (en) Inspection system, image discrimination system, discrimination system, discriminator generation system, and learning data generation device
KR101822404B1 (en) diagnostics system for cell using Deep Neural Network learning
US11580629B2 (en) System and method for determining situation of facility by imaging sensing data of facility
CN112955837B (en) Abnormality diagnosis device, abnormality diagnosis method, and program
WO2018078868A1 (en) Computer system, and method and program for diagnosing objects
CN110388879B (en) Inspection device
JP6356920B2 (en) Failure diagnosis apparatus, failure diagnosis method, and failure diagnosis program
JP7621586B2 (en) Production line smart monitoring system and monitoring method
JP5718781B2 (en) Image classification apparatus and image classification method
CN113095438A (en) Wafer defect classification method and device, system, electronic equipment and storage medium thereof
US20200065967A1 (en) Computer system, method, and program for diagnosing subject
JP7056259B2 (en) Inspection system, identification system, and classifier evaluation device
CN113052295B (en) Training method of neural network, object detection method, device and equipment
US20240046443A1 (en) Analysis apparatus, analysis system, analysis program, and analysis method
KR102782727B1 (en) Apparatus for dividing, tagging an image and for detecting defect of facilities using the same
CN112308148A (en) Defect category identification and twin neural network training method, device and storage medium
JP7095988B2 (en) Abnormality monitoring system, anomaly monitoring method and program
US11120541B2 (en) Determination device and determining method thereof
CN113139932A (en) Deep learning defect image identification method and system based on ensemble learning
WO2018179421A1 (en) Computer system, artificial object diagnosis method, and program
CN109711434A (en) Method, apparatus, apparatus and medium for obtaining and evaluating training data for VQA system
CN114187292B (en) Abnormality detection method, apparatus, device and storage medium for cotton spinning paper tube
WO2018179221A1 (en) Computer system, object diagnosis method, and program

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17903874

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17903874

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: JP