JP2007114967A - Process management apparatus and method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a process management apparatus that achieves data acquisition related to the same contents as a LAN, that is, operation information of the apparatus, when the arrangement, model, display mode, etc. of the apparatus or the display are changed, and its Provide a method.
An image obtained by photographing a plurality of devices and a display (patlight 6) attached to each device and displaying an operation state thereof is displayed on a display, and based on the display image frame. Create a layout diagram of the device. The operation information (color) displayed on the display unit is recognized from the image (white to green) of the display unit in the image frame (S3), and the operation information (white to green) recognized corresponding to the device on the layout diagram. Is displayed (S4 to S11). Even if there is a change in the device or the like, it can be dealt with by updating the layout diagram or the like, and the device operating state can be managed as in the LAN system.
[Selection] Figure 5

Description

  The present invention relates to a process management apparatus and method based on image recognition.

  Conventionally, in order to manage the work process, maintenance, etc. of various devices, machines, and equipment (hereinafter referred to as “devices”) installed in factories, individual operation information of these devices is acquired and based on the acquired information. Centralized management is performed to ensure efficient operation of the entire factory.

  FIG. 7 is a conventional general factory production system configuration diagram showing a part of a manufacturing factory as an example. A series of production processes are arranged in the factory. For example, parts supply apparatus group process (process I), automatic assembly apparatus group process (process II), inspection apparatus group process (process III), appearance inspection process, etc. Steps IV) are sequentially arranged. Further, a plurality of devices are operating in each process, and a facility programmable logic controller PLC is arranged for each device, and each device operates under the control of each PLC. In addition, an input / output device I / O is arranged in association with each device, and the input / output device I / O (data conversion) is connected to a device group PC for each process. Depending on the process, each PLC is connected to an MPLC (main programmable logic controller). The device group PC in each of the above steps is configured as a production LAN by a cable together with an information collection server and a PC in the factory / production management room.

  The manager of the factory can acquire the operation information of each device through the production LAN, that is, from each device or device group through the PLC and I / O, and further through the MPLC. For example, when a device with one of the processes malfunctions or fails, process control such as grasping the operation status of the device in the PC in the factory / production management room and transferring to another device It can be carried out.

  However, since the production LAN is constructed by routing a wire, the work time and cost for transferring to another device are great. If the transfer to other equipment is not enough, the manufacturing plant will change the layout, such as adding production lines, rearranging equipment, replacing old and new models, and adding / changing accessories. Is called. If a production LAN has already been constructed in the manufacturing factory, it is necessary to perform reconstruction, inspection, testing, etc. of the LAN with redrawing of the LAN cable every time the layout is changed. The LAN-related facility change work accompanying such a layout change is not a simple work and increases capital investment costs, and the factory operation must be stopped during the work. Therefore, such a layout change results in an increase in product cost.

  It is also conceivable to connect information exchange between the apparatus and the management center through a wireless LAN. However, the wireless LAN has the following problems when used for factory production management. That is, first, there is a large risk of information leakage during wireless communication. Further, it is difficult to avoid malfunctions and communication errors in the communication device, and the reliability may be reduced to a fatal level. Furthermore, each apparatus requires a communication facility, and there are restrictions on the number of units for constructing a wireless LAN.

Integrated monitoring and analysis apparatus and method for integrated monitoring and analysis of problems ranging from monitoring the operational status of various facilities such as production facilities in factories to grasping factors that hinder normal operation for each process Has been proposed (see Patent Document 1). In this integrated monitoring analysis, each PLC, each personal computer, management server, and personal computer for analysis are arranged in a central control room in the factory, and together with each system such as an actuator system and a line transport system, A local area network system is configured.
JP 2004-341822 A (paragraph [0024], FIG. 1)

There has been proposed a layout drawing production apparatus for automatically drawing a planar arrangement of a plurality of installation objects installed on an office floor to save labor for producing a layout drawing (Patent Document 2). This layout drawing production apparatus has a barcode that can be attached to each installation by printing the installation data with ink that can be recognized only in the infrared region, an infrared camera that images the layout drawing production target area, and an image taken by this camera. Input image processing means for detecting each barcode position in the layout drawing production target area in association with the installation data from the infrared image, and the barcode position and installation data of each barcode detected by the input image processing means Layout determining means for determining the layout of each installation based on the layout, and layout drawing output means for generating a layout drawing based on the determination of the layout determination means. The arrangement determining means has a database for storing various article data including the shape and size of the installation object for each installation object code.
Japanese Unexamined Patent Publication No. 11-213126 (paragraphs [0012] to [0017], FIG. 1)

  Therefore, there is a problem to be solved in that the layout of the apparatus is created and the operating state of each apparatus is recognized and displayed based on the image data captured by the camera, instead of actually routing the LAN cable.

  The object of the present invention is that there is a large amount of equipment-related costs that would occur if the conventional LAN cable was redrawn if there was a change in the arrangement, model, display mode, etc. of the device or display. It is an object of the present invention to provide a process management apparatus and method for achieving data acquisition related to the same contents as in the case of LAN, that is, operation information of the apparatus, while suppressing occurrence and stoppage of factory operation.

  In order to solve the above-described problems, a process management apparatus according to the present invention includes one or a plurality of cameras for photographing a plurality of apparatuses and a display attached to each of the apparatuses and displaying an operating state thereof, and the cameras A display capable of displaying an image obtained by photographing, layout creation means capable of creating or supporting creation of the layout of the device based on the image displayed on the display, and from the video of the display in the image And recognizing means for recognizing the operation information displayed on the display, and operation information display means for displaying the operation information recognized corresponding to the device on the layout on the display.

  According to this process management device, one or a plurality of cameras, a plurality of devices, and a display attached to each device and displaying the operating state are photographed, and images obtained by the cameras are obtained. It is displayed on the display. In the image taken by the camera, the size of the apparatus is shown different from the actual size due to the distance, but it can be converted into an image similar to that seen in plan by performing image processing. The layout creation means can create a layout of the apparatus based on the image displayed on the display or provides creation support. The layout creating means need not automatically create a layout. For example, it may be graphic creation support software capable of creating a layout in accordance with the actual size on a layer superimposed on an image by manipulating an input means. The recognizing unit can recognize the operation information in the apparatus based on the image about the display on the display. The operation information display means can display the previous operation information recognized corresponding to the device on the layout on the display.

  In this process management device, each of the cameras is a panoramic camera that obtains the images obtained by photographing all the devices, or one image so that the images obtained by photographing all the devices can be formed by combining. It can be assumed that there are a plurality of corner-installed cameras capable of photographing the device. When a panoramic camera is placed at the approximate center of all the devices, the entire device can be photographed with one device. Further, when a plurality of corner-installed cameras each capable of photographing a part of the apparatus are arranged, an image obtained by photographing all the apparatuses can be formed by combining the image data photographed by each.

  In this process management apparatus, the display device may be a patrol light or a matrix display device that can distinguish and display a plurality of operating states based on a light emission position, light emission color information, or a combination thereof. The bar-shaped palite has different colors for light emission depending on the height position. In principle, the luminescent color information can be combined with hue, brightness, and saturation, which are the three elements of color. However, in the processing of image data, it is possible to use a hue that is easily distinguished and recognized. This is true. When it is desired to display a large number of pieces of operation information separately, it is possible to use a matrix display that can control display independently in vertical and horizontal grids (cells).

  In this process management apparatus, it is preferable that the layout is created and registered based on the image displayed on the display when the apparatus is changed. When a change is made to the device, the layout is recreated, and the recreated device and its layout are registered. Once the layout is registered, the registered device and layout are assumed to be the actual device and its layout until the next device is changed, and the device operation information is recognized based on the images taken by the camera. The recognized operation information is displayed in correspondence with the registered layout. In the case of registering the display device as data regarding the type and the like, not only the device body but also the display device itself is included in the layout.

  In the process management apparatus for updating the layout, the display of the operation information on the display can be expressed as an attribute on the display of the apparatus displayed in the registered layout. On the display, the apparatus is displayed in a registered layout, for example, as a figure such as a rectangle. The operation information of each device is displayed as a display attribute such as the color, shading, hatching, and mark of the figure in accordance with the operation contents. Therefore, the administrator can check the operating state of the apparatus on the display only by looking at the display.

  In this process management apparatus, when the content of the display on the display device changes, the time of the change and the content of the display recognized on the display device can be recorded. When the display contents of the display unit change, the time and display contents are recorded, so that the administrator can grasp the normal operating duration and abnormal operating status history, and use it as data for improvement. be able to.

  In this process management apparatus, when the content of the display recognized by the display device changes, it is preferable to notify the necessary department to that effect. When the display content of the display has changed, it is when it has changed from a normal operating state to an abnormal operating state or vice versa, so that the administrator is notified by voice or video, Administrators can take quick action.

  Further, the process management method according to the present invention is based on an image including a plurality of devices photographed by a single or a plurality of cameras and a display attached to each device and displaying an operating state thereof. Displaying the layout on the display together with the operating information of the device recognized based on the display of the indicator in the image.

  According to this process management method, it is possible to recognize and display the layout of the device and the operating state of each device based on the image taken by the camera. It is possible to acquire the same contents as in the case of the LAN, that is, data relating to the operation information of the apparatus.

  Since the process management apparatus and method according to the present invention are configured as described above, if there is a change in the arrangement, model, display mode, etc. of the apparatus or the display, the image taken by the camera is displayed. A layout is created based on the image of the display unit in the image, the operation information displayed on the display unit is recognized, and the operation information recognized corresponding to the device on the layout is displayed on the display. It is possible to obtain the same contents as in the case of the LAN, that is, data related to the operation information of the apparatus, and display the operation information of the apparatus, while suppressing the generation of a great expense required when the cable is redrawn. Therefore, if there is a change in the device or its layout, etc., there is no need to redraw the LAN cable. A layout is created on the display based on the images taken by the camera, and the creation result is registered in the computer. In addition, a series of operations to display the operation information of the device corresponding to the device on the display can be executed without taking much time, and it is possible to avoid the stop of the factory operation due to the construction, resulting in the product. The manufacturing cost can be greatly reduced.

  Hereinafter, embodiments of a process management apparatus according to the present invention will be described with reference to the accompanying drawings.

  FIG. 1 is an explanatory view of the arrangement of cameras when a factory apparatus in a process management apparatus according to the present invention is targeted. According to the layout diagram of the camera in the factory 1 having a flat rectangular floor shown in FIG. 1A, the cameras 2a, 2b, 2c, and 2d are corner-installed cameras disposed at the four corners of the floor. Image data photographed by the cameras 2a to 2d can be combined and converted into image data obtained by photographing all devices with a single image, as will be described later. Instead of the cameras 2a to 2d arranged at the four corners, or in addition to the cameras 2a to 2d, a 360 degree panoramic camera 2e can be arranged in the center of the factory 1. The image of the panoramic camera 2e is developed into image data such as a map. The number of cameras installed can be increased or decreased according to the size of the factory 1 or the number of devices. A plurality of cameras 2a to 2e cover the floor area where the apparatus of the factory 1 is arranged. Generally, factories usually do not have pillars, and are suitable for capturing and laying out with cameras.

  FIG. 1B is an explanatory diagram showing a state where the cameras in the factory are photographed by showing the cameras at the four corners (illustrated only two units 2a and 2b are shown). Various devices are included in the factory 1 shown in FIG. 1B, and a large device 3, a medium device 4, and a small device 5 (all of which are represented by only one device as a representative). ) Are arranged. A patrol light 6 is attached to each of the devices 3 to 5 as a display for displaying the operating state of each device. In the example shown in the figure, the camera 2a can photograph the large-sized device 3, the medium-sized device 4, and its patrol light 6, and the camera 2b can photograph the small device 5 and its patrol light 6. The patrol light 6 is formed by arranging a plurality of lights that are turned on in different colors in the vertical direction, and the lights that are turned on according to the operating state can be selected individually or in combination. As an example of the operating state indicated by the display by the patrol light 6, for example, white indicates an operating state, red indicates an abnormal state, yellow indicates a recovery standby state, and green indicates that a part is being replaced. Can do.

  FIG. 2 is a diagram showing a composite image (FIG. 2A) obtained by combining camera images taken by a plurality of cameras, and a layout created based on the composite image (FIG. 2B). Since the actual camera image is an image viewed obliquely from above, there may be a change in size or distortion of each device according to the difference in perspective, but the image displayed on the display was viewed in plan by applying image processing It is possible to convert to an image similar to. The images taken by the cameras 2a to 2d at the four corners overlap with other cameras, which can be appropriately overlapped by image processing. An image obtained by the synthesis is captured by a computer and captured as an image. A captured image to the computer is shown in FIG.

  Based on the captured image shown in FIG. 2B, the layout creating means creates a layout diagram of the apparatus in the factory. Based on the captured image, an equipment outer frame 7 for the entire factory (floor) is formed, and the layout of each device is described therein. The layout creation means may automatically create a layout, or may be support software created manually. For example, figure creation software that can create a layout diagram that actually conforms to the computer by instructing on the layer superimposed on the image on the display via an input means such as a keyboard, mouse or pen input It can be. By giving a link with the captured image when giving an instruction on the display, it is possible to create a plan layout diagram of the factory while manually correcting it.

  Based on the captured image, the positions of the respective devices 3 to 5 in the factory are designated, and layout diagrams of the devices 3 to 5 are created. FIG. 3 is an example of a layout diagram created in this way. In the layout diagram, data of each of the devices 3 to 5 is attached, and a database is created for the attributes of the devices 3 to 5 necessary for process management such as an operating state. That is, a management (monitoring) target device is specified, and attributes such as a device name, a machine number, and an operating condition of each device are registered in the computer as necessary management information (database master registration (A)). In addition, the device name is described in text (B) in a frame indicating each device 3-5. Further, when the layout is changed, the positions and sizes of the individual devices 3 to 5 can be moved / changed (C) on the display.

  The layout diagram shown in FIG. 3 is stored in the computer together with the database master. Once stored, the computer recognizes as an arrangement in the layout diagram and as having registered database attributes until the device is changed. If there is a change in the apparatus, the image data is taken in every change, and the layout diagram and the database are updated by associating the position on the image with the apparatus. In the case of registering attributes such as the type of display (patite or matrix) in the database of the computer, the device can include not only the device itself but also the display.

  In the above example, the patrol light 6 is shown as a display in FIG. 1B, but the patrol light is a vertically long bar-shaped lamp determined in accordance with each predetermined device in the screen, and has a height. The light emission color is varied depending on the position. The position of the patlite is determined from the image data. Since the number of types that can be represented in the Patrite is limited, if you want to distinguish more operational information, form vertical and horizontal cells inside the box and display each cell. A matrix display capable of controlling (light) independently can be employed. An example of a matrix display is shown in FIG. In the matrix display 10, it is possible to increase the types and the number of lights 11 and 12 to be displayed. Considering the combination, a large number of operation information can be distinguished and displayed as the number of cases.

  Generally, a display device can distinguish and display a plurality of operating states according to the position of light, emission color information, or a combination thereof. In principle, the light emission color information can be combined with hue, brightness, and saturation, which are the three elements of color, but it is preferable to use a hue that is easily distinguished and recognized for the processing of image data. As described for the display of the patrol light 6 described above, the operating state of the apparatus can be displayed in colors such as white, red, yellow, and green.

  The display of the operation information of the devices 3 to 5 on the display can be expressed as an attribute on the display of the device displayed in the registered layout. As shown in FIG. 3, the factory apparatus is displayed on the display as a registered layout diagram, for example, in a figure such as a rectangle. In the example of FIG. 3, the operation information is represented by shading on the display shown in the figure, but the operation information of each device is divided according to the operation contents as display attributes such as the color of the figure, hatching, and marks. It is displayed separately. In the case of color display, it is preferable to display in the same color as the color recognized by the patrol light 6. The administrator can see the operating states of the devices 3 to 5 on the display together with their attributes, and can instantly check them.

  FIG. 5 is a block diagram showing an example of a process management apparatus according to the present invention. The process management apparatus 20 can be embodied as a computer (PC), and during operation of the apparatuses 3 to 5, a digital image frame photographed by regularly scanning with the cameras 2 a to 2 e is an immediately preceding digital image. The frame is stored in the memory 21. The memory 21 also stores the created layout diagram and the database master (A). The process management apparatus 20 includes a layout creation unit 22, a comparison unit 23, a recognition unit 24, and an operation information display unit 25 that can exchange data with the memory 21. A display 27 is provided as a peripheral device.

  By comparing the digital image frames photographed by the cameras 2a to 2e with the digital image frame photographed immediately before by the comparison means 23, the devices 3 to 5 whose positions are recognized are monitored. If the normal display of the display provided corresponding to the devices 3 to 5 does not change, the devices 3 to 5 are operating the same as the previous time, and if they are operating normally, their normal operating time can be measured It is. When comparing digital image frames, the display of a display provided for a certain device may change due to a failure of the devices 3 to 5 or a lack of parts. When a display change occurs, the recognition means 24 provided in the process management apparatus 20 reads the changed part as the data processing from the digital image data photographed by the camera, and the changed part is a display change of the display. If the display changes, it is possible to automatically recognize up to the operation information of the device that is the display content of the display.

  When the display content of the display device changes, the process management apparatus 20 records the change time recognized by the recognition means 24 and the display content of the display device. When the display contents of the display unit change, the time and display contents are recorded so that the administrator can grasp the normal operating duration and abnormal operating status history and use it as material for improvement. it can.

  The process management apparatus 20 includes a notification unit 26, and notifies the necessary department when the display content recognized by the display device has changed. When the display content of the display has changed, it is when it has changed from a normal operating state to an abnormal operating state or vice versa, so that the administrator is notified by voice or video, Administrators and related departments can respond quickly.

  FIG. 6 shows a flow of a series of management operations as an example. The cameras 2a to 2e are moved or scanned (scanned), and the display status of the display (patlight 6) is confirmed each time. For example, in the frame 1, the first to fourth machines of the device 5 are captured by a camera. Search is performed at 0.2 seconds per frame (step 1, abbreviated as S1, the same applies hereinafter), and frame 1 and frame 2 are compared (S2). The color of the patrol light 6 for the No. 4 machine is confirmed in the frame 2 (S3). If the patrol light 6 of Unit 4 is confirmed to be white (in operation) (S4), the portion corresponding to Unit 4 in the layout diagram is displayed in white (S5). Confirm that the Unit 4 patrol light 6 is abnormal (red) lighting (S6), recovery (yellow) lighting (S8), or green (part replacement) lighting (S10). The part corresponding to the fourth car is displayed in red (S7), yellow (S9), and green (S11). In the case of S6 to S11, the abnormal time of No. 4 is recorded (S12).

It is a figure which shows an example of arrangement | positioning of the camera and apparatus in a factory based on this invention. It is a figure which shows an example of the synthetic | combination image obtained from the image image | photographed with the camera, and the capture image to a computer. It is a figure which shows an example of the layout figure in this invention. It is the schematic which shows another form of a display. It is a block diagram which shows an example of the process management apparatus of this invention. It is a flowchart which shows an example of a series of management operation | movement in this invention. It is the schematic which shows an example of the conventional factory production system structure.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Factory 2a, 2b, 2c, 2d Corner installation camera 2e Panorama camera 3 Large apparatus 4 Medium apparatus 5 Small apparatus 6 Patlight (display) 7 Equipment outer frame 10 Matrix display 11, 12 Light 20 Process control apparatus 2 21 Memory 22 Layout creation means 23 Comparison means 24 Recognition means 25 Operation information display means 26 Notification means 27 Display

Claims (8)

  One or a plurality of cameras that shoot a plurality of devices and a display that is attached to each device and displays an operating state thereof, a display that can display images obtained by the cameras, and the display Layout creation means capable of creating or supporting creation of the layout of the device based on the displayed image; recognition means for recognizing the operation information displayed on the display from the video of the display in the image; and A process management device comprising operation information display means for displaying the operation information recognized corresponding to the device on the layout on the display.   The camera is a panoramic camera that obtains the images obtained by photographing all the devices in one unit, or each of the devices is photographed so that the images obtained by photographing all the devices can be formed by combining. The process management apparatus according to claim 1, comprising a plurality of corner-installed cameras.   2. The process management apparatus according to claim 1, wherein the display is a patrol light or a matrix display capable of distinguishing and displaying a plurality of operating states according to a light emission position, light emission color information, or a combination thereof.   The process according to claim 1, wherein the layout is created based on the image displayed on the display when the device is changed, and the device and its layout are registered. Management device.   The process management apparatus according to claim 4, wherein the display of the operation information on the display is represented as an attribute on the display of the apparatus displayed in the registered layout.   2. The process management apparatus according to claim 1, wherein when the content of the display on the display device changes, the time of the change and the content of the display recognized on the display device are recorded.   The process management apparatus according to claim 6, wherein when a content of the display recognized by the display device changes, a notification to that effect is given to a necessary department.   Based on an image including a plurality of devices photographed by a single camera or a plurality of cameras and a display attached to each device and displaying an operating state thereof, the layout of the device is displayed on the display in the image. An operation state management method comprising displaying on a display together with the operation information of the device recognized on the basis of the operation information.