CN120752517A - Fixed position imaging system for automated visual inspection - Google Patents

Abstract

An Automatic Visual Inspection (AVI) system in the present disclosure may position a container clamp such that a central container axis is optically aligned with a central image axis of a fixed-position imaging system. Once the container holder is positioned such that the central container axis of the container holder is aligned with the central image axis of the fixed position imaging system, the fixed position imaging system may simultaneously capture a first image of the first container and a second image of the second container. The fixed position imaging system may capture a first image of a first container from one perspective while the fixed position imaging system captures a second image of a second container from the same perspective. The AVI system may analyze the first image and the second image to examine the first subset of containers and the second subset of containers for the same set of one or more features.

Description

Fixed position imaging system for automated visual inspection

Cross Reference to Related Applications

Priority is claimed herein for U.S. provisional patent application No. 63/448,552 filed on 2 months 27 of 2023, the entire contents of which provisional patent application is incorporated herein by reference.

Technical Field

The present application relates generally to inspection of containers, and more particularly to imaging systems for automated visual inspection.

Background

In some contexts, such as quality control of manufactured pharmaceutical products, it is necessary to check containers (e.g., vessels, vials, syringes, cartridges, etc., and/or their contents) for the presence of various defects (e.g., cracks, defective seals, air gap measurements, plunger depth measurements, low fill, high fill, foreign particles, fibers, etc.). Under applicable quality standards, the acceptability of a given container or sample may depend on indicators such as the condition of the container, the presence of undesirable particles within the container, and the like. If the container has an unacceptable indicator, the container and contents may be rejected and discarded.

To handle the number of containers typically associated with pharmaceutical commercial production, product inspection tasks have become increasingly automated. Known Automatic Visual Inspection (AVI) systems (e.g., AVI system 100 of fig. 1) have been striving to overcome various obstacles in order to achieve good product fidelity without system complexity. The AVI system 100 receives a plurality of containers 102 via a conveyor 101. A subset of containers 102 (e.g., four containers) are transferred to a container gripper 104 of a turntable 103. The turntable 103 aligns a central container clamp axis 109 of the container clamp 104 with an imaging system central axis 108 of the imaging system using a mechanical mover 105.

In addition to the mechanical movers aligning the central vessel gripper axis 108 with the imaging system central axis 108, the AVI system 100 also requires a complex camera with a mechanical movement mechanism 105 to sequentially align the central image axis 108b of the camera with the mechanical movement mechanism 105 with the central vessel axis 109b, thereby capturing images 102b of each vessel 102a in turn. Thus, the camera with the mechanical movement mechanism 105, along with the complex optics 106 and the connection cable 107, are subject to associated physical stresses. The AVI system 100 often requires high maintenance due to millions of associated mechanical movement cycles of the camera with the mechanical movement mechanism 105.

Imaging systems need to reduce the mechanical movement of the associated camera, optics, and associated camera connections.

Disclosure of Invention

Embodiments described herein relate to fixed location imaging systems and Automatic Vision Inspection (AVI) systems that incorporate fixed location imaging.

As described herein, a system for capturing images of a plurality of containers includes a first fixed-position imaging system configured to capture a first image, a second fixed-position imaging system configured to capture a second image, and a transport mechanism configured to transport containers through the first fixed-position imaging system and the second fixed-position imaging system. The first fixed position imaging system, the second fixed position imaging system, and the transport mechanism are configured such that the first fixed position imaging system captures a first image of a first subset of containers while the second fixed position imaging system captures a second image of a second subset of containers from the same perspective relative to the containers.

A method for imaging a plurality of containers includes transporting the containers in front of a first fixed-position imaging system and a second fixed-position imaging system using a transport mechanism. The method also includes capturing a first image of a first subset of the containers using the first fixed position imaging system. The method further includes capturing a second image of a second subset of the containers using the second fixed position imaging system, wherein capturing the first image and capturing the second image occur simultaneously and from the same perspective relative to the containers.

A non-transitory computer readable medium having stored thereon computer readable instructions that, when executed by one or more processors, cause the one or more processors to control a conveyor mechanism to convey containers in front of a first fixed location imaging system and a second fixed location imaging system. The computer-readable instructions are further executable by the one or more processors to cause the one or more processors to simultaneously control, from the same perspective with respect to the container, (i) the first fixed position imaging system capturing a first image of a first subset of the containers, and (ii) the second fixed position imaging system capturing a second image of a second subset of the containers.

Novel fixed position imaging systems and Automatic Visual Inspection (AVI) systems incorporating fixed position imaging systems are provided. Novel methods for operating an AVI system are also provided.

Drawings

The skilled artisan will appreciate that the drawings described herein are included for illustrative purposes and are not limiting of the present disclosure. The drawings are not necessarily to scale, emphasis instead being placed upon illustrating the principles of the disclosure. It should be appreciated that in some instances, various aspects of the described embodiments may be shown exaggerated or enlarged to facilitate an understanding of the described embodiments. In the drawings, like reference numbers generally indicate functionally similar and/or structurally similar elements throughout the various figures.

Fig. 1 depicts a known Automatic Visual Inspection (AVI) system with mechanically positioned imaging system.

Fig. 2 depicts a top plan view of an example Automatic Vision Inspection (AVI) system with a fixed position imaging system.

Fig. 3A depicts a top plan view of an example Automatic Vision Inspection (AVI) system with a fixed position imaging system.

Fig. 3B depicts a top side perspective view of the example Automatic Visual Inspection (AVI) system of fig. 3A.

Fig. 3C depicts a top rear perspective view of the example Automatic Visual Inspection (AVI) system of fig. 3A.

Fig. 4 depicts a top plan view of an example Automatic Vision Inspection (AVI) system with a fixed position imaging system.

Fig. 5 depicts a high-level block diagram of an example Automatic Visual Inspection (AVI) system with a fixed-position imaging system.

FIG. 6 depicts an example method of implementing an Automatic Visual Inspection (AVI) system having a fixed location imaging system.

Skilled artisans will appreciate that elements in the figures are illustrated for simplicity and clarity and have not necessarily been drawn to scale. For example, the dimensions and/or relative positioning of some of the elements in the figures may be exaggerated relative to other elements to help to improve understanding of various embodiments of the present invention. Moreover, common but well-understood elements that are useful or necessary in a commercially feasible embodiment are often not depicted in order to facilitate a less obstructed view of these various embodiments. It will further be appreciated that certain actions and/or steps may be described or depicted in a particular order of occurrence while those skilled in the art will understand that such specificity with respect to sequence is not actually required. It will further be appreciated that certain actions and/or steps may be described or depicted in a particular order of occurrence while those skilled in the art will understand that such specificity with respect to sequence is not actually required. It will also be understood that the terms and expressions used herein have the ordinary technical meaning as is accorded to such terms and expressions by those skilled in the technical field as set forth above except where different specific meanings have otherwise been set forth herein.

Detailed Description

The various concepts introduced above and discussed in more detail below may be implemented in any of a variety of ways and the described concepts are not limited to any particular implementation. Examples of implementations are provided for illustration purposes.

The fixed position imaging system in the present disclosure may reduce or entirely eliminate complex mechanical movements of associated cameras, optics, camera connections, etc. Instead of capturing images 102b of each container 102a in sequence as in the known AVI system 100, the systems described herein may, for example, simultaneously control that a first fixed position imaging system captures a first image of a first subset of containers and a second fixed position imaging system captures a second image of a second subset of containers from the same perspective relative to the containers. The fixed position imaging system of the present disclosure may enable container inspection in milliseconds, while the known system 100 includes complex mechanical movements and thus may perform the same activities at a slower rate. Fixed position imaging systems can eliminate the high maintenance requirements and downtime due to various failure modes that result from millions of complex mechanical imager movement cycles in the known system 100.

As used in the context of this disclosure, "simultaneous" shall mean "substantially simultaneous. For example, the fixed position imaging system may capture the first image and the second image within a predetermined time (e.g., 50 milliseconds, 100 milliseconds, etc.). In any event, the fixed position imaging system will capture the first image and the second image without physically moving the fixed position imaging system or the container clamp between capturing the first image and the second image.

Fig. 2 is a top plan view of an Automatic Vision Inspection (AVI) system 200 with a fixed position imaging system 205. The AVI system 200 includes a transport mechanism 203 configured to position a container clamp 204 such that a central container clamp axis 209 is aligned with a central image axis 208 of a fixed position imaging system 205. Once the container gripper 204 is positioned such that the central container gripper axis 209 is aligned with the central image axis 208, the fixed position imaging system 205 may simultaneously capture the first image 202b1 of the first container subset 202a1 and the second image 202b2 of the second container set 202a 2. The fixed position imaging system 205 may capture a first image 202b1 from one perspective relative to a first central image axis 208a while the fixed position imaging system 205 captures a second image 202b2 from the same perspective relative to a second central image axis 208 b. In any event, the fixed position imaging system 205 captures the first image 202b1 and the second image 202b2 simultaneously without mechanical movement therebetween.

As shown in fig. 2, the first subset of containers 202a1 and the second subset of containers 202a2 may each include two containers 202. Although both the first subset of containers 202a1 and the second subset of containers 202a1 are shown in fig. 2 as including two containers 202, either the first subset of containers 202a1 or the second subset of containers 202a1 may include one or more containers 202. Thus, the AVI system 200 may simultaneously capture images 202b1, 202b2 in a shorter time than the time required for the AVI system 100 to mechanically align the imaging system 105 and capture a single image 102b of a single container 102 a.

The AVI system 200 may further include an illumination source 210. The illumination source 210 may be configured as a fixed position backlight. In any event, the AVI system 200 may energize the illumination source 210 prior to capturing the first image 202b1 and the second image 202b2. While the first and second images 202b1, 202b2 include only a portion of the respective container 202 (i.e., a portion of the syringe flange, syringe barrel, plunger, air gap, and a portion of the product within the syringe), the first and second images 202b1, 202b2 may include an overall profile view of the respective vessel 202a1, 202a 2.

The AVI system 200 may be used to inspect containers 202 in a final packaging area of, for example, an associated manufacturing facility. The AVI system 200 may also be used to inspect containers 202 in an inspection area within a manufacturing facility in addition to or instead of a final packaging area. For example, as compared to AVI system 100, which requires three to four seconds, AVI system 200 may be configured to inspect a given number of containers 202 in less than twenty milliseconds (i.e., AVI system 200 may be one hundred times faster than AVI system 100). In any event, the AVI system 200 may eliminate millions of mechanical cycles of the AVI system 100 that require mechanical movement of sensitive optics and millions of bends in the camera cable. Although the AVI system 200 is illustrated in fig. 2 with respect to a prefilled syringe, the AVI system 200 may also be applied to inspection of the assembly, labeling, and packaging stages of an associated manufacturing process. Similarly, the AVI system 200 may evaluate attributes regarding a combination product (e.g., a hand-held auto-injector, etc.) or associated packaging (e.g., label presence, label location, etc.).

The AVI systems 300a-C of fig. 3A-3C may be similar to the AVI system 200. The AVI systems 300a-c may include a transport mechanism 303 configured to position a container clamp 304 with a central container clamp axis 309 aligned with a central image axis 308 of the fixed position imaging system 305.

Fixed position imaging system 305 includes a first fixed position imaging system 305a having a first telecentric lens 306a and a first camera connection 307 a. For example, the first fixed position imaging system 305a may be oriented in a fixed position such that the first central optical axis 308a is aligned with the first central container holder axis 309a of the first container subset 302a1 using the first mirror 340 a. Although the first mirror 340a is illustrated in fig. 3A-3B as having a planar reflective surface oriented at a forty-five degree angle relative to the first central optical axis 308a, the first mirror 340a may be oriented at an angle relative to the first central optical axis 308a such that the first subset of containers 302a1 is included within the field of view of the first fixed position imaging system 305a. The first subset of containers 302a1 may include a single container (e.g., container 102a of fig. 1), a portion of two containers (e.g., a portion of containers 202a1, 202a2 of fig. 2), or a predetermined number of containers based on, for example, a desired container inspection speed and/or a predetermined image resolution. As an alternative to providing a plan view of the first subset of containers 302a1, the first mirror 340a may be rotationally oriented with respect to the first central optical axis 308a such that a perspective view of the first subset of containers 302a1 is included within the field of view of the first fixed position imaging system 305a (e.g., oriented to view the syringe flange from a predetermined view angle, oriented to view the vial seal from a predetermined view angle, etc.). First telecentric lens 306a can include any number and type of optical elements that can be configured to, for example, align a central imager axis of an image sensor of first fixed position imaging system 305a with first central optical axis 308 a. Thus, the central imager axis of the image sensor may be oriented in any fixed position relative to the first central optical axis 308 a.

Fixed position imaging system 305 includes a second fixed position imaging system 305b having a second telecentric lens 306b and a second camera connection 307 b. For example, the second fixed position imaging system 305b may be oriented in a fixed position such that the second central optical axis 308b is aligned with the second subset of containers 302a2 using the second mirror 340 b. The second subset of containers 302a2 may include a single container (e.g., container 102a of fig. 1), two containers (e.g., container 202a1 or container 202a2 of fig. 2), or a predetermined number of containers based on, for example, a desired container inspection speed and/or a predetermined image resolution. The second subset of containers 302a2 may include more or fewer containers than the first subset of containers 302a 1. As an alternative to providing a plan view of the second subset of containers 302a2, the second mirror 340b may be rotationally oriented with respect to the second central optical axis 308b such that a perspective view of the second subset of containers 302a2 is included within the field of view of the second fixed position imaging system 305b (e.g., oriented to view the syringe flange from a predetermined view angle, oriented to view the vial seal from a predetermined view angle, etc.). The second telecentric lens 306b can include any number and type of optical elements that can be configured to, for example, align the central imager axis of the image sensor of the second fixed position imaging system 305b with the first central optical axis 308 b. Thus, the central imager axis of the image sensor may be oriented in any fixed position relative to the first central optical axis 308 b.

In any event, once the AVI systems 300a-c align the central container clamp axis 309 with the central image axis 308, the fixed position imaging system 305 may simultaneously control the first fixed position imaging system 305a to capture a first image 302b1 of the first container subset 302a1 and control the second fixed position imaging system 305b to capture a second image 302b2 of the second container subset 302a 2. The first fixed position imaging system 305a may capture a first image 302b1 from one perspective with respect to a first central image axis 308a of the first container subset 302a1, while the second fixed position imaging system 305b captures a second image 302b2 from the same perspective with respect to a second central image axis 308b of the second container set 302a 2. The fixed position imaging system 305 captures the first image 302b1 and the second image 320b2 simultaneously without mechanical movement therebetween as would be required to sequentially capture the image 102b using, for example, the prior art AVI system 100.

The fixed position imaging system 305a may further include an illumination source 310 (e.g., a fixed position backlight, a turntable mounted backlight, a backlight incorporated into a container fixture, etc.). The fixed position imaging system 305 may include a fixed position illumination source 310 attached in a fixed position via, for example, a bracket 311.

Although AVI system 300a is illustrated in fig. 3A with respect to a prefilled vial, AVI system 200 may be applied to inspection of the assembly, labeling, and packaging stages of an associated manufacturing process. Similarly, AVI system 300a may evaluate attributes regarding a combination product (e.g., a hand-held auto-injector, etc.) or associated packaging (e.g., label presence, label location, etc.).

The AVI system 400 may be similar to the AVI systems 300a-C of fig. 3A-3C or the AVI system 200 of fig. 2. The AVI system 400 may include a transport mechanism configured to position the container clamp 404 with a central container clamp axis 409 aligned with a central image axis 408 of the fixed position imaging system 405.

The fixed position imaging system 405 includes a first fixed position imaging system 405a having a first telecentric lens 406a and a first camera connection 407 a. For example, the first fixed position imaging system 405a may be oriented in a fixed position such that the first central optical axis 408a is aligned with the first central container axis 409a of the first container subset 402a1 such that the first container subset 402a1 is included within the field of view of the first fixed position imaging system 405a. The first subset of containers 402a1 may include a single container (e.g., container 102a of fig. 1), a portion of two containers (e.g., a portion of containers 202a1, 202a2 of fig. 2), a profile view of an entire container (e.g., a profile view of containers 302a1, 302a2 of fig. 3A), or a predetermined number of containers based on, for example, a desired container inspection speed and/or a predetermined image resolution. As an alternative to providing a plan view of the first subset of containers 402a1, the first fixed position imaging system 405a may be rotationally oriented with respect to the first central optical axis 408a such that a perspective view of the first subset of containers 402a1 is included within the field of view of the first fixed position imaging system 405a (e.g., oriented to view the vial seal from a predetermined view angle, oriented to view the syringe flange from a predetermined view angle, etc.). First telecentric lens 406a can include any number and type of optical elements that can be configured to, for example, align a central imager axis of an image sensor of first fixed position imaging system 405a with first central optical axis 408 a. Thus, the central imager axis of the image sensor may be oriented in any fixed position relative to the first central optical axis 408 a.

The fixed position imaging system 405 includes a second fixed position imaging system 405b having a second telecentric lens 406b and a second camera connection 407 b. For example, the second fixed position imaging system 405b may be oriented in a fixed position such that the second central optical axis 408b is aligned with the second subset of containers 402a 2. The second subset of containers 402a2 may include a single container (e.g., container 102a of fig. 1), two containers (e.g., container 202a1 or container 202a2 of fig. 2), a profile view of the entire container (e.g., profile views of containers 302a1, 302a2 of fig. 3A), or a predetermined number of containers based on, for example, a desired container inspection speed and/or a predetermined image resolution. The second subset of containers 402a2 may include more or fewer containers than the first subset of containers 402a 1. As an alternative to providing a plan view of the second subset of containers 402a2, the second fixed position imaging system 405b may be rotationally oriented with respect to the second central optical axis 408b such that a perspective view of the second subset of containers 402a2 is included within the field of view of the second fixed position imaging system 405b (e.g., oriented to view the syringe flange from a predetermined view angle, oriented to view the vial seal from a predetermined view angle, etc.). The second telecentric lens 406b can include any number and type of optical elements that can be configured to, for example, align the central imager axis of the image sensor of the second fixed position imaging system 405b with the first central optical axis 408 b. Thus, the central imager axis of the image sensor may be oriented in any fixed position relative to the first central optical axis 408 b.

In any event, once the AVI system 400 aligns the central container clamp axis 409 with the central image axis 408, the fixed position imaging system 405 may simultaneously control the first fixed position imaging system 405a to capture the first image 402b1 of the first container subset 402a1 and the second fixed position imaging system 405b to capture the second image 402b2 of the second container subset 402a 2. The first fixed position imaging system 405a may capture a first image 402b1 from one perspective with respect to a first central image axis 408a of the first subset of containers 402a1 while the second fixed position imaging system 405b captures a second image 402b2 from the same perspective with respect to a second central image axis 408b of the second set of containers 402a 2. The fixed position imaging system 405 captures the first image 402b1 and the second image 402b2 simultaneously without mechanical movement therebetween as would be required to capture the images 102b in sequence using, for example, the prior art AVI system 100.

The fixed position imaging system 405 may further include an illumination source 410 (e.g., a fixed position backlight, a turntable mounted backlight, a backlight incorporated into a container fixture, etc.). The fixed position imaging system 405 may include a fixed position illumination source 410 attached in a fixed position via, for example, a bracket 411.

Although the AVI system 400 is illustrated in fig. 4 with respect to a prefilled vial, the AVI system 400 may also be applied to inspection of the assembly, labeling, and packaging stages of an associated manufacturing process. Similarly, the AVI system 200 may evaluate attributes regarding a combination product (e.g., a hand-held auto-injector, etc.) or associated packaging (e.g., label presence, label location, etc.).

Fig. 5 is a simplified block diagram of an example AVI system 500 that may implement various techniques related to training (and possibly verifying and/or authenticating) and/or using one or more neural networks or non-Machine Learning (ML) systems. The AVI system 500 may also be used to test/qualify non-ML AVI systems. In addition to or as an alternative to ML systems, AVI system 500 may include a "computer vision" algorithm that does not use ML, but instead uses fixed rules (e.g., empty vials, low fill, high fill, etc.).

The AVI system 500 may include one or more AVI neural networks. Once trained and authenticated, the AVI system 500 may be used in production to detect defects associated with containers and/or the contents of these containers. In a pharmaceutical context, for example, AVI system 500 may be used to detect defects associated with syringes, cartridges, vials, or other container types (e.g., scratched curls/seals of containers, cracks, scratches, stains, component missing, etc.), and/or to detect defects associated with liquid or lyophilized pharmaceutical products within a container (e.g., the presence of fibers, metal particles, and/or other foreign particles, color changes of a product, etc.). As used herein, "defect detection" may refer to classifying a container image as exhibiting or not exhibiting a defect (or a particular defect class), and/or may refer to detecting a particular object or feature (e.g., particle or crack) associated with whether the container and/or its contents should be considered defective, depending on the embodiment.

AVI system 500 includes a Visual Inspection System (VIS) 505 communicatively coupled to a computer system 520. VIS 505 includes hardware (e.g., illumination source 510, telecentric optics 506, etc.), firmware and/or software configured to capture digital images of a sample (e.g., a container holding a fluid or lyophilized substance). The VIS 505 may include any of the fixed position imaging systems 205, 305a-c, 405 described herein with reference to, for example, fig. 2-4, respectively, or may be some other suitable VIS.

For ease of explanation, AVI system 500 is described herein as training and validating one or more AVI neural networks using container images from VIS 505, and then performing AVI/defect detection using the trained/validated neural network(s). However, it should be understood that this need not be the case. For example, AVI system 500 may perform training and/or verification using container images generated by a plurality of different visual inspection systems in lieu of or in addition to VIS 505. Further, training/verification may be performed by another system, and AVI system 500 may then use the trained neural network(s) (e.g., during commercial production). In some embodiments, some or all of the container images for training and/or verification are generated using one or more off-line (e.g., laboratory-based) "simulation stations" that closely replicate important aspects of the commercial production line equipment stations (e.g., optics, lights, etc.), thereby expanding the training and/or verification library without causing excessive downtime of the commercial production line equipment.

VIS 505 may image each of the plurality of containers simultaneously. To this end, VIS 505 may include or operate in conjunction with a holding device (e.g., a conveyor, turntable, cartesian robot, carousel, star wheel, and/or any other holding device) that may move each container in turn into position for imaging and then remove the container once imaging of the container is complete. Although not shown in fig. 5, VIS 505 may include a communication interface and a processor to enable communication with computer system 520. In other embodiments (e.g., laboratory-based settings), VIS 505 includes a simpler retaining device (e.g., a table with holes covered by a glass plate).

Computer system 520 may be generally configured to control/automate the operation of VIS 505 and to receive and process images captured/generated by VIS 505, as discussed further below. The computer system 520 may be a general purpose computer specially programmed to perform the operations discussed herein, or may be a special purpose computing device. As seen in fig. 5, computer system 520 includes user interface 521, processing unit 522, and memory unit 523. However, in some embodiments, computer system 520 includes two or more computers that are co-located or remote from each other. In these distributed embodiments, the operations described herein with respect to processing unit 522 and memory unit 523 may be divided among multiple processing units and/or memory units, respectively.

The processing unit 522 includes one or more processors, where each processor may be a programmable microprocessor that executes software instructions stored in the memory unit 523 to perform some or all of the functions of the computer system 520 as described herein. For example, processing unit 522 may include one or more Graphics Processing Units (GPUs) and/or one or more Central Processing Units (CPUs). Alternatively or additionally, some of the processors in processing unit 522 may be other types of processors (e.g., application Specific Integrated Circuits (ASICs), field Programmable Gate Arrays (FPGAs), etc.), and some of the functions of computer system 520 as described herein may instead be implemented in hardware.

The memory unit 523 may include one or more volatile and/or nonvolatile memories. The memory unit 523 may include any suitable memory type or types, such as Read Only Memory (ROM), random Access Memory (RAM), flash memory, solid State Drive (SSD), hard Disk Drive (HDD), and so forth. In general, the memory unit 523 may store one or more software applications, data received/used by those applications, and data output/generated by those applications.

The memory unit 523 stores software instructions of the various modules that, when executed by the processing unit 522, perform various functions for the purpose of training, verifying, and/or authenticating one or more AVI neural networks. Specifically, in the example embodiment of fig. 5, the memory unit 523 includes an image analysis module 525 and a Visual Inspection System (VIS) control module 526. In other embodiments, memory unit 523 may omit one or more of modules 525, 526, and/or include one or more additional modules. Additionally or alternatively, one, some, or all of the modules 525, 526 may be implemented by a different computer system (e.g., a remote server coupled to the computer system 520 via one or more wired and/or wireless communication networks). Furthermore, the functionality of any of modules 525 and 526 may be divided among different software applications and/or computer systems. As just one example, in embodiments where the computer system 520 accesses a web service to train and use one or more AVI neural networks, the software instructions of the image analysis module 525 may be stored at a remote server.

The image analysis module 525 includes software that trains one or more AVI neural networks using images stored in the image library 530. The image library 530 may be stored in the memory unit 523, or in another local or remote memory (e.g., memory coupled to a remote library server, etc.). In addition to training, the image analysis module 525 may also implement/run the trained AVI neural network(s), such as by applying the image newly acquired by the VIS 505 (or another visual inspection system) to the neural network(s), possibly after performing some preprocessing on the image as discussed below. In various embodiments, AVI neural network(s) trained and/or run by image analysis module 525 may classify the entire image (e.g., defects versus no defects, or the presence or absence of a particular type of defect (e.g., bead scratch or bead defect in general), detect objects in the image (e.g., detect the location of foreign objects within the container image that are not bubbles), or some combination thereof (e.g., one neural network classifies the image and another performs object detection). As used herein, unless the context clearly indicates a more specific use, "object detection" refers broadly to techniques that identify specific locations of objects (e.g., particles, fibers, etc.) within an image and/or identify specific locations of features of a larger object (e.g., a scratched bead or seal, a crack or notch, etc. on a barrel of a syringe or cartridge), and may include, for example, techniques that perform segmentation (e.g., pixel-by-pixel classification) on a container image or image portion, or techniques that identify objects and place bounding boxes (or other bounding shapes) around those objects.

In embodiments where the AVI neural network(s) detect container defects, these defects may be associated with any suitable container feature(s). Referring to the example container of fig. 2-4, for example, a particular AVI neural network implemented by the image analysis module 525 may detect whether the container has a crack or stain, whether the flange is malformed, whether the needle shield is improperly positioned, whether the plunger or piston has any imperfections, whether the luer lock has any imperfections, whether the crimp is properly positioned and/or has any imperfections (e.g., a scratch), whether the flip cap is properly positioned and/or has any imperfections, and so forth.

The image analysis module 525 may run the trained AVI neural network(s) for verification, authentication, and/or inspection purposes during commercial production. In one embodiment, for example, the image analysis module 525 is used only to train and verify AVI neural network(s), and then the trained neural network(s) are transported to another computer system for authentication and inspection during commercial production (e.g., using another module similar to module 525). In some embodiments where the image analysis module 525 trains/runs multiple neural networks, the image analysis module 525 includes separate software for each neural network.

After enhancing the associated training images by adjusting brightness, vertical mirroring, adding noise and tilting images, and tilting bounding boxes, AVI neural network training may be performed on images from, for example, six vials (i.e., the training set may be multiplied by five). In general, deep learning may be used to detect defects in an image. Using the previously trained AVI neural network(s) further reduces the time required to set up an automatic inspection scheme for new products. AVI neural networks in the present disclosure may be implemented for high mix, low volume production scenarios (e.g., clinical operations or small lot products) and then use modern deep learning techniques (e.g., image analysis module 525 of fig. 5).

In some embodiments, VIS control module 526 controls/automates the operation of VIS 505 such that container images may be generated with little or no human interaction. The VIS control module 526 may cause the given fixed position imaging system to capture images of the container by sending commands or other electronic signals (e.g., generating pulses on control lines, etc.) to the given imager. VIS 505 may send the captured container image to computer system 520, which may store the image in memory unit 523 for local processing. In alternative embodiments, VIS 505 may be locally controlled, in which case VIS control module 526 may have fewer functions than described herein (e.g., handle image retrieval from VIS 505 alone), or may be omitted entirely from memory unit 523.

Fig. 6 is a method 600 of operating an Automatic Visual Inspection (AVI) system, which may be implemented by a processor (e.g., processing unit 522 of fig. 5) executing, for example, at least a portion of Visual Inspection System (VIS) control module 526 and/or image analysis module 525. For example, the AVI system may be similar to any of AVI system 200 of fig. 2, 300a-c of fig. 3A-3B, 400 of fig. 4, or 500 of fig. 5. In particular, the processing unit 522 may execute the VIS control module 526 to cause the processing unit 522 to align the central container clamp axis 209, 309, 409 of the container clamp 204, 304, 404 with the central image axis 208, 308, 408 of the fixed position imaging system 205, 305, 405, for example (block 640). The processing unit 522 may execute the VIS control module 526 to cause the processing unit 522 to power up the illumination sources 210, 310, 410, 510, for example (block 641).

The processing unit 522 may further execute the VIS control module 526 to cause the processing unit 522 to, for example, simultaneously capture the first image 202b1, 302b1, 402b1 of the first container subset 202a1, 302a1, 402a1 and the second image 202b2, 302b2, 402b2 of the second container subset 202a2, 302a2, 402a2 (block 642). The processing unit 522 may execute the image analysis module 525 to cause the processing unit 522 to, for example, analyze the first image 202b1, 302b1, 402b1 and the second image 202b2, 302b2, 402b2 to examine the first container subset 202a1, 302a1, 402a1 and the second container subset 202a2, 302a2, 402a2 for the same set of one or more features (block 643).

A method for imaging a plurality of containers includes transporting the containers in front of a first fixed-position imaging system and a second fixed-position imaging system using a transport mechanism. The method also includes capturing a first image of a first subset of the containers using the first fixed position imaging system. The method further includes capturing, using the second fixed position imaging system, a second image of a second subset of the containers from the same perspective as the first image was captured with respect to the containers and simultaneously.

The fixed position imaging system in the present disclosure may reduce the complexity of the AVI system. Fixed location imaging systems may also reduce lifecycle maintenance of AVI systems. Fixed position imaging systems can further improve quality inspection by reducing vibration and reducing lens errors.

Camera connection and signal management present design challenges in mobile AVI system 100. The fixed position imaging system in this disclosure does not include a mobile camera connection.

The fixed position imaging system may reduce design costs compared to the imaging system 105. The fixed position imaging system may increase the station speed compared to the AVI system 100. The fixed position imaging system may be one hundred times faster than the mechanical AVI system 100 of fig. 1.

Although systems, methods, devices, and components thereof have been described in accordance with exemplary embodiments, they are not limited in this regard. The detailed description is to be construed as exemplary only and does not describe every possible embodiment of the invention since describing every possible embodiment would be impractical, if not impossible. Numerous alternative embodiments could be implemented, using either current technology or technology developed after the filing date of this patent, which would still fall within the scope of the claims defining the invention.

Those skilled in the art will recognize that a wide variety of modifications, alterations, and combinations can be made with respect to the above described embodiments without departing from the scope of the invention, and that such modifications, alterations, and combinations are to be viewed as being within the ambit of the inventive concept.

Claims (20)

1. A system for capturing images of a plurality of containers, the system comprising:

A first fixed position imaging system configured to capture a first image;

a second fixed position imaging system configured to capture a second image, and

A transport mechanism configured to transport containers past the first fixed-position imaging system and the second fixed-position imaging system,

Wherein the first fixed position imaging system, the second fixed position imaging system, and the transport mechanism are configured such that the first fixed position imaging system captures a first image of a first subset of the containers and at the same time the second fixed position imaging system captures a second image of a second subset of the containers from the same perspective relative to the containers.

2. The system of claim 1, further comprising:

One or more processors configured to analyze the first image and the second image to examine the first subset of containers and the second subset of containers for the same set of one or more features.

3. The system of claim 1 or 2, wherein each of the transported containers is selected from the group consisting of a vial, a syringe, or a cartridge.

4. The system of any of claims 1-3, wherein the first fixed-position imaging system comprises a first imaging device having a first telecentric lens and the second fixed-position imaging system comprises a second imaging device having a second telecentric lens.

5. The system of any of claims 1-4, wherein the first fixed position imaging system comprises a first mirror configured to align a first central image axis with the first subset of containers, and wherein the second fixed position imaging system comprises a second mirror configured to align a second central image axis with the second subset of containers.

6. The system of any of claims 1-5, wherein the first fixed position imaging system comprises a first mirror oriented at a forty-five degree angle relative to the first central image axis, and wherein the second fixed position imaging system comprises a second mirror oriented at a forty-five degree angle relative to the second central image axis.

7. The system of any one of claims 1 to 6, further comprising:

A backlight oriented to emit light toward the first subset of containers, the first fixed-position imaging system, the second subset of containers, and the second fixed-position imaging system.

8. A method for imaging a plurality of containers, the method comprising:

transporting the container to the front of the first fixed position imaging system and the second fixed position imaging system using a transport mechanism;

Capturing a first image of a first subset of the containers using the first fixed position imaging system, and

A second image of a second subset of the containers is captured using the second fixed position imaging system, wherein capturing the first image and capturing the second image occur simultaneously and from the same perspective relative to the containers.

9. The method of claim 8, further comprising:

One or more processors configured to analyze the first image and the second image to examine the first subset of containers and the second subset of containers for the same set of one or more features.

10. The method of claim 8 or 9, wherein the first fixed-position imaging system comprises a first imaging device having a first telecentric lens and the second fixed-position imaging system comprises a second imaging device having a second telecentric lens.

11. The method of any of claims 8-10, wherein the first fixed position imaging system comprises a first mirror configured to align a first central image axis with the first subset of containers, and wherein the second fixed position imaging system comprises a second mirror configured to align a second central image axis with the second subset of containers.

12. The method of any of claims 8-11, wherein the first fixed position imaging system comprises a first mirror oriented at a forty-five degree angle relative to the first central image axis, and wherein the second fixed position imaging system comprises a second mirror oriented at a forty-five degree angle relative to the second central image axis.

13. The method of any of claims 8 to 12, further comprising:

light is emitted toward the first subset of containers, the first fixed-position imaging system, the second subset of containers, and the second fixed-position imaging system using a backlight.

14. The method of any one of claims 8 to 13, wherein the first subset of containers comprises two or more containers, wherein the second subset of containers comprises two or more containers.

15. A non-transitory computer-readable medium having stored thereon computer-readable instructions that, when executed by one or more processors, cause the one or more processors to:

Controlling the transport mechanism to transport the container in front of the first fixed position imaging system and the second fixed position imaging system, and

While controlling that from the same perspective with respect to the containers, (i) the first fixed position imaging system captures a first image of a first subset of the containers, and (ii) the second fixed position imaging system captures a second image of a second subset of the containers.

16. The non-transitory computer-readable medium of claim 15, wherein further execution of the computer-readable instructions by the one or more processors causes the one or more processors to further perform the operations of:

The first image and the second image are analyzed to examine the first subset of containers and the second subset of containers for the same set of one or more features.

17. The non-transitory computer readable medium of claim 15 or 16, wherein the first fixed position imaging system comprises a first imaging device having a first telecentric lens and the second fixed position imaging system comprises a second imaging device having a second telecentric lens.

18. The non-transitory computer readable medium of any one of claims 15-17, wherein the first fixed position imaging system comprises a first mirror configured to align a first central image axis with the first subset of containers, and wherein the second fixed position imaging system comprises a second mirror configured to align a second central image axis with the second subset of containers.

19. The non-transitory computer readable medium of any one of claims 15-18, wherein the first fixed position imaging system comprises a first mirror oriented at a forty-five degree angle relative to the first central image axis, and wherein the second fixed position imaging system comprises a second mirror oriented at a forty-five degree angle relative to the second central image axis.

20. The non-transitory computer-readable medium of any one of claims 15-19, wherein further execution of the computer-readable instructions by the one or more processors causes the one or more processors to further perform the operations of:

A backlight is controlled, the backlight being oriented to emit light toward the first subset of containers, the first fixed-position imaging system, the second subset of containers, and the second fixed-position imaging system.