Method and apparatus for inspecting solder paste using geometric constraints

[0002] This Application is a continuation in part of U. S. patent application Ser. No. 08/769,739, filed Dec. 19, 1996, entitled Method and Apparatus for in-line Solder Paste Inspection.

[0003] The present invention relates to machine vision and automated inspection systems, and more particularly to a method and apparatus for inspection of solder paste screen printed on printed circuit boards using a stencil.

[0004] The process of assembling printed circuit boards (PCBs) generally requires solder paste to be applied to the printed circuit board before components, such as integrated circuits and discrete devices, are placed and ultimately secured onto the board. Initially, the sticky solder paste temporarily holds the components in place on the PCB until the solder paste is melted or reflowed. When the solder paste is reflowed it forms both mechanical and electrical connections between the components and the board. If there is not enough solder paste, components may be inadequately mechanically and/or electrically connected to the PCB. If there is too much solder paste or if solder paste is deposited in the wrong place, an extra or incorrect electrical connection such as a short circuit may result, generally referred to as a “solder bridge”.

[0005] Automated processes for fabricating PCBs are generally known, including process steps for applying solder paste to printed circuit boards using screen-printing techniques. Prior to printing, a thin metal stencil or template is constructed with openings or apertures matching appropriate places on a particular PCB where paste should be printed. During printing the stencil is aligned to the PCB. The solder paste is then applied to the PCB through the stencil openings, typically using a squeegee to spread the paste. Then the stencil is lifted off of the PCB which is transported to its next manufacturing process step.

[0006] Automatic inspection or vision systems may be used in known processes to detect problems in the printing process, including problems associated with the stencil and problems associated with the application of paste to the PCBs. Stencil blockage may be detectable wherein dried solder paste or debris causes stencil openings to clog. It is desirable to detect stencil blockage by inspecting the stencil before any printed circuit boards are actually misprinted with solder paste. Stencil smearing may be detectable wherein solder paste smears around the stencil openings on a side of the stencil adjacent to the PCB. It is also desirable to detect stencil smearing by inspecting the stencil before any boards are actually misprinted with solder paste. Misalignment of the stencil and PCB may be detectable prior to distribution of solder paste by inspection of the stencil position relative to the PCB before printing thereon. The PCB can be inspected to detect incorrect solder paste distribution after printing. Automatic inspection may be used to determine if there is too little solder paste, too much solder paste, or solder paste in the wrong place on the printed circuit board.

[0007] The automated inspection of solder paste on PCBs is inherently difficult to accomplish. The solder paste is difficult to identify on the PCB because it is variable in terms of its appearance. The paste appearance varies over time, e.g. wet paste has a different appearance than dry paste. The paste has texture and the gray level reflectance of the paste is similar to solder pads with tinning and other circuitry on the PCB. Also, the three dimensional shape of the paste causes shadowing, making automated inspection difficult. The inherent difficulty is exacerbated by variability from one PCB to the next. For example, the color of boards of a single board type in one printing run can vary from light green to dark green or blue. In one printing run, the relative positioning of portions or all of a PCB of a single board type may vary with respect to the stencil or solder mask on the board. Additionally, all of the circuitry on a board is subject to dimensional variability. Perhaps most significantly, the solder paste deposited on a board can obscure solder pads beneath it making it impossible to tell either what is beneath the solder paste or what is the relative alignment of the solder paste to what is beneath it. For instance, if the solder paste is exactly the same size and shape as the solder pad beneath it, it may be impossible to tell whether there is perfect coverage or no coverage at all.

[0008] Known in-line or image automated inspection systems for solder paste inspection do not adequately overcome such difficulties. Some known systems typically only perform inspection based on an image of the PCB captured after the solder paste has been deposited thereon (referred to hereinafter as “post-only” systems). Known post-only systems typically use a vision/inspection tool that assigns grey scale values to pixels of a captured image of a PCB under inspection after printing. Pixels are classified as being not-paste or paste based on their grey scale value. Pixels are aggregated for particular areas of interest on the PCB, in a connectivity analysis in which pixels connected to one another and identified as being paste are labeled as belonging to the same lump of paste. Characteristics of interest related to the aggregation of pixels, such as area, center of mass, and size of enclosing rectangle, are determined and ultimately used by an operator to adjust parameters of the printing process to maintain the post-printing product within acceptable tolerances.

[0009] Unfortunately, color variation in the underlying areas of the PCB negatively affects the inspection process because grey scale values will vary according to the color variation of the underlying portions of the PCB, especially near the paste edge where the solder paste is thin. Variability in color/composition of the solder paste may also cause grey scale variations that negatively affect the inspection process. Similarly, known post-only systems may be negatively affected by lighting, which must be rigorously controlled in order to avoid negative affects on the inspection process due to shading or reflection of light from components of the system. In general, the yield and accuracy of known post-only solder inspection systems are not sufficient to make such systems useful or practicable.

[0010] Many known solder paste inspection systems are stand-alone (i.e. not “in-line”) stations dedicated to performing solder paste inspection apart from and subsequent to the screen printing process step(s). Such stand-alone systems include optics and lighting that are optimally configured for the inspection of solder paste. The optics and lighting dedicated to solder paste inspection represent costly, redundant hardware in the PCB fabrication process flow in that similar optics and lighting are necessary upstream in the process flow to effect registration or alignment in the screen printing stage. Furthermore, stand-alone solder paste inspection systems typically incorporate laser stripers and other very expensive and sophisticated hardware that is often difficult to justify in a cost/benefit analysis. Stand-alone systems disadvantageously introduce additional machine set-up time and maintenance, require additional factory floor space to implement, and introduce additional undesirable handling of PCBs to convey them through the stand-alone inspection stage. Generally, a stand-alone solder inspection system will represent a significant overall reduction in throughput in a PCB fabrication line.

[0011] The present invention provides a method and apparatus implemented in a screen printing process, for in-line inspection of solder paste on a printed circuit board using a before printing image (pre-application image) to normalize an after printing image (post-application image) of the printed circuit board. A stencil in the screen printing process is also inspected using a before printing image (pre-application) to normalize an after printing image (post-application) of the stencil. Existing lighting and optics used for alignment of the screen printing stencil to the printed circuit board are used for the solder paste inspection.

[0012] In an embodiment of the invention, solder paste on printed circuit boards is automatically inspected in an operation in which stencil and/or board design information and/or images of the stencil and board are acquired and stored before solder paste printing (pre-application image data). Images of the stencil and printed circuit board after solder paste printing are also acquired and stored (post-application image).

[0013] A registration technique is used to align the pre-application and post-application images. Registration aligns a coordinate system associated with one image, e.g. the pre-application image, to a coordinate system associated with another image, e.g. a post-application image, a model image or model information.

[0014] In an embodiment, the pre-application image and the post-application image are subtracted to provide a post-application material image that represents information related to the solder paste located on the printed circuit board. Alternatively, the post-application image is thresholded to provide the post-application material image.

[0015] In a preferred embodiment, instead of examining both on-pad and off-pad regions of the printed circuit board, the invention essentially processes only the on-pad regions (i.e., regions of interest) for the position of the paste over the on-pad regions. In a preferred embodiment, therefore, the method segregates substantially any portion of the on-pad regions in the post-application material image using a mask that is created from the pre-application image. The on-pad information is preferably represented in an on-pad image. For instance, the on-pad mask is logically ANDed with the post-application paste image to form an on-pad image representative of the position of solder paste on the pads.

[0016] The on-pad region information is then analyzed to determine the degree to which the material overlays each of the pads. It is recognized by the invention that processing essentially only one region as opposed to the dual processing of several embodiments of the method decreases the processing requirements of the method while still substantially inspecting the paste. It also minimizes confusion caused by the off-pad regions of a variety of substrates, and thus the preferred method can be used in a wider range of applications.

[0017] The on-pad image is converted, using thresholding operations with different respective thresholds, to a binary image representative of solder paste on the pads of the printed circuit board. The thresholds are locally adjusted, that is, every pixel in the image has its own threshold, based on the pre-application image, so that when the pre-application image gray scale value is nearer to the paste gray scale value the threshold is (linearly) smaller. The binary image is then analyzed using a connected components analysis method as known in the art, or by using a known vision tool to measure paste distribution and/or to determine information regarding the solder paste, such as, area, center of mass, orientation, etc.

[0018] In one embodiment, a registration technique is used to register the acquired pre-application image to model information acquired during a training procedure to generate the pre-application image data.

[0019] Features of the invention include provision of an automated inspection system, optionally used in-line during screen printing of PCBs, that avoids redundant hardware and associated cost by effecting inspection using existing lighting and/or optics present in the screen printing equipment for alignment purposes. The method for inspection according to the invention can be implemented in systems having existing optics and lighting used for alignment purposes or it can be retrofitted into systems having mechanical or other alignment means by adding the necessary lighting and optics in support of the inspection process. The technique implemented according the invention normalizes each board by an image of that same board without paste and therefore avoids problems associated with prior art post-only inspection systems resulting from variations in board color, solder mask shifting and/or board or component dimension variability.

[0020] Various problems with prior art solder paste inspection systems are overcome by the method and apparatus according to the invention, including problems resulting from pads being covered by paste so that they are not visible or otherwise discernable, and problems resulting from an inability to differentiate between pads that have no paste on them and pads that are exactly 100% covered by paste. The method and apparatus according to the invention benefit from implementation of a registration mechanism which facilitates a determination as to whether the paste is in the correct location. Implementation of the registration mechanism in conjunction with a subtraction procedure facilitates differentiation between 0% and 100% coverage. Additionally, the solder paste inspection implementation according to the invention is flexible and extensible in that the registration, subtraction and masking procedures facilitate use of different algorithms for inspecting paste on pad, circuitry, bare board, mask, etc.

[0021] The foregoing and other features and advantages of the present invention will be more fully understood from the following detailed description of illustrative embodiments, taken in conjunction with the accompanying drawing in which:

[0022] FIG. 1 is a schematic diagram of components of a screen printing system as known in the art;

[0023] FIG. 2ais an illustration of stencil apertures corresponding to printing regions for receiving solder paste;

[0024] FIGS. 2bis an illustration of blocking regions where solder paste blockage is looked for in the stencil apertures of FIG. 2a;

[0025] FIGS. 2cis an illustration of smearing regions where smearing is looked for in the stencil apertures of FIG. 2a;

[0026] FIG. 2dis an illustration of runtime images of the apertures of FIG. 2aincluding the blocking regions of FIG. 2b and the smearing regions of FIG. 2c;

[0027] FIG. 3 is a flow diagram of process steps implementing stencil side inspection for a method and apparatus for in-line solder paste inspection according to the invention;

[0028] FIG. 4 is a flow diagram of process steps implementing printed circuit board side inspection for a method and apparatus for in-line solder paste inspection according to the invention;

[0029] FIG. 5ais an illustration of a pre-image with defined on-pad regions and off-pad regions;

[0030] FIG. 5bis an illustration of an on-pad difference image representative of the position of solder paste on solder pads;

[0031] FIG. 5cis an illustration of an off-pad difference image representative of the position of solder paste on areas of the board other than the pads;

[0032] FIG. 5dis an illustration of an image representative of solder paste on the printed circuit board;

[0033] FIG. 6 is a flow diagram of process steps implementing stencil side inspection for an embodiment of the method and apparatus for in-line solder paste inspection according to the invention;

[0034] FIG. 7 is a flow diagram of process steps implementing printed circuit board side inspection for an embodiment of the method and apparatus for in-line solder paste inspection according to the invention; and

[0035] FIG. 8 is a flow diagram of process steps implementing printed circuit board side inspection for an embodiment of the method and apparatus for in-line solder paste inspection according to the invention.

[0036] The present invention is useful in the context of solder paste screen printing systems known in the art. Solder paste printing using screen printers is typically effected on bare printed circuit boards (PCBs) in a manufacturing production line prior to placement of components on the board. The bare PCB is placed on or transported to a screen printing station 10, such as illustrated in FIG. 1, whereat the PCB 12 is held firmly in place on a support 14 by either vacuum or mechanical means. The screen printing station includes an upper portion that houses a stencil 16 having apertures that correspond to areas of interest on the PCB, such as solder pads, where solder is to be disposed.

[0037] Optics 18, such as movable telecentric optics, a CCD camera 20 and flash lighting 22 all known in the art are used in conjunction with a vision system, such as a Cognex 5000 Series vision system and vision tools known in the art, to capture images of stencil 16 and board 12 fiducials to effect registration, i.e. alignment of the stencil to the board. It should be noted that the placement of lighting 22, the camera 20 and optics 18 is a function of the particular vision system configuration and application as known in the art, and as provided herewith FIG. 1 is not intended to be illustrative of any particular implementation or application. The illustrative system effecting the functionality described hereinafter incorporates a vision system processor 23 receiving signals, typically captured image signals, from the camera 22. The vision system processor 23 has access to a memory 25 in which it stores information processed in effecting the inspection system according to the invention.

[0038] Generally, in the screen printing process registration or alignment between the stencil and the PCB is accomplished, such as by a method involving registration of fiducials as described in commonly owned U.S. Pat. No. 5,550,906 which is incorporated herein by reference, and the stencil 16 is lowered or actuated to a position proximate to the PCB. The solder paste is deposited on a side of the stencil 16 away from the PCB and is pushed through apertures or openings in the stencil onto solder pad sites on the PCB 12 by means of a squeegee (not shown), that traverses the stencil at an appropriate speed while exerting appropriate pressure. The stencil 16 is then lifted or actuated away from the PCB 12, leaving solder paste deposits on pad sites corresponding to the apertures in the stencil.

[0039] More specifically, in an illustrative in-line solder paste inspection implementation according to the invention, prior to closing the fixture (i.e. moving the stencil proximate to the PCB for printing), the vision system captures images of areas of interest on both the stencil and board (the side of the stencil facing the board being somewhat of a “mirror image” of the board side facing the stencil). Substantially simultaneously, the vision system captures images of fiducials on the stencil and board for alignment purposes, and images of sites that are subject to inspection on the stencil and the board. Further, after printing of the solder paste on the PCB the stencil is actuated away from the PCB and post-printing images of the stencil and the PCB are captured to effect the inspection process as described in detail hereinafter.

[0040] Inspecting a given stencil and a given board may involve multiple images depending on the image size, the stencil and board size and the number of sites on a stencil and board that the user wants to inspect. In typical usage, multiple sites on each of the stencil and the board are inspected. Therefore, capturing of an image on each of the stencil and the PCB, both before and after printing, is effected as many times as is necessary to capture images of each of the relevant sites on a particular board in a particular inspection cycle. In a typical PCB manufacturing process it is most expedient to “tile” the stencil and PCB into a plurality of sites so that only images of relevant portions of the stencil and PCB are captured during stencil and board inspection in a given cycle. Tiling minimizes memory storage requirements for a given substrate or board under inspection.

[0041] Furthermore, it is preferable that the tiles of the stencil and PCB be captured in a round-robin fashion so that not every site is inspected on a given board in a given cycle. That is, processing and fabrication line speed is enhanced when inspection according to the invention is implemented such that only a portion of all of the relevant sites are captured and inspected on a given board, while another portion of the relevant sites are inspected on a successive board and still another portion are inspected on still another board. In this manner, an inspection cycle (i.e. inspection of every site on a given board type) is effected over a plurality of boards under inspection. The round robin inspection in conjunction with tiling facilitates better statistical analyses.

[0042] In the illustrative embodiment of the invention described hereinafter, stencil inspection is described first, however, it should be appreciated that PCB inspection as described hereinafter can be effected prior to stencil inspection. In the present implementation it is preferable to consider the stencil first because certain efficiencies can be achieved in the training and inspection processes. For example, information obtained in a training process for inspection of the stencil can optionally be used in a training process for inspection of the board, as described in detail hereinafter. Similarly, information derived in a registration process for the stencil can be used as a starting point to simplify and enhance the registration process for the PCB in the PCB inspection process, also described in detail hereinafter.

[0043] Prior to undertaking any inspection of the stencil, the vision system must be trained, as known in the art. Training is handled “off-line”, i.e. prior to any inspection run, and in the case of stencil inspection basically involves constructing a model of the stencil by identifying stencil characteristics based on a “golden sample” or known good stencil. The stencil characteristics identified during training are related to sites of interest, in this case apertures in the stencil corresponding to the locations of solder pads on the PCB. Training in this illustrative embodiment is effected using vision tools with training capabilities, such as Cognex Blob Analysis Tool, which runs on the vision system used in the inspection system.

[0044] For each aperture 28, 28′, such as illustrated in FIG. 2a, a data structure is constructed during training including a unique identifier for the aperture and characteristics of the aperture. Characteristics relating to and stored in association with each uniquely identified aperture include an aperture width, aperture height, and an X-Y location of the center of the aperture. For apertures with corner rounding (e.g. 28′), radial values indicative of curvature are also stored. When the Cognex Blob Analysis Tool is used for training, center of mass as determined by the Blob Analysis Tool as known in the art, is used as the X-Y location. Those skilled in the art will appreciate that median, mode or center of boundary box can also be used as the X-Y location.

[0045] For each uniquely identified aperture a blockage region is specified during training, as illustrated in FIG. 2b. The blockage region 30, 30′ is a region interior to the aperture 28, 28′ which will be inspected for occlusion resulting from residual solder paste remaining on the stencil. Additionally, for each uniquely identified aperture a smear region is specified during training, as illustrated in FIG. 2c. The smear region 32, 32′ is a region exterior to the aperture 28, 28′ which will be inspected for the presence of solder paste resulting from residual solder paste remaining on the stencil.

[0046] During a run-time inspection of the stencil as described in detail hereinafter, smearing and blockage might appear as illustrated in FIG. 2d. The extent of the open aperture 34, 34′ may be impinged upon or restricted by blockage 36, 36′ which will be detected as it resides within the specified blockage region. Likewise, smearing 38, 38′ of solder paste will be detected on regions of the stencil exterior to the aperture which are within the specified smear region. Although not illustrated in FIGS. 2a-2d, the blockage region and smear region may be specified so as to incorporate a “don't care” region of several pixels between the outer edge of the blockage region and the inner edge of the smear region. Such a don't care zone is useful in that it removes from consideration the visual appearance at the stencil edge. The visual appearance at the stencil edge may be highly variable and extremely sensitive to lighting changes, squeegee position during printing, and aspects of the mechanical process used to form the aperture in the stencil.

[0047] The stencil inspection process flow in the solder paste inspection procedure according to the invention is illustrated in FIG. 3. Initially an image of the stencil underside or side to be positioned adjacent to the PCB is acquired 40 before solder paste is printed, to obtain pre-printing image data (at times referred to herein as “stencil pre-image data”). The pre-image can be acquired immediately before printing a board in-process or it can be obtained well in advance of the screen printing process. While the pre-printing image or pre-image data in this illustrative embodiment is acquired in-line using the camera and optics available for registration of the stencil and PCB, as discussed hereinbefore, the data can be acquired as data files from automated machinery used to fabricate the stencil (or PCB). Similarly, the pre-image data can alternatively be synthesized from a geometric description of the stencil, including information about the location and size of each aperture, combined with calibration information required to calibrate the geometric description of the stencil to pixel size(s) and aspect ratio(s) of the camera imaging system that is used to acquire post-printing image data.

[0048] Inspecting a stencil may involve multiple images depending on the image size and stencil size and the number of sites on a stencil that the user wishes to inspect, thus acquisition or construction of stencil pre-image data is effected for each site on a stencil that is to be inspected. Subsequent steps in the inspection process are performed as many times as necessary to effect inspection of each site for which pre-image data (and post-image data) is acquired.

[0049] The image acquired by the camera imaging system is registered 42 to the model of the stencil stored during training. This initial registration effectively aligns the selected sites where the apertures are on the stencil image acquired before printing (the pre-image), with the selected sites of the apertures on the model stencil image stored during training. Registration compensates for any differences in the coordinate systems of the respective acquired images. This initial registration is accomplished by performing two one-dimensional registrations, one in the X and one in the Y, on projections, as known in the art, of the model and stencil images. Registration is achieved when there is maximum correlation between the projections in each dimension. The initial registration can alternatively be effected by vision tools known in the art, such as described in U.S. Pat. No. 5,548,326 entitled EFFICIENT IMAGE REGISTRATION which is incorporated herein by reference. Registration can be implemented as well, with other vision tools using fiducials on the stencil.

[0050] After the pre-image data is a acquired and after registration of the pre-image and the model image, printing of the solder paste can be effected.

[0051] Thereafter, the camera acquires an image of the stencil underside after solder paste 44 has been printed on the circuit board (the “stencil post-image”). The stencil post-image corresponds to the same stencil or section of stencil that was acquired before the screen printing of the solder paste.

[0052] The stencil pre-image and the stencil post-image are then registered to one another 46. Registration of the pre-image to the post-image can have several degrees of freedom, such as translation (2 degree), rotation, scale, skew, etc. In the present implementation, rotation, scale and skew are minimized and registration (both pre-image to model image and pre-image to post-image) is effected for translation only. A preferred method for pre-image to post-image registration is to use the technique of Efficient Image Registration (U.S. Pat. No. 5,548,326) which can align two images of an object that has undergone a manufacturing process. However, other techniques such as Cognex's Normalized Correlation Search tool, Cognex's Caliper Tool, or Cognex's CNL-PAS tool could also be used to align the two images. It should be appreciated that pre-image to post-image registration works best if there are no changes in lighting between acquisition of the respective images.

[0053] The registration step 46 allows information from the bare stencil (pre-image) to be combined with information from the stencil after screen printing with paste (post-image). The two aligned images, i.e. the pre-image and the post-image, can be effectively combined by optionally subtracting one from another 48 yielding a difference image with pixel values that are signed quantities (i.e. positive or negative). The positive value of pixels in the difference image represent blockage when those pixels appear in the blockage region of the aperture. The negative value pixels represent smear when those pixels appear in the smear region.

[0054] The difference image is then separated into two images for separate processing of positive value pixels and negative value pixels. An expeditious method of separating the difference image is to duplicate the image to form a positive value image and a negative value image, and then replace all of the negative pixels in the positive value image with don't cares (i.e. zeros) and replace all of the positive value pixels in the negative value image with don't cares. All pixels in the negative image can then be replaced with their absolute values for mathematical convenience, and the positive value image and the absolute value image can be separately processed.

[0055] The subtraction step 48 is optional because it may not be necessary to subtract and obtain a difference image (representative of difference in gray value) in order to clearly distinguish locations where there is paste from locations where there is no paste. In situations where there is significant contrast in color (and consequently significant disparity in gray value) between paste and non-paste areas, a binary image can be generated based on discriminating one gray value (or range of gray values) as representative of paste and another gray value (or range) as representative of other than paste, i.e. stencil background. The alignment effected by registration between the pre-image and the post-printing binary image clearly indicating paste locations, facilitates detection of paste in the blockage and smear regions (defined during training).

[0056] The difference image (if necessary) or the binary image contains information clearly indicative of the location(s) of solder paste on the stencil or in the aperture(s). It is desirable to separately process aperture information in order to detect blockage and stencil background information in order to detect smear. A mask is generated using stencil training information to define in-aperture and off-aperture regions. It should be appreciated that the mask can also be generated by thresholding the pre-image. The mask is applied 50 to the difference (or binary) image to mask out the aperture area from the stencil background area so that those two groups of image regions can be processed separately.

[0057] Analysis of the in-aperture region and the off-aperture region 52 can then be effected by using a connected components analysis as known in the art. Alternatively, Cognex's Blob Analysis Tool can be used to measure paste distribution on the stencil. Cognex's Blob Analysis Tool provides information such as area, center of mass, orientation, moments, and extent. In this illustrative embodiment, area measurements of blockage and smear are the measurements of primary interest. The regions of interest for applying the Blob Analysis Tool are the registered stencil openings for paste blockage and around the stencil openings for paste smearing. The exact locations of the openings are known because the aperture locations are known from the pre-image acquired before screen printing and the relative position changes of the apertures are known from the post-image acquired after solder paste screen printing.

[0058] Printed circuit board inspection described in detail hereinafter is effected in a substantially similar manner as the stencil inspection described in detail hereinbefore. As with the stencil, prior to undertaking any inspection of the PCB, the vision system must be trained, as known in the art. Training for the PCB is also handled “off-line” prior to any inspection run. Like in the case of stencil, PCB training basically involves constructing a model of the PCB by identifying PCB areas of interest and characteristics. The PCB characteristics identified during training are related to sites of interest, which in the PCB case are solder pads corresponding to apertures in the stencil.

[0059] As with the stencil side, PCB inspection training is effected by acquiring an image of a known good entity, in this case a PCB, and constructing a model of characteristics of interest, in this case solder blobs on solder pads. Generally, PCB training is used to define a model image based on a registration window within a field of view. The model of the PCB constructed during training will result from consideration of where the solder is supposed to go as defined by a baseline location of the pads of interest and location(s) on the pad where solder is properly deposited.

[0060] As with the stencil side training, training for the PCB in this illustrative embodiment is effected using vision tools with training capabilities, such as Cognex Blob Analysis Tool, which runs on the vision system used in the inspection system.

[0061] For each registration window selected within a field of view, a data structure is constructed during training including a unique identifier for the pads of interest. Further, characteristics for each of the pads are stored in the data structure in association with the pads unique identifier, including: the pad's width, height, X-Y location, and any corner curvature information. As with the stencil training procedure, PCB training for pad information is effected using a vision tool such as Cognex's Blob Analysis tool or other vision tool having training capabilities as known in the art.

[0062] In addition to information relating to the pads, training for the printed circuit board inspection may involve gathering information relating to solder bricks or blobs deposited on a known good PCB. In such a case, where certain of the information relating to the solder is a function of characteristics of the stencil used to deposit the solder, the efficiency of the process can be enhanced by using, in the training for the PCB, certain information obtained in training for the stencil. Specifically, a given solder brick can be identified and correlated to a known aperture on the stencil already trained for. Width and height information for respective apertures can be used as width and height information for corresponding solder depositions in training for the PCB. Similarly, the X-Y location of the respective apertures can be used for a starting point for determining the X-Y location of the corresponding pads in gathering characteristics or training for the PCB. The training process for the PCB may be implemented to utilize virtually all of the characteristics relating to and stored in association with each uniquely identified aperture, including the aperture width, aperture height, and X-Y location, and for apertures with corner rounding the radial values indicative of curvature can be used. It should be appreciated that the training information relating to the stencil is a “mirror image” of the information as it would relate to the PCB, therefore in order to use the stencil information it must be processed accordingly. It should also be appreciated, as previously mentioned, that training for the PCB can be done as with the stencil, by using a vision tool such as Cognex's Blob Analysis tool or other vision tool having training capabilities as known in the art.

[0063] The PCB inspection process flow is illustrated in FIG. 4. Initially an image of the PCB is acquired 60 before solder paste is printed, to obtain pre-printing image data (at times referred to herein as “PCB pre-image data”). As with the stencil, the image can be acquired immediately before printing a board in-process or it can be obtained well in advance of the screen printing process. While the pre-printing image or PCB pre-image data in this illustrative embodiment is acquired in-line using the camera and optics available for registration of the stencil and PCB, as discussed hereinbefore, the data can be acquired as data files from automated machinery used to fabricate the PCB, as known in the art.

[0064] Inspecting the PCB, as with the stencil, may involve multiple images depending on the image size and board size and the number of sites on the board that the user wishes to inspect. Thus acquisition or construction of PCB pre-image data is effected for each site on the board that is to be inspected. Subsequent steps in the inspection process are performed as many times as necessary to effect inspection of each site for which pre-image data (and post-image data) is acquired.

[0065] The pre-image, acquired by a camera imaging system, is registered 62 to the model of the PCB stored during training. This initial registration effectively aligns the selected sites of interest on the PCB image acquired before printing (the pre-image), with the selected sites of interest on the model PCB image stored during training. This initial registration can be efficiently undertaken using a vision tool effecting normalized correlation as known in the art, which allows linear changes in brightness to be ignored when registering the model to the pre-image. As with the initial registration in the stencil inspection, initial registration in the PCB inspection can be implemented as well, with other vision tools using fiducials on the stencil. Similarly, the initial registration can be effected by vision tools known in the art, such as described in the referenced U.S. Pat. No. 5,548,326 entitled EFFICIENT IMAGE REGISTRATION.

[0066] After the pre-image data is acquired 60 and registered 62, printing of the solder paste on the PCB can be effected.

[0067] After solder paste has been printed on the circuit board, the camera acquires an image of the PCB (the “PCB post-image”) 64. The PCB post-image corresponds to the same PCB or section of PCB that was acquired before the screen printing of the solder paste.

[0068] The PCB pre-image and the PCB post-image are then registered to one another 66. Again, a preferred method for pre-image to post-image registration is to use the technique of Efficient Image Registration (U.S. Pat. No. 5,548,326) which can align two images of an object that has undergone a manufacturing process. However, other techniques such as Cognex's Normalized Correlation Search tool, Cognex's Caliper Tool, or Cognex's CNL-PAS tool could also be used to align the two images. The registration step 66 allows information from the PCB before printing (PCB pre-image) to be combined with information from the PCB after screen printing with paste (PCB post-image).

[0069] The two registered or aligned images, i.e. the PCB pre-image and the PCB post-image can be effectively combined by optionally subtracting one from another 68 yielding a difference image with pixel values that are signed quantities (i.e. positive or negative). Since solder paste is generally darker than printed circuit board pads and brighter than printed circuit board background, the positive value pixels in the difference image represent solder paste on the pads and the negative value pixels represent solder paste off the pads and on the printed circuit board background. Zero value pixels may result in the difference image, which represent either locations without solder paste or locations where the solder paste happens to have the same gray value as the printed circuit board proximate to it. Zero value pixels are “don't care” pixels, i.e. of no interest.

[0070] The difference image is then separated into two images for separate processing of positive value pixels and negative value pixels. An expeditious method of separating the difference image is to duplicate the image in order to form a positive value image and a negative value image. This is accomplished by replacing all of the negative pixels in the positive value image with don't cares (i.e. zeros) and replacing all of the positive value pixels in the negative value image with don't cares. All pixels in the negative image can then be replaced with their absolute values to form an absolute value image. Thus the positive value image constitutes a solder paste/pad image representative of solder paste on pads and the absolute value image constitutes a non-pad/paste image representative of solder paste off the pads and on the printed circuit board background.

[0071] As with the stencil side processing, the subtraction step 68 is optional because it may not be necessary to subtract and obtain a difference image (representative of difference in gray value) in order to clearly distinguish locations where there is paste from locations where there is no paste. In situations where there is significant contrast in color (and consequently significant disparity in gray value) between paste on pad and non-pad/paste areas, a binary image can be generated based on discriminating one gray value (or range of gray values) as representative of paste on pad and another gray value (or range) as representative of other-than-pad paste areas, i.e. PCB background. The alignment effected by registration between the pre-image and the post-printing binary image clearly indicating paste locations, facilitates inspection and detection of paste locations (as defined during training).

[0072] The difference images (if necessary) or the binary image contains information clearly indicative of the location(s) of solder paste. It is desirable to mask out the pads from the PCB background so that it is possible to separately process pad information and non-pad or background information. That is, for purposes of further processing and analysis to determine the location of solder on pads and the location of solder off of the pads (likely causing bridges between fine pitch or closely spaced pads), masks are used to select out and independently process pad information and PCB background information. A mask is generated and applied 70 using PCB training information to define on-pad regions 80 and off-pad regions 82 as illustrated in FIG. 5a. Masking in this illustrative embodiment is based on the relative gray value of areas of the PCB, and is accomplished by setting a threshold on the PCB pre-image and designating all pixels in that image having a gray value greater than (or equal to) the threshold to be on-pad regions 80, and designating all pixels with gray values less than (or equal to) the threshold to be off-pad regions 82. Optionally, a connected components analysis as known in the art can be effected on the on-pad regions to ensure that the pads are a minimum size.

[0073] In a combining operation 72, the on-pad mask is logically ANDed with the solder paste/pad image to form an on-pad difference image, illustrated in FIG. 5b, representative of the position of solder paste on the pads 84. The off-pad mask is logically ANDed with the non-pad/paste image to form an off-pad difference image, illustrated in FIG. 5c, representative of the position of solder paste on areas of the board other than the pads 86.

[0074] The on-pad difference image and the off-pad difference image are converted, using thresholding operations with different respective thresholds, to respective first and second binary images. The thresholds are locally adjusted, that is, every pixel in the image has its own threshold, based on the pre-image, so that when the pre-image gray scale value is nearer to the paste gray scale value the threshold is (linearly) smaller. The paste gray value is determined by statistically sampling known paste pixels at runtime.

[0075] In the present illustrative embodiment, the off-pad thresholds are calculated as a fraction of the filtered post-image and the on-pad thresholds are calculated as a fraction of the filtered pre-image, i.e. the gray scale values of each pixel in the respective image are each multiplied by a fractional value. The fractional values are determined empirically such that the locally calculated thresholds will be effectively empirically derived as a function of the lighting, type of paste, board color and factors affecting the vision system. The thresholds can be calculated locally based on a fraction of the pre-image or a filtered pre-image gray scale value. Local thresholding is described in detail in commonly owned, copending U.S. patent application Ser. No. 08/716,779 filed Sep. 24, 1996 by Nichani entitled SYSTEM OR METHOD FOR DETECTING DEFECTS WITH A SEMI-OPAQUE ENCLOSURE, which is incorporated herein by reference. Alternatively the thresholds can be an empirically derived fixed quantity.

[0076] The respective first and second binary images are in turn combined by a logical OR operation to form a single binary image, illustrated in FIG. 5d, representative of solder paste 88 on the printed circuit board.

[0077] The single binary image is then analyzed 74 using a connected components analysis method as known in the art, or by using a known vision tool to measure paste distribution and/or to determine information regarding the solder paste, such as area, center of mass, orientation, etc. From the paste area distribution and two sets of image registrations, other useful quantities can be derived, as known in the art, such as indications of bridging between solder blobs and print offset(s).

[0078] Bridging 90, the misplacement of solder in a manner that results in an erroneous electrical connection between adjacent pads or between a pad and conductive surface proximate thereto as illustrated in FIG. 5d, is determined by calculating the boundary of the paste region and measuring its proximity to neighboring pads.

[0079] Print offset 92, also illustrated in FIG. 5d, is a deviation in position between where the paste was actually printed relative to where it was supposed to be printed as defined at training time. There are two sets of information generated relating to two “types” of print offset in the system according to the invention.

[0080] A first print offset calculation relates offset of the center of mass of each of the individual solder blobs to the center of mass of its respective or corresponding solder pad, determining offset therebetween. The first print offset calculation makes available an offset for each of the individual solder blobs.

[0081] A second print offset calculation determines an aggregate offset of all of the solder paste blobs inspected relative to where they are supposed to be printed on their respective solder pads. The second print offset calculation effectively correlates a synthesized, mirrored stencil image to the single binary image representing solder on the board, enabling a “global” view of the results of the printing process. Such an aggregate view can be achieved as well by taking an average of all of the individual offsets determined according to the first print offset calculation(s).

[0082] Again, Cognex's Blob Analysis Tool can be used to measure paste distribution on the PCB. Cognex's Blob Analysis Tool provides information such as area, center of mass, orientation, moments, and extent. Illustrative regions of interest for applying the Blob Analysis Tool on the PCB are the registered solder pads and center of mass of the solder blobs for detecting print offset, and around the solder pads, i.e. between adjacent solder pads for detecting bridging. The exact locations of the pads are known from the pre-image acquired before screen printing and the relative position of the solder paste to the pads is known from the post-image acquired after solder paste screen printing.

[0083] It should be appreciated that vision tools implemented in conjunction with the inspection methodology described hereinbefore can be used to inspect for various other aspects related to the screen printed material, i.e. other than bridging and print offset, such as dross presence (i.e. extra paste not associated with any solder pad deposition), total paste coverage or the like.

[0084] Those skilled in the art will appreciate that some, or all, of the steps of subtraction, masking, thresholding, ANDing and ORing described hereinbefore can be combined and effected as hardware implementations, software implementations or a combination thereof, and that although a vision system is described implemented with a Cognex vision system and vision tools, the invention can be implemented on other platforms, such as general purpose computers running software and/or configured with hardware to effect the functionality described herein.

[0085] The local thresholding described herein can also be accomplished in various ways, such as by taking the square root of the pre-image, or by computing some linear or non-linear function of a respective image (i.e. model image, pre-image or post-image), or by using a constant.

[0086] It should be further appreciated with respect to the foregoing, that it is not necessary to acquire pre-printing image information before every printing operation and that a database of information can be stored and accessed to obtain pre-printing image information. The database of information can be from a captured image, downloaded from a Computer Aided Design tool or otherwise acquired. Likewise, for purposes of efficiency and expediency it is not necessary to acquire a post-printing image after every printing operation. Post-printing information can be acquired periodically, as necessary to keep the process within some appropriate user defined control limits.

[0087] While the system described herein acquires and stores an entire pre-image it will be appreciated that instead of storing an entire pre-image portions of an image constituting useful information can be stored and used to synthesize a pre-image during runtime. Similarly, limited useful information from the captured pre-image, such as a gray level histogram, can be stored and used to determine gray levels in the post-image for binarizing.

[0088] Additionally, while two aligned images, i.e. the pre-image and the post-image, are described herein as being effectively combined by optionally subtracting one from another yielding a difference image with pixel values that are signed quantities (i.e. positive or negative), it should be appreciated that analogs of the pre and post images, such as filtered or re-mapped images can be processed and combined to yield a “difference image” that has unsigned pixel values

[0089] Furthermore, while other operations are described herein utilizing the pre-image and the post-image, it should be appreciated that any of the images described herein can be subject to further processing, such as by filtering using a gaussian filter, median filter, smoothing filter, morphological filter or the like known in the art, in order to improve image quality.

[0090] Although a training process is described herein using a specified, known in the art vision tool (i.e. Cognex Blob Analysis Tool) for obtaining characteristics relating to the apertures for the stencil side inspection and the solder pads for the PCB side inspection, it should be appreciated that a data base of information relating to the characteristics of the stencil and the board can be obtained using other vision tools, such as Cognex Caliper Tool or from a CAD tool or other machinery that requires or acquires and has information relating to the characteristics of interest, such as automated tooling used to cut the stencil or other tooling used to fabricate the PCB. Alternatively, training information can be manually obtained by an operator and entered into the system.

[0091] It will be appreciated that although the invention described in an illustrative embodiment herein effects inspection of solder paste applied to printed circuit boards, it can be similarly implemented in processes to effect inspection of other materials such as screen printed adhesives, conductive adhesives or the like, applied on substrates other than printed circuit boards. Furthermore, although the invention is described in an “in-line” implementation with a screen printing process, the invention can be implemented and/or the inventive concepts herein modified to be used to inspect solder paste or adhesives applied in other ways. That is, efficient, robust inspection can be effected as described hereinbefore using a before image and an after image, regardless of how the material added to the after image is applied.

[0092] While the method and apparatus described herein use a masking operation that involves application of a threshold to a pre-printed image to form on-pad and off-pad masks, it should be appreciated that other means of masking or mask generation can be implemented in order to distinguish between the areas of interest and other areas that may be inspected.

[0093] Although in the method and apparatus described hereinbefore on-pad and off-pad images are described as logically ANDed with respective masks, and two threshold difference images are described as logically ORed to effect a single binary image, it will be appreciated that other means of combining the respective images can be effected to arrive at images or an image representative of the logically combined images.

[0094] FIGS. 6-8 illustrate the inspection process flow in the solder paste inspection of one region of interest, such as on-pad regions, as opposed to the dual processing stencil inspection of on-aperture and off-aperture regions illustrated in FIG. 3 and the dual processing PCB inspection of on-pad regions and off-pad regions illustrated in FIG. 4. This embodiment of the invention analyzes a single region of interest on each substrate, such as the on-pad region, to determine the position of the solder on that single region of interest. For instance, the in-aperture region is analyzed on a stencil to determine if there is solder blockage in the aperture, while the on-pad region is analyzed on a PCB to determine the position of the solder on the pads.

[0095] It is recognized by this embodiment of the invention that efficiency is achieved by processing essentially only one region as opposed to the dual processing of FIGS. 3 and 4. It is efficient because this embodiment ignores, to the extent possible, the region that is less essential to inspect, which also is the region that typically generates data that is more difficult for the invention to interpret. In the PCB inspection, the more essential region to inspect is the on-pad region and the less essential region to inspect that is ignored is the off-pad region.

[0096] In the previous embodiment, each region is separated from the difference image, where the difference image contains ideally only non-zero grey values of pixels representing paste. As previously described, the pre-paste application image is aligned and then subtracted from the post-application paste image to generate the difference image. Ideally, in the difference image, the grey values of the regions where paste was not applied equal zero and all non-zero grey values correspond to paste. The difference image will approach this ideal situation only if the pre-application image and the post-application image are successfully registered and if the grey values are consistent between the pre-application image and the post-application image at all locations except where paste is deposited. Grey values are consistent when the grey value at each location in the pre-application image and the post-application image are substantially the same value.

[0097] Alternatively, if thresholding is used instead of subtraction, the proper identification of the regions depends upon the successful registration of the post-application image and the mask, which is generated using the pre-application image data. Registration is sensitive to the consistency of the grey values of the pre-application image and the post-application image, but to a lesser degree than subtraction.

[0098] Practically, the grey values in the pre-application image and post-application image do not remain the same. The method employs the masking and the thresholding previously described to accommodate the variability of the grey values. On the PCB example, thresholding and masking can accommodate limited changes, such as minor variations in lighting, that typically occur on pads. Owing to the different nature of the regions, the thresholding cannot accommodate substantially common problems in the off-pad region, such as common misalignments, shadows caused by paste, and more than minor lighting system changes.

[0099] One principal difference between off-pad regions and on-pad regions is that the off-pad regions typically differ from one off-pad region to the next (e.g., each off-pad region contains different circuits), while the on-pad regions are more homogeneously shaped and are often evenly spaced from each other (i.e., pads are typically rectilinear shapes arranged in rows). Because of the variability among off-pad regions and the homogeneous nature of the on-pad regions, when the post-application image is misaligned from the pre-application image, it is typically misaligned by an integer number of pad regions (i.e., the pre-application image is offset from the post-application image in one direction by the space of a pad(s)). The homogeneous nature of the pads contributes to the misalignment.

[0100] When misaligned, the aligned off-pad regions (e.g. two circuits) do not correspond with each other. The different circuitry of the off-pad regions within the post-application image appears as non-zero values in the difference image. The resulting difference image contains non-zero values that look like paste to the method, but do not represent paste.

[0101] Similarly, the off-pad regions cause discrepancies in the difference image when the light source of the optical system is positioned differently while acquiring the pre-application image versus the post-application image. In addition, the metallic nature of the circuitry in the off-pad regions causes specular reflections which, as known in the art, typically image inconsistently within a sequence of images. Evaluating only one region can minimize the detrimental effects of other manufacturing conditions, such as the texture of a substrate, for example. Thus, the method and apparatus can be used with a wider variety of substrates.

[0102] The difference image, particularly in the off-pad regions, contains non-zero grey values that, therefore, appear as paste, but instead are grey value differences caused by misalignments, or other manufacturing conditions. These grey values are more difficult to interpret. Consequently, this embodiment eliminates consideration of those grey values. The invention recognizes generally that inspection of a more periodic region is preferred while the more varied region should be ignored when possible.

[0103] Even though evaluation of the off-pad region is eliminated, this embodiment still generates the majority of the essential inspection information in the PCB example. For instance, paste distribution on the pads can still be determined. The position of the paste can indicate that offset and bridging may be a problem, but they cannot be determined with certainty.

[0104] In the stencil example of the previous embodiment, the on-aperture image and the off-aperture image were processed separately. In this embodiment, the stencil process inspection flow remains the same except only one region, as opposed to both regions, is generated (steps 108 and 110) and analyzed 112 as indicated in FIG. 6, where like numerals designate like steps.

[0105] FIG. 7 illustrates the PCB inspection process flow in the solder paste inspection of the on-pad regions, as opposed to the dual processing on-pad and off-pad regions illustrated in FIG. 4, where like numerals denote like steps. Therefore, FIG. 7 eliminates part of step 68, generating the off-pad image, and eliminates step 72 of combining the on-pad and off-pad images to form a single image. Further, one region, as opposed to both regions, is generated (steps 116 and 118) and analyzed 120. Otherwise, the printed circuit board inspection described in detail hereinafter is effected in a substantially similar manner as the stencil inspection described in detail hereinbefore.

[0106] Further, FIG. 7 makes express what is inherent in the dual processing inspection of FIGS. 3, 4, and 6, i.e. the pre-application image data can be obtained in more than one manner, where the pre-application image data consists of the pre-application image associated with data of the regions of interest.

[0107] As explained earlier, the vision system can be trained with a “golden sample”, and the PCB/solder/stencil charateristics can be identified (e.g. solder blobs and/or solder pads on the PCB corresponding to the locations of apertures in the stencil), thereby generating a model. The model is then aligned with a pre-application image of the PCB/stencil to produce the pre-application image data, where the pre-application image can be acquired in-real time or from stored data files. The registration compensates for any differences in the coordinate systems of the respective acquired images.

[0108] Alternatively, as explained earlier, the pre-application image is evaluated using vision tools previously referenced, e.g. Cognex's Blob Analysis Tool, to construct a data structure for each pad/solder bump/aperture identified by a user, therefore generating the pre-application image data. Evaluating the pre-application image eliminates the need to align a model to the pre-application image. Consequently, the registration of the pre-application image to the model is eliminated in FIG. 7.

[0109] FIG. 8 illustrates an additional embodiment of the PCB inspection process flow in the solder paste inspection of the on-pad regions, where like numerals denote like steps. FIG. 8 expressly shows using thresholding instead of the subtraction step 68 of FIG. 7 as hereinbefore described. Further, FIG. 8 makes express what is inherent in the optional thresholding of the dual processing inspection, which is that the thresholding can be completed at any point in the process. For example, the masks are applied to select out and independently process pad information 122, and the thresholding is completed in conjunction with the analysis of the on-pad region 124.

[0110] Otherwise, the printed circuit board inspection is effected in a substantially similar manner as the printed circuit board inspection described with reference to FIG. 7.

[0111] Although the invention has been shown and described with respect to exemplary embodiments thereof, various other changes, omissions and additions in the form and detail thereof may be made therein without departing from the spirit and scope of the invention.