Nordson_WhitePaper_WP8901_Deep Learning.pdf - 第3页
Figure 2. Blob analysis incorrectly divi d ed this good corner fi ll into sev eral sm aller pieces and reported a false neg ative resul t. The m anufacturer desire d a solution that could confirm the presence of absence …
recognize objects in this image? What are they? Where are they? How big are they?” Finally,
segmentation tries to associate each point (pixel) in the image with an object. Segmentation tries to
draw a closed border around each detected object. Any pixe l inside the border is part of that object. Any
pixel not in an object is background.
Figure 1. Object detection problem s can be di vided into 3 sub-types of increasing complexity:
classifi cation, detection, and segmentation.
Corner fill inspection
A memory manufacturer uses fill under the corners of ICs to bond the package to the substrate. They
needed to inspect for the pres ence or absence of fill and ensure that there is neither too much, nor too
little. They required a solution that could measure the length of the corner fil l and compare it to
specifications.
Traditional methods of corner fill inspection, such as blob analysis, are challenged by the lack of gray
level specifici ty in this application. Blob analysis attempts to find a continuous blob within a certain
intensity or contrast range and sometimes breaks larger blobs into separate small er blobs. Figure 2
shows an exampl e of corner fill in which bl ob analysis incorrectly found multiple smaller blobs, all too
short to meet specification, rather than a single good one, and reported a false negative resu lt. This is an
example of a problem that is challenging for AOI but relatively easy for humans, who can readi ly see that
the fill is continuous in the example. Results from inspection with blob analysis were i nconsistent and
unreliable, with many false negatives. The manufacturer sought a more reliable approach.
Figure 2. Blob analysis incorrectly divided this good corner fill into several smaller pieces and reported a
false negative result.
The manufacturer desired a solution that could confirm the presence of absence of the corner fill and
measure its size – a classic example of object detection, mid-level in complexity among classification,
detection and segmentation as described above. Research in the use of deep learning for object
detection has made dramatic progress in recent years, driven by demand across a variety of
applications, including facial recognition and autonomous driving. Autonomous driving shares som e
requirements with the corner fill application. It needs to be fast, i.e., it nee ds to detect objects, such as
pedestrians and other cars in nearly real-time. It needs to determine how big the object is, and where it
is in the field of view, with enough accuracy to avoid a collision. I t does not need to precisely define the
edges of the object.
A deep learning algorithm for object detection that has gained wide acceptance, is used in autonomous
drivi ng applications. Earlier approaches to object detection repurpose classifiers to perform detection,
while this approach frames object detection as a regression problem to spatially separated bounding
boxes and associated class probabilities. A single neural network predicts bounding boxe s and class
probabilities directly from full images in one evaluation. Since the whole detection pipeline is a single
network, it can be optimized end-to-end directly on detection performance. The approach applies a
single neural network to an image, divides the image into regions, and predicts bounding boxes that are
weighted by class probabilities. It shares the same network architecture across all classes, which
simplifies programming and speeds inferencing. The network can be trai ned on a personal computer
with a single GPU (g raphics processor). Once trained, inferencing can run on a device as simple as a
mobile phone.
Corner fill i s located below the corners of a relatively flat, rectangular package wher e it is not easy to see
from any one point of view. Some corner fill inspection systems use a top-down camera and a mirr or to
view all sides of the package as it rotates – an approach that adds time and complexity to the data
acquisition process. The system used in this work is the Nordson TEST & INSPECTION SQ3000™ Multi-
Function System for AOI, SPI and CMM powered by M ulti-Reflection Suppression™ (MRS™) sensor
technology) incorporates a uni que optical sensor originally designed for three-dimensional inspection
and metrology using phase shift profilometry. The sensor v iews the inspection target simultaneously
through four side-view cameras positioned off the normal axis at azimuths of 0°, 90°, 180°, and 270°. For
corner fi ll i nspection, t he side-view cameras can instantly acquire images of all four sides, without
mirrors or rotating the sample (Figure 3).
Figure 3 The MRS sensor in the inspection system uses 4 cameras arranged around the sample to
provide detailed views of all sides. The corner fill is located under the corners of the package where it
cannot be seen from any single point of view.
The sy stem was trained and tested with a set of 72 images. 62 images were used for training and 10 for
validation. Corner fill was labeled in the training images and the training was run on a standard personal
computer with dual GPUs. One simplified model was able to perform all necessary tasks, powered by
deep learning. The bounding box proved to be sufficiently accurate for corner fill measurements (Figure
4). The program runs smoothly and delivers a simplified user experience.
Figure 4. Examples of corner fill measurements using the bounding box reported by the deep learning
object detection algorithm. The left image shows good corner fill meeting the length specification. The
right image shows corner fill too short.
Conclusion
We have described the use of an AOI system (Nordson TEST & INSPECTION MRS-Enabled SQ3000™) and
an associated deep learning algorithm to inspect corner fill in an electronics assembly application. The
results confirm robust performance in detecting the presence or absence of corner fill and measuring its
length. The system was easy to train and runs readily on a standard PC. Integration of the dee p learning
algorithm into standard system software would allow factory engineers to train networks that could
then be run l ocally for inspection. There are many more potentiall y valuable applications for deep
learning object detection in SMT and semiconductor appli cations that we are actively pursuing.