Automatic Calibration of the Optical System in Passive Component
Inspection
Sungho Suh and Moonjoo Kim
Samsung Electro-Mechanics, 150 Maeyoung-ro, Suwon, Korea
{sh86.suh, moonjoo.kim}@samsung.com
Keywords:
Automatic Calibration of the Optical System, Passive Component Inspection, Gain, White Balance Ratio.
Abstract:
A passive component inspection machine is to obtain a image of a passive component by using a specific
lighting and camera, and to detect defects on the image of the component. It inspects all the aspects of the
component based on the image which is captured by using the lightings and cameras. The number of the
lightings and cameras are proportional to the number of the component aspects. To detect the defects of the
component effectively, the difference between the image quality by each camera should be minimized. Even
if the light conditions are calibrated automatically, the average intensities of the images are different because
of influence of Bayer filter which is used in CCD camera in the passive component inspection machine. More-
over, there is one more problem that the range of the light intensity cannot cover the range of the component
reflectance. Sometimes, it is needed to calibrate a gain value and white balance ratios of the camera manually.
In order to solve the problems, we propose an automatic calibration method of the optical system in passive
component inspection machine. The proposed method minimizes the influence of Bayer lter, does not use
any initial camera calibration, and find the optimal values for the overall gain and white balance ratios of red,
green, blue colors automatically. To reduce the influence of Bayer filter, we perform to find the optimal values
of all colors balance ratio iteratively and formulate a relation between the overall gain and the white balance
ratios to control all the parameters automatically. The proposed method is simple and the experimental results
show that the proposed method provides faster and more precise than the previous method.
1 INTRODUCTION
A passive component is a component that con-
sumes, accumulates, and emits electric power sup-
plied from the outside. It means that the part is in-
capable of an active function. It includes various
types of chips: MLCC (Multi-Layer Ceramic Capac-
itor), BLCC (Boundary Layer Ceramic Capacitor),
VLC (Vertically Laminated Capacitor), EMC (Elec-
tro Magnetic Compatibility), etc. One of the pro-
duction processes for the passive components is a
visual inspection that inspect a chip visually. Prior
to visual inspection, electrical properties inspection
finishes to ship the chip without any visual defects.
A passive component inspection machine (Liu et al.,
2007) (Kim et al., 2013) (chieh Tseng et al., 2006)
(chieh Tseng et al., 2009) is to capture an image of
the passivecomponentby using a specific lighting and
camera, and to judge whether the chip has defects.
The passive component inspection machine is shown
in Figure 1 and the flow of the machine is shown in
Figure 2. It inspects all the aspects of the compo-
nent based on the image captured by the lightings and
cameras. The number of the lightings and cameras are
proportional to the number of the component aspects.
For effective detection, the differencein image quality
by each camera should be minimized. More and more
different image quality between cameras can cause
higher false-positive and false-negative. Therefore, it
is important that the inspection machine should se-
cure uniformity of the image quality and calibrate its
optical system to minimize the difference of the im-
age quality between each camera. The number of the
lighting for inspection is 12 based on two lanes six
sides machine which is shown in Figure 2, and the
number of the lighting channel is 128. And the num-
ber of the used cameras is also twelve and it is nec-
essary to calibrate the overall gain value, red, green,
blue balance ratios. Previously, those are calibrated
by operators manually. Thus, obtained image qual-
ities vary as shown in Figure 3 because the calibra-
tion by each operator is different. Moreover, it takes
around 240 minutes averagely per a machine to set
the optical system manually. In order to improve the
230
Suh S. and Kim M.
Automatic Calibration of the Optical System in Passive Component Inspection.
DOI: 10.5220/0006165402300237
In Proceedings of the 12th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2017), pages 230-237
ISBN: 978-989-758-225-7
Copyright
c
2017 by SCITEPRESS Science and Technology Publications, Lda. All rights reserved
Figure 1: Vision system of passive component inspection.
Figure 2: A flow of inspection machine.
existing manual calibration method, we used an auto-
matic light calibration method of the optical system
to make sure of uniform image quality and reduce the
setting time.
The previous method makes the light channels
grouping and set the directional ratio of lighting.
Based on body area of the chip, user sets a target
value of image intensity and change the light value to
converge into the target value in order of red, green,
and blue. However, the average image intensities are
changed because the cameras in the inspection ma-
chine are CCD cameras which use Bayer filter (Hubel
et al., 2004). The spectral response of CCD camera
is shown in Figure 5. In Figure 5, wavelengthes are
duplicated between red and green, and green and blue
colors. Due to the duplicated wavelengths, blue in-
tensity is not only changed, but also green intensity in
the image can be changed when the blue light value
is changed, and green intensity is not only changed,
but also red and blue intensities are affected when the
green light value is changed.
Furthermore, reflectance ratio is different per each
type, model of chip, lot number, capacity. Even
Figure 3: Chip images of four sides.
Figure 4: (a) brightness control by black level offset (b)
Contrast control by gain.
though the lighting value is used as maximum value,
the average intensity can be less than the target value.
Or the average intensity can be higher than the target
value even though the lighting value is set as mini-
mum. In the case, the previous method considers the
automatic lighting calibration as failed and need to
calibrate again manually. It requires more often man-
ual calibration, since there are lots of types of the pas-
sive components and various capacity differences.
There are two ways to improve amount of light
normally. First way is to use brighter lens or change
the light which can reflect more light on the target
object. Second way is to amplify the electrical signal
from output of the camera sensor by increasing gain
and white level of camera sensor. Without changing
hardware configuration, we should use second way.
There are two values to amplify the output power of
image sensor: Black level and white level. Black level
is to change brightness and white level is to change
contrast. In other words, black level is to adjust the
offset in Input-Output graph which is shown in Figure
4 (a). However, it is normally used as initial value
because dynamic range can be reduced and noise level
can be increased when the offset is changed. On the
other hand, gain and white level change the gradient
of the graph in Figure 4 (b). The gain value functions
to change the overall gradient and is represented in
log scale. But the white level is functioned as white
balance ratio in color sensor, changes the ratio of R,
G, B colors, and is represented in integer scale.
As mentioned above, image intensity is deter-
mined by the overall gain value and the white balance
ratios of the camera besides lighting conditions. To
reduce variables, the previous method fixes the gain
value for each camera and measures the combination
of ND filter and back light. Through the measure-
Automatic Calibration of the Optical System in Passive Component Inspection
231
Figure 5: wavelength of color channels.
ment, the white balance ratio values are fixed by man-
ual setting. To fix these values, twelve cameras, based
on the 2 lanes 6 sides machine, are required to cal-
ibrate manually. However, it takes around 180 min-
utes for the initial calibration. In addition, the failure
to convergeon target value frequently occurs since the
determined gain value is not proper to all the compo-
nents. In the case of failure, ratio value of the failed
color should be balanced manually. It also takes lots
of time and the operator feels difficulty to adjust the
values.
In this paper, we propose an automatic calibration
method of the optical system in passive component
inspection machine. The proposed method minimizes
the influence of Bayer filter, does not use any initial
camera calibration, and find the optimal values for the
overall gain and white balance ratios of red, green,
blue colors automatically. To reduce the influence
of Bayer filter, we perform to find the optimal val-
ues of all colors balance ratio iteratively and formu-
late a relation between the overall gain and the white
balance ratios to control all the parameters automati-
cally. Due to the iterate steps, the required time can
slow down, but it does not need the initial calibration
process and any manual calibration. As a result, it can
be calibrated automatically for more various models
and lots.
The rest of this paper is organized as follows. The
details of the automatic calibration method with cam-
era control are proposed in Section2. In Section 3, the
experimental results are presented. The experiments
are performed on the actual inspection machines. The
paper concludes in Section 4 with description of our
future work.
2 AUTOMATIC CALIBRATION
OF OPTICAL SYSTEM WITH
CAMERA CONTROL
2.1 Automatic White Balance Ratio
Calibration
To minimize the influence of Bayer filter and to con-
verge to the target value by using similar lighting
value, all colors of white balance ratios need to be
adjusted. First of all, initial light calibration needs
to be performed. If the initial light calibration suc-
cessfully converges to the target value, the average of
lighting values of all aspects can be calculated. Af-
ter setting the average lighting value to all sides, the
optimal white balance ratio value can be obtained. It
can make image intensities matching with the target
value. It means that image intensities converge to the
target value under same lighting condition. If the im-
ages of all sides have same lighting condition, the in-
fluence of Bayer filter can be reduced. At this time,
the camera calibration needs to be set similar white
balance under similar lighting condition.
To minimize the influence of Bayer filter, we cal-
ibrate the white balance ratio in order of blue, green,
red and recalibrate the blue one. Thereby, all cameras
can converge to the target value with similar lighting
condition. If there is a big difference between initial
white balance ratio value and the optimal value, the
image intensity can be higher or lower than the tar-
get value even though the lighting value reach maxi-
mum value or minimum value. In the case, the white
balance ratio of the failed camera and color should
be adjusted that the initial lighting ca be successfully
calibrated. The process of the white balance ratio cal-
ibration is as follows.
Step 1. Initial Automatic lighting calibration
Step 1.1. Performing iterative lighting calibration
for matching with the target value
Step 1.2. When the initial lighting calibration is
failed, Changing balance ratio of the failed camera
and color for matching with the target value with fixed
lighting value
Step 2. Calibration of blue balance ratio
Step 2.1. Setting the light value of blue channel
for all the cameras as the average value of the initial
lighting values from Step 1
Step 2.2. Performing iterative calibration for blue
white balance ratio with fixed blue light value from
Step 2.1
Step 2.3. Performing the lighting calibration of
green channel
Step 3. Calibration of green balance ratio
VISAPP 2017 - International Conference on Computer Vision Theory and Applications
232
Step 3.1. Setting the light value of green channel
for all the cameras as the average value of the initial
lighting values from Step 2.3
Step 3.2. Performing iterative calibration for
green white balance ratio with fixed green light value
from Step 3.1
Step 3.3. Performing the lighting calibration of
red channel
Step 4. Calibration of red balance ratio
Step 4.1. Setting the light value of red channel
for all the cameras as the average value of the initial
lighting values from Step 3.3
Step 4.2. Performing iterative calibration for red
white balance ratio with fixed red light value from
Step 4.1
Step 4.3. Performing the lighting calibration of
blue channel
Step 5. Re-calibration of blue balance ratio
Step 5.1. Setting the light value of blue channel
for all the cameras as the average value of the initial
lighting values from Step 4.3
Step 5.2. Performing iterative calibration for blue
white balance ratio with fixed red light value from
Step 5.1
Step 5.3. Performing the lighting calibration of all
channels
2.2 Automatic Overall Gain Value
Calibration
If the overall gain value is bigger or smaller than the
optimal value, the captured image intensity cannot
match with the target value even though the white bal-
ance ratios reach maximum or minimum values. And
if the gain value is changed, the image intensity of
all channels also changes. In order to keep the im-
age intensity with changing the overall gain, all chan-
nels of white balance ratios should be changed. And,
running time of the automatic calibration can increase
hugely if we use iterative method for finding the opti-
mal value of the overall gain. Therefore, we propose
an automatic overall gain value calibration by using
relation between gain and balance ratios. The relation
is released in the camera manual.(reference) Through
a verification experiment in Section 3, we verify the
relation. The relation is formulated as follows.
y
Red
= 10
0.0359G/20
B
Red
64
x
Red
y
Green
= 10
0.0359G/20
B
Green
64
x
Green
y
Blue
= 10
0.0359G/20
B
Blue
64
x
Blue
(1)
where (y
Red
,y
Green
,y
Blue
) is the output power of red,
green, blue channels, (x
Red
,x
Green
,x
Blue
) is the input
power of all channels, the unit of all the powers is
dB. G is the setting value of overall gain. Originally,
the gain is represented by dB unit, but represented by
integer scale in this formula. (B
Red
,B
Green
,B
Blue
) is
the white balance ratios of all channels, the minimum
value should be set as 64. Using these formulas, we
need to obtain the relation between the variation of
gain and the variation of each white balance ratio. To
obtain those, we can represent the formulas as fol-
lows.
y
Red
= 10
0.0359(G+G/20
B
Red
+ B
Red
64
x
Red
y
Green
= 10
0.0359(G+G)/20
B
Green
+ B
Green
64
x
Green
y
Blue
= 10
0.0359(G+G)/20
B
Blue
+ B
Blue
64
x
Blue
(2)
where G is the variation of the gain value, and
(B
Red
,B
Green
,B
Blue
) are the variations of the
white balances of all channels. By using (1) and (2),
the relations can be rewritten as
G =
20
0.0359
log
10
B
Red
B
Red
+ B
Red
(3)
And using (3), we can calculate the relation be-
tween red and green balance ratios, the relation be-
tween red and blue balance ratios. Thus, we can
rewrite as
B
Green
= B
Red
B
Green
+ B
Red
B
Blue
= B
Red
B
Blue
+ B
Red
(4)
Using (3) and (4), we can obtain the deviation of
the corresponding gain value and the deviations of the
balance ratios of other channels when the red white
balance ratio value is adjusted. Thus, the overall gain
and other channels of balance ratios can be changed
to maintain the image intensity when the white bal-
ance ratio reaches maximum or minimum value. By
changing the gain and other channels of balance ra-
tios, the optimal white balance ratio and gain can be
obtained.
3 EXPERIMENTAL RESULT
In this section, we verify the formulas from Section 2
and evaluate the performanceof the proposed method.
And we compare the results of the proposed method
to the results of the previous work.
Automatic Calibration of the Optical System in Passive Component Inspection
233
3.1 The Experimental Verification of
Correlation between Image Sensor
Gain and the White Balance Ratio
The gain value is defined differently by a manufac-
turer of the image sensor, normally it is separated into
two types.
Gain
dB
= A× Gain
raw
(5)
Gain
dB
= A× log
Gain
raw
B
(6)
In (5), both system gain (Gain
dB
) and raw gain
(Gain
raw
) are defined in dB scale, thus the system
gain is increased in integer scale. In (6), the sys-
tem gain is increased in exponential scale by raw gain.
Also, the system gain is generally set to be increased
in integer scale by balance ratios. In this section,
we verify the output signal value of the input signal
(I
cameraout
/I
sensorout
) and the relation between the
gain and balance ratios.
In the experiment, Basler Ace 640-GC cam-
era (Basler Vision Technologies, 2015) is used to
verify the relations. For the camera, Gain
dB
and
I
cameraout
/I
sensor
o
ut
are as follows.
Gain
dB
= 0.0359 × Gain
raw
I
cameraout
I
sensorout
10
0.0359Gain
raw
20
I
cameraout
I
sensorout
BalanceRatio
raw
64
(7)
Before verifying the relations, we perform an ex-
periment to check a linearity of camera sensor. The
property of the linearity is related to generate current
from quantum efficiency of sensor and convert it to
voltage. Gain can determine how much amplify the
electrical signal from the process. Thereby, we can
predict the result by changing the amount of light if
the linearity is verified. In the experiment, white LED
dome is used and balance ratios are determined by
putting pure white color target. In the experiment of
the linearity, the current for lighting, the gain value,
and balance ratios are kept constantly and exposure
time of camera sensor is changed from 100µs to 10µs
by decreasing 10µs in order to control the input power
of sensor. And, we check that the intensities of the
ten images are increased linearly. The experimental
result is shown in Figure 6. We can figure out that the
intensities of the images are increased by the exposure
time of camera linearly in Figure 6(a). And the result
of linearity by linear fitting is shown in Table 1. The
value of R-square is 0.9999, and the result of fitting is
reliable. Based on the result, the differences between
Table 1: Result of line fitting for linearity.
Equation y = A+ B× x
Adj.R-Square 0.9999
A(intercept) -2.0158
B(slope) 2.2253
Table 2: Result of curve tting for increasing the gain value.
Equation y = A+ B× 10
0.0359x/20
Adj.R-Square 1
A(intercept) -0.2613
B(slope) 24.3031
the measured intensity and fitting value is displayed in
Figure 6(b) and non-linearity is calculated by a equa-
tion as follow.
Non linearity(%) =
| +
max
| + |
max
|
maximum
× 100
(8)
where +
max
is the maximum deviation of positive,
max
is the maximum deviation of negative, and
maximum is the maximum measured value. The re-
sult of the non-linearity is 0.17%, it means that the
input of camera sensor and the output have a linear
relation. Thus, we can conclude the output intensity
can be increased linearly by the amount of input light.
To verify the relation between the gain and the bal-
ance ratios, we perform three experiments:1) verify-
ing a relation between the gain and output image in-
tensity, 2) a relation between the balance ratios and
output image intensity, 3) a relation between the gain
and balance ratios. To check the influence of the gain
value to the output image intensity, the gain value is
set as operational factor by increasing from 100 to
550, other factors are set as controlled factors. White
LED dome light is also used in the experiment.
I
cameraout
= A+ B× 10
0.0359×Gain
raw
20
(9)
The curve fitting result of (9) to the measured data
is shown in Figure 7 and Table 2. In the result, the
output value is increased nonlinearly, and the curve
fitting result is completely matched with (9).
Next experiment is to verify the relation between
the balance ratios and image intensity. The balance
ratios are composed of three channels;red, green, and
blue. Therefore, when the influence of a specific
channel of the balance ratios is measured, the other
channels of the balance ratios are set as 0 to remove
their influences. The experiment measures the image
intensity by increasing the balance ratio with an in-
crement in 20. The relation to be verified is as follow.
VISAPP 2017 - International Conference on Computer Vision Theory and Applications
234
(a) Result of linearity test (b) Result of Non-linearity analysis
Figure 6: Result of linearity test.
Figure 7: Relation graph for Image intensity and the gain
value.
I
cameraout
= A+ B×
BalanceRatio
raw
64
(10)
If the relation between balance ratio and the inten-
sity has a linearity, 10 can be verified. The measured
outputs and the result of fitting (10) for each channel
are shown in Figure 8 and Table 3. Also, the results
show that all channels of the balance ratios are linear
and (10) is matched to the results. In the results, each
slope of channel is different because it is determined
by spectral response of camera sensor.
Last experiment in this subsection is to verify how
the gain and the balance ratios impact on image inten-
sity. In the experiment, the gain value is fixed as 150,
250, and 350 for three times of measuring the inten-
sity. As same as the previous experiment for the bal-
ance ratios, when the influence of a specific channel
of the balance ratios is measured, the other channels
of the balance ratios are set as 0 to remove their influ-
ences. The verifying relation is as follow.
I
cameraout
= A+ B×10
0.0359×Gain
raw
20
×
BalanceRatio
raw
64
(11)
Using (11), each channel of the balance ratio is
matched to the measured result. The fitting is a type
Figure 8: Relation graph for Image intensity and the balance
ratios.
Table 3: Result of line fitting for increasing each balance
ratios.
Equation y = A+ B× (x/64)
Red Adj.R-Square 0.9999
Red A(intercept) -0.5166
Red B(slope) 25.9076
Green Adj.R-Square 0.9999
Green A(intercept) -0.80093
Green B(slope) 35.1681
Blue Adj.R-Square 0.9999
Blue A(intercept) -0.4936
Blue B(slope) 24.1601
of surface fitting. If the data is placed on the surface,
the relation can be verified. The result is shown in
Figure9 and Table 4. In the result, the fitting results
are perfectly placed on the surface. Therefore, we can
verify the relation between the gain value and the bal-
ance ratios. And we can understand why color is not
balanced when the gain value is adjusted because each
color channel has a different factor.
Automatic Calibration of the Optical System in Passive Component Inspection
235
(a) Red (b) Green (c) Blue
Figure 9: Result of Changing the gain and the balance ratios.
Table 4: Result of surface fitting for increasing gain and
each balance ratios.
Equation y = A+ B × 10
0.0359y
20
x
64
Red Adj.R-Square 0.9999
Red A(intercept) -0.1234
Red B(slope) 4.3491
Green Adj.R-Square 0.9999
Green A(intercept) -0.1708
Green B(slope) 6.3044
Blue Adj.R-Square 1.0000
Blue A(intercept) -0.1324
Blue B(slope) 4.6417
Table 5: Comparison for average required time.
Average required time(sec)
The previous work 55.6
Manual calibration 602.3
The proposed method 60.1
3.2 The Experimental Results of the
Proposed Method
In this subsection, we evaluate the performance of the
proposed method with real products and compare the
results of the proposed method to the results of the
previous work and the manual calibration. The ex-
periments measure the required time, a deviation of
lighting values between all cameras, and a deviation
of average gray scale value for each camera. In our
experiment, seven machines which are used in actual
field are used to measure the time and quality. And
it is applied to 50 different lots. Since the results of
manual calibration for the gain and balance ratios can
be different by each operator, an expert in understand-
ing how to calibrate adjusts the camera setting.
The result of required time is shown in Table 5. In
Table 5, the previous method can be finished within
shortest time, and the manual calibration which cali-
brate the image deviation within 10 gray scale value
Figure 10: The result of the previous work.
Figure 11: The result of the manual calibration.
takes almost ten minutes. When the operator in field
calibrates it, it takes over than twice. The proposed
method takes longer than previous method, but it
takes hugely less than the manual calibration. How-
ever, the required time for the proposed method just
includes the succeeded cases except the failed cases.
Next comparison is the deviation of lighting val-
ues and average gray scale values. If the deviation
of lighting values is huge, the deviation of gray scale
value for all the color channels is increased due to im-
VISAPP 2017 - International Conference on Computer Vision Theory and Applications
236
Table 6: Comparison for deviations of light values and gray-scale.
deviation of light value(R,G,B) gray-scale(R,G,B)
The previous work (80,81,46) (9.6,19.6,13.4)
Manual calibration (74,44,26) (8.7,9.0,10.1)
The proposed method (9,10,6) (3.9,4.1,5.1)
Figure 12: The result of the proposed method.
pact of Bayer filter. In addition, in the case that the de-
viation of lighting value is high butit is converged,the
camera calibration which should make similar gray-
scale values with same light values is not proper. The
deviation of gray-scale values is measured on the cen-
ter area of body in the product. The deviation of light
values and the average gray-scale values are shown
in Table 6. The result of the previous work does not
include the failed cases. Because the previous work
does not calibrate the gain value and the balance ra-
tios, the deviation of light values is huge and the de-
viation of average gray-scale values is over than 10.
Moreover, 35 of the total 50 lots are failed to con-
verge to the target, and it needs to be calibrated man-
ually. The manual inspection gives better image qual-
ity within 10 difference of gray-scale values, but the
deviation of the light values is large. It means that
the difference of colors is improved but the camera
calibration is improper. On the other hand, the pro-
posed method can provide the deviation of the gray-
scale values within 5 and the deviation of the light
values within 10. And, all the lot with the proposed
method are successfully calibrated. The results of the
calibrations are shown in Fig 10, Fig 11, and Fig 12.
4 CONCLUSIONS
In this paper, an automatic calibration method of the
optical system has been proposed for passive compo-
nent inspection. The proposed method calibrated the
gain and white balance ratios of the inspection cam-
eras using the relation between the gain and the bal-
ance ratios. The proposed method set the gain and
the balance ratios to make the obtained images sim-
ilar with same light conditions and make the differ-
ences of the image intensities and the light values
minimized. In the experiment, we compared the pro-
posed method to the previous method and the manual
calibration method. The proposed method gave better
performance than previous work and the manual cali-
bration method. It took less time to calibrate the opti-
cal system and minimized the difference of the image
intensities.
Since the experiment was performed for 50 lots,
we have a plan to experiment more various models
and lots. Furthermore, we need to reduce the re-
quired time for the calibration and apply the calibra-
tion method to other camera models.
REFERENCES
Basler Vision Technologies (2015). User Manual for GigE
Cameras.
chieh Tseng, C., fu Lai, M., and song Lee, P. (2006). Image
Inspection System for Defect Detection of Multilayer
Ceramic Capacitors. International Conference on In-
telligent Information Hiding and Multimedia.
chieh Tseng, C., Wu, J.-H., and Liao, B.-Y. (2009). Defect
Detection of Skewed Images for Multilayer Ceramic
Capacitors. International Conference on Intelligent
Information Hiding and Multimedia Signal Process-
ing.
Hubel, P. M., Liu, J., and Guttosch, R. J. (2004). Spatial fre-
quency response of color image sensors: Bayer color
filters and Foveon X3. SPIE Electronic Imaging.
Kim, D.-H., tae Bae, J., bok Kim, T., and hyeon Kim, G.
(2013). Electrostatic inductive absorption type inspec-
tion table for electronic components.
Liu, D., Braden, D., De, V. R. V., Hawkes, M. V., and
Nebres, J. V. (2007). Inspection machine for surface
mount passive component.
Automatic Calibration of the Optical System in Passive Component Inspection
237