Current Issue

Journal of Korea Technical Association of the Pulp and Paper Industry - Vol. 52 , No. 3

[ Original ]
Journal of Korea Technical Association of the Pulp and Paper Industry - Vol. 52, No. 2, pp.3-11
Abbreviation: J. Korea TAPPI
ISSN: 0253-3200 (Print)
Print publication date 30 Apr 2020
Received 30 Aug 2019 Revised 21 Nov 2019 Accepted 28 Feb 2020
DOI: https://doi.org/10.7584/JKTAPPI.2020.04.52.2.3

Identification and Detection of Surface Defects of Outer Package Printed Matter Based on Machine Vision
Shuo Wang1, ; Jiangfeng Xu2 ; Fangzhou Wang1 ; Shenshen Ruan1
1School of Information Engineering, Zhengzhou University, Zhengzhou, Henan, 450001, China, Student
2School of Information Engineering, Zhengzhou University, Zhengzhou, Henan, 450001, China, Professor

Correspondence to : E-mail: shuow_ws@126.com (Address: School of Information Engineering, Zhengzhou University, No. 100, Kexue Avenue, Zhengzhou, Henan, 450001, China)


Abstract

In the printing process, the surface of the outer package printed matter is prone to defects such as chromatic aberration and misting, leading to a defective product. It is necessary to take an effective method to identify and detect surface defects in printed matter. In this paper, the application of machine vision technology in the detection of defects in outer package was studied. Firstly, a machine vision image acquisition device based on charge coupled device (CCD) camera was designed; secondly, the obtained image was processed by graying the average method, then OTSU binarization was performed after denoising by median filtering, and finally an improved differential matching method was designed for defect detection. The example analysis showed that the clear defect image of the outer package could be obtained by using the method proposed in this paper. The detection rate of the method in detecting 2,000 images reached 99.4%, and the average detection time was 103 ms. In the detection of 5,000 images, the detection rate of the manual detection method was 81%, but the detection rate of the method proposed in this paper was 99.1%. The experimental results proved the reliability of the method and provided some theoretical basis for the further application of defect detection technology based on machine vision in the print industry, which was conducive to the good development of the print industry.


Keywords: Machine vision, print, defect detection, image processing

1. Introduction

The outer package of various products is a very common item in people’s lives. Consumers’ demands for quality tend to increase over time. However, in the printing process, the outer package printed matters may have defects such as chromatic aberration, misting, scratches, stains, etc. due to insufficiency in terms of technology, equipment, environment or other aspects.1) If these defects are not detected in time, defective products will be produced, resulting in waste of production resources and damaging the image and reputation of the company in case of circulation to the market. Therefore, in order to control the appearance of defective products, it is necessary to detect timely and effectively surface defects during the printing process to avoid the occurrence of unqualified products. With the development of automation technology, the emergence of defect detection methods based on machine vision has greatly improved the efficiency of defect detection,2) which can acquire image of objects through cameras, optical lenses and other equipment and then achieve accurate identification of defects by processing the images. Defect detection technology based on machine vision has been widely applied and studied. Meng et al.3) processed the image of the collected car hose through HALCON algorithm according to machine vision and then identified the hose. It was found through experiments that the method could accurately predict the surface defects of the hose. Wang et al.4) combined with machine vision and sensor coupling, collected images by charge coupled device (CCD) camera and processed the images by VS2010 and Halcon library. The results showed that the method could successfully identify gears with surface defects and achieve gear automation defect detection, which had a high application value. Min et al.5) designed a railway track surface defect detection method based on machine vision. Firstly, the image acquisition device equipped with light emitting diode (LED) light source was used to acquire the orbit image, then the image was optimized by the morphological process, and finally the defect feature was obtained by tracking the direction chain code. It was found through experiments that the detection speed of the method could reach 2 m/s, which effectively realized the real-time automatic detection of railway tracks. Wang et al.6) studied the detection of raw silk defects. Firstly, an image acquisition system based on machine vision was developed, then defect segmentation and feature extraction were performed through threshold processing and morphological operations such as opening, hole fill up, and image subtraction, and finally defect types were identified by using back propagation (BP) neural network. The experiment proved that the method could accurately detect and identify raw silk defects. In this paper, a defect identification detection method for outer package printed matter based on machine vision was designed. The results of this study confirmed the possibility of applying machine vision technology for defect detection in the printing industry.


2. Materials and Methods
2.1 Detection of defects on the surface of outer package print

The traditional defect detection method mainly relies on manual detection, i.e., directly comparing printed matter and standard manuscript through human eyes. Manual detection has many disadvantages:

(1) The rate of missed detection is high. The long-term repetitive work of the human eye is prone to fatigue and missed inspection.

(2) The quality standards are different. Manual detection relies mainly on the subjective judgment of individuals.7) Different people has different standards for defects, so the standards for qualified products are also different.

(3) Efficiency is low. The manual detection speed is slow, the resource consumption is large, and the detection efficiency is low. Due to the fast printing speed, manual inspection requires off-line inspection instead of performing defect detection directly on the machine, reducing production efficiency.

With the development of technology, the emergence of defect detection technology based on machine vision makes up for the various deficiencies of manual detection. It replaces human eyes with machines, collects the target image through computer, and then identifies the image defects by digitizing the image with high speed, precision and degree of automation. The technology has strong objectivity and lower cost compared to manual detection, which has been widely used in medical testing,8,9) and parts testing,10) etc.

2.2 Defect detection method based on machine vision

Machine vision defect detection first needs to obtain an image of the object to be inspected through some devices. The image acquisition device used in this paper was shown in Fig. 1.


Fig. 1. 
Image acquisition device.

CCD camera has been widely used in image acquisition of industrial inspection. In this paper, area array CCD camera (TXG12c, Baumer Company, Germany) was used, and its parameters were shown in Table 1. It had high sensitivity and could suppress overexposure. Its lens (M3514-MP2, Computar Company, Japan) were used for focusing the detected object on the sensor device of the camera, and its parameters were shown in Table 2. The lighting tool was LED (LDR-75SW2-LA1, CCS Company, Japan). The illumination mode was coaxial light sources illumination.

Table 1. 
The parameters of TXG12c area array CCD camera
Sensor type Sony ICX445
Resolution 1296×960 px
Color process filter RGB Bayer Mosaic
Pixel size 3.75×3.75 μm
Transmission rate 1,000 Mbits/s

Table 2. 
The parameters of Computar M3514-MP2 optical lens
Maximum imaging size 8.8×6.6 mm
Aperture range F1.6-F16C
Minimum object distance 0.3 m
Control (aperture and focus) Manual

2.3 Image processing and defect detection
2.3.1 Image graying

In the case of external packaging printing, most of them are multicolored, and it is difficult to analyze them in this state. In order to make it easier to detect defects on the printing area, it is necessary to first convert the multicolor image to gray. In this paper, the average value method was used to perform image graying. Firstly, the average values of three components R, G, and B of the image was calculated and then the obtained average was used to replace the original pixel, that is:

R=G=B=R+G+B3[1] 
2.3.2 Median filtering denoising

The median filtering method is used to remove noise from the image of outer package printed matter. The principle of the method is: first, select a template consisting of several pixels from the image to be processed, measure the gray value of each pixel in order to get the media value, and then replace the original pixel gray value. It is assumed there is a noisy image, f(i,j), its pixel gray value is xij,i,jI2, and the selected template is S. The process of median filter denoising is:

gi,j=Medxij=Medxi+r,j+s,r,sS,i,jI2[2] 

where g(i,j) represents the image after denoising.

2.3.3 Image binarization

Image binarization can make an image of black and white, reduce the amount of data in the image, and highlight the target outline. In this paper, the maximum inter-class variance (OTSU) method11) was used to achieve binarization. OTSU method which was put forward by Nobuyuki Otsu in 1979 can realize image segmentation based on the idea that the variance between the target area and background area was maximum. It is assumed that there is an image I with a size of M×N, and the gray value of the pixel point (i,j) is expressed as f(i,j). The binarization process can be expressed as:

hi,j=0,fi,jt255,fi,j<t[3] 

where h(i,j) represents the gray value of point (i,j) after binarization, 0 for black and 255 for white, and t represents the segmentation threshold, which can divide the image into background and target. The optimal threshold is obtained by OTSU.

The gray level of image I is expressed as L, C={0,1,...,L-1}, the occurrence probability of gray level l is expressed as ρl,ρl=1M×Ncountfi,j=l,fi,jI,lC, the target is C0=(0,1,...,t), and the background is C1=(t+1,t+2,...,L-1). The proportion of C0 in the image is expressed as w0=ρC0=l=0tρt=wt, the average grayscale is u0=l=0tlρtw0=utwt, where , ut=l=0tlρl and the variance is σ02=l=0tl-u02ρlw0. The proportion of C1 in the image is w1=ρC1=l=t+1L-1ρt=1-wt, the average grayscale is u1=l=t+1L-1lρlw1=uT-ut1-wt, where uT=uL-1=l=0L-1lρl, and the variance is σ12=l=t+1L-1l-u12ρlw1.

The variance between classes can be expressed as σB2=w0w1u1-u02, and the overall variance can be expressed as σT2=σB2+w0σ02+w1σ12. The computational formula of the optimal threshold is l=ζargmaxλt, t0,1,,L-1, where ϛ indicates the correction factor and λt=σB2σT2 indicates the judgment condition.

2.4 Defect detection

In this paper, the method of difference matching was used to detect the defect of the processed image, that was, the two images were subtracted to obtain the shadow image to find the difference between the images. It is assumed that the standard image is s(x,y), the image of the printed product to be inspected is w(x,y), and difference image is d(x,y). The calculation process of the difference matching is: d(x,y)=|w(x,y)-s(x,y)|.

The hard threshold and the soft threshold are introduced to solve the artifact problem in the shadow matching. It is assumed that the image after binarization is f(x,y), the hard threshold is fthred(x,y), and the soft threshold is fsoft(x,y). The new threshold for the image can be expressed as:

f'x,y=gthredx,y,gthredx,ygsoftx,ygsoftx,y,gthredx,y<gsoftx,y[4] 

Effective removal of image artifacts can be achieved by using new f' (x,y).


3. Results and Discussion

In order to verify the effectiveness, the method proposed in this paper needed to be verified by experiments. First, the defect detection was performed by taking an outer package printed matter as an example. The images were shown in Fig. 2, where the left side of the figure was the standard image, and the right side was the defect image.


Fig. 2. 
Standard and defect images.

It was found that the outer package printed matter was a color image. In order to achieve accurate detection of the defect, the average value of R, G, B values in the image was calculated by the average method firstly, and then the image was processed by graying, as shown in Fig. 3.


Fig. 3. 
The grayscale image.

Since the image collected by the CCD camera contained a certain amount of noise, which might affect the result of the defect detection, the image needs to be denoised, and the image processed by the median filtering method was shown in Fig. 4.


Fig. 4. 
The denoising image.

After obtaining the denoised image, in order to further reduce the amount of data in the image and highlight the defect information, the optimal threshold of binarization was obtained by using the OTSU method proposed in this paper, and then the binarization operation was performed, as shown in Fig. 5.


Fig. 5. 
The binarized image.

Finally, the improved difference matching method was used, and a clear defect image was obtained.

It could be found from Fig. 6 that the method proposed in this paper could accurately identify and detect the defects in the printed matter of the outer package. The image defect could be clearly found through the difference matching, thereby realizing the detection of the defect. To further analyze the effectiveness of the method, 2,000 sheets of printed matter were tested. The outer package printed matter was coated white board paper with a size of 787 mm×1,092 mm, printing gloss no lower than 88% and 15-28% ink absorption; its technical requirement satisfied GB-T 10335.4-2004. There were 500 standard images and 1,500 defective images. The test results were shown in Table 3.


Fig. 6. 
The defect image.

Table 3. 
The test results of outer package printed matter
Number of
samples
Number of
false
positives
Detection
rate
Average
detection
time
2,000 12 99.4% 103 ms

From Table 3, it could be found that in the detection of 2,000 printed matter images using the method proposed in this study, the number of false positives was 12, the overall detection rate was 99.4%, and the average detection time was also short. It took only 103 ms to complete defect detection for the images, which could not be realized by traditional manual detection methods. From Fig. 7, i.e., the partial defect image detected, it could be found that the method can effectively identify defects, no matter how large or small, or thick or light, on the outer package printed matter, which proved the reliability of the method.


Fig. 7. 
Partial defects detected.

The difference between the traditional manual detection method and the method proposed in this paper was compared using samples of different sizes. The sample size was 10, 50, 100, 200, 500, 1,000, 2,000 and 5,000 respectively, and the proportion of standard images in each group of samples was 10%. The comparison of the detection rates of the two methods was shown in Fig. 8.


Fig. 8. 
Comparison of the detection rate.

It could be found from Fig. 8 that when the number of samples was small, the detection rates of manual detection and machine vision detection both reached 100%. As the number of samples increased, the detection rate of manual detection began to decrease. The larger the number of samples is, the lower the detection rate is. However, the detection rate is high in machine vision inspection. When the sample size is 5,000, the detection rate of manual detection is 81%, while the detection rate of machine vision detection is 99.1%, which proved the reliability of the detection method based on machine vision in this paper.

Machine vision technology which combines electronics, automatic control, artificial intelligence, image processing and other technologies12) can solve the shortcomings of manual detection. It has a wide range of applications in industrial defect detection13) for its high detection quality and high speed. In the production process of outer package printed matter, the appearance of defects is inevitable, therefore the detection of defects of printed matter is crucial to prevent it become defective products.

Clear defect images could be obtained through this method (Fig. 6). The average detection time of the method was very short, 103 ms, and the detection rate was high, reaching 99.4%. Compared with the traditional manual detection method, it was found that the advantage of the machine vision based detection method proposed in this paper was more obvious in the defects detection of large sample size. In the detection process of large sample, the detection rate of manual detection gradually declined due to fatigue and other reasons. In the detection of 5,000 samples, the detection rate of the manual detection dropped to 81%, but the detection rate of machine vision method changed little, and the accuracy of 99.1% was maintained, which proved the reliability of the method proposed in this paper.


4. Conclusions

This study analyzed the application of machine vision in the recognition and detection of defects of outer package printed matter and found that the designed method had short detection time and high detection accuracy in the defect detection of outer package printed matter. Compared with the traditional manual detection method, it had a higher detection rate and good performance in the detection of large samples. This study provided some theoretical support for the application of machine vision technology in the defect detection of outer package printed matter, which was conducive to the further development of detection technology of defects of printed matter.


Literature Cited
1. Xing, S. Y., Liu, Y. Q., and Zheng, Y. S., Research on the inspection and mark on the defects of printed matter based on template matching algorithm, Appl. Mech. Mater. 713- 715:377-380 (2015).
2. Yuan, X. C., Wu, L. S., and Peng, Q., An improved Otsu method using the weighted object variance for defect detection, Appl. Surface Sci. 349:472-484 (2015).
3. Meng, F., Ren, J., Wang, Q., and Zhang, T., Rubber hose surface defect detection system based on machine vision, IOP Conf. Ser.: Earth Environ. Sci. 108:022057 (2018).
4. Wang, Y., Wu, Z., Duan, X., Tong, J., Li, P., Chen, M., and Lin, Q., Design of gear defect detection system based on machine vision, IOP Conf. Ser.: Earth Environ. Sci. 108:022025 (2018).
5. Min, Y., Xiao, B., Dang, J., Yue, B., and Cheng, T., Real time detection system for rail surface defects based on machine vision, EURASIP J. Image and Video Process. 2018(1): 3 (2018).
6. Wang, C., Li, J., Chen, M., He, Z., and Zuo, B., The obtainment and recognition of raw silk defects based on machine vision and image analysis, J. Text. I. 107(3):316-326 (2016).
7. Yang, Y., Zhao, L., Wang, S., Cao, P., Liu, D., Li, L., Yan, L., Li, C., Xie, S., Li, Y., and Chen, Y., A high-resolution detecting system based on machine vision for defects on large aperture and super-smooth surface, Proc. SPIE-Int. Soc. Opt. Eng. 9446:94462N (2015).
8. Wu, C., Wang, Y., He, Z., Zhang, H., and Zhou, X., Research on foreign insoluble particulate detection method for medicinal solution based on machine vision, Chin. J. Sci. Instrum. 36(7):1451-1461 (2015).
9. Li, X., Qiao, T., Pang, Y., Zhang, H., and Yan, G., A new machine vision real-time detection system for liquid impurities based on dynamic morphological characteristic analysis and machine learning, Measurement 124:130-137 (2018).
10. Hu, F., He, Z., Zhao, X., and Zhang, S., A novel vision-based mold monitoring system in an environment of intense vibration, Meas. Sci. Technol. 28(10):105906 (2017).
11. Ren, Y., He, P., Wang, H. L., Cen, Z. J., Feng, P., and Wei, B., Compressed sensing and Otsu’s method based binary CT image reconstruction technique in non-destructive detection, Nucl. Sci. Tech. 26(5):63-68 (2015).
12. Vodrahalli, K. and Bhowmik, A. K., 3D computer vision based on machine learning with deep neural networks: A review, J. Soc. Inf. Display 25(11):676-694 (2017).
13. Zhang, J., Zhang, C., Guo, S., and Zhou, L., Research on tool wear detection based on machine vision in end milling process, Prod. Eng. 6(4-5):431-437 (2012).