Current Issue

Journal of Korea Technical Association of the Pulp and Paper Industry - Vol. 52 , No. 4

[ Article ]
Journal of Korea Technical Association of the Pulp and Paper Industry - Vol. 52, No. 4, pp.28-37
Abbreviation: J. Korea TAPPI
ISSN: 0253-3200 (Print)
Print publication date 30 Aug 2020
Received 26 Jun 2020 Revised 12 Aug 2020 Accepted 14 Aug 2020
DOI: https://doi.org/10.7584/JKTAPPI.2020.08.52.4.28

An Algorithm for On-Line Image Segmentation of Multiple Paper Defects Based on Fast Two-Dimensional Threshold Method
Yun-hui Qu1, 2, ; Wei Tang1 ; Bo Feng3
1Department of Electric and Control Engineering, Shaanxi University of Science & Technology, Xi’an, Shaanxi, 710021, Professor, People’s Republic of China
2Computer Teaching and Research Section, Xi’an Medical University, Xi’an, Shaanxi, 710021, Professor, People’s Republic of China
3Department of Electric and Control Engineering, Shaanxi University of Science & Technology, Xi’an, Shaanxi, 710021, Lecturer, People’s Republic of China

Correspondence to : † E-mail: nan-nan_1951@163.com (Address: Department of Electric and Control Engineering, Shaanxi University of Science & Technology, Xi’an, Shaanxi, 710021, People’s Republic of China)

Funding Information ▼

Abstract

The two-dimensional threshold segmentation algorithm is time-consuming and cannot detect multiple paper defects. Considering these problems, a fast two-dimensional threshold method for on-line segmentation of multiple paper defects was proposed. The image subtraction operation was used first to segment the image with multiple paper defects. Then dividing two-dimensional threshold into two one-dimensional thresholds was used to accelerate the segmentation. At the same time, according to the gray distribution characteristics of the paper defect image after subtraction, the search range of the threshold was narrowed, and the selection of the optimal threshold was further accelerated. Experimental results showed that the algorithm proposed in this paper can accurately and quickly detect multiple paper defects, and could be effectively used for real-time detection of the paper production process.


Keywords: Background subtraction, multiple paper defects, two-dimensional threshold

1. Introduction

With the development of pulp and paper industry, the appearance quality of paper is more and more concerned by modern enterprises.1) In the paper industry, the flaws that do not meet the technical requirements of paper quality are called paper defects. With the rapid development of papermaking production automation, the speed of modern paper machine can reach 1,500 m/min and the width can exceed 10 m.2,3) With the increase of the speed and the width of the web, the probability of paper defects in the process of making is also increasing, which brings great challenges to the detection technology of paper defects.

At present, the paper defect detection system used in the production lines generally use linear-array CCD cameras to collect the paper image sequence, and then use industrial computer to detect and classify the defects. The hardware diagram of the paper defect detection system is shown in Fig. 1.4,5)


Fig.1. 
Schematic diagram of paper defect detection system.

The paper defect detection system based on machine vision can detect the paper defects continuously and accurately, which plays an important role in improving the production efficiency and product quality of the paper machine and improving the competitiveness of enterprises.

At present, the commonly used methods for detecting paper defects include threshold-based, region-based, edge detection based and specific theory-based segmentation method. Among these methods, the threshold-based segmentation method is favored by many scholars because of its simple calculation, high efficiency and fast speed.6,7)

The traditional threshold segmentation is mostly based on one-dimensional histogram, which cannot use the spatial information of the image. When the target occupies a very small area and the contrast of paper object is low, the phenomenon of false segmentation is serious. Many researchers proposed the threshold segmentation methods based on two-dimensional histogram.8-10) The two-dimensional histogram threshold segmentation methods use the gray value of the pixel itself and the correlation of its neighborhood, which reduce the impact of noise on the segmentation, and have a good segmentation effect. However, after expanding from one-dimensional to two-dimensional, the time complexity is bound to increase, which is not suitable for real-time detection of paper defects in paper production line. Moreover, when there are multiple paper defects with different gray levels in the same paper image, the traditional threshold detection methods are also difficult to select the threshold.

In order to solve the problems that the two-dimensional threshold segmentation algorithm is time-consuming and the traditional threshold algorithm cannot detect multiple paper defects, an improved algorithm was proposed in this paper. Firstly, the paper image was subtracted by subtraction method. Secondly, the two-dimensional threshold was decomposed into two one-dimensional thresholds for optimal solution. And then, the two one-dimensional optimal thresholds were combined into two-dimensional optimal thresholds. Which not only improves the speed and accuracy of threshold selection, the accuracy of segmentation, but also effectively solves the problem of multi-paper defects segmentation. In the experiment, programs were compiled by Visual Studio 2010+Open CV for simulation, and the results showed that the algorithm can accurately and quickly detect multiple paper defects and can be effectively used in the paper production process.


2. Materials and Methods
2.1 Threshold segmentation of two-dimensional histogram

In view of the real-time consideration, the threshold method is most commonly used in paper defect detection. The threshold method based on one-dimensional histogram does not use the spatial information of the image. When the paper defects occupy the very small area of the image, the phenomenon of false segmentation is serious. Especially when the contrasts of paper defects are low, the results of segmentation have a large deviation. The threshold method based on two-dimensional histogram can use part of the spatial position information to get better segmentation results than one-dimensional threshold method, which lays a good foundation for the extraction of paper disease areas and paper disease classification in the later stage.11)

Let I(x,y) is the image with gray level of L, f(x,y)=i is the gray value of (x,y) point, g(x,y)=j is the average gray value of (x,y) point neighborhood, and a binary (i,j) is formed. The threshold vector (t,s) can divide the two-dimensional histogram of the image into four regions, as shown in Fig. 2.:


Fig. 2. 
Two dimensional histogram domain.

Among them, areas I and III are target and background areas respectively, areas II and IV are noise and edge points.

Otsu algorithm is also known as the maximum inter class difference method, which basic idea is to select the best threshold to segment the image into multiple parts, so that the maximum inter class variance of each part of the image after segmentation. Two-dimensional Otsu segmentation algorithm is described as follows12,13):

1) The total number of pixels of the image is as follows:

N=i=0L-1ni[1] 

Where, the probability of occurrence of pixels with gray value i is as follows:

Pi=niN[2] 

2) Let f(x,y)=ig(x,y)=j,a binary (i,j) is formed.

3) Let the occurrence number of binary (i,j) is fij; The probability density of the binary is Pij, Pij=fij/N, i=1,2,…,L;Where, N is the total pixel number of the image.

4) The probability corresponding to the target and background is Po and Pb. When the threshold is (s,t), Po and Pb can be calculated by Eqs. 3-4.

Po=i=0tj=0spij[3] 
Pb=i=t+1L-1j=s+1L-1pij[4] 

Let Co and Cb are mean vectors of target and background areas,which can be calculated by Eqs. 3, 5 and 6.

Co=Coi,CojT=i=0tj=0sipijPo,i=0tj=0sjpijPoT[5] 
Cb=Cbi,CbjT=i=t+1L-1j=s+1L-1ipijPb,i=t+1L-1j=t+1L-1jpijPbT[6] 

The total gray value CT of the image is shown in Eq. 7:

CT=CTi,CTjT=i=0L-1j=0L-1ipij,i=0L-1j=0L-1jpijT[7] 

The dispersion matrix is defined as Eq. 8:

St,s=PoCo-CTCo-CTT+PbCb-CTCb-CTT[8] 

The trace of the dispersion matrix is defined as Eq. 9:

trSt,s=Po×Coi-CTi2+Coj-CTj2+Pb×Cbi-CTi2+Cbj-CTj2[9] 

The maximum (t,s) is the best segmentation threshold.

2.2 Fast two-dimensional threshold method for paper defect segmentation

In the traditional two-dimensional threshold method, the selection method of the best combination threshold is the exhaustive method, which needs to search in the whole gray level range to find the best threshold that meets the maximum of Eq. 9. Although this method can ensure the accuracy of segmentation, the time complexity is O(L4), which is time-consuming and not conducive to the use of industrial online detection.14) In addition, when there are multiple paper defects with different gray levels in the same paper image, it is difficult to segment multiple paper defects directly using threshold method. In view of the above problems, a fast two-dimensional threshold segmentation method of multiple paper defects is proposed. The flow is shown in Fig. 3.


Fig. 3. 
Algorithm flow chart.

2.2.1 Image processing of multiple paper defects based on subtraction

Before threshold segmentation, subtraction is performed first. That is using the difference between the paper defects image and the image template without defect to obtain the subtracted paper image.15) The contrast image before and after the subtraction is shown in Fig. 4. For some images with gray value less than normal gray value, such as dirty points (as shown in Fig. 4(d)), the gray value of the paper defect part will be negative after the subtraction. In order to solve the problem that the gray value is negative after subtraction, the absolute value of the difference is taken as the output. In this way, the gray value of all the paper defect areas after subtraction operation will be greater than the background area (as shown in Fig. 4(g)). In the process of threshold segmentation, only a suitable threshold value needs to be selected to segment multiple paper defect areas.


Fig. 4. 
Impression drawing of background subtraction.

2.2.2 Fast 2-D threshold paper defects image segmentation

The traditional two-dimensional threshold segmentation uses the exhaustive method to select the threshold, which needs double cycle. When the gray level of the image is L, the time complexity is O(L4). The calculation and time complexity are too high, which is not conducive to the online paper defects detection. Therefore, this paper proposes a decomposition idea to reduce the time complexity, and according to the gray distribution range of the paper defect area after subtraction, reduce the gray search range, improve the speed and accuracy of threshold selection, and meet the requirements of online detection of paper defect image. The specific methods are as follows:

1) Decomposition of the two dimensional threshold

The two-dimensional threshold method is decomposed into two one-dimensional optimal thresholds for solution,and then combined into two-dimensional thresholds. According to the definition of two-dimensional threshold, the threshold t is obtained from the original image f(x,y), and the threshold s is obtained from the neighborhood mean image g(x,y). The two one-dimensional thresholds t and s are solved respectively by the following methods:

Set threshold t to divide the image into two categories: objective and background. The mean value is uo and ub respectively, and the probability of occurrence is po and pb respectively. Then the variance between the two categories is shown in Eq. 10.

sp=pouo-u2+pbub-u2=popbuo-u2[10] 

Where, u is the mean value of the paper defects image after subtraction.

Set pi is the probability of occurrence of gray-scale i, the intra class variance of objective and background is shown in Eq. 11 and Eq. 12 respectively. And the total intra class variance is shown in Eq. 13.

so=i=0tpii-uo2[11] 
sb=i=t+1L-1pii-ub2[12] 
sin=pos0+pbsb[13] 

The best segmentation should make the variance between the object and background class sp maximum, and the variance within intra class sin minimum. Let S=sp/sin. Then the segmentation threshold t at S maximum should be the optimal threshold of f(x,y).

Similarly, the same operation is performed on g(x,y) to obtain the best segmentation threshold s.

Two one-dimensional thresholds, t and s, are calculated by the above method instead of the two-dimensional Otsu algorithm. This method can not only reduce the time complexity of the two-dimensional threshold algorithm, but also reduce the space complexity, which is conducive to the real-time detection of the production line.

2) Limitation of threshold range

After subtraction, the gray value of the paper defect area must be greater than the mean value of the background, and in the paper defect image, the paper defect area must be far less than the background area. Therefore, when selecting the threshold value, the lower limit of the threshold value is set as the mean value of the subtraction image. In addition, because the gray value of the paper defect area must not be greater than the maximum gray value of the image, the upper limit of the threshold value is set as the maximum gray value of the paper defect image. This will greatly reduce the range of threshold selection, reduce the number of cycles, and speed up the two-dimensional threshold selection.


3. Results and Discussion

In productive processing of paper, the common paper defects include dirty spots, holes, and cracks. Among them, the dirty spots and holes belong to relatively high contrast paper defects, cracks belong to low contrast paper defects. The gray values of dirty spots are lower than that of paper background, and the holes and cracks are higher than background. The three kinds of paper defects and several different kinds of paper defects in the same image were used as the test objects in the paper. Programs were compiled by Visual Studio 2010+Open CV to compare the proposed method with 1-D OSTU and 2-D OSTU algorithms. The environment used in the experiment was: Windows 10 Home Edition, Intel® core™ I7-7500u CPU, 8G DDR4 2400.

The segmentation algorithm and optimal segmentation threshold of the algorithm proposed in this paper were shown in Table 1:

Table 1. 
The segmentation effect of this algorithm
Paper defects Image after subtraction Image after segmentation Segmentation threshold
(49,46)
(64,62)
(54,55)
(37,44)

3.1 Comparison of segmentation effect

In this paper, three algorithms were respectively used to segment the paper defect image with holes and the multi paper defects image, and the segmentation effect was shown in Table 2. (Because there are crack and dirty spot in the image of multiple paper defects, the segmentation results of two kinds of paper defects were not shown in the experimental results).

Table 2. 
Comparison of segmentation results of three algorithms
Image after subtraction 1-D Otsu 2-D Otsu Proposed

It can be seen from the contrast effect in Table 2 that when the paper defect area was large and the contrast was obvious, the segmentation effect of the three algorithms was ideal. And for the segmentation of paper defects with low contrast and small area, the segmentation effect of one-dimensional Otsu algorithm was poor in the lower part of contrast, while the segmentation effect of proposed in this paper was close to that of two-dimensional Otsu algorithm, which was obviously better than that of one-dimensional Otsu algorithm.

In order to further verify the performance of the algorithm proposed in this paper, peak signal-to-noise ratio (PSNR) was used to quantitatively compare the segmented images of the three algorithms. The definition of PSNR is as follows:

PSNR=101g2552/MSE[14] 

Where, MSE is the mean square error. The smaller MSE value is, the smaller the difference between the segmented image and the contrast image is, and the more accurate the segmentation is. The MSE calculation formula is as follows:

MSE=1MNi=1Mj=1NIi,j-I'i,j2[15] 

The PSNR comparison values of the three algorithms are shown in Table 3:

Table 3. 
PSNR comparison of three algorithms
Image OSTU 2-D OSTU Proposed
Dirty spot 21.1141 23.8925 23.7647
Multi defects 18.1538 21.7923 21.3267

From the results in Table 3, it can be seen that the PSNR of the algorithm proposed in this paper was higher than that of one-dimensional Otsu algorithm and close to that of two-dimensional Otsu algorithm, which was consistent with the comparison and evaluation results of visual effects.

3.2 Comparison of time complexity

The time complexity of three threshold segmentation algorithms was shown in Table 4:

Table 4. 
Comparison of the operation time of three algorithms (unit: s.)
Image OTSU 2-D OTSU Proposed
Dirty spot 0.04676 0.4128 0.04821
Hole 0.04673 0.3875 0.04877
Crack 0.04765 0.5526 0.04867
Crack+Dirty spot 0.04657 0.6563 0.04735

From the results in Table 4, it can be seen that the time complexity of the algorithm proposed in this paper was far lower than that of the two-dimensional Otsu threshold segmentation algorithm, which was close to the one-dimensional Otsu segmentation algorithm, so as to achieve the purpose of acceleration. When there were multiple paper defects in the image, the segmentation time was basically the same as that of a single paper defect, and the time complexity did not increase significantly. The algorithm proposed in this paper can basically meet the needs of the web defects detection.


4. Conclusions

In this paper, a fast two-dimensional threshold method for on-line segmentation of multiple paper defects was proposed. Firstly, the image subtraction operation was used to segment the multi paper defect image. After subtraction, the two-dimensional threshold was divided into two one-dimensional thresholds to accelerate. At the same time, according to the gray distribution characteristics of the paper defect image after subtraction, the search range of the threshold was narrowed, and the selection of the optimal threshold was further accelerated. This method not only improved the speed and accuracy of threshold selection, the accuracy of segmentation, but also effectively solved the problem of multi paper image segmentation. The experimental results show that the algorithm can accurately and quickly detect multiple paper defects of the production line, and can be effectively used for real-time detection of the paper production line.


Acknowledgments

This work was partially supported by Scientific Research Project of Shaanxi Provincial Education Department (17JK0645). We sincerely thank for the funding of the project.


Literature Cited
1. Wei, A. J., Li, Q., and Tang, W., An improved paper defects denoising method based on gray associated with neighborhood characteristics, China Pulp & Paper 28(1):44-47 (2013).
2. Qu, Y. H., Tang, W., and Wen, H., Paper defects de-noising algorithm based on homomorphic filtering and discrete cosine transform, China Pulp & Paper 37(5):45-49 (2018).
3. Shan, W. J. and Tang, W., Multivariable dimension-reduction and synergic control strategy for cross-directional basis weight of papermaking process, Journal of Korea TAPPI 51(2):76-87 (2019).
4. Wu, G. A. and Wu, D., Application of industrial camera on paper making, Paper and Paper Making 35(07):8-16 (2016).
5. Wang, B., Tang, W., Dong, J. X., and Wang, F., Study on the drive parameters of a high precision basis weight control valve, Journal of Korea TAPPI 49(3):41-56 (2017).
6. Zhou, Q., Chen, Y., and Shen, T. Y., Review of paper defect detection system based on machine vision technology, China Pulp & Paper 35(5):72-79 (2016).
7. Feng, B., Tang, W., and Qu, Y. H., Dynamic threshold setting scheme of paper disease detection based on fuzzy logic, Packaging Engineering 41(03):218-223 (2020).
8. Wu, Y. Q., Zhu, L., and Wu, S. H., Fast iterative algorithm for image threshold segmentation based on two-dimensional Arimoto gray entropy, Journal of South China University of Technology (Natural Science Edition) 44(05):48-57 (2016).
9. Zhao, H., An, W. S., and Yang, T., Hyperbolic two dimensional Otsu threshold segmentation algorithm, Computer & Digital Engineering 47(08):2033-2038 (2019).
10. Hamed, A., Luiz, E. V. S., and Silva, A. C. M. O., Two-dimensional dispersion entropy: An information-theoretic method for irregularity analysis of images, Signal Processing: Image Communication 75:178-187 (2019).
11. Xu, C., Huang, F. H., and Mao, Y. Z., An improved two-dimensional Otsu thresholding segmentation method, Application of Electronic Technique 42(12):108-111 (2016).
12. Gao, H. J., Wang, L., and Gong, W. Y., Two-dimensional Otsu fast image segmentation based on modified CS algorithm, Communications Technology 50(12):2698-2703 (2017).
13. Qu, Y. H., Tang, W., and Feng, B., Web inspection algorithm for low contrast paper defects based on artificial bee colony optimization, Journal of Korea TAPPI 52(2): 43-51 (2020).
14. Liu, J. and Jin, W. D., Fast method for 2D threshold segmentation algorithm based on inter-class and intra-class variances, Journal of Southwest Jiaotong University 49(05):913-919 (2014).
15. Qu, Y. H., Tang, W., and Feng, B., On-line detection and classification method based on background subtraction and SVM, Packaging Engineering 39(23):176-180 (2018).