Real-time surface defects inspection of steel strip based on difference image

CONG Jia-huia, YAN Yun-huia, ZHANG Hai-anb, LI Juna a School of Mechanical Engineering & Automation, Northeastern University, Shenyang, China b The No.3 Steelmaking Plant of Angang Iron and Steel Co., Anshan, China 114021. 110004;

ABSTRACT
A method of difference image to inspect real-time defects for cold rolled steel strip is proposed, which is based on subtract arithmetic of image. That is, shooting a scene at different time subtraction or image of the same scene at different waveband subtraction. This paper outlines a subtraction operation between the gathering images and the standard images. The standard image selection utilizes sequence extraction technique, which is to extract background image as a standard image from the multi-frame continuous real-time processing of images, and the standard image is self adaptive update. In the course of image defect inspection, we divided the difference image into small regions and inspected them respectively with the character of defects being remarkable in part image. Through the experiment analysis, conditions can be obtained to judge if any defects exist. Experiment on five typical defects (wrinkles, inclusion, weld, holes and serrated edges) were done. The results show that this method can meet the requirements of defect inspection and a higher rate of correct identification can be achieved.
Keywords: steel strip, defect inspection, difference image, self-adaptive update

1.

INTRODUCTION

As one of the main product forms in steel industry, the cold rolled strips have become the essential material of trades such as automobile, mechanical manufacturing, chemical, shipbuilding, aeronautics and astronautics. Its surface quality has direct influence on the quality of product. How to inspect these defects real-timely and acquire the information of defects is essential to control and improve the quality of steel strip. Therefore, it has become the main focus of steel works recently[1]. In early 1990s, many Chinese steel enterprises relied on local universities and research institutes to start the research of steel strip surface defect detection and classification[23]. Even though they have got some achievements, the on-line monitoring system still fails to meet the real-time requirements of most real production due to the limitations of hardware and software. The paper studies real-time surface defect inspection for cold rolled steel strip based on CCD[4]. A method of difference for surface defects inspection is presented. A difference image is obtained by subtracting pixel by pixel both the

International Symposium on Photoelectronic Detection and Imaging 2007: Related Technologies and Applications, edited by Liwei Zhou, Proc. of SPIE Vol. 6625, 66250W, (2008) 0277-786X/08/$18 doi: 10.1117/12.790865

Proc. of SPIE Vol. 6625 66250W-1

acquisition images and the criterion images. And then the results of difference image are analyzed and the judgment conditions of defects image are made out.

2. METHODOLOGY
2.1. Defect inspection principle
This system inspects defects based on difference image. The algorithm[5] can be described as image subtraction of the same scene shooting at different time or at different wavebands. The difference image provides different information between images. That can be used to direct dynamic monitoring, moving target inspecting and tracking image background eliminating and objects recognizing etc. The mathematical expression of difference image[6] is:

0, A( x, y ) B( x, y ) < 0 f ( x, y ) = A( x, y ) B( x, y ), A( x, y ) B( x, y ) 0

(1)

Where A(x, y) and B(x, y) are input images, and the B(x, y) is also the background image. In this paper it is just the criterion image, A(x, y) is the forthcoming recognized image and the f(x, y) is the output image. In order to adapt to the defect inspection of the system, the previous mathematical expression of difference image can be improved, as shown in Eq (2).

f ( x, y ) = A( x, y ) B( x, y )

(2)

In ideal condition, if the current frame image has no defects, the grey of every pixel in difference image would be zero. But if has defects, the grey of corresponding defects pixel which is in the difference image would be not zero. 2.2. Criterion image selection The criterion image as the background image participates in operation. Its quality will influence the performance of the whole system which inspects defects directly. The steel strip image in gathering process will be greatly affected by light and other environment factors. Besides, the difference of every rolled steel strips surface quality is great. Even within the same rolled steel strip, surface quality of two pieces far away from each other is impossible to be identical. Thus, if we, regardless of actual circumstance, save a standard picture in the calculator memory in advance in the course of production, and take that picture as the standard image to inspect and discriminate gathered images, then the misjudgment rate will increase greatly. Therefore, this paper made use of a kind of sequence to withdraw the selection carrying on a standard picture. That is to extract the background image as criterion image from several consecutions gathering on real-time[7]. The test results indicate this method resolved the selection problem of standard picture, made the misjudgment rate reduce greatly. Because the continuous images of steel strips have recorded the majority of moving and changing information in a period of time. Therefore, an ideal method to take steel background is to analyze on data in the longer time range and make full use of related information. According to this thought, we can carry analysis of variety regulation of each pixel along the time axis and select suitable point in the whole sequence to recover the background according to statistical regularity.

Proc. of SPIE Vol. 6625 66250W-2

Define image sequence as I ( x, y, i ) , where x, y are the space coordinates, i is frame number ( i 1L N ), N is the total frame number of the sequence. The bright component of the sequence is I L ( x, y, i ) , and Change Detection Mask (CDM)

responds grey change between the neighboring frames.

d , if d T CDM ( x, y , i ) = , d = I L ( x, y, i + 1) I L ( x, y, i ) 0 , if d < T

3 Among them, threshold value T is used to remove the noise. To fixed coordinates position (x, y), CDM ( x, y, i ) can be expressed as the function i, and it records pixels located in the position (x, y) change curve along the time axis. We may act this curve partition according to whether x is larger than zero. The static part inspected was expressed as set of {S j ( x, y ) , I j M } . As shown in figure 1, the beginning point and the end point of S j are
ST j and EN j respectively. We choose the longest static partition of the next step in each position ( x, y )

corresponding {S j } and record this partition center point the corresponding frame number as M ( x, y ) .
CDM, (i)

Is,

57

\ ,r,, -- (\ tIsH.
EN, IsAl I

Frame NumFer

Fig.1 The function of grey frame difference change along time axis

Finally the point of frame M ( x, y ) is used to fill the corresponding position of steel strips background. This logic can be described as the under formula:
M ( x, y ) = ( ST ( x, y ) + EN ( x, y )) / 2

4 5

B( x, y ) = I ( x, y, M ( x, y ))

Among them, ST ( x, y ) and EN ( x, y ) corresponds to the beginning point and end point of the longest static partition and B( x, y ) is the standard image of reconstruction steel strip. This topic is based on the above method to extract the standard image from the surface image of steel strips. Flowchart is shown in figure 2.

Proc. of SPIE Vol. 6625 66250W-3

First frame image Each pixel tracks the maximum length which is zero Next frame Set the start and end frame label of section where value is zero consecutive Traverse the entire image and set background data Y Obtain the standard image Record the intermediate mark of the maximum length

Reach the sequence end

Fig.2 The flow chart of distill standard image by sequence method

2.3. The standard picture set

There is no standard picture when a volume of steel strip starts rolling. So the picture should be gathered continuously before the standard picture acquisition. We use the sequence law above to extract standard image, which is the second basis of detecting defects of volume strip. Meanwhile, these images are temporarily stored in the memory and CPU spare time is spent to detect them. If the continuous acquisition quantity of images is too low, the standard image extraction may be affected because of the individual image quality problem; if the continuous acquisition quantity of images is too high, it will cost a lot of time and occupy a larger memory to extract the standard image, thus affecting the acquisition and processing speed. A good many experiments prove that the most appropriate method is to gather 10 pieces of images for the system, which not only avoids problems brought about by too much and too little pieces, but also guarantees the quality and deficiency of the standard picture and the detection of strip images.

2.4.

Standard picture update

After obtaining the standard picture, we can compare the steel strips in the rolling process with it to complete defect detection. However, the scene will change with the environment conditions, such as the illuminating ray change and rolling speed change. It will enable the grey of the steel strip image change considerably. If the standard image doesnt update for a long time, there would make no defects images be wronged by images of a flawed and increase the rate of misjudgment, affecting production. To solve this problem, we use real-time updating of the standard image, after a time alternation to acquire new standard image to replace the former one. Standard pictures update similarly existent the question of update interval. If excessive update occupies more system resources, it will reduce the speed of detection and recognition; if the update is too slow, it will affect the correct rate of the defect detection. In this paper, a lot of experiments confirmed that, using the following methods update is better. After continuous detection of 30 images, we extract an image using sequence law and carry on the difference image operation with the preceding standard image. If the test result shows this new picture is non- flawed, the standard image is remained, if the difference exists, original standard picture will be replaced by current picture and be used as the new standard picture to continue defect detection.

Proc. of SPIE Vol. 6625 66250W-4

3.

DATA PROCESSING

Strip image recognition environment is a complex industrial scene, the scene is worse in the condition of high-temperature air, dust, vibration, ambient light changes, oil and so on. The steel strips that need to be detected are located on the continuously moving production line, and besides there are variety defects in the strip surface, such as feet, serrated edge, or inclusions. The differences among those defects are relatively large, the characteristic of some defects is obvious on the one hand, but on the other hang it is vague, which makes defect detection more difficult[89]. For example, the strip surface often has small linen, scratched defects etc. Although the corresponding pixels value of such defects differ from other pixels grey value, the connected grey histogram, because of its limited proportion in the whole image, doesnt have significant features. However, if the whole strip images are segmented into several small regions averagely[10], these defects can be found more clearly in his small areas. Relatively its proportion is large, corresponding histogram peak was also quite noticeable. After a large amount of analysis of the defects image of steel surface, it can be found that the use of small regional defect detection bring very good results.

(a)

(b)

(c)

(g)

(h) Fig.3 detection for steel strip defects (1:1.5)

(i)

Proc. of SPIE Vol. 6625 66250W-5

As shown in figure 3, chart (a), (b), (c) are three standard pictures with withdraws, Figure (d) is the treat of the detecting standard image, (e) (f) are defect images to be tested of a mixture containing wrinkle, (g), (h), (i) are obtained picture respectively (d), (e), (f) are image after three standard pictures (a), (b), (c) are carried on the difference image operation, more clearly, we carried on anti-color processing to the difference image. In order to make the flawed steel strip picture to be picked out correctly, this paper has separately carried on the large amount of statistical data analysis. The regional division is done in accordance with the size of 20 20 , besides according to grey values, the scope of which is respectively large than 30,40,50,55, 60, the data of every region are brought in order from large to small, and the curves are mapped.
>50 350>3 >40 >60 >00 SObS >40 >60

lObs

>50 450'5 >40

>66

>60 >55
>40

lobs

>40 >60

>50

1.1

1.00.90.80.70.60.50.4-

0.1 -

0.0>1 40 50

6,5

a)No defects
8

(b) Serrated edge

II
0.80.70.80.50.40.30.2-

II

II

11111
0.9 0.8 0.7 0.8 0.5

0.4.
0.3

0.2.
0.1

0.0-

0.0

20 50 75 100 125 50 170200220250275300325350375400425

o 20 50 75 100 125 ISO 170200220250275300325350375400425

20,1201>'. 18

20x20 0i

(c) Wrinkle

dWeld

Proc. of SPIE Vol. 6625 66250W-6

20x201X

0,

On

(0

6,)

0 20 50 75 100125105 178200225250275005025050075

>55

501

>500

>50

>ll

1.11

1111111111 11111

o 25 50 75 100 125 100 170200225250275300325350375400425

2Ox2OV

eInclusion

(f) Linetype and color table

Fig. 4 Scale figures of steel strip differenced image during different grey range in small region

As shown in figure 4 (the curve of the figure is the schematic diagram drawn in accordance from the samples of a variety of defect images and non-defect images), the abscissa represents 20 20 divisional region and the vertical coordinate represents within various grey scope, the grey value quantity is proportional to the total grey value. The different color curve represents the different gradation value scope. Comparing the five charts of Figure 4, we can see that judging a picture whether exists defects can judge the following condition whether are established: (1) whether has the region which proportion is equal to 1; (2) whether the grey is more than 30 and the proportion value of region is bigger than 0.35; (3) whether the grey is more than 40, the proportion value is bigger than 0.8 and the region number is larger than 0; (4) whether the grey is more than 60, the proportion is bigger than 0 and the region number is larger than 300. If in the image detection process, a frame of the image satisfied the conditions for an arbitrary, we can determine this frame of the image is flawed, then can store it and carry on the following processing. On the contrary, if the four conditions are not met, we can determine the image is non-flawed, and there would be no need to store the handling operations.

4.
The experimental results, as shown in table 1.

RESULTS

By using the above judgment for the experimental test, each type of defect image of the sample is 30, in all 150 samples.

Table 1 Experimental data of defect detection

Types of defects

Sample size

Correct Identification of the samples 29 30

Misjudge the number of samples 1 0

Recognition rate

Total recognition rate

No defects Serrated edge

30 30

96.7% 100% 99.3%

Proc. of SPIE Vol. 6625 66250W-7

Wrinkle Inclusion Weld

30 30 30

30 30 30

0 0 0

100% 100% 100%

5. Conclusion
(1) The experiment proves that the method using the difference image to detect defects is feasible. Different images and the steel strip surface defect judgment conditions are intrinsically linked. (2) This paper presents the sequence method using the standard picture extraction, and the standard image self-adaptive update will reduce the misjudgment rate greatly. (3) Defect detection using small regional law through the test data confirmation is very practical.

ACKNOWLEDGEMENT
The study is supported by Important Foundation Study Prophase Study Expert Item Fund of Ministry of Science and Technology of China (No. 2003CCA03900) and Joint Fund of Iron and Steel Research Founded by National Natural Science Fund of China and Shanghai Baosteel Group CorporationNO.50574019.

REFERENCES
1. 2. 3. 4. Franz Pernkopfa, Paul OLearyb. Image Acquisition Techniques for Automatic Visual Inspection of Metallic Surfaces [J], NDT&E International, 2003, (36):609617. Alper Baykut, Alper Atalay, Aytl Eril and Mustafa Gler. Real-time Defect Inspection of Textured Surfaces [J], Real-Time Imaging, 2000 (6):17-27. Xu K. On-line inspection of surface defects of cold rolled strips based on image processing [J], Iron and Steel, 2002, 37 (12):61-64. Ville Hyvarinen, Raimo Silvennoinen, Kai-Erik Peiponen Tapani Niskanen. Diffractive Optical Element Based Sensor for Surface Quality Inspection of Concave Punches [J], European Journal of Pharmaceutics and Biopharmaceutics, 2000, 49: 167-169. 5. 6. C.X.Zhou. A Fast Human Face Detection Method Based on Difference Image and Multi-templates Matching [J], Application Research of Computers, 2004(5):197-199. G.Placidi, M.Alecci, A.Sotgiu. Theory of Adaptive Acquisition Method for Image Recognition from Projections and Application to EPR Imaging [J]. Journal of Magnetic Resonance, 1995, 108:50-57.

Proc. of SPIE Vol. 6625 66250W-8

7. 8. 9.

Gupt S. Masound O. Martin RFK, etal. Detection and classification for vehicles [J].IEEE Transactions on Intelligent Transportation Systems, 2002, 3(1):37-47. H.Zheng, L.X.Kong, S.Nahavandi. Automatic Inspeciton of Metallic Surface Defects Using Genetic Algorithms [J], Journal of Materials Processing Technology, 2002 (125-126):427-433. Topi Menp, Markus Turtinen and Matti Pietikinen. Real-time Surface Inspection by Texture [J], Machine Vision Group, Real-time Imaging, 2003 (9):289296.

B.Y. Lee a, Y.S. Tarng. Surface Roughness Inspection by Computer Vision in Turning Operation [J], International Journal of Machine Tools & Manufacture, 2001(41): 12511263.

Proc. of SPIE Vol. 6625 66250W-9