Determination of seal coat deterioration using image processing methods

Construction and Building Materials 53 (2014) 273–283

Contents lists available at ScienceDirect

Construction and Building Materials

journal homepage: www.elsevier .com/locate /conbui ldmat

Determination of seal coat deterioration using image processingmethods

0950-0618/$ - see front matter � 2013 Elsevier Ltd. All rights reserved.http://dx.doi.org/10.1016/j.conbuildmat.2013.11.090

⇑ Corresponding author. Tel.: +90 272 2281311.E-mail address: [email protected] (S. Çetin).

Mustafa Karas�ahin a, Mehmet Saltan b, Sedat Çetin c,⇑a Engineering Faculty, Department of Civil Engineering, _Istanbul University, _Istanbul, Turkeyb Engineering Faculty, Department of Civil Engineering, Suleyman Demirel University, Isparta, Turkeyc Engineering Faculty, Department of Civil Engineering, Afyon Kocatepe University, Afyon, Turkey

h i g h l i g h t s

�We developed new system using digital image processing techniques.� Bleeding deterioration on seal coat was determined accurately.� This new system will encourage development in seal coat.� This research will aid pavement engineers for pavement ratings.

a r t i c l e i n f o

Article history:Received 10 July 2013Received in revised form 12 November 2013Accepted 26 November 2013

Keywords:Seal coatDigital image processingBleedingDeterioration

a b s t r a c t

Seal coat is the most commonly used asphalt pavement type due to its low initial construction cost andease of application in countries such as Turkey, Australia, South Africa and New Zealand. Seal coat dete-rioration occurs over time because of the effect of various factors such as weather, traffic, etc. The deter-mination and assessment of deterioration is an important components of pavement managementsystems (PMS). This article presents, digital image processing (DIP) techniques as effective and reliablemeasurement techniques for the determination of bleeding deterioration in seal coats. The developedtechnique was applied to a total of 140 images, taken from four survey sites in four different HighwayDistricts. These images were obtained with an image acquisition device that was developed to takeimages for this study. Each image was classified in one of two categories, namely, bleeding or satisfactory.One hundred seal coat images were classified as bleeding surfaces and the others were satisfactory sur-faces. The edge detection algorithm was developed using the image processing toolbox of Matlab soft-ware. Aggregate edge patterns of bleeding or satisfactory seal coat surfaces differ significantly.Therefore, in this study was examined the edges of aggregate particles using seal coat images. The resultsshow that bleeding deterioration on seal coat was determined accurately using the developed algorithmin the scope of study. The results also indicate that this system is a promising tool in seal coat surfacecondition evaluation, potentially aiding pavement engineers in prioritizing seal coat projects in a quan-titative rather than qualitative manner.

� 2013 Elsevier Ltd. All rights reserved.

1. Introduction by seal coat road pavement [9]. Seal coat is also used to as a pre-

Seal coat is an economical flexible pavement type which wasconstructed single or double layer aggregate-bitumen applied ona prime sealed granular pavement surface. Seal coats enable awaterproof surface to protect sub layers and smooth and high-skid-resistance surface for vehicles comfort and to protect thepavements against the damaging effects of the traffic and climate[1–6]. Seal coats are constructed on an unbound granular base incountries such as Turkey, South Africa, Australia and New Zealand[6–8]. In Turkey, 300,000 km of 385,000 km rural road network,and also 48,929 km of 64,865 km state highway network is formed

ventive and maintenance alternative for purposes for bituminoushot mix pavements [10–11].

A certain time after a road has been opened to loading, climate,environmental factors, the use of unsuitable material, improperconstruction and design can cause deterioration which adverselyaffects driving comfort and the safety of the seal coats over thegranuler base. The most common types of deterioration on sealcoats are bleeding and raveling [12,13]. Bleeding refers to the riseof excess binder to the surface of the seal coat and is generally dis-tinguished by black patches of excessive binder appearing on thepavement surface. In other words, a bleeding surface has a smooth,slick, shiny and glass-like appearance where the aggregates are lessvisible [12,14–26].

Nomenclature

i illumination functionr reflectance functionf image functionF color imageg gray scale imageX 1, 2, . . ., NY 1, 2, . . ., MM number of rowN number of columnf(x, y, 1) matrice related to red bandf(x, y, 2) matrice related to green bandf(x, y, 3) matrice related to blue bandø empty setB structural element� dilation process operator� erosion process operator

� opening process operator� closing process operatormy the mean value of the new coordinate systemCy the covariance matrice of the new coordinate systema (m�1)/2 (non-negative integer)b (n�1)/2 (non-negative integer)h 0, 1, 2, 3, . . .. . .M�1k 0, 1, 2, 3, . . .. . .N�1fg filtered imageI input imageO output imageImin the smallest gray level value of input imageImax the largest gray level value of input imageOmin the smallest gray level value of output imageOmax the largest gray level value of output imageb gradient

274 M. Karas�ahin et al. / Construction and Building Materials 53 (2014) 273–283

Monitoring and evaluation of the deterioration is an importantpart of pavement management systems. Data collection on pave-ment surface conditions has traditionally been performed bytrained people who walk or drive along the road and visually ob-serve and record information on the condition of the road. Thismethod of visual observation not only results in a waste of moneyand time but also puts the safety of the personnel at risk. In addi-tion, the physical state of the personnel performing the survey mayalso affect the results. For this reason, objective methods are re-quired to directly measure pavement conditions.

Today, image processing techniques, the use of which has in-creased over time, have shown a marked improvement in parallelwith the advances in computer technology. Their scope of applica-tion has been steadily extended to various areas such as astron-omy, remote sensing, medicine, electronics, biology, mineralogyand nanotechnology. In addition, the use of these techniques hasresulted in advantages in terms of both time and economics.

In recent years, there has been a large increase in the number ofstudies conducted on image processing in highway engineering.These studies can be divided into two groups: the evaluation andclassification of road pavement deterioration on the one hand,and of aggregate shape properties on the other hand. Studies onroad pavement deterioration such as those on cracks constitute alarge part of the studies related to image processing in highwayengineering [27–33] in these studies, crack types have been iden-tified and a classification method has been developed. When thestudies on road pavement deteriorations were analyzed, it wasestablished that they had focused on hot bituminous or rigid pave-ment materials and also we just one study found related to thedeterioration of seal coat surfaces [34].

The main objective of this study is to determine the most com-monly observed deterioration of seal coats, bleeding, through themethod of image processing. For this purpose, first an image acqui-sition device was developed to take images of seal coat surfaces.Then, benefiting from expert opinions for pavement surface bleed-ing detection, 100 images were taken of bleeding seal coat sections,and, for comparison, 40 images were taken of satisfactory surfaces.An analysis was performed on all the images with an algorithmdeveloped for this study and the images were classified.

2. Seal coat image acquisition device

An image can also be defined as the two-dimensional map ofthe three-dimensional (3-D) view. In light of this definition, this

mathematical expression represents the image of an object pointin x, y, z coordinates at any (t) time:

f ðx; y; z; t; kÞ ¼ iðx; y; z; t; kÞ rðx; y; z; t; kÞ ð1Þ

Daylight is usually not very well suited for image processing interms of illuminating a scene because the color and the intensityof the light changes with the time of day, the time of year and theweather conditions. Image processing systems are adversely af-fected by situations in which uncontrolled light cannot be avoided[35]. From the equation of the image (Eq. (1) [36]) we see thatthe illumination function is under the influence of a light source.It may be an artificial or a natural light source. The use of the sunas a source of natural light for image acquisition poses two prob-lems. One of them, as can be understood from the definition ofthe image, is the (t) time and (k) the wavelength in the illuminationfunction. The analysis of an image taken at the same location at dif-ferent times may reveal different values due to the fact that theintensity of the light from the sun (k) varies at any (t) time. The sec-ond problem is the presence of shadows. Sunlight leads to the for-mation of shadows on aggregates at different points. Shadow isperceived as an artificial edge of the image during the processing.Therefore, it is thought that the results obtained are not fully accu-rate. This can be seen clearly in Fig. 1.

A closed system made of wood was designed to eliminate thisunfavorable situation. An artificial light source was used becauseit was a closed system. 2 � 70 W metal halides were used as asource of artificial light. The light source and the design of the woo-den system were determined after several trials, and a seal coatimage acquisition device was developed (Fig. 2). The device hasfour tires and also images taken with this device is taken only de-vice stationary. It does not work in motion. The device ensures thatall the factors (camera, height, zoom, angle, lighting, and resolu-tion) are kept constant in each of the images taken for the analysisof the seal coat surface (Table 1).

3. Digital image processing techniques

The methods and procedures employed to determine the aggre-gate edge images with high accuracy within the scope of this studyare described in more detail below:

3.1. Binary image, gray scale image, color image

The brightness level of each pixel of the digital image is referredto as the gray level. The range of gray level is determined by the

Fig. 1. Sunlight leads to the formation of shadows on aggregates at different points.

Table 1Camera features used in this study.

DSLR camera setting parameter Value

Dimensions 3872 � 2592Horizontal resolution 300 DPIVertical resolution 300 DPIBit depth 24Compressed bits/pixel 4F-stop f/4Exposure time 1/50 s.ISO speed ISO-200Focal length 26 mmMax. aperture 3.9Flash mode No flash35 mm focal length 39

M. Karas�ahin et al. / Construction and Building Materials 53 (2014) 273–283 275

number of bits (m ? the number of bits) in which the brightnessvalue of each pixel is encoded. In the simplest case, the pixels takevalues of 0 or 1. Images composed of these pixels are called binaryimages. The brightness value of each pixel in the binary image isencoded as m = 1 bit, which means that two colors are dominantin the binary image in terms of the gray level: black (0) and white(1) [36].

Images in which each pixel is denoted as m > 1 are referred to asmonochromatic (grayscale) images. Each pixel of grayscale imagescommonly used in applications is represented as 8 bits of data perpixel (m = 8). In this type of image, each pixel consists of differentgrayscale values (brightness levels) G = 28 = 256 and the range of

Fig. 2. Seal coat image acqui

gray level values is expressed as G = {0, 1, 2, . . ., 255}. As a generalrule, level ‘0’ and level ‘255’ correspond to black and white respec-tively while all the intermediate levels correspond to different graylevels [36].

A color image is composed of the mixture of gray-level imagesencoded as red, green and blue in different proportions. The red(R), green (G) and blue (B) color model is based on the Cartesiancoordinate system. Unlike grayscale images, a color image is repre-sented by 24 bits of data per pixel on computer screens. Repre-sented as a matrix, a natural color RGB image is a combination ofthree matrices each of which has the magnitude of (N �M) andis represented as {f(x, y, k)|x = 0, 1, 2, . . ., N; y = 0, 1, 2, . . ., M;k = 1, 2, 3} in the matrix equation. In general, each of these matri-ces represents a grayscale image with 256 gray levels for each pixel[36].

The process of converting a color digital image to a correspond-ing grayscale image consists of scaling the grayscale images corre-sponding to each color band in the RGB color model. The scalingprocess thus depends on the brightness values of the color image,as expressed by Eq. (2) [36].

g ¼ 0:299 � f ðx; y;1Þ þ 0:587 � f ðx; y;2Þ þ 0:114 � f ðx; y;3Þ ð2Þ

All images used in this study (bleeding and satisfactory) were ob-tained with the seal coat image acquisition device, which was ad-justed to produce the same image size (3872 � 2592 pixels) andcolor image (Fig. 3a and c). The color images were converted tograyscale images using Eq. (2). Because the time of grayscale imageprocessing is substantially less compared to the time of processing

sition device and detail.

Fig. 3. (a) Bleeding seal coat surface color image, (b) gray scale image, (c) satisfactory seal coat surface color image and (d) gray scale image.

276 M. Karas�ahin et al. / Construction and Building Materials 53 (2014) 273–283

color image. There was no loss of information during the conversionprocess (Fig. 3b and d).

3.2. Morphological image processing

The word morphology is commonly used to denote a branch ofbiology dealing with the study of the form and structure of plantsand animals. In image processing, the word morphology is used torefer to mathematical morphology. Mathematical morphology is amethod for the identification and presentation of the geometricaland topological properties of a shape, such as the boundaries, skel-eton (dome or basin) and convexity of an image [37].

In this study, morphological operations such as opening, closingand dilation were applied to the images belonging to the bleedingand satisfactory surfaces. Below are some brief mathematicaldescriptions of these operations:

3.2.1. Morphological dilationDilation is a process used to grow or thicken binary images. The

size and shape produced by this thickening process is determinedby a shape referred to as a structuring element [37].

Mathematically, the dilation process can be explained on thebasis of set operations. The dilation of A by B is denoted by A � Band expressed by Eq. (3). In terms of image processing, the firstterm (A) is the image and the second term (B) is the structuring ele-ment and the structuring element is generally much smaller thanthe image [37].

A� B ¼ fzjðbBÞz \ A–Øg ð3Þ

3.2.2. Opening–closingThe process of opening smoothes the contours of an object and

removes thin protrusions. Closing tends to narrow smooth sectionsof contours as well; however, contrary to the opening operation, itfuses narrow breaks and long thin gulfs, eliminates small holes and

fills gaps in contours. The opening of A by the structuring elementB is denoted by ‘A � B’ and expressed by Eq. (4) [37].

A � B ¼ ðA� BÞ � B ð4Þ

What is conducted here is the erosion of set A by the structuringelement B followed by the dilation of the result by the structuringelement B. Similarly, the closing of A by the structuring element Bis denoted by ‘A � B’ and expressed by Eq. (5) [37].

A � B ¼ ðA� BÞ � B ð5Þ

The procedure carried out here firstly involves the dilation of A bythe structuring element B followed by the erosion of the result bythe structuring element B [37].

3.3. Principle component analysis (PCA)

Principal component analysis is a statistical method widelyused for filtering noise, grouping systems, face recognition, imagecompression, pattern recognition in high-dimensional data pro-cessing, photogrammetry and remote sensing, image enhance-ment, screening of an image of three color components, changedetection, image merging, reduction of the number of componentsto be classified prior to the classification and feature extraction inartificial neural networks [38–40].

The basis of principal components analysis is to reduce the sizeof datasets with high numbers of correlated variables by preserv-ing the variables in the dataset as far as possible [41]. The aim hereis to calculate the most significant basin which expresses the data-set. Principal component analysis in digital images is not a spatialtransformation of the geometric properties of an image, but a sta-tistical analysis on the radiometric (spectral or color) properties ofthe image [39]. The color image is formed by the transmission ofthree grayscale images successively encoded as R (Red), G (Green),and B (Blue) onto a screen. Fig. 4a shows the band sequence of thecolor image and Fig. 4b shows the joint distribution of pixels in athree dimensional color space [36,39].

Fig. 4. (a) The band sequence of the RGB image and (b) the joint distribution of pixels in a three dimensional color space [36,39].

M. Karas�ahin et al. / Construction and Building Materials 53 (2014) 273–283 277

If each pixel in Fig. 4a is taken as 3 � 1 dimensional vectors (x)containing the brightness values in the related bands, the mean va-lue of any vector x is calculated by Eq. (6) [39]:

mx ¼ Efxg ¼ 1K

XK

k¼1

xk ð6Þ

The covariance matrix of the three-dimensional distribution inFig. 4b is calculated with Eq. (7) [39].

Cx ¼ Efðx�mxÞðx�mxÞTg ¼ Cx

¼ 1K � 1

XK

k¼1

ðxk �mxÞðxk �mxÞT ð7Þ

The transformation matrix (A), being the size of 3 � 3, is denoted by(Eq. (8)) [37]:

y ¼ Ax ð8Þ

Since the main purpose is to eliminate the correlation between thedata after the transformation, the covariance matrix in the y1-y2-y3coordinate system must be diagonal, that is, the non-diagonal ele-ments (covariances) of the matrix should be zero (Eq. (9)) [39].

Cy ¼ Efðy�myÞðy�myÞTg ð9Þ

my ¼ Efyg ¼ EfA � xg ¼ A � Efxg ¼ A � 1K

XK

i¼1

xk ¼ A �mx

The covariance matrix of the new coordinate system is reorganizedand (Eq. (10)) is obtained as [39];

Cy ¼ EfðA � x� A �mxÞðA � x� A �mxÞTgCy ¼ EfAðx�mxÞðAðx�mxÞÞTgCy ¼ EfAðx�mxÞðx�mxÞT ATg ð10Þ

Fig. 5. Linear transformation function used for the contrast stretching operation[36].

Cy ¼ AT � A � Efðx�mxÞðx�mxÞTgCy ¼ AT � A � Cx

In addition, Cy is a diagonal matrix and the main diagonal elementsof this matrix are eigenvalues of matrix Cx (Eq. (11)) [37]. In otherwords;

Cy ¼

k1 0k2

��

0 kn

26666664

37777775 ð11Þ

The non-diagonal elements of this covariance matrix are 0, there-fore, there is no correlation between the elements of the vector ‘y’[37,39].

3.4. Spatial domain filter

Filtering in the spatial domain is essentially carried out on thebasis of the regional neighborhood of pixels which form the image.For this procedure, filter patterns in certain sizes referred to as spa-tial filters are utilized [42].

In general, a linear filtering operation on an MxN-dimensional fimage performed with a mxn sized mask is calculated by Eq. (12):

fgðh; kÞ ¼Xa

s¼�a

Xb

t¼�b

wðs; tÞf ðhþ s; kþ tÞ ð12Þ

3.5. Contrast stretching

Contrast is a degree of difference between the lightest and dark-est colors in an image in terms of brightness. The contrast stretch-ing method aims to increase the contrast in every part of an imageby using the grayscale range at maximum level. Increasing the con-trast allows for the easy detection of objects and details in an im-age. Fig. 5 shows the most typical linear transformation functionused for the contrast stretching operation [36].

Making use of the definition which expresses a line passingthrough two points, the mathematical expression of this lineartransformation function is denoted by Eq. (13) [36].

I � Imin

Imax � Imin¼ O� Omin

Omax�Omin) ½O� OminðImax � IminÞ

¼ ½I � ImaxðOmax � OminÞ

O ¼ b½I � Imin þ Omin ð13Þ

b ¼ Omax � Omin

Imax � Imin

Fig. 6. Roberts cross convolution masks [43].

278 M. Karas�ahin et al. / Construction and Building Materials 53 (2014) 273–283

3.6. Edge detection

The edge represents a set of interconnected pixels located onthe border of two regions [42]. There is a direct relationship be-tween the physical properties of objects and their edges. Therefore,the edges of an object contain a great deal of information about thevarious physical properties of an image. Roberts edge filters wereused to detect aggregate edge points on the bleeding seal coat sur-face images while canny edge detection filters were used to detectaggregate edge points on the satisfactory seal coat surface images.All edge detection filters were applied to the images and the mostappropriate filters were selected for both conditions.

3.6.1. Edge detection by roberts filterRoberts operator quickly and easily performs a 2-D spatial gra-

dient on an image [43]. It displays the edge points corresponding tohigh spatial gradient regions. The pixel values of each point in anoutput image represent the estimated absolute magnitude of thespatial gradient in the input image. As shown in Fig. 6, the Robertsoperator is composed of a pair of 2 � 2 convolution filters.

A filter is simply formed by the 90� rotation of the other mask.These filters are designed to find the maximum point of continuousedges in the direction at 45� on a pixel grid. Filters can be appliedseparately to the input image of gradient components in eachdirection referred to as Gx and Gy. Afterwards, these can be com-bined together to find the absolute magnitude of the gradient ateach point and the orientation of that gradient. The magnitude ofthe gradient is calculated by Eq. (14) [37]:

rf ¼ magðrf Þ ¼ ½G2x þ G2

y 1=2

ð14Þ

Or can be denoted by Eq. (15) in the form of the absolute value.

rf jGxj þ jGyj ð15Þ

The orientation of the gradient vector is also an important quantity.The orientation angle of vector rf on the (x, y) coordinates can bedenoted by a(x, y). a(x, y) from the vector analysis is expressed byEq. (16) [37]:

aðx; yÞ ¼ tan�1 Gy

Gx

� �ð16Þ

Here, the angle is measured according to the x-axis. The orientationof an edge on the (x, y) coordinates is perpendicular to the gradientvector at this point [37].

Fig. 7. Flow chart of a

3.6.2. Edge detection by Canny filterThe Canny edge detection method was developed in 1986 by

John F. Canny [44]. This method is often used as a strong edgedetection technique in image processing applications. The stepsof this method:

1. The image is smoothed using a Gaussian filter with a standarddeviation defined for noise reduction.

2. The regional gradient, gðx; yÞ ¼ ½G2x þ G2

y 1=2

and edge direction,a(x, y) = tan�1 (Gx/Gy) are calculated for each point. Gx andGy values are calculated using any of the ‘Sobel’, ‘Prewitt’ or‘Roberts’ methods. Edge points are defined as local maximumpoints in the gradient direction.

3. Edge points detected by the application above create camber(hill) in the gradient magnitude image. Then, the algorithmtraces a path along the top of this hill and the pixels that arenot actually available at the peak of the hill are set to zero insuch a way that it gives a thin line in the output image. This pro-cess is also known as the suppression of the non-maximumpoints. The peak pixels T1 and T2 (T1 < T2) are subjected tothresholding using two threshold values. Any peak pixel thathas a value greater than T2 is referred to as a strong edge pixel.Any peak pixel between T1 and T2 is referred to as a weak edgepixel.

4. Finally, the algorithm combines weak pixels that are 8-con-nected to the strong pixels and in this way it performs edgelinking [42].

The deteriorations under study are different from one another interms of their features and formation. When the images of the bleed-ing and satisfactory seal coat surfaces were analyzed, the mostimportant difference between the two was found to be the aggregateamount on the surfaces. Due to the lack of any deterioration on thesatisfactory surface, the aggregate amount on its images is higherthan that on the images of the bleeding surface. As the deteriorationoccurs, that is, as a seal coat surface undergoes a change from thecondition of ‘‘satisfactory’’ to ‘‘deteriorated’’ the amount of aggre-gate particles on the surface decreases. Therefore, the intentionwas to create a comparison criterion with the detection of the aggre-gate edges in the images for the two types of surfaces. For this pur-pose, two different algorithms which detect the aggregate edgesboth on the bleeding and satisfactory surfaces were written usingthe above-mentioned image processing methods. The working prin-ciple of the algorithm and the results are given below:

4. Implementation of the proposed algorithm

4.1. Edge detection algorithm for images of bleeding surfaces

In this section, Fig. 7 presents the flow diagram of the algorithmdeveloped in the scope of this study for the seal coat surfaces withbleeding deterioration. In addition, the step-by-step application of

pplied algorithm.

Fig. 8. (a) The output image obtained from as a result of image subtraction andenhancement and (b) binary image obtained from (Fig 8a).

Fig. 9. (a) Image obtained from on the binary image (Fig 8a) to remove the noise(small object), (b) image was obtained as a result of morphological closingoperation on (Fig. 9a) and (c) the aggregate edge output image belong to bleedingdeterioration surface and the pixel information of the aggregate edge points.

M. Karas�ahin et al. / Construction and Building Materials 53 (2014) 273–283 279

the algorithm and the results obtained from these applications aredescribed below:

Step 1: The color images of the bleeding surface obtained fromthe seal coat image acquisition device were first converted tograyscale images and then performed on this image (Fig. 3aand b).Step 2: At this stage, the morphological opening operation wascarried out on Fig. 3b to bring to the fore the objects in theimage. A disk-shaped structuring element with a diameter of50 pixels was used for this operation. The intensity values ofthe image were changed by applying an image enhancementmethod (intensity transformations function) on the imagesobtained from the opening operation which subtracts the non-uniform background from the original. The image obtainedfrom the opening operation was subtracted from the originalimage. An image enhancement method was applied to thisimage. In this way, a new output image with changed intensityvalues was obtained. As a result of all these operations, theobjects in the new output image in Fig. 8a have become morevisible than in Fig. 3b.Step 3: Each image was converted to an image consisting of pix-els with the brightness values of black and white, that is, it wasconverted to a binary image. Here, as a result of the transforma-tion performed, white represented objects (aggregates) in theimage while black represented the other parts. The outputimage obtained as a result of this process is given in Fig. 8b.Step 4: The morphological opening operation was performed onthe binary image (Fig. 8b) to remove the noise (smaller objects).A disk-shaped structuring element with a radius of 2 pixels wasused for this operation. After the morphological opening opera-

Fig. 10. Flow chart of applied algorithm.

Fig. 11. (a) Gray scale image obtained before the application of the principalcomponents analysis and (b) gray scale image obtained after the application of theprincipal components analysis.

Fig. 12. (a) The image obtained from after image enhancement method and (b) theoutput image obtained from after average filter method.

Fig. 13. The output image obtained from the morphological operations.

280 M. Karas�ahin et al. / Construction and Building Materials 53 (2014) 273–283

tion, the images with a number of connected components lowerthan 300 pixels were eliminated. The reason why both thesetwo operations were performed to eliminate smaller objectswas to select the most appropriate values for the 100 bleedingimages used in this study. When the opening operation wasperformed on its own, it was determined that some otherobjects were also eliminated in some images in addition tothe smaller objects as a result of the structuring elementselected and therefore, the edge images failed to reflect theaccurate state of the object. The output image obtained fromthis step is shown in Fig. 9a.Step 5: At this step, the morphological closing operation wasperformed. A disk was used as the structuring element for thisoperation. As a result of this operation, the edges of the objectsspecified in the red circle in Fig. 9a were closed. The final statusof the objects is shown in the red circle in Fig. 9b. Through this

Fig. 14. The aggregate edge output image belong to satisfactory surface and thepixel information of the aggregate edge points.

Fig. 15. Total number of edge pixel distribution for bleeding and satisfactory sealcoat surface.

M. Karas�ahin et al. / Construction and Building Materials 53 (2014) 273–283 281

process, the edge points of an object can be detected moreaccurately.Step 6: The final step is the one in which the edges of theobjects were detected. For this purpose, ‘Roberts’ was used asan edge detection operator [37]. As a result of this operation,the edge pixels of the objects took the value ‘1.’ Then, the edgepixels with a value of 1 in the image were summed and eachimage of bleeding was recorded. Fig. 9c shows the edge imagesof the objects obtained from the edge detection operator andthe pixel information of the aggregate edge points.

4.2. Edge detection algorithm for satisfactory surface images

In this section, Fig. 10 presents the flow diagram of the algo-rithm developed in the scope of this study for the satisfactory sealcoat surfaces with no deterioration. In addition, the step-by-stepapplication of the algorithm and the results obtained from theseapplications are described below:

Step 1: All the color images of satisfactory surfaces used in thisstudy were taken by the seal coat image acquisition device(Fig. 3c).Step 2: In this step, a principal components analysis (PCA) wasperformed on the color image (Fig. 3c). The image obtainedfrom the analysis was converted to grayscale images and afterthis stage, all the processing steps were performed on thisimage. The image change is presented before (Fig. 11a) and after(Fig. 11b) the application of the principal components analysis.It can be seen that the image in Fig. 11b is clearer.Step 3: This step consists of the implementation of imageenhancement and filtering operations which make the imagemore suitable after the PCA. The operation performed herechanged the intensity values (Fig. 12a) of the image (Fig. 11b).This made the details of the objects in the image as visible aspossible. Afterwards, an average filter was used on the imagein Fig. 12a [42] and this image was blurred. The purpose of thisoperation is to suppress false details. The output image after theoperation is given in Fig. 12b.Step 4: In this step, the gray-scale morphological image pro-cessing method was performed on the image in Fig. 12b. First,the morphological dilation operation and then the morphologi-cal opening operation was performed on the image. An octago-nal-shaped structuring element with a diameter of 3 pixels wasused for this operation. The output image obtained from themorphological operations is shown in Fig. 13.Step 5: This is the last step in which the edges of the objects inthe image are detected. ‘Canny’ was used as the edge detectionoperator [44]. ‘1’ represents the pixels of the edges in theobtained edge images. Then, the edge pixels with value 1 inthe image were summation and were recorded for each imageof the satisfactory surface. Fig. 14 shows the aggregate edgeimages obtained from the edge detection operator and the edgepixel information of the aggregate edge points.

5. Results and discussion

A total of 140 images were taken of the bleeding and satisfac-tory surfaces by the seal coat image acquisition device at 4 differ-ent locations. 100 of them were of surfaces with bleedingdeterioration and the others, were of satisfactory surfaces withno deterioration. The images of bleeding deterioration were takenby the seal coat image acquisition device from examination pointswhere the deterioration occurred on the seal coat surface. For com-parison, images were taken from examination points where nodeterioration had occurred on the seal coat surface.

All the images taken by the seal coat image acquisition device,developed for this study, have the same features. When the imagesof the bleeding and satisfactory seal coat surfaces were analyzed,the most important difference was found in the distribution ofthe aggregates on the surfaces. Therefore, it was thought that acomparison criterion could be established for the detection ofaggregate edges in the images for the two types of surfaces. For thispurpose, the aggregate edge images for both types of surfaces(bleeding and satisfactory) were detected by the above-mentionededge detection algorithms. The total number of aggregate edge pix-els (the sum of 1s) was recorded separately for each image as a re-

282 M. Karas�ahin et al. / Construction and Building Materials 53 (2014) 273–283

sult of the implementation of the edge detection algorithms forbleeding and satisfactory surfaces.

The results obtained are given in Fig. 15. When they were ana-lyzed, it was determined that the number of aggregate edges onthe seal coat surfaces where bleeding occurs was lower than thaton the satisfactory surfaces, as expected. The reason for this is thatthe bleeding deterioration observed on the pavement results in areduction in the number of aggregates on the surface. Since nodeterioration occurs on satisfactory surfaces, a reduction in thenumber of aggregates is not expected. In addition, looking atFig. 15, it can be seen that the total numbers of aggregate edge pix-els of the bleeding and satisfactory surfaces respectively are similarto each other within their own groups. It can also be seen that theresults obtained for both types of surfaces indicate within-groupconsistency.

6. Conclusion and recommendation

The purpose of this study was to investigate the detectability ofthe most common type of deterioration, bleeding, by using imageprocessing techniques as an effective and reliable measurementtechnique. For this purpose, images were taken of seal coat sur-faces where bleeding had occurred and of satisfactory surfaceswhere no deterioration was observed, to compare using the sealcoat image acquisition device.

Two different edge detection algorithms were applied on theimages obtained. A classification was made based on the totalnumber of aggregate edge pixels obtained from the implementa-tion of the two algorithms.

Through the pavement management system, the limited budgetwill be used at the optimum level by establishing the most appro-priate annual maintenance program of first priority in order todetermine roads, methods and the period of time in which theintervention will take place. Further research (more images andlocations) is required before this system included in pavementmanagement system. Therefore, with this system will be devel-oped, it can be determined whether maintenance work will be per-formed on a surveyed route and on which road it will be performedprimarily in case of a survey of multiple road segments.

For the algorithm developed for this study to be successful,images should be taken by the seal coat image acquisition deviceof seal coat surface routes immediately after the road is openedup to traffic, that is, of satisfactory surfaces where no deteriorationhas occurred, and a data base should be established on the roadanalyzed. Since deteriorations will occur on the road over timedue to various reasons, periodical surveys should be carried outon the seal coat surface. The images of the seal coat surfaces canbe analyzed using the algorithm and these can in turn provideinformation about the status of the road.

In this study, a successful operation was carried out using theimage processing method to analyze the bleeding deterioration ob-served on seal coats. Therefore, it is suggested that this study willplay an important role in the realization of a faster, more effectiveand reliable decision-making process for highway engineers.

References

[1] Whiteoak D. The shell bitumen handbook. England: Shell Bitumen; 1990. p.287–290.

[2] Griffith A, Hunt E. Asphalt cement chip seals in Oregon. Oregon department oftransportation, State research project no: 546, Oregon, US, 2000. p. 1.

[3] Karas�ahin M, Agar E. An assessment about chip seals. In: 4th National asphaltsymposium, Ankara, Turkey, 2004. p. 131–140 [in Turkish].

[4] Wood TJ, Olson RC. Rebirth of chip sealing in Minnesota. Transp Res Rec JTransp Res Board Natl Acad 2007:260–4.

[5] Karas�ahin M, Saltan M, Gürer C, Çetin S, Aktas� B, Uz VE. Chip seal applicationsin Turkey and world. In: Proceeding of the 5th national asphalt symposium,Ankara, Turkey, 2009(a). p. 153–162 [in Turkish].

[6] Gürer C. Determination of parameters affecting seal coat performance anddevelopment a performance model. PhD Dissertation, Suleyman DemirelUniversity, Graduate school of applied and natural sciences, Civil engineeringmain science, Isparta, Turkey, 2010.

[7] Distin T. Spray sealing practice in South Africa. In: 1st sprayed sealingconference, Adelaide Australia, 2008.

[8] Holtrop W. Sprayed sealing practice in Australia. In: 1st sprayed sealingconference, Adelaide, Australia, 2008.

[9] <http://www.kgm.gov.tr> [access 01.01.11].[10] Labi S, Sinha KC. Effectiveness of highway pavement seal coating treatments. J

Transp Eng 2004;130(1):14–23.[11] Karas�ahin M, Gürer C. Chip seals. In: 106th education seminar of asphalt day,

Konya, Turkey, 2007. p. 75–116 [in Turkish].[12] Gransberg D, James DMB. Chip seal best practices. In: Chip seal performance

measures, NCHRP synthesis 342, Transportation research board, Washington,D.C., 2005. p. 56–60.

[13] Lee J, Shields T, Ahn HJ. Performance evaluation of seal coat materials anddesign, FHWA/IN/JTRP, Indiana department of transportation office of researchand development, 2011/5. p. 125.

[14] Jackson DC, Jackson NC, Mahoney JP. Westside chip seal study (1989). ReportNo WA-RD 229.1, Washington state department of transportation,Washington, U.S., 1994, p. 37.

[15] Jahren CK, Bergeson KL, Al-hammadi A, Çelik S, Lau G. Thin maintenancesurfaces. Phase one report, Center for transportation research and educationIowa State University, 1999.

[16] Yamada A. Asphalt seal coat treatments. California: San Dimas Technology andDevelopment Center San Dimas; 1999.

[17] Johnson A. Best practice handbook on asphalt pavementmaintenance. Minneapolis: Minnesota Technology Transfer Center; 2000.

[18] Wade M, Desombre R, Peshkin D. High volume/high speed asphalt roadwaypreventive maintenance surface treatments. Final Report No. SD99-09, SouthDakota department of transportation office of research, 2001.

[19] ARRA Basic Asphalt Recycling Manual. Asphalt recycling and reclaimingassociation, U.S. department of transportation, Federal highwayadministration, U.S., 2001.

[20] NDOR. Pavement maintenance manual. Nebraska department of roads,Nebraska, 2002.

[21] MoDOT. Distress identification and treatment selection for asphalt surfaces.MoDOT research, development and technology, 2002.

[22] Freeman TJ, Pinchoo DA, Ren H, Spiegelman CH. Analysis and treatmentrecommendations from the supplemental maintenance effectiveness researchprogram (Smerp). Report No. FHWA/TX-03/4040-3, Texas department oftransportation, U.S., 2003, p. 178.

[23] Maher M, Marshall C, Harrison F, Baumgaertner K. Context sensitive roadwaysurfacing selection guide. FHWA-CFL/TD-0x-004, Federal highwayadministration central federal lands highway division, U.S., 2005.

[24] Bahia H, Jenkins K, Hanz A. Performance grading of bitumen emulsions forsprayed seals. In: 1st sprayed sealing conference, Adelaide Australia, 2008.

[25] AAPA. Treatment of flushed or bleeding surfaces. Pavement work tips-No 7,Australian asphalt pavement association, Austroads, 2010.

[26] <http://www.pavementinteractive.org/article/bituminous-surface-treatments/>[access 10.10.11].

[27] Offrell P, Magnusson R. In situ photographic survey of crack propagation inflexible pavements. Int J Pavement Eng 2004;5(2):91–102.

[28] Monem TA, Amr AO, Maghoub H. Asphalt crack detection using thermography.University of central florida-center for advanced transportation systemssimulation, information, proceeding, ITC 108 A 2005-06-01, 2005.

[29] Lee HD, Kim JJ. Development of a manual crack quantification and automatedcrack measurement system. Final project TR- 457, Iowa department oftransportation and iowa highway research board, 2005, p. 15.

[30] Bray J, Verma B, Li X, He W. A neural network based technique for automaticclassification of road cracks. In: International joint conference on neuralnetworks Sheraton Vancouver wall centre hotel, Vancouver, BC, Canada, 2006.

[31] Mustaffara M, Lingb TC, Puanb OC. Automated pavement imaging program(Apip) for pavement cracks classification and quantification – aphotogrammetric approach. The international archives of thephotogrammetry, Remote sensing and spatial information sciences, (XXXVII.Part B4) Beijing, 2008.

[32] Zhang J, Sha A, Sun ZY, Gao HG, Pavement crack automatic recognition basedon wiener filtering. In: Critical issues in transportation system planning,development, and management proceedings of the ninth internationalconference of chinese transportation professionals, 2009.

[33] Rababaah H, Vrajitoru D, Wolfer J. Asphalt pavement crack classification: acomparison of GA, MLP, and SOM. 2010, <http://www.cs.iusb.edu/~jwolfer/pubs/78-rababaah.pdf>.

[34] Gransberg DD, Karaca I, Burkett WR. Quantifying seal coat surface conditionusing digital image processing based on information theory. Int J PavementEng 2002;3(4):197–205.

[35] Erhardt A. Theory and applications of digital image processing. University ofApplied Sciences; 2000. p. 54.

[36] Kızılkaya A. Image processing lecture notes, Pamukkale University,engineering faculty, department of electrical and electronics engineering,Denizli, Turkey, 2009 [in Turkish].

[37] Gonzalez RC, Woods RE. Digital image processing. 2nd ed. U.S.: Prentice-Hall;2002.

[38] Simith LI. A tutorial on principal components analysis 2002:27.

M. Karas�ahin et al. / Construction and Building Materials 53 (2014) 273–283 283

[39] Akça MD, Dogan S. The principal components transformation on digitalimages. J Maps 2002;129:1–5 [in Turkish].

[40] Shih FY. Image processing and pattern recognation fundamental andtechniques. Hoboken, New Jersey: John Wiley & Sons Inc.; 2010.

[41] Jolliffe IT. Principal component analysis. 2nd ed. Newyork, U.S.: Springer-Verlag; 2002.

[42] Gonzalez RC, Woods RE, Eddins SL. Digital image processing using Matlab. 1sted. U.S.: Prentice-Hall; 2004.

[43] <http://www.robotics.technion.ac.il/courses/advanced_laboratory/lab7/arl_7_read.pdf> [access 05.01.13].

[44] Canny J. A computational approach to edge detection. IEEE Trans Pattern AnalMach Intell 1986;8(6):679–98.