Implementation of Computer Vision in Asphalt Damage Identiﬁcation on

the Trans-Sumatera Road

Soleh Darmansyah

, Rika Rosnelly

and Hartono

Computer Science Masters Study Program, Faculty of Engineering and Computer Science, Potential Utama University,

Medan, Indonesia

Informatics Engineering Study Program Faculty of Engineering, Medan Area University, Medan, Indonesia

Keywords:

Road, Asphalt, Machine Learning, Decision Tree, K-Nearest Neighbor.

Abstract:

Roads are infrastructure made to facilitate land transportation in connecting one area to another. In general,

roads in Indonesia use asphalt as a material in the road construction process. The cross-Sumatra route is one

of the accesses that plays an important role in increasing economic progress in areas that connect areas on

the island of Sumatra. The development of computer vision using various image recognition classiﬁcation

methods results in more accurate data accuracy. The Decission Tree and K-Nearest Neighbor methods in

image recognition classiﬁcation of asphalt damage can be a solution in identifying damage and measuring the

area of damage through machine learning from images taken from the ﬁeld. The design and implementation of

making applications is continued using the Decission Tree method using python as a programming language.

Asphalt damage conditions are divided into three classiﬁcation categories of asphalt damage, namely mild,

moderate and severe. The results of the identiﬁcation can be used as a report or ﬁeld survey of the damage

conditions that occur on the Sumatra route. The accuracy value of the training is carried out using a dataset of

560 images. The Decission Tree method can get light damage 99.3damage, the accuracy value is 99.3for light

damage is 79.12accuracy from Machine learning carried out in this study show the highest accuracy value

obtained from the Decission Tree method in identifying road damage.

1 INTRODUCTION

In road inspections it is usually carried out by ofﬁcers

who go into the ﬁeld by measuring and recording the

types of damage that occur in the ﬁeld and then tak-

ing pictures as report material. Road damage caused

by various factors causes road conditions to become

damaged quickly and is not feasible, thus endangering

road users crossing the road.Potholes, cracks and dis-

tortions are most commonly found on Sumatran high-

ways. This is caused by several things, namely the

condition of the soil structure, asphalt cracks that are

left, standing water or ﬂooding and landslides. The

trans-Sumatra route has land routes that often experi-

ence damage problems. Pruning process Along with

the advancement of technology, the use of humans in

surveys in the ﬁeld can be supported by technology

that suits the needs in the ﬁeld, making it easier to

make data and reports. With the development of com-

puter vision which is from the development of com-

puter science technology that can work like humans

(Dompeipen et al., 2021), the inspection and identi-

ﬁcation of roads that were previously done manually

can be done with digital image processing and digital

image processing candetermine the type and measure

road damage.

Factors that damage the road that occurs in the

ﬁeld. in Indonesia there are four classes of roads in

accordance with Law no. 22 of 2009 concerning road

trafﬁc and transportation. Until now, Public Works

Department workers and project consultants have not

used computer technology. During the process of

identifying road damage, it is still done manually to

ﬁnd and determine the type of road damage. The

measurement process is carried out manually using a

simple measuring device (roll meter) with human as-

sistance. Detecting holes manually requires a lot of

effort and time (Aparna et al., 2022).

Research by Agus Irawan, Adi Pratomo, Mey

Risa, Heldiansyah (2016) entitled ”Design of Road

Asphalt Damage Detection System Through Video

Using Fast Fourier Transform” In research designing

a road asphalt damage detection system (Ouma and

Hahn, 2017) via video can make it more cost effec-

102

Darmansyah, S., Rosnelly, R. and Hartono, .

Implementation of Computer Vision in Asphalt Damage Identiﬁcation on the Trans-Sumatera Road.

DOI: 10.5220/0012444500003848

Paper published under CC license (CC BY-NC-ND 4.0)

In Proceedings of the 3rd International Conference on Advanced Information Scientiﬁc Development (ICAISD 2023), pages 102-106

ISBN: 978-989-758-678-1

tive, faster and more safe in the implementation of ob-

servation and evaluation of the condition of the road.

This study uses video footage of road asphalt and then

extracts it into image frames. By utilizing several ar-

eas the result of the sum of the Fast Fourier Transform

values (Ma et al., 2021) in the image is used as a dam-

age feature to classify the image of asphalt roads into

good, moderate, slightly damaged and heavily dam-

aged categories.

Computer vision is a science that uses image

processing (Oppong et al., 2022) to make deci-

sions based on images obtained from sensors. In

other words, computer vision aims to make the sys-

tem work automatically by ”seeing” images/videos

(Azhar et al., 2016). Common frameworks that are

commonly used in computer vision are: image ac-

quisition, pre-processing, feature extraction, detec-

tion/segmentation, high-level processing, and deci-

sion making. Seeing the complexity of the acquisi-

tion process up to decision making, the research in-

cludes two major stages, namely feature extraction

for edge detection (Vijayarajeswari et al., 2019) be-

tween objects and its application for object recogni-

tion, namely to identify the type of damage (Riana

et al., 2022) to asphalt roads, such as in the form of

potholes, corrugated, cracking and raveling and edge

scouring (Ouma and Hahn, 2017).

Asphalt damage image identiﬁcation from the

dataset is divided into three categories of asphalt dam-

age classiﬁcation, namely mild, moderate and severe.

(Chitale et al., 2020) With these three categories,

damage to asphalt with potholes, cracks and distor-

tion can be identiﬁed and processed through a ma-

chine learning digital image recognition system (Feng

et al., 2023). Machine learning is a computational

model that is currently widely used in various ﬁelds.

The Gabor ﬁlter is used to analyze the texture

(Hendrawan et al., 2019) or obtain features from the

images used in the dataset with various types of as-

phalt damage images to process the images to be

tested (Permadi et al., 2021). Then proceed with Dis-

crete Cosine Transform in image compression.

2 RESEARCH METHODS

In this study the authors carried out the research

method described in Figure 1 with the initial stages

starting from taking image acquisition, then entering

the Machine Learning Process starting from Prepro-

cessing, Segmentation, Classiﬁcation Methods and

applications displaying the results and extent of dam-

age.

Figure 1: Research Method.

2.1 Data Collection

Asphalt image collection is done by taking pictures

directly in the ﬁeld so as to get various types of dam-

age that occurs on the asphalt. The number of image

data tested was 550 data and the training data tested

were 300 images. From the results of the inspection

carried out, it was found that three types of damage

occurred, namely severe, moderate and light. To as-

sess the type of road damage using the SDI (Surface

Distress Index) method, the road performance scale is

obtained from visual observations (Singh and Gupta,

2019). The images obtained will be continued with

the segmentation process and preprocessing will be

carried out.

Figure 2: Result Type of Road Damage.

2.2 Preprocessing

The preprocessing stage is the stage of the digital im-

age processing process before the image features are

extracted. The process carried out using the GLCM

method is one of the methods used for texture fea-

ture extraction in images (14, ). The features used in

GLCM, GLCM features have a total of fourteen fea-

tures and this study uses ASM features of contrast,

difference, homogeneity, correlation, second corner

moment (ASM), and energy (Leonardo, 2020). The

author uses the ASM feature and uses this entropy

feature to detect the type of asphalt damage with the

equation used in the ASM feature as follows:

ASM =

∑

i=1

∑

j=1

p(i, j)

(1)

H = −

∑

i=1

∑

j=1

p(i, j)log

p(i, j) (2)

Implementation of Computer Vision in Asphalt Damage Identiﬁcation on the Trans-Sumatera Road

103

2.3 Segmentation

Segmentation is performed to divide an image into

homogeneous regions based on certain similarity cri-

teria. (Li et al., 2022) The segmentation process uses

the Gabor Filter method. Gabor ﬁlters can be used to

detect asphalt damage. Gabor ﬁlter is one of the ﬁlters

that is able to simulate the characteristics of the hu-

man visual system in isolating certain frequencies and

orientations from the image (Coulibaly et al., 2022).

Gabor ﬁlter in extracting local features from images

(Waluyo et al., 2023). The Gabor ﬁlter equation can

be seen from the equation below.

G(x, y; f , θ, ψ, Y ) = exp −

′2

2σ

cos2π f x

′

+ ψ

(3)

Where:

G(x, y; f , θ, ψ, Y ) : Gabor ﬁlter values at coordinates

(x, y).

′

= x cos θ + y sin θ and y’ = -x sin θ + y cos θ :

Rotation of coordinates with respect to angle θ.

f : The spatial frequency of the Gabor ﬁlter.

θ : Gabor ﬁlter orientation (in radians).

ψ : Gabor ﬁlter phase.

Y : Gabor ﬁlter aspect ratio.

σ : Gabor ﬁlter Gaussian variance.

Figure 3: Process Result Changing Image.

2.4 Classiﬁcation Method

The classiﬁcation method in this study uses the two

Decission Tree methods and the K-Nearest Neighbors

Method to ﬁnd the highest accuracy results. From

the results of the accuracy values of the two meth-

ods used, the method that produces the highest accu-

racy value will be used to create an application sys-

tem that can measure the type of damage and extent

of asphalt damage. The K-Nearest Neighbor method

is a new data grouping classiﬁcation method based on

the distance of the new data with some of the closest

data or called data/neighbors (Coulibaly et al., 2022).

The K-Nearest Neighbors method is a method used to

classify objects based on training data that have the

closest distance to the test data object (Arfani, 2021).

The steps with the KNN method are started by enter-

ing: training data, training data labels, k, test data. All

testing data must be calculated the distance to each

test data with the Euclidian distance formula as in the

following equation.

d(x, y) =

∑

i=1

− y

)

(4)

Where:

= sample data

= test data or training data

i = data variable

d(x,y) = dissimilarity/distance

n = data dimension

The Decision Tree method is a type of classiﬁ-

cation that represents the shape of a tree structure.

Where each node represents the attribute, the branch

represents the value of the attribute, and the leaves

represent the class.

Figure 4: Process Result Changing Image.

Table 1: The results of the accuracy of training and test-

ing of image data carried out produce a comparison of the

accuracy values.

Categories K-NN Decission Tree

Lightly 97.1% 99.3%

Medium 87.8% 98.6%

Heavy 81.1% 99.3%

3 DATA AND APPLICATION

TESTING

On the initial page before starting the application, the

user can enter images to measure the extent of damage

to asphalt using the built application.

The machine learning process on images can rec-

ognize the type of image to then enter the process of

classifying the type of damage, asphalt damage cate-

gory and the system can calculate the area of asphalt

damage based on the pixel conversion value in meters

area unit.

The highest precision value of the type of groove

damage with the heavy damage category and the type

ICAISD 2023 - International Conference on Advanced Information Scientiﬁc Development

104

Figure 5: Initial Appearance of the Application.

Figure 6: Detection Results and Damage Area.

Figure 7: Precission-Recall Curve.

of curly damage with the moderate damage category.

Accuracy shows the accuracy of the results ob-

tained from the closeness of the value obtained with

the actual value. Precision is the suitability of the

data taken with the required information. Recall is

success in getting information back. F1-score is a

comparison of the average precision and recall (Qudsi

et al., 2020), (Kurniawan and Barokah, 2020). Precis-

sion Recall Curve results from the highest class label

Table 2: Precission-Recall Value Results.

No Damage Damage Precission

Type

Code

Category Type Recall

1 BA Groove Weight 0.995

2 BD Distortion weight 0.752

3 BH Severance Weight 0.507

4 BK Obesity Weight 0.202

5 BL Hole Weight 0.675

6 RH Broken Thirst 0.309

7 RL Light Hole 0.679

8 SD Moderate Distortion 0.655

9 SH Being Thirsty 0.995

10 SK Moderate Obesity 0.000

11 SKER Medium Curly 0.995

12 Straight Cracking 0.240

get the type of asphalt damage with heavy grooves,

medium wear and medium curly getting the highest

value, namely 0.995 while for light damage distor-

tion, light obesity and moderate obesity have a value

of 0.000 because the results of the image dataset did

not ﬁnd the type of damage that read by the system.

In taking images in the ﬁeld, it is necessary to

carry out technical training so that the images can be

appropriate and produce the best accuracy values, so

that the system can produce asphalt damage areas ac-

cording to the type and category of asphalt damage.

The shooting technique can be done by taking a 45

degrees angle towards the image object and can use

shooting with a slanted technique. In accordance with

the results of the accuracy of the KNN and Decission

Tree methods, the researchers decided to use the De-

cision Tree method as an image classiﬁcation method

because it has better results and can develop accord-

ing to the type of asphalt damage.

4 CONCLUSIONS

In taking image acquisition, it is necessary to pay at-

tention to the distance of taking, the quality of light-

ing at the location and the perspective of the direction

of shooting because these factors become obstacles

that occur when the system conducts data training and

tests system results on applications in identifying the

type of damage and predicting the extent of the size

of the damage that occurs on asphalt. The Decission

Tree method and the K-NN method are widely used

to carry out image processing classiﬁcation processes

and are applied to machine learning as a classiﬁcation

model. The Decision Tree method is able to provide

results in the form of a decision tree which can be

Implementation of Computer Vision in Asphalt Damage Identiﬁcation on the Trans-Sumatera Road

105

used as a reference in observing decisions made based

on the category of damage that occurs and is tested

by the system. Implementation of the best method

for the accuracy of asphalt damage identiﬁcation re-

sults is made into a web-based application that can be

run ofﬂine or online. Based on the overall accuracy

test the results of the decision tree method show an

accuracy value of 98.6% and the KNN method gets

an overall accuracy value of 74.07%. This result is

obtained from the category of heavy damage to as-

phalt. The types of asphalt damage carried out in the

tests were carried out using images of groove dam-

age, distortion, fatness, wear and tear, curl, holes, and

cracks. The comparison of classiﬁers results that the

decision tree classiﬁer gives the best accuracy value

from KNN.

ACKNOWLEDGEMENTS

The researcher would like to thank all parties involved

in this research, this research can be a solution and

simplify the ﬁeld survey process for damaged road

conditions so as to simplify and speed up calculations

in predicting the need for asphalt repair materials in

the ﬁeld.

REFERENCES

Aparna, Y., Rai, R., Gupta, V., Aggarwal, N., and Akula,

A. (2022). Convolutional neural networks based pot-

holes detection using thermal imaging. Journal of

King Saud University - Computer and Information

Sciences, 34:578–588.

Arfani, M. (2021). Implementasi computer vision pada de-

teksi dini kebakaran area spbu berbasis segmentasi.

Azhar, K., Murtaza, F., Yousaf, M., and Habib, H. (2016).

Computer vision based detection and localization of

potholes in asphalt pavement images. Institute of

Electrical and Electronics Engineers Inc.

Chitale, P., Kekre, K., Shenai, H., Karani, R., and Gala, J.

(2020). Pothole detection and dimension estimation

system using deep learning (yolo) and image process-

ing. IEEE Computer Society.

Coulibaly, S., Kamsu-Foguem, B., Kamissoko, D., and

Traore, D. (2022). Deep convolution neural network

sharing for the multi-label images classiﬁcation. Ma-

chine Learning with Applications, 10:100422.

Dompeipen, T., Sompie, S., and Najoan, M. (2021). Com-

puter vision implementation for detection and count-

ing the number of humans,”jurnal. Teknik Informatika,

16:65–67.

Feng, L., Zhang, L., Gao, Z., Zhou, R., and Li, L. (2023).

Gabor-yolonet: A lightweight and efﬁcient detection

network for low-voltage power lines from unmanned

aerial vehicle images. Frontiers in Energy Research,

10.

Hendrawan, Y., Widyaningtyas, S., and Sucipto, S.

(2019). Computer vision for purity, phenol, and

ph detection of luwak coffee green bean,”telkomnika.

Telecommunication Computing Electronics and Con-

trol, 17:3073–3085.

Kurniawan, Y. and Barokah, T. (2020). Klasiﬁkasi penen-

tuan pengajuan kartu kredit menggunakan k-nearest

neighbor. Jurnal Ilmiah Matrik, 22:73–82.

Leonardo, L. (2020). Penerapan metode ﬁlter gabor untuk

analisis ﬁtur tekstur citra pada kain songket. Jurnal

Sistem Komputer dan Informatika (JSON, 1:120.

Li, Y., Bi, Y., Zhang, W., Ren, J., and Chen, J. (2022). Color

image edge detection using multi-scale and multi-

directional gabor ﬁlter.

Ma, X., Shakoor, M., Vasiukov, D., Lomov, S., and Park, C.

(2021). Numerical artifacts of fast fourier transform

solvers for elastic problems of multi-phase materials:

their causes and reduction methods. Computational

Mechanics, 67:1661–1683.

Oppong, S., Twum, F., Hayfron-Acquah, J., and Missah,

Y. (2022). A novel computer vision model for medici-

nal plant identiﬁcation using log-gabor ﬁlters and deep

learning algorithms. Computational Intelligence and

Neuroscience.

Ouma, Y. and Hahn, M. (2017). Pothole detection on as-

phalt pavements from 2d-colour pothole images using

fuzzy c-means clustering and morphological recon-

struction. Automation in Construction, 83:196–211.

Permadi, D., Widianto, W., and Hudayat, Y. (2021). Anali-

sis kondisi permukaan perkerasan jalan dengan meng-

gunakan metode survey sdi dan rci serta penanganan-

nya.

Qudsi, N., Asmara, R., and Syulistyo, A. (2020). Identi-

ﬁkasi citra tulisan tangan digital menggunakan con-

volutional neural network (cnn. Seminar Informatika

Aplikatif Polinema, page 48–53.

Riana, D., Rahayu, S., Hadianti, S., Frieyadie, F., Hasan,

M., Karimah, I., and Pratama, R. (2022). Identiﬁkasi

citra pap smear repomedunm dengan menggunakan k-

means clustering dan glcm. Jurnal RESTI (Rekayasa

Sistem dan Teknologi Informasi, 6:1–8.

Singh, G. and Gupta, P. (2019). Performance analysis of

various machine learning-based approaches for detec-

tion and classiﬁcation of lung cancer in humans. Neu-

ral Computing and Applications, 31:6863–6877.

Vijayarajeswari, R., Parthasarathy, P., Vivekanandan, S.,

and Basha, A. (2019). Classiﬁcation of mammo-

gram for early detection of breast cancer using svm

classiﬁer and hough transform. Measurement: Jour-

nal of the International Measurement Confederation,

146:800–805.

Waluyo, W., Isnanto, R., and Rochim, A. (2023). Compar-

ison of mycobacterium tuberculosis image detection

accuracy using cnn and combination cnn-knn. Jur-

nal RESTI (Rekayasa Sistem dan Teknologi Informasi,

7:80–87.

ICAISD 2023 - International Conference on Advanced Information Scientiﬁc Development

106