Dalarna University's logo and link to the university's website

du.sePublications
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • chicago-author-date
  • chicago-note-bibliography
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Objective Assessment of Loose Gravel Condition using Machine Learning with Audio-visual Observation
Dalarna University, School of Information and Engineering, Microdata Analysis.
2024 (English)Doctoral thesis, comprehensive summary (Other academic)
Description
Abstract [en]

A well-maintained road network is essential for sustainable economic development, providing vital transportation routes for goods and services while connecting communities. Sweden's public road network includes a significant portion of gravel roads, particularly cost-effective for less populated areas with lower traffic volumes. However, gravel roads deteriorate quickly, leading to accidents, environmental pollution, and vehicle tire wear when not adequately maintained. The Swedish Road Administration Authority (Trafikverket) assesses gravel road conditions using subjective methods, analysing images taken during snow-free periods. Due to cost constraints, this labour-intensive process is prone to errors and lacks advanced techniques like road profilometers.

This thesis explores the field of assessing gravel road conditions. It commences with a comprehensive review of manual gravel road assessment methods employed globally and existing data-driven smart methods. Subsequently, it harnesses machine hearing and machine vision techniques, primarily focusing on enhancing road condition classification by integrating sound and image data.

The research examines sound data collected from gravel roads, exploring machine learning algorithms for loose gravel conditions classification with potential road maintenance and monitoring implications. Another crucial aspect involves applying machine vision to categorise image data from gravel roads. The study introduces an innovative approach using publicly available resources like Google Street View for image data collection, demonstrating machine vision's adaptability in assessing road conditions.

The research also compares machine learning methods with manual human classification, specifically regarding sound data. Automated approaches consistently outperform manual methods, providing more reliable results. Furthermore, the thesis investigates combining audio and image data to classify road conditions, particularly loose gravel scenarios. Early feature fusion using pre-trained models significantly improves classifier accuracy.

The research proposes using cost-effective devices like mobile phones with AI applications attached to car windshields to collect audio and visual data on gravel road conditions. This approach can provide more accurate and efficient data collection, resulting in real-time mapping of road conditions over considerable distances. Such information can benefit drivers, travellers, and road maintenance agencies by identifying problematic areas with loose gravel, enabling targeted and efficient maintenance efforts, and minimising disruptions to traffic flow during maintenance operations.

Place, publisher, year, edition, pages
Borlänge: Dalarna University, 2024.
Series
Dalarna Doctoral Dissertations ; 29
Keywords [en]
Gravel road condition assessment, Loose gravel, Sound analysis, Machine learning, Image analysis, Audio analysis, Image and audio data fusion
National Category
Infrastructure Engineering Reliability and Maintenance
Identifiers
URN: urn:nbn:se:du-47915ISBN: 978-91-88679-59-8 (print)OAI: oai:DiVA.org:du-47915DiVA, id: diva2:1831420
Public defence
2024-03-28, room Clas Ohlson, campus Borlänge, 13:00 (English)
Opponent
Supervisors
Available from: 2024-02-23 Created: 2024-01-25 Last updated: 2024-02-23Bibliographically approved
List of papers
1. A Review of Intelligent Methods for Unpaved Roads Condition Assessment
Open this publication in new window or tab >>A Review of Intelligent Methods for Unpaved Roads Condition Assessment
Show others...
2020 (English)In: 2020 15th IEEE Conference on Industrial Electronics and Applications (ICIEA), 2020, p. 79-84Conference paper, Published paper (Refereed)
Abstract [en]

Conventional road condition evaluation is an expensive and time-consuming task. Therefore data collection from indirect economical methods is desired by road monitoring agencies. Recently intelligent road condition monitoring has become popular. More studies have focused on automated paved road condition monitoring, and minimal research is available to date on automating gravel road condition assessment. Road roughness information gives an overall picture of the road but does not help in identifying the type of defect; therefore, it cannot be helpful in the more specific road maintenance plan. Road monitoring can be automated using data from conventional sensors, vehicles' onboard devices, and audio and video streams from cost-effective devices. This paper reviews classical and intelligent methods for road condition evaluation in general and, more specifically, reviews studies proposing automated solutions targeting gravel or unpaved roads.

Keywords
unpaved roads, machine learning, road condition monitoring, data quality, sensors
National Category
Computer and Information Sciences
Research subject
Research Profiles 2009-2020, Complex Systems – Microdata Analysis
Identifiers
urn:nbn:se:du-35459 (URN)10.1109/ICIEA48937.2020.9248317 (DOI)000646627000014 ()2-s2.0-85097521958 (Scopus ID)
Conference
2020 15th IEEE Conference on Industrial Electronics and Applications (ICIEA)
Available from: 2020-11-23 Created: 2020-11-23 Last updated: 2024-01-30Bibliographically approved
2. Classification of the Acoustics of Loose Gravel
Open this publication in new window or tab >>Classification of the Acoustics of Loose Gravel
Show others...
2021 (English)In: Sensors, E-ISSN 1424-8220, Vol. 21, no 14, article id 4944Article in journal (Refereed) Published
Abstract [en]

Road condition evaluation is a critical part of gravel road maintenance. One of the assessed parameters is the amount of loose gravel, as this determines the driving quality and safety. Loose gravel can cause tires to slip and the driver to lose control. An expert assesses the road conditions subjectively by looking at images and notes. This method is labor-intensive and subject to error in judgment; therefore, its reliability is questionable. Road management agencies look for automated and objective measurement systems. In this study, acoustic data on gravel hitting the bottom of a car was used. The connection between the acoustics and the condition of loose gravel on gravel roads was assessed. Traditional supervised learning algorithms and convolution neural network (CNN) were applied, and their performances are compared for the classification of loose gravel acoustics. The advantage of using a pre-trained CNN is that it selects relevant features for training. In addition, pre-trained networks offer the advantage of not requiring days of training or colossal training data. In supervised learning, the accuracy of the ensemble bagged tree algorithm for gravel and non-gravel sound classification was found to be 97.5%, whereas, in the case of deep learning, pre-trained network GoogLeNet accuracy was 97.91% for classifying spectrogram images of the gravel sounds.

Place, publisher, year, edition, pages
Switzerland: MDPI, 2021
Keywords
gravel roads; loose gravel; ensemble bagged trees; sound analysis; road maintenance; GoogLeNet
National Category
Computer and Information Sciences
Identifiers
urn:nbn:se:du-37811 (URN)10.3390/s21144944 (DOI)000677020000001 ()34300684 (PubMedID)2-s2.0-85110519300 (Scopus ID)
Available from: 2021-08-02 Created: 2021-08-02 Last updated: 2024-01-30Bibliographically approved
3. Gravel road classification based on loose gravel using transfer learning
Open this publication in new window or tab >>Gravel road classification based on loose gravel using transfer learning
2023 (English)In: The international journal of pavement engineering, ISSN 1029-8436, E-ISSN 1477-268X, Vol. 24, no 2, p. 1-8, article id 2138879Article in journal (Refereed) Published
Abstract [en]

Road maintenance agencies subjectively assess loose gravel as one of the parameters for determininggravel road conditions. This study aims to evaluate the performance of deep learning-based pretrainednetworks in rating gravel road images according to classical methods as done by humanexperts. The dataset consists of images of gravel roads extracted from self-recorded videos andimages extracted from Google Street View. The images were labelled manually, referring to thestandard images as ground truth defined by the Road Maintenance Agency in Sweden (Trafikverket).The dataset was then partitioned in a ratio of 60:40 for training and testing. Various pre-trainedmodels for computer vision tasks, namely Resnet18, Resnet50, Alexnet, DenseNet121, DenseNet201,and VGG-16, were used in the present study. The last few layers of these models were replaced toaccommodate new image categories for our application. All the models performed well, with anaccuracy of over 92%. The results reveal that the pre-trained VGG-16 with transfer learning exhibitedthe best performance in terms of accuracy and F1-score compared to other proposed models.

Place, publisher, year, edition, pages
Taylor & Francis, 2023
Keywords
Convolutional neural networks; transfer learning; deep learning; loose gravel; gravel road maintenance; road condition assessment
National Category
Other Civil Engineering
Identifiers
urn:nbn:se:du-43181 (URN)10.1080/10298436.2022.2138879 (DOI)000882702900001 ()2-s2.0-85141687244 (Scopus ID)
Available from: 2022-11-13 Created: 2022-11-13 Last updated: 2025-01-08
4. A multimodal deep learning approach for gravel road condition evaluation through image and audio integration
Open this publication in new window or tab >>A multimodal deep learning approach for gravel road condition evaluation through image and audio integration
2024 (English)In: Transportation Engineering, E-ISSN 2666-691X, Vol. 16, article id 100228Article in journal (Refereed) Published
Abstract [en]

This study investigates the combination of audio and image data to classify road conditions, particularly focusingon loose gravel scenarios. The dataset underwent binary categorisation, comprising audio segments capturinggravel sounds and corresponding images. Early feature fusion, utilising a pre-trained Very Deep ConvolutionalNetworks 19 (VGG19) and Principal component analysis (PCA), improved the accuracy of the Random Forestclassifier, surpassing other models in accuracy, precision, recall, and F1-score. Late fusion, involving decisionlevelprocessing with logical disjunction and conjunction gates (AND and OR) in combination with individualclassifiers for images and audio based on Densely Connected Convolutional Networks 121 (DenseNet121),demonstrated notable performance, especially with the OR gate, achieving 97 % accuracy. The late fusionmethod enhances adaptability by compensating for limitations in one modality with information from the other.Adapting maintenance based on identified road conditions minimises unnecessary environmental impact. Thismethod can help to identify loose gravel on gravel roads, substantially improving road safety and implementing aprecise maintenance strategy through a data-driven approach.

Place, publisher, year, edition, pages
Elsevier, 2024
Keywords
Gravel road maintenance, Data fusion, Sound analysis, Machine vision, Machine, Learning
National Category
Architectural Engineering
Identifiers
urn:nbn:se:du-48032 (URN)10.1016/j.treng.2024.100228 (DOI)2-s2.0-85184492304 (Scopus ID)
Available from: 2024-02-13 Created: 2024-02-13 Last updated: 2024-06-27

Open Access in DiVA

fulltext(952 kB)214 downloads
File information
File name FULLTEXT01.pdfFile size 952 kBChecksum SHA-512
72bfbbdc9660255e462ccbd955ee191982cd82764e1d6c1c5a4651ed154b3a2ce22321a9dac1d00d03b58404fa202343d993ef8bfca10a67e119e539df1f5ed6
Type fulltextMimetype application/pdf

Authority records

Saeed, Nausheen

Search in DiVA

By author/editor
Saeed, Nausheen
By organisation
Microdata Analysis
Infrastructure EngineeringReliability and Maintenance

Search outside of DiVA

GoogleGoogle Scholar
Total: 214 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

isbn
urn-nbn

Altmetric score

isbn
urn-nbn
Total: 1110 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • chicago-author-date
  • chicago-note-bibliography
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf