Dissertations and Theses @ UNI


Dissertation (UNI Access Only)


Machine learning; Automobiles--Motors--Maintenance and repair--Classification;


This study presents an empirical investigation of the performances of machine learning algorithms applied to classify engine issue reports. In order to determine the best classification method, three powerful classification algorithms were tested: Support Vector Machine (SVM), Random Forest, and Extreme Gradient Boosting (XGBoost). The dataset for this research was provided by a large vehicle manufacturing company. The data consists of 25,000 randomly selected engine issue reports processed by engine experts. Hyperparameters tuning was performed using 10-fold cross-validation to improve the classification power of the models. The classification models were compared by using receiver operating characteristic (ROC) curves and area under the ROC curve (AUC) values. To investigate the significance of any differences between the AUC scores of the models, five repetitions of a two-fold (5x2-fold) cross-validation paired t-test was performed. 95% confidence interval was calculated and used to assess whether changes in AUC between the best model and random or perfect classifiers are statistically significant. The test results indicated that the hyperparameters tuning significantly improved the performance of all three algorithms. The comparison of the tuned models showed that the XGBoost classifier (AUC = 0.9466) performed significantly better than Random Forest (AUC = 0.9289), SVM (AUC = 0.9256), and random guess (AUC = 0.5), but its performance is worse than the perfect classifier (AUC = 1.0). The results indicated that machine learning algorithms (specifically XGBoost, Random Forest, and SVM) could be used as decision-support aids to improve the process of classifying engine issue reports.

Year of Submission


Degree Name

Doctor of Industrial Technology


Department of Technology

First Advisor

Shahram Varzavand, Chair

Second Advisor

Sadik Kucuksari, Co-Chair

Date Original


Object Description

1 PDF file (viii, 92 pages)



File Format


Off-Campus Download