Early Detection of the Advanced Persistent Threat Attack Using Performance Analysis of Deep Learning

English. publishedVersion

English. One of the most common and critical destructive attacks on the victim system is the advanced persistent threat (APT)-attack. An APT attacker can achieve its hostile goal through obtaining information and gaining financial benefits from the infrastructure of a network. One of the solutions to detect a unanimous APT attack is using network traffic. Due to the nature of the APT attack in terms of being on the network for a long time and the fact that the system may crash due to the high traffic, it is difficult to detect this type of attack. Hence, in this study, machine learning methods of C5.0 decision tree, Bayesian network, and deep learning are used for the timely detection and classification of APT-attacks on the NSL-KDD dataset. Moreover, a 10-fold cross-validation method is used to experiment with these models. As a result, the accuracy (ACC) of the C5.0 decision tree, Bayesian network, and 6-layer deep learning models is obtained as 95.64%, 88.37%, and 98.85%, respectively. Also, in terms of the critical criterion of the false positive rate (FPR), the FPR value for the C5.0 decision tree, Bayesian network, and 6-layer deep learning models is obtained as 2.56, 10.47, and 1.13, respectively. Other criterions such as sensitivity, specificity, accuracy, false-negative rate, and F-measure are also investigated for the models, and the experimental results show that the deep learning model with automatic multi-layered extraction of features has the best performance for timely detection of an APT-attack comparing to other classification models.